sb0 changed the topic of #m-labs to: ARTIQ, Migen, MiSoC, Mixxeo & other M-Labs projects :: fka #milkymist :: Logs http://irclog.whitequark.org/m-labs
<cr1901_modern>
rjo: Oh right. I guess it's just a coincidence the pullup i/o in the flash proxies match the qspi data lines
<sb0>
rjo, oh yes, probably, good catch
<sb0>
though fixing this sounds like a mess, since there are s/h constraints on CLR...
<sb0>
sigh, xilinx
<sb0>
also what you propose will result in a reduced range of tx clk/sys clk phases, in addition to the non-determinism: "When CLR (reset) is deasserted, the output clock transitions from Low to High on the first edge after the CLR is deasserted, regardless of the divide value. Therefore, BUFGCE_DIV output clocks are always aligned, regardless of the divide value"
<sb0>
I think I'll just use another MMCM
attie has quit [Ping timeout: 240 seconds]
attie has joined #m-labs
<sb0>
sayma-1 still pings after running overnight ...
futarisIRCcloud has quit [Quit: Connection closed for inactivity]
<rjo>
sb0: no. sayma-3. i'd like to check the hmc830 with clock. and then at some point merge spi2 into master.
attie has quit [Remote host closed the connection]
attie has joined #m-labs
<sb0>
rjo, sorry I messed up a ftdi command and reflashed sayma-3
<sb0>
rjo, I'll try the 100M clock tomorrow
<rjo>
sb0: just now?
<sb0>
yes, just now
<rjo>
sb0: ack. can i go again?
<sb0>
rjo, yes
<rjo>
sb0: another question. the si5324 output phase is undetermistic. do we use it on the satellites for RTIO?
<sb0>
it's undeterministic?
<rjo>
sb0: and that's also a bit problematic since it is undeterministic for kasli when locked to an external reference.
<sb0>
Joe and I tested it with the early transceiver test code and found it deterministic
<rjo>
sb0: yes. i don't see how we can synchronize the output dividers (nc12_ls, n1_hs)
<sb0>
there is r
<sb0>
even that skew control register to tune the phase
<rjo>
sb0: that's why i am asking. i am surprised that you didn't see anything.
<sb0>
isn't there some internal control of those dividers?
<sb0>
it would actually be very problematic if it were undeterministic
<rjo>
sb0: internal control to what?
<rjo>
the fact that vco and output are an integer of the input doesn't factor into any of the settings
<rjo>
sb0: maybe i'm overlooking something.
<sb0>
something built into the chip that resets the dividers in a deterministic manner
<rjo>
sb0: things like "Input to output phase skew after an ICAL is not controlled and can assume any value" seem to generally confirm that it's not defined. and the skew control is onlye between the two outputs.
attie has quit [Ping timeout: 248 seconds]
rohitksingh_work has quit [Read error: Connection reset by peer]
attie has joined #m-labs
<rjo>
and 6.2.4 in the family datasheet.
<rjo>
you could use the output skew adjust, but only if you measure it, and only if the nc12_ls are 1 (which they can be in our case).
rohitksingh_work has joined #m-labs
<sb0>
hm, we did test that...
<sb0>
let me check what values were there
<rjo>
and that won't work either because the vco phase is much more undetermined.
<sb0>
no, we used higher values
<sb0>
ffs that's annoying
<sb0>
we can put a fpga mmcm before it, and adjust it to set the phases
<sb0>
though it does seem to have constant skew in practice, already
<rjo>
sb0: but that mmcm won't even come close to the resolution required. and you still need to measure the phase which is tricky due to the jitter on that clock.
<rjo>
sb0: maybe it works in practice. as you said you and joe tested that last year. but i'd really like to know how that works.
<sb0>
on ultrascale the mmcm can do fine phase shifts on one output
<sb0>
let me check 7-series
<rjo>
sb0: down to 20 bit resolution like the n3/n2 dividers of the si5324?
<sb0>
rjo, also possible on 7-series. 1/56's of the VCO period, so 10's picoseconds
<sb0>
maybe that's why the white rabbit people are using their own PLL, though I did ask one of the developers about it and didn't get that answer
<rjo>
that would kill it for DRTIO between two DDS. their phase would be undeterministic by those 10s of ps.
<rjo>
for me that was the single most important thing that needed to be checked with the drtio demonstrator way back. let's check it again.
<sb0>
yes...
<sb0>
though, for the most time-sensitive applications people want to supply a dedicated clock (and not use the si5324 output), so the 10's ps would be absorbed into the synchronous timing budgets and not be an issue
hartytp has joined #m-labs
<hartytp>
sb0: ^ not sure I agree with that last statement
<hartytp>
about "most sensitive"
<rjo>
sb0: not for any of the "jitter cleaner" applications that end up feeding a dac/dds clock.
<hartytp>
for applications where one doesn't care about this determinism this is obviously a non-issue
marmelada has joined #m-labs
<hartytp>
but, any time where one wants phase determinism, one wants it at the degree level or better
<hartytp>
(few degree phase determinism is not very useful IMO)
<sb0>
hartytp, what I'm saying is, the si5324-clocked signals (with 10's ps indeterminism) would be reclocked with a deterministic clock
<hartytp>
say our typical DDS/SAWG RF is in the region of 400MHz (give or take a factor of 2)
<hartytp>
that's 7ps
<hartytp>
per degree
<hartytp>
right, I get that.
<hartytp>
The point I was getting at is that this would completely nix the idea of using the recovered clock as a reference for anything where one wants phase determinism
futarisIRCcloud has joined #m-labs
<hartytp>
e.g. it would basically make Sayma only useful with an external 100MHz ref. That's a shame since, IMO, being able to recover a pretty low noise from the DRTIO fibre is a really nice feature of Sinara
<rjo>
hartytp: much worse. 16 bit phase of a ~300 MHz DDS from SAWG is 50fs.
<hartytp>
not a killer though
<marmelada>
rjo: what's your setup for urukul+kasli? I want to run your example
<sb0>
hartytp, this is a strong argument for using the 7044 that you recommended
<sb0>
if it doesn't also have this problem (can't find much in the datasheet yet)
<hartytp>
rjo yes
<hartytp>
well, it has this problem if you use the r divider/prescaler
<hartytp>
that's true
<rjo>
marmelada: opticlock variant, connect once of the clock outputs to the urukul mmcx, connect urukul-eem0 to kasli-eem5 and urukul-eem1 to kasli-eem4 (see the target definition).
<hartytp>
rjo: to use the HMC7044 to get the determinism, you'd also have to have it on Sayma AMC, right? Otherwise, you add the indeterminism on the AMC, which then passes it on to the RTM
<rjo>
hartytp: anything that does not do very special gymnastics to clear the input, feedback and output dividers in the si5324/hmc7044 in a defined way won't achieve it.
<hartytp>
plus, you'd need to avoid having any Kasli in the tree between the master and Sayma
<rjo>
hartytp: yes.
<marmelada>
rjo: +100 MHz clock to SMA?
<sb0>
i f
<rjo>
marmelada: currently it uses the si5324 free running.
<rjo>
currently == current default code
<marmelada>
ok
<hartytp>
rjo: is that right? If you use the HMC7044 as a straightforwards integer n all then it's deterministic, right. The edges of the output clock are always aligned with the input clock by the PLL
<hartytp>
?
<hartytp>
AFAICT, the indeterminism only comes in if you divide the input clock down
<hartytp>
multiplying is fine
<rjo>
hartytp: i don't know. does it clear input and output dividers at the same time? those do divisions.
<hartytp>
well, you don't have to use an input divider with the HMC7044
<rjo>
yes. clearing the feedback divider (the "multiplier") is not needed.
<marmelada>
rjo: I tried to use I2C (self.i2c_switch0.set(7)), however I get an exception that I2C device does not ack address
<hartytp>
or the output divider
<hartytp>
if you do, I agree you're toast potentially
<hartytp>
hmm....actually, you do probably want the output divider
<marmelada>
is i2c implemented or is there some kind of preliminary code there?
<hartytp>
point taken
<rjo>
marmelada: your code? no idea. is the switch fine?
<rjo>
marmelada: what do you mean by "implemented"? we use i2c to set up the si5324.
<rjo>
but that's it.
<marmelada>
I tried to use artiq function
<marmelada>
mine code works
<hartytp>
(otherwise you need one PLL to clean the DRTIO clock and one PLL to generate the DAC clock, you can't use the nice scheme of dividing the DAC clock to get the RTIO clock)
<marmelada>
*my
<rjo>
marmelada: i never tried the artiq kernel interface. just what we do in the runtime.
<rjo>
marmelada: i'll give it a try.
<rjo>
hartytp: ack.
<hartytp>
hmm...the 7044 does have a nice SYNC input, so probably something one can do. but not clean. i hate phase
<hartytp>
anyway, this could have a big input on the way we clock things (I was planning to try only using the DRTIO clock as the reference to all my RF) so it would be good to redo that test
<sb0>
the WR PLL starts to look good...
<hartytp>
sb0, rj0: nice work on the ethernet!
<sb0>
but iirc it is limited to one frequency (they're using the fine tuning signal on a VCXO)
<marmelada>
rjo: also, can I just change definitions of ttls in device_db from outputs to inouts? bc I haven't seen any declaration of direction in misoc or migen files...
<hartytp>
sb0: need to re-read that, but IIRC the limitations are: 1. doesn't work well with different DRTIO frequencies (fixed frequency oscillator to avoid dividers) 2. needs its own PLL so can't use an IC like the 44 to do all in one
<hartytp>
sb0: so the major roadblock to sayma is now the SDRAM AFAICT. All else seems to work (apart from the 830, which is not crucial)
<hartytp>
what's the plan for the RAM?
<sb0>
it needs several parts, it's a FPGA-controlled DAC driving a VCXO
<rjo>
marmelada: the gateware is defined the target.
<hartytp>
that's basically just implementing an integer-N PLL in the FPGA to be cheap, right?
<hartytp>
and to have a nice digitally tuneable loop filter
<hartytp>
rather than doing it in analog
<hartytp>
830 etc does the same, right?
<hartytp>
well, no, not the 830 at 100MHz ish. But, any standard integer-N all driving a VCXO
<hartytp>
PLL
<hartytp>
sb0: re SDRAM
<rjo>
marmelada: doesn't work here either. but i'm not sure it's supposed to work. let me check on kc705
<sb0>
I don't think it's particularly cheap. they use it as a jitter filter to replace si5324-style chips
<sb0>
on transceiver recovered clocks
<hartytp>
Am I right in thinking that the big changes in the location of the working regions could be due to the same divider issue that rj0 spotted on the ethernet
<hartytp>
?
<rjo>
sb0: does the NRT I2C kernel API work currently?
<hartytp>
in that case, since we tune the delays at startup, they're nothing to worry about
<sb0>
and it's fixed at 125MHz, since WR only does GbE
<rjo>
hartytp: no.
<sb0>
rjo, it's unittested so it should
<rjo>
sdram is something else afaict.
<sb0>
there's a kc705 unittest that talks to the i2c switch
<hartytp>
rjo: the changes in the location of the working region (working in the loosest possible sense)? Or, the gaps in the working region? Or, both?
<rjo>
hartytp: and i don't think _florent_ and sb0 should look at it until they have boards where the power supplies are fixed.
<hartytp>
sb0: right, but conceptually at least, I think that's just a way of implementing an integer-N pll. I don't think it's anything special unless I've missed sometihgn
<rjo>
hartytp: changes in the location of the working region could be. but that would not be an issue.
<hartytp>
rjo: right, just want to check I understand what's going on and know what I can safely ignore versus worrying about. Those shifts has surprised me since I expected the FPGA timing to be deterministic
<rjo>
hartytp: not the noise which looks like SI
<hartytp>
rjo. ack
<hartytp>
rjo: that sounds like a good plan to me.
<hartytp>
in that case, the action plan is to chase Greg up to make sure
<hartytp>
1. we get the MMCM plan that fixes the supplies asap
<hartytp>
2. he looks at the SDRAM on a scope to check for SI issues
<rjo>
hartytp: the shift could also be the IDELAY which has its own clock and maybe some other phase again. but i have no idea about that.
<hartytp>
if neither of those turn anything up then we can relook at the gateware. Happy with that plan?
<hartytp>
okay
<rjo>
s/MMCM/Exar/, yes.
<hartytp>
right
<hartytp>
marmelada: can you poke Greg to remind him to do those two things, please?
<hartytp>
I want to see some RF from Sayma
<marmelada>
hartytp: ok
<hartytp>
thanks!
<hartytp>
:)
<rjo>
marmelada: it works on kc705. i'll check why it doesn't on kasli.
<hartytp>
sb0: once the media converter Greg's posted to me arrives I'll patch my board and test your new ethernet code
<sb0>
hartytp, good
<sb0>
you have no other media converter right now?
<hartytp>
FYI, if you have time it might be worth you adding an AC termination resistor to the ethernet trace you added. That might increase the eye quite a bit and make things more reliable for you (it's a very quick fix to apply)
<sb0>
FYI I tested the standalone design with SAWG, without SAWG, and the drtio master, and ethernet worked with the current code
<hartytp>
looking at the photos, it looks like the wire Greg added for his rework was a bit shorter than the one you added, so he may have had fewer SI issues. Could be part of the reason things worked better for him?
<hartytp>
okay
<sb0>
only one board though
<sb0>
things work better here than for greg. he could not get rx to work.
<hartytp>
k
<sb0>
it seems quite reliable here now
<hartytp>
remind me how the ethernet is done. You're using the SATA connector and a SATA to SFP adapter, right?
<sb0>
no lost packets, no long-term breakage
<hartytp>
okay. Well, I'd be interested to see if an AC termination widens that eye up
<sb0>
no breakage from different bitstream content
<rjo>
marmelada: just the wrong address
<hartytp>
I can check that here if you tell me how to do an eye scan from your code
<sb0>
yes SATA port 0 to 1000-BASE-X direct
<marmelada>
rjo: if I modify Opticlock variant I have to manually modify device_db to reflect chagnes or is there some generated device_db?
<sb0>
you can cut a sata cable and solder it but I don't recommend it, it's very fragile
<hartytp>
right, so I need the SATA to SFP adapter. That's what Greg is posting me
<hartytp>
sb0: right, I can do that but Gregs board is in the post
<marmelada>
rjo: hm? I thought 0x70 should work
<sb0>
yes those PCBs Greg have made are very nice
<sb0>
also they can power SFP modules
<hartytp>
in any case, might as well wait for the EXAR fix as well (I can't do anything with Sayma until mem test passes so not much point rushing atm)
<marmelada>
hartytp: 8 are tested and taken by technosystem to install heatsinks and 2 missing sfp cages
<sb0>
cjbe, that needs special transceiver code. it's implemented for ultrascale, but not kasli (not needed for satellites). it's otherwise done in the runtime etc.
<sb0>
ultrascale is also not thoroughly tested yet
<marmelada>
they also took the untested ones to install panels
<hartytp>
Great! We're expecting three from this batch, do you think they'll ship this week?
<marmelada>
I see no reason why not
<hartytp>
(even one would be a big help for the current DRTIO tests we're doing atm)
<hartytp>
Whoo! Nice job.
<cjbe>
sb0, understood. Do you have an ETA for that for Kasli? (Master + 2 satellites is part of the DRTIO contract)
<hartytp>
(Kasli is a really cool board BTW, I'm very happy with it)
<hartytp>
marmelada: also, what about the BP adapter
<sb0>
cjbe, it's in another contract. will come later.
<marmelada>
btw, I installed small heatsink on kasli v1.0 we have in our lab and temps dropped ~5 deg
<hartytp>
really want to test that out as well
<hartytp>
any eta
<hartytp>
?
<hartytp>
yes, the temperature is still something I'm concerned about.
<hartytp>
wonder if we should have gone for the pcb mount heat sink.
<marmelada>
hartytp: layout is nearly finished, however I still wait for an answer from technosystem where will they produce the board
<sb0>
cjbe, another problem is the sfp1 routing on kasli. do you have a 1.1 board, where iirc this is fixed?
<marmelada>
bc I need to know stackup to set rules for diff impedance
<sb0>
we don't have them yet...
<marmelada>
sb0: did you have problems with sfp1?
<sb0>
iirc it didn't pass some tests so I didn't touch it
<marmelada>
if those were my tests I must have made some kind of mistake then, after a while I was able to run ibert at 6,25 Gbps on all channels
<marmelada>
probably I didn't reset ibert correctly
<sb0>
didn't rjo find that the differential pair was too close to something else?
<cjbe>
sb0: pg.2 of my DRTIO RFQ: "6. Repeat of above tests with one master and >=2 satellites."
<marmelada>
sb0: yup, however these were control lines so ~dc
<sb0>
cjbe, well, we don't have transceiver code to test that right now. but there you can see the code to handle multiple links, and the deterministic latency is done by the transceiver part
<sb0>
cjbe, and that transceiver code is part of the "kasli master" contract
hartytp has quit [Ping timeout: 260 seconds]
<marmelada>
rjo: bump, if I modify Opticlock variant I have to manually modify device_db to reflect chagnes or is there some generated device_db?
<sb0>
marmelada, you have to modify device_db
<marmelada>
ok
<cjbe>
sb0, and do you have an ETA on that? The DRTIO contract relies on this, and I would prefer to get that contract signed off before the end of March.
<rjo>
hartytp: please consider that we also need those boards to develop and test gateware for you. you don't want to have hardware you can't use because we don't have that hardware to make it usable for you.
<hartytp>
rjo: ack
<hartytp>
there are 8, so I figured that would be enough for us to have 3 and you to have 5
<hartytp>
but, if there is a good reason why you need more than that then we can wait a bit
<hartytp>
at least 1 more board asap would be a help though
<sb0>
cjbe, I'll also test the ultrascala code but only some SFPs work on Sayma for unknown reasons, so that looks like another PITA
<hartytp>
sb0: did you file an issue about that?
<hartytp>
marlemada: will be interested to see how hot Kasli runs with all 12 EEM connectors running LVDS
<sb0>
could be transceiver electrical settings, that needs more investigation
<hartytp>
and a sensible airflow (e.g. in a rack with other EEMs also putting out heat, rather than on a desk with a large fan dedicated to Kasli)
<rjo>
hartytp: sounds good. if they ship them that way, sb0 needs one, you get three, and i get four.
<hartytp>
maybe will be fine, but could be an issue
<hartytp>
sounds fine to me
<hartytp>
let's just make sure we coordinate with marmelada and technosystem to ensure that's how it goes (otherwise, there seems to be a tendency for boards to be shipped according to a Monte-Carlo method)
<hartytp>
rjo: so with the DRTIO phase determinism.
<hartytp>
1. *if* we could rely on a fixed RTIo frequency it would be easy
<hartytp>
use an integer-n pll with no dividers and job's a good un
<hartytp>
for the RTM, we could use the HMC7044 with a VCXO
<sb0>
but yeah I'll open an issue
<hartytp>
to get the clean rtio clock, we put the VCXO through a low noise clock buffer (ADCLk925) before putting it into the PLL
<hartytp>
that allows us to get the RTIO clock out directly without going through any dividers
<hartytp>
also, FWIW, an advantage of the WR approach of doing the PLL on the FPGA (VXCO + DAC) is that it allows one to support very low loop BWs with an aggressive LF cut-off. that can be hard to implement in analog without instabilities
<hartytp>
sbo: acl
<hartytp>
ack
<hartytp>
if we could get Sayma running at 125MHz RTIO clock (1GSPS data rate) which is my preferred option anyway and agree that that's the only configuration we support with phase determinism, then we could do this like WR, which might be better...
<marmelada>
rjo: File "/home/pawel/miniconda3/envs/artiq-dev/lib/python3.5/site-packages/misoc/cores/spi2.py", line 294, in __init__ cs.reset = C((1 << n) - 1) NameError: name 'n' is not defined
<rjo>
marmelada: update it.
<marmelada>
hm, I thought conda update from file would update it
<marmelada>
File "/home/pawel/artiq-dev/artiq/artiq/gateware/rtio/sed/fifos.py", line 29, in __init__ fifo_cls = AsyncFIFOBuffered NameError: name 'AsyncFIFOBuffered' is not defined
<rjo>
sb0, whitequark: fyi, since package versions in the dev channel supersede the main channel i removed the dev label from all pyqtgraph and llvmlite-artiq versions.
hartytp has joined #m-labs
<hartytp>
_florent_ did you test SC1 on Sayma with the new 7043 code?
<hartytp>
I don't expect anything I changed to make that work, but let me know if there are any more tests you want me to run
<hartytp>
also, did you have a chance to look at the swerwb timing issues sb0 identified?
futarisIRCcloud has quit [Quit: Connection closed for inactivity]
<hartytp>
just want to make sure it doesn't get forgotten about -- these small bits of unreliability on Sayma are a real pain, and fixing the some of the timing issues could hel
<hartytp>
help
<rjo>
whitequark: ^ i fixed that build issue (it wasn't conda as you were quick to cast blame). now could you fix your new artiq-board recipes?
hartytp has quit [Client Quit]
<sb000>
rjo, ok
<sb000>
i've actually used post place, but I could have used post route as well (waiting a bit longer...)
<sb000>
niche use case anyway, not worth slowing down regular builds
<rjo>
ok. then i'll rmeove post-synth and post-place. the post-route is nice and serves as a template to add your own checkpoints.
<whitequark>
I'll fix the packages in release-3 in a moment
<rjo>
whitequark: hmm. i can't pin point it anymore. i did several things at the same time and did not notice that the failure mode changed until i applied the final fix. could have been anything from cleaning up the dev/main labels on conda, bumping jesd204b, cleaning up sources, packages, tarballs caches, upgrading conda-build and conda.
<rjo>
i throught it was the labels but i'm not sure anymore.
<whitequark>
I'm pretty sure that qualifies as "it was conda".
<whitequark>
if not a specific bug then the general flakiness of its design.
<GitHub177>
artiq/release-3 d0d150d whitequark: conda: add back py_ prefix in dependencies.
<GitHub38>
[artiq] gkasprow commented on issue #854: I used my code and default drive strength. Now I'm compiling version with 16mA drive strength. @sbourdeauducq rework was on Rx side but the problem exists on Tx side. https://github.com/m-labs/artiq/issues/854#issuecomment-368547794
cjbe_ has quit [Read error: Connection reset by peer]
<cjbe__>
sb0: the Kasli DRTIO Master + Satellite examples at the moment do seem to generate a 'fine' rtio clock (rtiox4). Can you explain to me how I should set up the clocking to do this?
<sb0>
cjbe__, I would suggest MMCM with delay compensation (so that the input is aligned with the output), input on the "rtio" clock (which is the jitter-cleaned output of the si5324), and one 4x output
<sb0>
the si5324 "rtio" clock goes to the transceiver, so the mmcm cannot be inserted there... but delay compensation should work
<rjo>
sb0: is tha apparent lockup after "INFO(board_artiq::hmc830_7043::hmc830): waiting for lock..." new? no register dump, no panic, just lockup? is it a serwb lockup? happens with both master and spi2 on sayma-3
<GitHub130>
artiq/spi2 7f05083 Robert Jordens: kc705: switch backplane spi to spi2
<GitHub130>
artiq/spi2 e87af7b Robert Jordens: hmc830: be explicit about SPI mode selection
<GitHub130>
artiq/spi2 d84f573 Robert Jordens: firmware, sayma: port converter_spi to spi2...
<rjo>
bb-m-labs: force build --branch=spi2 artiq
<bb-m-labs>
build #2118 forced
<bb-m-labs>
I'll give a shout when the build finishes
<cjbe__>
sb0: understood. Can you point me to where the RTIO clock is created in (for example) the Kasli Master example?
<cjbe__>
I see the Si5324 output connects to the QPLL (this is the 'jitter-cleaned output' of the Si5324 you were referring to I assume). However I don't see how the RTIO clock domain is driven from this