<sb0>
rjo, do we distribute another clock for RTIO or do we run everything off the DAC clock?
<whitequark>
should we use the new i2c core for that?
<whitequark>
or just the existing bitbang stuff
<sb0>
bitbang
<whitequark>
ok
<whitequark>
sb0: so in the CSR generator there's this code:
<whitequark>
r += "\tr <<= "+str(busword)+";\n\tr |= MMPTR("+hex(reg_base+4*byte)+");\n"
<whitequark>
why does it shift by `busword` but advance the pointer by 4 bytes always?
<whitequark>
oh, do peripherals with byte-sized bus have layout like... D0 XX XX XX D1 XX XX XX etc ?
<sb0>
yes
<rjo>
sb0: i'd prefer to run as much as possible of the RTIO clock since that's our timebase.
<rjo>
sb0: the tricky thing with any other clock (other than the link-clock) is that it needs to be a multiple of the RTIO clock or else we need to do some funny measurement and realignment. like we have to do for sysref.
<sb0>
how many AMC clocks do we have?
<sb0>
anyway we'd just have the RTIO clock on AMC, correct?
<rjo>
there is something funny about tclka being restricted somehow (it's also "fabric clock") and then tclkc and tclkd come from the right side.
<rjo>
i think so yes.
<rjo>
at most. but that's also not a given.
<sb0>
we can also use another AMC clock as time reference signal, should the transceiver sync become problematic for some reason
<rjo>
that's nasty.
<sb0>
(the Chris scheme which doesn't rely on fixed transceiver latency)
<sb0>
I mean, plan it in the hardware, we can ignore it
<rjo>
you can always trade fixed transciever latency for an additional synchronization signal.
<sb0>
?
<rjo>
that's what he does.
<sb0>
all I'm proposing is support this signal on the boards in case it's really required, we should try to ignore it
<rjo>
if you distribute "an RTIO clock" on the AMC backplane, you also need to distribute a synchronization signal that marks a certain cycle. or otherwise you have to match the drtio link latencies so that they always uniquely and reliably identify a certain cycle.
<sb0>
yes, we can probably do that with fixed transceiver latency
<sb0>
what exactly is "nasty"?
<rjo>
not just fixed. but also "good values" so that they reliabley identify a certain cycle and don't come close to the edges.
<sb0>
yes
<rjo>
the nasty thing is that you have to calibrate the link latencies w.r.t. the clock path lengths and adjust them.
<sb0>
we need a configurable delay element somewhere
<rjo>
not if the drtio link provides both clock and marker.
<sb0>
so no RTIO clock is distributed?
<sb0>
<rjo> sb0: i'd prefer to run as much as possible of the RTIO clock since that's our timebase.
<rjo>
yes. i would not distribute a separate RTIO clock.
<sb0>
iirc ultrascale supports using the idelays for internal paths
<rjo>
in that quote "the RTIO clock" meaning "the link recovered RTIO clock".
<sb0>
okay
<rjo>
i am fine with cleaning it up (as long as the clean-up is fixed latency).
<sb0>
makes the backplane requirements simpler too
<sb0>
yes, this si5324 chip has fixed delay
<sb0>
configurable even
<rjo>
nice.
<sb0>
i'm actually quite happy with this si5324, it's pretty well designed as far as modern chips go
<sb0>
okay, now the DAC clocks
<rjo>
sure. it's nice and small and certainly good enough for the digital stuff.
<sb0>
I suppose they are derived from the same oscillator as the RTIO clock?
<sb0>
and doesn't behave in stupid, buggy or obscure ways, unlike e.g. the ad9914
<sb0>
or ftdichips
<rjo>
ultimately yes. we demand exacltly fixed ratio, fixed phase.
<sb0>
okay but then the phase tweaking problem is there again
<rjo>
absolutely.
<rjo>
a) measure alignment, b) delay if s/h violated and store that delay c) shuffle samples
<sb0>
okay, we need a good way to detect s/h violation points
<rjo>
that shuffling would be within a coarse RTIO cycle. alternatively we could delay in an elastic buffer up to the next coarse cycle. that might be easier.
<sb0>
we're going to put those ADI HMCxxxx clock chips on the Sayma RTMs, right?
<rjo>
the measurement would be an IDELAY scan of SYSREF and then choose the middle, store that value, reuse on next boot.
<rjo>
yes.
<sb0>
ok, so they can feed a divided DAC clock that the FPGA can check for alignment with the RTIO clock#
<rjo>
sysref is just fine for that.
<sb0>
ok. the clock chips have many (buffered) outputs, we should use them
<rjo>
ah. no for c) (the shuffling) we need to keep in mind the varying phase of sysref between boots. so we need shuffling within the ~8 parallel samples.
<rjo>
well yes. at least one deviceclock for transcievers and the fabric between eb and xcvr. and sysref.
<sb0>
can't we control the sysref phase?
<sb0>
and how would we measure it anyway?
<rjo>
with the mechanism that oxford sketched. yes. but that comes at a cost.
<rjo>
as i said above.
<rjo>
the measurement would be an IDELAY scan of SYSREF and then choose the middle, store that value, reuse on next boot.
<rjo>
idelay scan plus tdc timestamp.
<rjo>
would a 128 bit wide DDR3 be hard? apart from needing more chips and more wires?
<sb0>
okay I'm not sure if I understand this
<sb0>
wasn't the Oxford plan to fix sysref at the source, distribute to each RTM, then have this clock chip distribute to each DAC chip?
<rjo>
if by "fix sysref" you mean the entire string of the sync signals to and through the fpgas, the first hmc7043 that re-times it and then the clock distribution chip, then yes.
<rjo>
the consensus from my e-mail exchange with tom after the meeting warsaw was that their method would restrict the clock frequencies that they distribute too much (or otherwise would require yet more logic and questions if that refclock and rtio clock are not commensurate).
<rjo>
that was their "active sysref alignment" approach. the "measure sysref" approach was the one i described above. its downside is the need for an idelay scan and the need to shuffle samples.
<sb0>
ok, so it's the system I described above, but the sysref/rtio clock phase varies at every boot?
<sb0>
for 128-bit DDR3, we have to check if there are enough HP I/O pins
<sb0>
do we have a direct connection between an external master oscillator (at x GHz) and DAC clock inputs, or do we abandon this idea?
<rjo>
yes. and in the oxford design you wouldn't need the measurement of sysref and the sample shuffling.
<whitequark>
bb-m-labs: force build --props=package=rust-core-or1k conda-lin64
<GitHub1>
[misoc] whitequark pushed 1 new commit to master: https://git.io/vPTfh
<GitHub1>
misoc/master 04481ea whitequark: Add Rust output to the CSR generator.
<bb-m-labs>
build forced [ETA 1h01m21s]
<bb-m-labs>
I'll give a shout when the build finishes
kuldeep has joined #m-labs
<rjo>
i don't know. there seem to be two different fall-back clocking approaches. clock by 100 MHz externally with sma or clock directly with x GHz externally.
<rjo>
strictly worse at the relevant carrier offsets. but afaict they are good enough to not degrade.
<sb0>
unless we want to have a dozen SMA inputs and let the user deal with feeding them with the appropriate phases. but this obviously sounds like a horrible mess.
<rjo>
yes. but iirc the "clocking through rtm panel" at 100 MHz (or whatever) is still there.
<rjo>
replacing the rf backplane clock. but i think a simple passive splitter is fine there.
<sb0>
this is just connecting the master clock chip (the one that generates the xGHz DAC, sysref, and metlino TX RTIO clock) to an external reference, correct?
<rjo>
i'd still clock the drtio tx on sayma from the (cleaned) recovered rtio clock. that should make debugging a bit simpler and separates the clocking dependencies.
<GitHub39>
[misoc] whitequark pushed 1 new commit to master: https://git.io/vPTUj
<GitHub39>
misoc/master 806125b whitequark: libbase: fix definition of offsetof()....
<rjo>
and i think on metlino i'd do the same: it's either the grandmaster where the root rtio clock is generated and distributed over the links or it receives rtio clock from its upstream.
<sb0>
the metlino needs a reference clock that is derived from the same oscillator as the DAC to transmit DRTIO data
fengling has joined #m-labs
<rjo>
which mode of metlino? the core device or a satellite?
<sb0>
core
<rjo>
then correct.
<sb0>
clocking DRTIO TX on Sayma from the recovered clock avoids one CDC with det-lat issues
<sb0>
and allows RTT measurements (with e.g. DDMTD) on the metlino
<rjo>
in practice it will most likely be a 100 MHz distribution amplifier feeding the core device metlino, each rf backplane, and standalone saymas.
rohitksingh_work has quit [Quit: Leaving.]
<rjo>
exactly. that's why i wouldn't clock the drtio tx on sayma from what the fpga gets through "its" dac/adc clock dist/divider chip.
<rjo>
about the drtio design: afaics the "switch" feature set (enumeration etc.) would be integral and not optional as already the core device would need its functionality.
<sb0>
there are no issues with this scheme, since we don't have any other CDC
<rjo>
yes. metlino satellites don't need to see any other clock.
<sb0>
it simply recovers the RTIO clock from its fiber, cleans it, and uses it for its TX instead of the locally generated clock used in core mode
<rjo>
yes
<sb0>
so, the nasty part is aligning a RTIO clock with the DAC clock/sysref
<sb0>
also, when we have two crates, will they get different sysrefs?
<rjo>
it would need to have stuff to do the ddmtd measurements though.
<rjo>
sysref is local to each sayma.
<sb0>
can't we use a PLL for DDMTD?
<sb0>
okay then I don't understand
<sb0>
you are proposing to do the same as it is done on the FMC card?
<rjo>
sure. but DDMTD needs to happen at each switch (each metlino) independent of the mode.
<sb0>
I mean FPGA PLL
<rjo>
yes.
<larsc>
rjo: it really depends on your application which one you want to us hmc7044 or ad9528. But the hmc7044 has a few more featurs
<sb0>
hm
<sb0>
sysref tagging will still need ~200ps precision right?
<rjo>
sb0: yes. testing it with the fmc's also pushes all the right "risk mitigation" buttons.
<sb0>
will that work across two crates and meters of fiber?
<rjo>
sb0: "tagging" to within 1/2.4GHz = 416ps yes.
<sb0>
hmm
<sb0>
so we need that sort of precision for the time synchronization between all cards
<sb0>
(over DRTIO)
<sb0>
what if the sysref phase falls at the limit of two zones that cause different sample reshufflings?
<rjo>
that's what i described above.
<sb0>
where?
<rjo>
you scan it the first time. on next boot you use that value to track.
<sb0>
how does it help?
<rjo>
scan, decide, store, track.
<sb0>
won't the sysref phase be completely different on the next book?
<sb0>
*boot
<rjo>
let's say t_rtio = 8ns, t_dac = 1/2.4GHz.
<rjo>
then, on your first virgin boot, you do:
fengling has quit [Quit: WeeChat 1.4]
<rjo>
measure a sysref edge w.r.t. an rtio edge by an idelay "eye" scan to be d_sysref~4.5*t_dac. but that's not allowed, so you choose (once!) 5*t_dac, you shuffle samples by 5, store "4.5" and "5" and next boot you scan around 4.5 and make the same decision. this works as long as this is reproducible to within +-.5 t_dac.
<rjo>
sorry. you store "0.5" and "1". next boot, you measure 9.4 but choose 10. shuffle by 10.
<rjo>
maybe "slip" is better than "shuffle" here.
<sb0>
is the sysref phase consistent between boots?
<rjo>
no.
<sb0>
you are assuming that its inconsistency is an integer multiple of the DAC clock?
<rjo>
yes.
<rjo>
but that comes with the "integer PLL" thing they are guaranteeing. then the hmc7044 is just a divider.
<sb0>
where is the "1" that you store coming from?
<rjo>
since you stored your choice to round 4.5 up to 5, you'd also round up 9.4 to 10.
<rjo>
maybe better description:
<sb0>
yes, I understand that
<rjo>
you'd store -.5
<sb0>
but not what you store
<rjo>
just the delta between your measurement and the decision.
<rjo>
yes. you round each time. just the first time (virgin setup) you start with delta=0 initialized.
<sb0>
okay
<sb0>
has anyone ever done this?
<rjo>
don't know. but afaict there is no other way to do it with the fmc's.
<rjo>
there are a couple +- mixups in that sentence of mine above. but you get the idea.
<sb0>
yes
<rjo>
so the other two advantages of this approach over the oxford one is a) its required for the fmcs and b) its testable with the fmcs.
<rjo>
*are
<rjo>
how accurately do you get the ddr3 trained with an idelay scan?
<rjo>
and what we could do when the drtio clock is too loose to reproducibly scan sysref, without changing the hardware, we can clock the entire rtio side from the deviceclock (or better from the clock that comes into the hmc7044). we would need a (very relaxed) version of the same delay measurement and rounding scheme.
<rjo>
you'd assume fixed phase, scan the drtio-to-deviceclock delay and round it with the same tracking mechanism.
<sb0>
the disadvantage is, if it doesn't work, then no hardware works
<rjo>
which "it"?
<sb0>
idelay scan + decision storage
<rjo>
for the original version with the rtio clock from the link?
<rjo>
we can then do the other approach with rtio clock from deviceclock.
<sb0>
ok
<rjo>
we would still have to measure sysref to within one dac clock cycle w.r.t. deviceclock (but not w.r.t. potentially jittery drtio_recovered anymore).
<sb0>
should I order another adi fmc and plug it? I suppose the sooner we try this the better
<rjo>
i think we can proove this without another dac card. but yeah. we'll need that further down the road for the other logic test anyway.
<rjo>
*prove
<sb0>
right, we can just try synching between a TTL and a DAC output
<rjo>
and in any case (prove with or without two dac cards or for the logiq stuff) we want a mixer.
<sb0>
minicircus?
<sb0>
if one input is a TTL I suppose we need to attenuate it and maybe filter it
<rjo>
minicircus or usrp b100 fwiw. the latter bein a nice toy that you can chat about with the dragonlabs folks as well.
<whitequark>
minicircus?
<larsc>
circuits
<rjo>
a 13 dBm mixer would already be optimal 1 V rms
<rjo>
larsc: no, circus.
<larsc>
oh
<rjo>
larsc: that's a decade old tradition in the labs i have worked in.
<rjo>
;)
<larsc>
btw. the more fmc boards you buy the better (for me) ;)
<sb0>
b100 "Please note that this product is now End-of-Life (EOL), and is no longer available for sale through Ettus Research, and is not recommended for use in new designs or in new projects."
<rjo>
the stuff is good but you get the same feel of going to a pharmacy. they have everything, most stuff is ok, but it's all expensive.
<rjo>
ah.
<rjo>
let me decide which i'd suggest instead.
<rjo>
larsc: sure. are you actually working on zynq/zc705 plus adc-on-fmc or is it some other combination?
<larsc>
everything really, even altera fpgas
<larsc>
if it has jesd I work on it
<rjo>
sb0: LVTTL + balun + 13 dBm mixer looks good to me.
<rjo>
but you don't profit from *-FMC-EBZ specifically, right?
<rjo>
or with termination maybe a 10 dBm mixer. but it doesn't matter that much.
<rjo>
"even altera" ...
<larsc>
well, I could claim that any increases in sales are due to my involvement ;)
<rjo>
i wonder, since greg said they have floating point DSPs, what would one do with them other than a FPU? i don't really see the use for FIR/FFT/IIR etc.
<larsc>
but I don't profit directly of it, that's right
<rjo>
larsc: ok. but please feel free to claim that due to your involvement there will be lots of AD jesd adcs and dacs in devices that we are spinning.
<rjo>
and hmc/ad clock chips.
<rjo>
larsc: so, tell me about that subclass 0.5 stuff. what was the best work-around?
<rjo>
there is an interesting (for me at least) insight that many people don't really notice. in subclass 1, you don't get "absolute" det-lat, you only get it w.r.t. the slower of the two deviceclocks on the link. but in many cases you'd actually need it w.r.t. some other reference or "absolute time".
<rjo>
whitequark: are you in the lab by chance?
<whitequark>
rjo: I'm not
<sb0>
so, which clock chip are we using?
<rjo>
whitequark: ack.
<whitequark>
rjo: I'll be in the lab tomorrow, what do you need?
<rjo>
sb0: dunno. i haven't read that deeply into the two.
<rjo>
whitequark: i think i can pull myself out of the mud.
<rjo>
whitequark: i accidentally enabled the "forbidden" short instruction mode on the clock chip...
<larsc>
rjo: usually you'd align your clocks and sysref so that the det-lat is actually with the faster clock
<larsc>
but of course your latency is a function of clock and non-clock related parts
<rjo>
larsc: i'm referring to the indeterminism of the phase of the slower deviceclock (w.r.t. to "absolute time" or "something else").
<rjo>
larsc: i know that that's hardly the fault of jesd or subclass 1. but it is still a problem. at least for us.
<sb0>
so Ettus is now a NI company and they're running a "Vivado HLS challenge"?
<larsc>
rjo: you mean the relative phase to an external event?
<rjo>
sb0: where? their rfnoc?
<rjo>
larsc: yes.
<larsc>
rjo: what about resetting the clockchip dividers to this event so the outputs are in phase?
<sb0>
yes
<rjo>
if you say in your fabric, before the jesd stuff and in another clock domain, that you want to output sample x at time y (for a dac), then you need to figure out the phase of the fpga deviceclock w.r.t. that other clock.
<larsc>
well if you have two async clocks, you have two async clocks
<rjo>
sb0: yeah. that rfnoc is a cute idea. but what that amounts to is weird: they are building a "fpfpga"...
<sb0>
so it's not just a verilog/vhdl code generator?
<rjo>
larsc: yes. i am not hilding jesd accountable. but if your time reference is not based on the phase of the fpga deviceclock, then that indeterminism hits you hard.
rohitksingh has joined #m-labs
<rjo>
sb0: i thought they did the wiring at runtime.
<rjo>
sb0: if it is actually a *hdl generator, then they are doing exactly what NI tried with their *RIO and (barf) labview.
<rjo>
sb0: i thought it was a huge crossbar with a bunch of somewhat runtime-configurable processing blocks connected to it.
<rjo>
looks like it is a mixture of the two. you can choose what modules you want to have on the crossbar.
<GitHub1>
[misoc] whitequark pushed 1 new commit to master: https://git.io/vPTCI
<GitHub1>
misoc/master 20fdb77 whitequark: Rust CSR: mark all remaining consts as pub.
<rjo>
sb0: if you want a versatile tool to measure this phase stuff, go for the good old usrp1 (745$) + one basicrx ($80). otherwise any mixer plus filters should do.
<GitHub130>
[artiq] jordens pushed 8 new commits to phaser: https://git.io/vPTu0
<GitHub130>
artiq/phaser 740b61a Robert Jordens: ad9516_regs.h: fix B_COUNTER_MSB
<GitHub130>
artiq/phaser 7f0dffd Robert Jordens: kc705: mirror clk200 at user_sma_clock_p
<GitHub130>
artiq/phaser a9426d9 Robert Jordens: kc705: single ended rtio_external_clk...
<sb0>
rjo, no redpitaya?
<GitHub175>
[artiq] whitequark pushed 2 new commits to master: https://git.io/vPT2m
<GitHub175>
artiq/master 3263def whitequark: Rust: use generated CSR functions.
<whitequark>
wow, that did *wonders* for code size
<whitequark>
OR1K really has some rather bloated code...
<whitequark>
sb0: why is CSR_RTIO_CRG_PLL_RESET_ADDR an option in the first place?
<whitequark>
does the RTIO core sometimes not use a PLL?
<whitequark>
actually, CSR_RTIO_CRG_BASE is also an option, when does it ever make sense to build ARTIQ without RTIO?
<sb0>
RTIO_CRG
<whitequark>
ok, that makes more sense but the question stands
<sb0>
so, yes you can use potentially it without a PLL
<sb0>
*potentially use it
<sb0>
the papilio pro/pipistrello didn't have one for a while
<whitequark>
ok, so this is basically for porting
<sb0>
what the PLL essentially gives you is the high frequency clock for the high resolution TTLs
<sb0>
which are optional
<sb0>
yes
<whitequark>
ok, I'll remove the ifdef then, if we ever need it again we can introduce rustc --cfg new-board-here
<sb0>
why is it so difficult to do conditional compilation in rust?
<rjo>
_florent_: afaict we can't use the vco on the ad9154-fmc-ebz: bypass etc are not wired.
<whitequark>
it's not particulalry difficult
<whitequark>
but every option makes the matrix of possible builds larger
<rjo>
_florent_: it'll probably oscillate happily. but by no means good. the ldo shouldn't be stable.
<sb0>
why is it a problem?
<whitequark>
so, instead of writing more support code for rust, that ultimately makes the codebase messier, I would like to not do that
<rjo>
_florent_: we certainly can't use the pll. there is no loop filter.
<whitequark>
well, can you say for sure if builds with !defined(CSR_RTIO_CRG_BASE) still work? or did some other part bit rot or never support that config in first place?
<whitequark>
this is why I would rather use coarsely grained configuration: the entire matrix can then be tested on the buildserver
<sb0>
it does, yes, I used it for drtio_transceiver_demo
<whitequark>
okay
<sb0>
drtio_transceiver_demo also sets a number of other relatively unusal defines
mumptai has joined #m-labs
<rjo>
sb0: for redpitaya, i am pretty sure they messed up the analog side before the ADCs pretty royally. at least from what i have seen. and i don't know that ADC well enough (i do know the one in the usrp1). also there is no gnuradio support for that stuff. we'd have to write a lot of code.
<sb0>
*unusual
<whitequark>
there's also no direct equivalent of #ifdef, since there is no preprocessing stage
<whitequark>
but it has a worse ergonomics in this case
<whitequark>
since you can only use it to define toplevel items (functions in this case) and not twiddle parts of function body
<whitequark>
you can always write if cfg!(...) { foo() } else { bar() } but foo() and bar() must always exist, and they won't if those are generated CSRs for peripherals that are absent
<sb0>
there will be the case where some peripherals are only on some targets, and this needs to be dealt with
<whitequark>
sure
<rjo>
sb0: is there still that windfreak thing in the lab?
<sb0>
rjo, yes
<whitequark>
like I said, this is possible, I merely want to have *less* conditional compilation, not removing it entirely
<rjo>
whitequark: ok. could i ask you to -- if possible tomorrow -- find that and connect that to the ad9154-fmc-ebz J1 instead of (currently) kc705 user_sma_clock_p? and its usb to lab.m-labs.hk.
<whitequark>
rjo: okay
<whitequark>
rjo: connect *what*
<whitequark>
oh, windfreak thing? I have no idea what that is
<rjo>
alternatively, sb0: remind me what clock freq i can spit out of the kc705 sma_user_clock?
<rjo>
whitequark: i think it's usb-bus powered.
<whitequark>
rjo: ah
<rjo>
whitequark: yes. bus powered or stand-alone on 7V.
<sb0>
whitequark, it can be USB powered, if there are any ports left on the buildserver
<sb0>
(and power left)
<rjo>
sb0: do you know from the top of your head whether I can output 500 MHz from a clock capable pin of the kc705?
<whitequark>
there still are...
<sb0>
rjo, 500 should be fine yes
<sb0>
the BUFG can take 625MHz, and the IO at least 600
<sb0>
rjo, do we send the 2.4GHz clock on the RTM backplane?
<sb0>
there's this 100MHz input, then we generate a 125MHz DRTIO clock from it, and some other clock that goes to the HMC704x on the Sayma RTMs
<sb0>
what is the frequency of that other clock?
<rjo>
whitequark: ok. thx. when would you do that reconnection?
<rjo>
sb0: ok. then i can at least run the dac at 500 MHz and 1x interpolation first.
<rjo>
sb0: there would be 100 MHz on the backplane.
<whitequark>
rjo: by 2016-09-30 08:00 UTC, I think
<sb0>
so there is no clock chip on the Metlino RTM?
<sb0>
or simply a buffer.
<sb0>
hm
<sb0>
if we want 125 for the transceivers, we may want to generate it externally
<rjo>
whitequark: ok. ack.
<rjo>
sb0: on the metlino rtm there would be a low noise clock distribution thing or splitter.
<sb0>
but if the RTIO clock is 125
<sb0>
where do we go from 100 to 125?
<sb0>
and why is it 100 to begin with and not 125?
<rjo>
100M - splitter - (rf-bp, sayma) - pll - 2.4 G - fanout/divider
<sb0>
what reference clock do we use for the DRTIO transceivers?
<rjo>
that fanout should then have at least thre connections to the fpga, one jesd deviceclock, one rtio_from_100M, one sysref.
<sb0>
the MGT PLLs have only a few possible m/d values, so the reference clock frequency is very constrained
<rjo>
yes. that's why i'd do two clocks from the divider, one deviceclock for the transcievers and one rtio/fabric.
<rjo>
that's also in-line with the jitter/delay uncertainty problem if IBUFDS_GTX.
<sb0>
which divider?
<sb0>
the only transceiver that requires a precise refclk that is originating from the same oscillator as the one for the core metlino
<sb0>
other use cases can take an on-board crystal
<rjo>
for the drtio transcievers i'd either use the cleaned rx recovered clock (which would be the same as the rtio_clock in one of the "measure sysref" scenarios) or whatever you can get from the clock fanout.
<sb0>
all transceivers require a refclk, even when receiving
<rjo>
"divider" = hmc7044 or equiv.
<rjo>
i'd guess the si* cleaner will oscillate by itself, as will the pll feeding the hmc7044
<whitequark>
si4564? yes, you can tell it to free-run
<sb0>
no, that won't work. what we need to do is crystal -> MGT PLL RX -> CDR -> Si5324 -> MGT PLL TX
<sb0>
"crystal" can be any source within some hundred ppm of 125MHz
<sb0>
but I guess putting some cheap oscillator on the board is easiest
* rjo
imagines the current top thread on the artiq mailinglist unfolding on "artiq.stackoverflow.com" ;)
<whitequark>
hm, no one told him to use conda, my oversight..
<rjo>
sb0: yes. isn't that the kc705 setup for mini-drtio? i'd just do very much the same thing on sayma.
<sb0>
for the root metlino, however, we need to do 125MHz in phase with DAC clock -> MGT PLL TX
<sb0>
mini-drtio is one-way
<rjo>
sb0: you'd get 100 M and make 125 MHz locally.
<sb0>
define "locally"
<rjo>
yes. the same modulo that last "-> MGT PLL TX"
<sb0>
since we have some sort of clock chip to distribute the 100MHz, can't it make 125 as well?
<rjo>
with your fpga PLL if that's not too jittery for DRTIO 10G or the si5324 that it has anyway.
<sb0>
I'm not sure if the FPGA PLLs can be used for clocking transceivers. if it were the case, why would there be dedicated transceiver clock pins?
<sb0>
yes, we can probably use the Si5324.
<rjo>
sb0: ok. then the root metlino would just make 125 MHz on fpga, clean it up with the si5324 and use it for the gtx.
<sb0>
we can save a PLL and have the Si5324 do 100 -> 125
<rjo>
putting another clock onto the backplane does not seem to solve any issue, right? since you can get a good 125 MHz from the hmc7044, with known phase w.r.t. sysref etc even .
<sb0>
I'm talking about the Metlino right now
<sb0>
for the Sayma sysref, we can use any old clock, since the transceiver CDR will syntonize it to the root metlino clock anyway
<rjo>
afaict that should always just use the cleaner, right?
<sb0>
*Sayma refclk
<sb0>
yes with that scheme the Metlino has its Si5324 always used
<sb0>
either for cleaning up the CDR clock in satellite mode, or for doing 100->125 and routing the clock to transceiver pins in core mode
<rjo>
ack. for sayma we'd also want a local oscillator on board so that we are at least a bit independent of the rtm.
<sb0>
the sayma should also have an on-board 125MHz crystal for its RX PLL
<rjo>
100 MHz because there are very good oscillators there but not so much at 125 MHz.
<rjo>
i even think we could do f_rtio = 200 MHz ...
<rjo>
sb0: there need to be more clocks from the rtm to sayma.
<rjo>
jesd deviceclock and that "rtio_from_100M"
<rjo>
and the adcs and dacs also need the sysref and their deviceclock.
<rjo>
and it gets more interesting if we can't use the same sysref for adc and dacs then the fpga needs to get both of them.
<sb0>
jesd device clock?
<sb0>
and what is rtio_from_100M and why?
<sb0>
if that's what I think it is, rtio_from_100M is generated from 100M by the Si5324 of the root metlino, and recovered by the transceiver of all other boards
<rjo>
we discussed all of that above already afaict. jesd device clock is what you use to clock the jesd transcievers, rtio_from_100M is the rtio_clock in case rtio_from_drtio is too loose to tag sysref accurately.
<sb0>
wouldn't rtio_from_100M be sent on AMC?
<rjo>
i don't think that helps.
<sb0>
it is generated on the metlino, why would a round trip on RTM help? the most direct connection between metlino and sayma is AMC
<rjo>
the metlino does not get the good 100 MHz clock.
<rjo>
from the backplane.
<rjo>
at least it should not need to.
<sb0>
it gets it from its RTM right?
<sb0>
or some clocking module
<rjo>
the metlino is not double width.
<rjo>
or, it doesn't need to be.
<sb0>
so anyway the RTIO clock is sent on AMC, I have absolutely no idea why you would like to send it via RTM
<rjo>
there is no round trip on the rtm.
<rjo>
the linkage i described above does not round trip.
<sb0>
the RTIO clock is generated by the Si5324 on the Metlino
<sb0>
you also just said that the Metlino does not have RTM
<rjo>
no. i didn't say that.
<sb0>
don't only double width cards have RTM?
<rjo>
the metlino is single-width, it doesn not have an RTM connector. there can and should still be an RTM behind it. to do the clock dist.
<sb0>
sigh
<sb0>
so anyway
<sb0>
that's beside the point
<rjo>
just go and read up on what the racks look like.
<sb0>
sigh.
<sb0>
point is, a single width card doesn't have a RTM connector.
<rjo>
i don't see why this is beside the point.
<rjo>
yes.
<sb0>
so. if the clock is generated on the metlino, how do you send it to the sayma, if not by AMC?
<rjo>
as i said, i wouldn't send it through amc.
<sb0>
so how?
<rjo>
in the case that rtio_from_drtio is too loose, you take it from your rtm.
<sb0>
very fine, but how does it get to RTM in the first place?
<rjo>
via the 100 Mhz on the rf backplane.
<sb0>
and how do you make it 125?
<rjo>
up to 2.4 GHz with the PLL down to 125 MHz with hmc7044.
<rjo>
as i said. the good thing is that if you drive your rtio CD from thac clock, you don't have to measure sysref that well anymore. and you only have to measure the phase between rtio_from_drtio and rtio_from_100M which is not that tricky anymore.
<rjo>
but it makes sayma much more dependent on its rtm.
<sb0>
how is that less tricky? you still need to measure to less than a DAC clock cycle
<sb0>
to ensure inter-card sync
<rjo>
right. the possible shifts between the two rtio_from_* are at t_dac granularity.
<rjo>
but the sysref measurement would be easy.
<sb0>
how does the metlino get 100MHz?
<sb0>
we need to get 100M both to the RF backplane and the metlino
<rjo>
why on the metlino?
<sb0>
to clock the root DRTIO transceiver
<rjo>
on the root metlino yes.
<rjo>
front panel.
<sb0>
meh
<sb0>
why not make it double width and have RTM=
<rjo>
i for one don't want to also have metlino dependent on RTM unless really needed.
<rjo>
this way you would always need racks with rtm and rf-bp
<sb0>
but those two SMAs that need to be connected together on the same device are confusing for the user
<rjo>
let's assume someone makes a nice card that does not pack 8 dac and 8 adc and rtm but only so much.
<sb0>
plus, if they connect it with a long cable, does it behave?
<rjo>
they will come from a commmon upstream dist amplifier afaics.
<sb0>
can we have an internal connection somehow?
<rjo>
that's the same problem with two crates.
<sb0>
maybe pass a cable through the RTM hole
<rjo>
sure.
<sb0>
yes but two crates a) is a more advanced use case b) there is a clearer rationale for the users
<rjo>
to me those things easily outweigh the alternative of making metlino double-width and requiring rtm+rf-backplane everywhere.
<rjo>
but yeah. maybe joe wants it that way as well. file an issue.
<sb0>
ok, I'm fine with an internal cable
<sb0>
remember that Joe bothered me one whole afternoon about ARTIQ not working because he didn't connect the RTIO clock
<rjo>
but check with joe and greg. they were discussing that for a few seconds while i was "on mental standby".
<sb0>
so an external cable that has to be connected to the same device, well...
<sb0>
plus it does look ugly
<rjo>
i wouldn't phrase it that way. the root metlino+sayma rack needs two 100 MHz connections, one for rtio, one for the rf backplane.
<rjo>
they come from the same dist amp in the lab.
<sb0>
so you then require a dist amp for a single-crate system
<rjo>
as do a couple dozen other devices that are all connected to 100 MHz.
<sb0>
what devices?
<rjo>
other oscillators, counters, sdr equipment, spectrum analyzers, etc.
<rjo>
in the lab it doesn't make a difference.
<rjo>
if you have 100 MHz, you tend to also have another 100 MHz port.
<sb0>
meh
<sb0>
does it make sense to have a builtin 100MHz oscillator? for fully standalone operation
<rjo>
i'd put some regular oscillator on board.
<rjo>
* on metlino.
<rjo>
same as sayma.
<sb0>
how does it get distributed to the hmc70xx?
<sb0>
shouldn't it be on the clock module?
<rjo>
i don't know whether it is useful to run sayma without a good 100 mhz clock (for the rtm).
<rjo>
but i think it is useful to have some oscillator on the amc. at least for bring-up and testing intially.
<rjo>
but yes. the 100 mhz fall-back osc should be on the clock module in that case.
mumptai has quit [Ping timeout: 244 seconds]
<whitequark>
sb0: can you explain why clock.c sets the timer to 0x7fffffffffffffffLL initially?
mumptai has joined #m-labs
_florent_ has quit [Ping timeout: 272 seconds]
_florent_ has joined #m-labs
mumptai has quit [Quit: Verlassend]
<GitHub24>
[misoc] whitequark pushed 1 new commit to master: https://git.io/vPkX0
<GitHub24>
misoc/master 8806d93 whitequark: Rust CSR: avoid dead code warnings.