<sb0>
_florent_, how is the jesd core coming along?
rohitksingh has joined #m-labs
<_florent_>
sb0: the model is almost done, I'm now going to do the rtl layer by layer (I expect it to be fast since the main difficulty was to clearly understand the jesd specification), and I'll start testing next week
<_florent_>
sb0: first step will be to test the transceiver with prbs (the ADI devices have integrated prbs checker), then code group synchronization, initial lane alignment, ...
<rjo>
sb0: rtio_coarse and ref_clk have the same freq and fixed alignment. we assume that fpga on rtio_coarse meets s/h w.r.t. ref_clk in on the hmc chip.
<GitHub158>
[artiq] sbourdeauducq pushed 1 new commit to release-2: https://git.io/viBEZ
<rjo>
then, there is a fixed delay from a ref_clk edge to the next dac_clk (a.k.a deviceclock). the hmc chip emits sys_ref at a fixed delay after ref_clock to meet dac_clk s/h.
<rjo>
ref_clk is generated from an oscillator disciplined to some other oscillator. and the rtio_coarse is disciplined (through drtio) to the same oscillator.
<sb0>
what is the difference between sys_ref and ref_clk?
<rjo>
one is the jesd sync mechanism, the other is the local low-noise oscillator.
<sb0>
ok, then this local ref_clk needs to be distributed to one DAC cycle throughout the system?
<sb0>
how does the hmc chip determine the delay to apply? what if the phase of ref_clk lies near the boundary where it selects a delay that reaches one DAC clock cycle and a delay that reaches the next?
<rjo>
yes.
<rjo>
that needs to be scanned and monitored with the dac notifying when it crosses dac clock cycles.
<rjo>
but once.
rohitksingh1 has quit [Quit: Leaving.]
<sb0>
well, how does this help us?
<sb0>
we might as well distribute the JESD sys_ref
<rjo>
as well? that's three times as much distribution (also needed for the adcs) and then it still needs to be shifted to meet at every chip.
<rjo>
and you still need to measure it.
<sb0>
okay, so it just makes sys_ref distribution somewhat easier
rohitksingh has joined #m-labs
<rjo>
not "just"
<rjo>
and it ensures known relationship w.r.t. rtio
<sb0>
again, the rtio/ref_clk phase will have to be tuned
<rjo>
yes. as i said. and the two other phases as well.
ylamarre has joined #m-labs
<rjo>
i am not impressed either. measuring sys_ref needs fewer chips and less phase fiddling. and this becomes a bit nasty if rtio_coarse != ref_clk.
rohitksingh has quit [Quit: Leaving.]
ylamarre has quit [Ping timeout: 264 seconds]
rohitksingh has joined #m-labs
<larsc>
there is one more thing, Xilinx says if you want DET-LAT you need two clocks to the FPGA in case your lanerate is over 6.6GHz. One clock as the reference for the GTX, another going to a normal clock input for sampling sysref and for clocking the jesd logic
<larsc>
I don't know whether that is just a precaution from their side or whether this is actually necessary
<rjo>
larsc: oh thanks for chiming in. i was about to ask you questions.
<rjo>
larsc: do you have a reference for that? their jesd204b core?
<rjo>
i am inexperienced about how to clock the actual transcievers. i always assumed that there would be a slow clock into the fpga (its device clock) and you would clock fabric from it and the transcievers would happily generate their line clock from that clock.
<larsc>
rjo: their jesd204b manual, and an email clarifying on the reason.
<larsc>
rjo: the traceiver has a reference clock for initial CDR synchronization
<larsc>
this is necessary so the CDR does not lock onto a harmonic
<larsc>
this is a special clock input
<larsc>
only one diff pair per region, iirc
<larsc>
this clock can also be used for clocking the logic (aka deviceclock)
<larsc>
as long as it is less than 165MHz
<larsc>
but for best phase noise you actually want the refclock as fast as possible
<larsc>
max is 625M or something like that
<larsc>
so it is always a good idea to design with two clock inputs
<rjo>
larsc: for ultrascale that number if TBD....
<larsc>
right
<larsc>
they said it is because you can't meet s/h on sysref
<larsc>
so it is something you could check using the timing tools
<rjo>
larsc: but CDR? doesn't that only apply to ADC-FPGA?
<larsc>
oh right
<larsc>
sorry
<larsc>
I'm mostly working on ADCs atm
<rjo>
larsc: no you are fine. i just don't known. they do write it as if it applies to whatever you are doing.
<larsc>
but same story, you need a reference to generate the output clock at 6G or whatever
<rjo>
tx or rx.
<larsc>
and this reference comes from the same special pins
<larsc>
but even more with TX you want to reduce your PLL phase noise so a faster ref is always better
<rjo>
aha. so the restriction actually arises from the need to locate sys_ref accurately enough (needs high refclock).
<larsc>
to sample sysref you reference needs to be lanerate/40
<rjo>
larsc: but noise on sys_ref and deviceclock_fpga does not matter as long as it meets s/h timing. it is irrelevant for output noise.
<larsc>
rjo: exactly
<larsc>
you have a PLL in the FPGA
<larsc>
this PLL is fed by a reference clock
<larsc>
the faster this reference clock the better the phasenoise
<larsc>
and the fastest is typically > lanerate/40
<rjo>
is that so bad that it could add enough noise to fail eye openings?
<larsc>
so having two clock inputs one for the refclock and the other for lanerate/40 is useful
<larsc>
rjo: if everything else is well designed probably not
<larsc>
but BER will go up
<larsc>
wheter it is stil low enough is something you need to measure
<rjo>
sure.
<rjo>
so. since we are building our own sys_ref measurement mechanism and won't use their core at all, this does not apply directly but might hit us all the same.
<larsc>
it depends
<larsc>
I've not independently tried to verify why s/h fails
<larsc>
go could be that the path is simply too long
<larsc>
in which case you can compenstate using a delay
<larsc>
but it could also be that there is too much uncertainty on the path in which case you cant really fix it
rohitksingh has quit [Quit: Leaving.]
<rjo>
larsc: but i dont understand why there is a "max frequency for refclk as core clock". especially if the example they give for split clock is 200/400 MHz
<rjo>
ah. refclk goes only to the gtx and glblclk goes to the gtx and the core_clk...
<rjo>
so it is actually either a max freq limitation of tx/rxusrclk or core_clk.
<rjo>
blah. core_clk is linerate/40, fixed. and ref_clk needs to be faster to ensure gtx eye spec.
<rjo>
and tx/rxusrclk is just the fabric side of the gtx and thus == core_clk == linerate/40.
<rjo>
therefore shouldn't this split-clock thing be generally applicable to all >6 GHz transciever setups independent of whether its jesd or something else?
<larsc>
rjo: the issue is that the refclock is a special clock net and routing it to a GP pin for sampling sysref causes the issue
<larsc>
if you don't care about det-lat there is no problem
<larsc>
it's really just for sampling sysref
<rjo>
hmm.
<larsc>
for other designs you can generate the deviceclock from the GTX PLL by downdividing it by 40
<larsc>
but that will have no known phase relationship regards to the external sysref
<larsc>
so you can't use it for det-lat
<larsc>
this is explained a few pages later as subclass 0 setup
<rjo>
ok. i think i get it. to summarize: a) ref_clk to the gtx needs to be fast for jitter b) core_clk should be fast and low jitter to properly tag sys_ref c) you can't get a low jitter route from ref_clk to fabric
<larsc>
yes
<larsc>
except I don't know if c) is true
<larsc>
needs to be check against the timing tools
<rjo>
yes. that unknown phase relationship for the fabric clock if divided somewhere other than in the external clock fanout is the same problem we have.
<rjo>
by timing tools, which tool are you referring to? their BERR tester?
<rjo>
or their eye-scanner?
<larsc>
s/h check
<rjo>
larsc: the "usual" timing check after p&r?
<larsc>
yes
<rjo>
ok
<larsc>
just to make sure whether s/h always fails or whether you can tune the phase between sysref and clock so that it actually works
<rjo>
ack
<larsc>
xilinx e.g. also recommends that you sample sysref on the negative edge, whereas I'd recommend to use the clockchip to adjust the phase so that s/h is max at the posedge
<rjo>
larsc: a little detail that i am not sure about. you have an adc sampling at f, it decimates to f/8 internally, the latency from analog to decimated data (all before the jesd phy) is obviously deterministic. now the jesd stuff works at f/8, deviceclock_adc is f. does sys_ref need to be w.r.t. f/8 or f?
<larsc>
rjo: good question
<larsc>
ideally the adc should reset it's decimator state so that is aligns with sysref as sampled by the deviceclock
<larsc>
oh right
<larsc>
the deviceclock is f
<larsc>
so, yes, sysref needs to be aligned to f
<rjo>
larsc: yes. we would do that. we would sample sys_ref with an iserdes at e.g. 1 GHz and then scan the phase (either idelay or with the fanout) to decide whether the sysref edge is at the odd or even cycles of the 2 GHz fast device clock.
<rjo>
larsc: but that doesn't matter. the decimator does not need to be reset.
<larsc>
it does
<rjo>
also the interpolation factor does not appear in the frequency formulas for sys_ref.
<rjo>
ah. you mean the f/4 datapath clock up to and including the jesd should be reset?
<rjo>
at least in the chip we are looking at currently (ad9154) all the jesd clocking including sys_ref is w.r.t. data_clock (which is the slower sample clock)
<rjo>
but you are right. the data_clock phase matters for latency.
<larsc>
and what is the relationship to the external CLK+- input?
<rjo>
that input (the deviceclock) is dac_clk.
<larsc>
ok
<larsc>
but isn't that clock used to sample sysref?
<rjo>
yes. i guess. but then -- since most of the datapath is at data_clk -- they have to introduce a few sample delay after the interpolation stage as well (in addition to the delay after the jesd).
<rjo>
so they need to introduce a coarse delay early and additionally a finer delay later.