Hi Sebastien (cc'ing ARTIQ list for commentary/archival purposes), For the RTIO clocking, I'm currently planning to put a 2.5V LVDS clock on the USER_CLK_P/USER_CLK_N SMA connectors directly on the KC705 board (goes to an MRCC pair in I/O Bank 15, pins L25 and K25). This will come from an LTC6957-2 clock generation chip on a separate small board, which takes a reference sine wave and converts it to LVDS with very low added phase noise. We will use a phase-locked oscillator (PLO) referenced to the 10 MHz maser signal to generate the RTIO clock, and then an ultra-low-phase-noise multiplier will multiply this frequency by 24 to generate the DDS SYSCLK. In this way we can ensure that the DDS SYSCLK/SYNC_CLK and the RTIO clock are always reliably phase-locked to each other. Another option would be to use a PLO at the DDS SYSCLK frequency and then employ a 24x divider, which would give better phase noise performance at the RTIO clock frequency, but a) I don't think the PLO is the limit on the RTIO clock phase noise, and b) the divider system is more e xpensive and would require purchase orders.
For the Magtrap experiment, we will clock the RTIO core at 100 MHz and the DDS chips at 2.4 GHz -- this is optimal in terms of where the DDS spurs are located with respect to the Mg+ transition frequencies. For other experiments, people may use higher frequencies. Above 3 GHz we haven't been able to get the DDS clocks to sync reliably, though. This may work better with the FPGA-based syncing because even if the process is a bit random, you can just send individual SYNC pulses until the SYNC_CLK phase is as desired and then stop. The edge timing at the SYNC_IN pin is probably the main limitation there. For the DDS clocking, Raghu has done some experiments to see how long the DDS SYNC_CLKs maintain synchronization if they are not constantly receiving synchronization pulses (both at 2.5 GHz and 3 GHz SYSCLK), and it seems they have a tendency to lose lock with each other at around 1, 2, or 3 hours after synchronization. This is not a big deal necessarily, but it does mean that if we use the FPGA-based synchronization it will have to be a calibration experiment that gets repeated on a regular basis (or at least, the FPGA should be able to check for DDS synchronization before and after a given kernel runs and flag an error if the DDS lost sync during the experiment, and then run a resynchronization routine). The current DDS sync design I envision involves using one 3.3V LVCMOS line from the FPGA to send a synchronization signal, which will go to a 1:12 LVCMOS clock distribution chip on the FMC backplane. Unfortunately we cannot use LVDS because all the available I/O banks need to run at a 3.3V VCCO in order to talk to the interface logic of the DDS chips (e.g. the parallel bus etc). Using 2.5V LVCMOS to drive 3.3V inputs degrades our noise margin to basically zero, which given our desire for >100 MHz bus speeds doesn't seem like a sensible tradeoff. Putting level translators in will be a nightmare in terms of introducing skew and latency, especially for bidirectional signals. I don't think that using LVCMOS instead of LVDS will make a big difference, though, as the 3.3V LVCMOS rise times (based on IBIS) are ~300-400 ps, and I don't think we can do much better since we have to convert to 3.3 V LVCMOS in the end to talk to the DDS SYNC_IN pin. The clock distribution chip has a similar rise time spec. Each output from the clock distribution chip will be dedicated to one DDS. On the DDS cards themselves, there will be an analog switch which allows the clocking signal to be either passed through to the DDS SYNC_IN or blocked (SYNC_IN pulled low after the switch), depending on a DDS chip select signal. The same analog switch will use another channel to pass the DDS SYNC_CLK back to the FPGA for monitoring. It will also have two more channels that are connected together right next to the DDS, so that the FPGA can send pulses and measure a round-trip time from FPGA inputs to DDS and back for deskewing if needed. I would imagine that the board-to-board variations would be sufficiently small, though, that one could manually perform a calibration of the propagation delays for the setup with DDS clocks synced and then just give those numbers to the FPGA to use as offsets....then one wouldn't need to use a round-trip signal every time. I am making provision on the DDS cards themselves to do the hardware-only syncing using the SYNC_OUT/SYNC_IN method if preferred for some reason, so that can be a backup option. Let me know your thoughts! Best, Daniel _______________________________________________ ARTIQ mailing list https://ssl.serverraum.org/lists/listinfo/artiq
