Thanks Robert for your thoughtful comments on the hardware, they are indeed very much appreciated. Replies inline below:
> What is the mission statement here? From the level of complexity already > required for this implementation, it looks like this hardware will require a > lot > of work and will be around for many years. If that is the case, then it would > be disappointing if it ends up (again) being an ad-hoc mechanical design with > boards mounted in random places in a box, lots of flying wires, rf-splitters, > and flimsily mounted daughterboards that can not be accessed when > mounted and not exchanged without opening the custom box. I don't think > that should be considered "good enough". You are correct that this falls a bit in a no-man's-land between "do nothing" and "make it really robust and nice". The "do nothing" approach, essentially just using the clock team's backplane, would work fine for everything except the DDS syncing using the FPGA. The motivation for this design was to try to enable the FPGA-based DDS syncing, since this appears to be a feature that people want to have. Since this requires a high-bandwidth-capable connector to the DDS boards (or in any event something better than a pin header) because of rise time/jitter issues, the thought was to go to a new connector, and once you are there changing up the buses is not a great deal of additional work. However, I don't have the time to design a nice solution, something like a rack-mounted chassis with slots for DDS cards, and there is also then the issue of getting signals from the FPGA card to the chassis. We had a lot of discussions about this several months ago, and it was decided that having a DDS/TTL backplane directly on the FMC connector in the same crate as the KC705 was the best solution for now -- people didn't like the idea of giant SCSI cables as we currently have, and the high-speed serial interconnection option was deemed to have too many questions about latency, signal integrity, etc for a first-generation design (although likely this is where we want to head in the future). This leaves us with an FMC backplane solution, with DDS cards plugging directly in, ugly though that may be in some ways. We need to add in some solution for DDS syncing, so engineering of some sort will have to be done. One option would be to use the existing clock backplane and just add an external board with SMA cables for the syncing stuff. Another would be to include the DDS syncing stuff on the backplane and then use an additional high-speed connector (e.g. SATA with short cables, or SMP coax) to carry the sync signals to and from the DDS cards. Another alternative is to just make an FMC-SCSI adapter card (similar in functionality to the daughtercard on the current HFGUI FPGAs) and use ribbon cables to connect to our existing DDS/TTL boxes. This was proposed initially, and it was voted down. Joe is now suggesting we do this so we have the ability to use our existing hardware while people migrate; I think it can be made to happen in time for Sebastien's visit > > - The interface between the FPGA and the TTL riser card is > > designed in such a way that one can very easily replace the current > > TTL riser card with a card that accepts up to 14 differential pairs > > for high-speed serial signaling from the FPGA, with no modifications > > to the FMC backplane. A replacement card could be designed to allow > > these lines to talk to remote DDS/TTL outputs, or to other boards in a > > rack employing a distributed RTIO system, both of which are proposed > > next-generation hardware ideas for ARTIQ. This “future-proofs” the > > design to some extent. > > To what extent? I would probably use the GTX transciever(s) for that even if > they are a bit of a grey box. I was planning to route the GTX transmitter and receiver to this connector as well. In any event, if you don't think this is worthwhile I don't have to bother, but it's basically for free to have the lines run as differential pairs. Probably not much crosstalk effect since they already run this way for a much longer distance on the KC705 board. > > All the signals save two are sent through the FMC connector as > > single-ended LVCMOS, which allows us to get more signals through. > > Hmm. I don't have a good feeling for how much the coupling of the pairs on > the board and in the connectors will cause problems here. I don't either, except to say that the clock team are using this method on their backplane and so far I haven't heard complaints? The outputs of the FPGA can be configured for fast or slow rise times, and if crosstalk is an issue we can go to the slower rise time to reduce crosstalk. Otherwise there doesn't appear to be much one can do about this other than a) using LVDS and accepting many fewer signal lines or b) using LVDS with the OSERDES and then deserializing on the backplane, which sounds like a giant engineering mess. > > All the other DDS-related signals (which come from the FPGA as > > LVCMOS) are immediately converted on the FMC backplane into LVDS > > signals. These signals are either routed individually to each DDS > > card (the “DDS SELECT” signals), or are sent to a double-terminated > > multipoint LVDS parallel bus (those labeled “Y” under “shared bus”) > > that is shared by all DDS cards. All signals are converted between > > 3.3V LVCMOS and LVDS using the SN65MLVD080 chip, a transceiver > > designed for driving heavily-loaded double-terminated buses in > > backplane applications with good signal integrity. The choice of this > > bus design, instead of the old LVCMOS bus used in the HFGUI > > backplanes, should allow for faster signaling, lower bus error rates, > > and greater noise immunity – in general, it’s more robust and in > > keeping with modern bus design. > > But this bus is not a simple multi-drop lvds bus. Would the > (unterminated) stubs from the backplane to the transcievers on the riser > cards not cause trouble? These stubs appear as distributed capacitance along the line because the length of the bus and the length of the stubs are both electrically small, given our max 100 MHz data clock (max allowed by the DDS chips). This means that the impedance of the transmission line needs to take the capacitive loading into account when doing the geometric layout. This is a well-studied problem and there are good solutions. The rise times of the LVDS drivers are intentionally slowed to reduce the high-frequency content of the output signals and preserve signal integrity. > > All riser cards will be connected using SAMTEC HSEC8-DV connectors on > > the FMC backplane. These connectors are designed to allow very high > > speed single-ended and/or differential signaling (>8 GHz 3dB cutoff). > > The riser cards will have gold fingers at the edge (like a PCI card or > > a RAM module) and will just plug in directly with no second connector > > on the riser card itself. The connectors will have board locks to > > help hold the riser cards in place on the FMC backplane. > > The 50 pin (this design needs more pins AFAICT) HSEC8 already has ~50 N > mating force. This is quite an adventure without pcb guide rails for the riser > cards and well designed and dense mechanical mounts for the backplane... I had not considered insertion force, and you are right that this is large. It seems that most high-speed connectors have similar mating force requirements. If we continue to use pin headers, we can avoid this but will have to find a solution (e.g. separate SATA connector with cable, or coax cable) to send in timing-critical signals for the syncing process. > > Each DDS riser card will have one AD9914 DDS. The DDS SYSCLK (3.5 > > IMHO putting more than one DDS (two or four) on each "riser" would make > stuff quite a bit simpler and more compact. Fewer connectors, fewer > mechanical parts. Simpler cabling. Totally achievable; I had just been using the previous design from the clock team. Putting two DDS chips per riser card would definitely give us more room to maneuver on the backplane. > > return loss and small form factor. The DDS chip, amplifier, SMP > > connectors, and nearby parts will be covered with a shielding cavity > > (modified 10-CBS) with small holes to allow for SMP connector entry > > and exit. This will reduce the crosstalk of the DDS outputs between > > DDS cards, which is currently an issue for the clock team. If there > > is enough physical room, external finned heat sinks will be soldered > > on the back of the DDS cards near the chips to help with thermal > > control and stability. > > I would suspect that this might not be sufficient given the power density and > typical air flows. A heatsink on the DDS might be desirable. My intention had been to put a heat sink soldered to the ground pad on the back of the card beneath the DDS, which is connected to the pad of the DDS with a number of thermal vias (per the DDS design). Thermal resistance to the die is actually lower this way than with a heat sink on top of the package. > > The DDS chip SELECT signal will enable the propagation of the RD, WR, > > IO_UPDATE, and MASTER_RESET from the bus to the DDS chip. As with the > > current generation of DDS cards, a SN74CBTLV3245A octal bus switch > > will either pass or block these signals between the LVCMOS outputs of > > the LVDS transceiver and the DDS chip, depending on the state of the > > DDS SELECT signal. Unlike with the previous system, each DDS card > > will have a dedicated DDS SELECT signal, so multiple DDS chips can > > receive an IO_UPDATE simultaneously. > > > > The profile pins and the DRCTL, DRHOLD, and OSK pins require a latch > > on each DDS card, since the signal levels must be held for proper > > functionality but these lines are driven from a shared bus. Since it > > is possible that one might wish to update multiple DDS cards at the > > same time, but have them running separately numbered profiles (or have > > one engaged in a digital ramp and another not) > > Why is IO_UPDATE not in the same category as DRCTL, DRHOLD, and OSK? > AFAICT it has the same (timing) characteristics. You are correct that the timing is the same, but DRCTL, DRHOLD, OSK, and the profiles need to be latched for each DDS (i.e. values held while the bus talks to other DDS chips), while IO_UPDATE should not be latched. > > The TTL riser card will host 28 individual digital I/O lines. The > > direction of each line (input or output) will be controlled by an > > array of mechanical switches (DIP switches), which can either be on > > the TTL riser card itself or can be placed remotely and wired back to > > the card with a short commercial ribbon cable (e.g. on the outside of > > the FPGA crate, for easy access). These direction switches will > > control the direction of both the voltage translators and the > > LVCMOS-LVDS transceiver channels. > > Can't we use a couple of latches to be able reconfigure the direction from > software? Replace the DIP switches with latches and hook them up to either > the ttl bus itself or the dds data bus (and a few more select signals). > Grouping > the direction by e.g. 4 would reduce the parts count further. Yes, at the cost of one or two digital signals this would be totally doable. My thought had been that since changing the direction of a hardware line is not typically done on the fly (most stuff typically runs full duplex, i.e.), it would not be necessary to do in software. Usually it involves you unplugging and replugging cables, so why not a mechanical switch? Also, grouping by direction is fine but might end up limiting some people if they want inputs from disparate locations. The parts are small and the part-to-part skew is low, so there's not really a major cost to going more granular. > > drive the transceiver input via the ‘125 buffer (so the signal is sent > > back to the TTL riser card). Because there are only 2 output enables > > on the ‘25244 buffer (one for each set of 4 channels), we will use two > > chips per breakout board and just not wire outputs 2-4 and 6-8 (inputs > > grounded). > > Wouldn't it be ok to keep this TTL direction grouping of 4 and propagate it > all > the way through instead of trying to build everything for single-channel > granulatity? Again, the cost of single-channel granularity is very small so it seemed like a nice feature to have for flexibility and to avoid wasting lines. _______________________________________________ ARTIQ mailing list https://ssl.serverraum.org/lists/listinfo/artiq
