Hi Sam,

I'm not familiar with the music board, but I wonder if it would benefit
from tuning the clock vs data skew on the adc->fpga links to tune the data
capture for reliability.

Are you coming to the CASPER workshop? If so I suggest bringing a music
board and maybe somebody can donate some time to investigate your problem.

Cheers
Jack

On Thu, Aug 3, 2017, 10:57 PM Sam Gordon <[email protected]> wrote:

> Hi all,
>
> I seek your insight into an issue which my collaborators and I have been
> troubleshooting for some time. It has to do with a 1024 channel Roach2
> spectrometer which uses the MUSIC DAC/ADC board. The Roach is clocked at
> 256 MHz, which it derives from a 512 MHz input to the MUSIC board (taken
> from one of the ADC channels). After making several duplicates of the
> system (hardware, firmware and software), we noticed that a few of them
> (something like one out of six) exhibit spurious voltage spikes in the ADC
> time streams, which are visible via snap block readout. The effect of the
> glitches is to increase the overall amplitude noise level at the channel
> outputs, so we'd like to minimize or eliminate them if at all possible.
>
> Here is a rundown of observations and some attempts at troubleshooting so
> far:
>
> Signs and symptoms:
> - Voltage spikes in the digitized time streams of multitone frequency
> combs. The glitches appear to be non-periodic, and vary in amplitude over
> the full scale of the ADCs. They are present in both the I and Q channels,
> but can be worse in a single channel. They are also observed, to a lesser
> extent, when the I/Q inputs are terminated in 50 ohms. No glitches are
> observed in the DAC output, which is looped directly back to the ADCs
> during these measurements. The glitches only present in waveforms
> containing more than a single frequency.
>
> - On an afflicted system, the glitches disappear when the clock is either
> slowed to < ~430 MHz, or increased above ~530 MHz. On a system which is not
> afflicted, the glitches can be induced by slowing the clock to below ~500
> MHz.
>
> - They have not been remedied by swapping MUSIC boards, although their
> characteristics differ somewhat between different board combinations. When
> a MUSIC board is taken from an afflicted system and used with a reliable
> Roach, no glitches are observed.
>
> - Glitches are not observed in five systems which have had their FPGA fans
> removed and replaced with heat pipe assemblies. However, one of these
> systems exhibited glitches when the mounting bolts on the FPGA heat sink
> were overtightened by a small amount. When they were loosened, the glitches
> subsided immediately.
>
> Attempts at troubleshooting:
> - Firmware: Identical firmware used on each of the systems. Given the
> correlation with clock rate, timing is a likely culprit. Using MSSGE, the
> firmware compiles with a timing score of zero. However, the best case
> positive slack is quite small (a few picoseconds). I've attempted to
> increase this by using pblocks in PlanAhead, as well as the get lucky
> approach with SmartXplorer, without much success. The ADC MMCM is
> configured for high bandwidth, in accordance with a previous post on MUSIC
> board glitches.
>
> - Hardware: As mentioned above, the glitches haven't been observed when
> running with heat pipes instead of fans. While watching a live stream of
> the ADC snap readout on one of the afflicted systems, I unplugged the power
> leads to the FPGA fan. The spikes gradually subsided over a few seconds.
> When the fan was powered on again, they returned on the same time scale.
> This effect was not reproducible on an identical system.
>
> - We've had no luck using different synthesizers as clock sources,
> filtering, attenuating or otherwise isolating the clock line to the MUSIC
> boards.
>
> Much appreciated,
> Sam Gordon
> Astrophysics PhD candidate
> Arizona State University
>
> --
> You received this message because you are subscribed to the Google Groups "
> [email protected]" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
>

-- 
You received this message because you are subscribed to the Google Groups 
"[email protected]" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To post to this group, send email to [email protected].

Reply via email to