Hi, Jeff, Matt pointed out to me that the crystals on the ROACH2 are pretty nice (100 MHz +/- 25 ppm). Maybe this one misbehaving ROACH2 has a bad crystal (or a badly soldered crystal)?
FWIW, when the PAPER correlator used FPGA-based X engines, the X engines ran asynchronously from the F engines at 5% faster than the F engines (210 MHz vs 200 MHz). I don't really understand why they ran asynchronously. IMHO, it would have made things far simpler if the F engines fed a clock to the X engines so that they could run synchronously. Dave On Jul 8, 2013, at 1:20 PM, Haoxuan Zheng wrote: > Hi Casper, > Is there any way to find out the FPGA clock rate precisely? We are suspecting > that our ROACH2s are not running at the frequency (202MHz) as we specified in > the design. We tried the python function in katcp, but it does not look > reliable enough. We hacked that function and increased the time between > counter reads to 30 seconds (instead of 2 seconds), and the reading is still > not that reliable. They all read roughly around 200.2MHz, and if that's the > case, then we must be doing something wrong regarding the clocks and our > 202MHz did not go into the design. We are using internal FPGA clocks with no > ADC or anything. > > Background: > We are testing 4 ROACH1 F-engine to 4 ROACH2 X-engine 10GBE set up, 16 > connections in between. Each F-engine is sending to all 4 X-engines, and each > X-engine is receiving from all 4 F-engines. We have a very simple sending and > receiving FIFO buffer logic. The thing we see is that one particular X-engine > always has all 4 buffers filled up in a fixed amount of time (32K buffer > fills up in ~270 seconds). We swapped a lot of thing around to test which > part is wrong in the system, and we nailed it down to something physical in > that X-engine, and the fact that it fills up indicate that X-engine clock is > running slower than we set it to, because we set X-engine clock to be 202MHz > vs 200MHz on F-engine. Therefore we would like an definitive measurement of > the actual X-engine clock that is running. > > Plot explanation: > It's plotting one of the four FIFO buffer filling up over 270 seconds. The > vertical axis is the literal number of numbers currently stored in the FIFO. > This FIFO should on average get 1 number every 8 clocks from the F-engine > (200MHz) receiver, and should get 1 number pumped out every 8 clocks on > X-engine (202MHz), so this long term build up definitely suggest that the > clocks are not what we think. > > > Thanks a lot and sorry for the long email! > > Jeff > <for_email.png>
