Jack, When you were having problems calibrating the qdr at high clock rate, how did you go about removing half a clock cycle latency from the interface?
Thanks, Matt On Wed, Mar 11, 2015 at 12:27 PM, Matt Strader <mstra...@physics.ucsb.edu> wrote: > I have my own fork of mlib_devel (https://github.com/mstrader/mlib_devel), > which is nearly identical to the current ska-sa repo. > > Thanks, > Matt > > On Wed, Mar 11, 2015 at 12:04 AM, Andrew Martens <and...@ska.ac.za> wrote: > >> I played around with ROACH2 QDR calibration a few months ago and the >> following must be noted; >> >> The physical trace length between FPGA and QDR SRAM is a fixed length. >> This fixed length equates to a different number of clock cycles, >> depending on clock speed. >> The interfacing firmware assumes a fixed number of clock cycles latency. >> The calibration routine thus only works for a certain range of clock >> speeds. >> >> In your case it looks like (at least) one of the interfaces is writing >> (or reading) incorrectly, but is compensating by inserting or removing a >> multiple of half a clock cycle of delay when reading (or writing). When >> reading/writing via the other interface, a different number of compensating >> half clock cycles is inserted. >> >> Which repo are you using? >> >> Regards >> Andrew >> >> On Wed, Mar 11, 2015 at 8:15 AM, David George <dgeorges...@gmail.com> >> wrote: >> >>> A small chip-in - QDR doesn't work below 120 MHz: >>> http://www.cypress.com/?id=4&rID=31542 >>> >>> Regards >>> >>> On 10 March 2015 at 22:17, Matt Strader <mstra...@physics.ucsb.edu> >>> wrote: >>> > Hi Jack, >>> > >>> > Thanks for your reply. I think I understand better what's going on. >>> I was >>> > thinking in terms of 64-bit words that get written or read each cycle, >>> but >>> > looking in qdr_cal(), I should probably think of two 32-bit words read >>> or >>> > written each cycle. Then, perhaps I shouldn't be too surprised if the >>> order >>> > of the two words read out doesn't match my intuition. >>> > >>> > Initially, I was compiling at 100 MHz, but the qdr_cal always failed. >>> Then >>> > I tried 200 and 250 MHz, and the qdr_cal succeeds. >>> > >>> > My tests go like this. For the 200 MHz design, >>> > I call qdr_cal, and it returns True. >>> > >>> > Then I write this sequence of 64-bit values via the CPU: >>> > [0, 1, 2, 3, 4, 5, ..., , 1048574, 1048575] >>> > >>> > I read back the sequence from the CPU, and it agrees. >>> > [0, 1, 2, 3, 4, 5, ..., , 1048574, 1048575] >>> > >>> > I read it in fabric and get >>> > [1048575<<32, 0<<32, 1<<32, 2<<32, 3<<32, 4<<32, ..., 1048574<<32, ] >>> > >>> > >>> > Then I try the other direction, by writing a sequence of 64-bit values >>> in >>> > fabric: >>> > [0,1,2,3,4,5, ... , 1048574, 1048575] >>> > >>> > I read back the sequence in fabric and it agrees: >>> > [0,1,2,3,4,5, ... , 1048574, 1048575] >>> > >>> > I read it via the CPU: >>> > nBytesPerSample=8 >>> > struct.unpack('>{}Q'.format(nSamples), >>> > roach.read('qdr0_memory',nSamples*nBytesPerSample)) >>> > and get: >>> > [1<<32, 2<<32, 3<<32, 4<<32, 5<<32, ..., 1048575<<32, 0<<32] >>> > >>> > So reading and writing only by the CPU is consistent, and reading and >>> > writing only in fabric is consistent, but when I do one by the CPU and >>> the >>> > other in fabric, the two 32 bit words get swapped, and the sequence is >>> > offset by one value. I actually see the same offset in a Roach1 >>> version of >>> > the design as well. It all seems consistent, not intermittent. >>> > >>> > Thanks, >>> > Matt >>> > >>> > >>> > >>> > On Mon, Mar 9, 2015 at 10:42 AM, Jack Hickish <jackhick...@gmail.com> >>> wrote: >>> >> >>> >> Hi Matt, >>> >> >>> >> The qdr_cal routine calibrates the relationship of clock and data >>> signals >>> >> in the link between the FPGA and QDR signals. Basically it writes test >>> >> patterns and reads them back looking for glitches. It then changes >>> the delay >>> >> of IODELAY blocks so that data read from the QDR is captured reliably. >>> >> If this calibration script runs successfully, I'm surprised you still >>> see >>> >> glitches with 32 bits of data swapped between read and write (both >>> from >>> >> PPC). This would (surely?) count as a glitch and cause the calibration >>> >> routine to fail(?). Do you only see strange behaviour when you read >>> from the >>> >> FPGA fabric, or do straight write/reads from the CPU also misbehave? >>> Is the >>> >> behaviour the same on every read / write (i.e., not intermittent)? >>> >> >>> >> My experience with the QDR is that at high clock speeds (not sure how >>> >> high, but I see this at 312 MHz) the read latency of the QDR >>> interface isn't >>> >> 10 clock cycles as it should be even with the minimum IODELAY >>> configuration, >>> >> and there are no reliable qdr_cal solutions. I solved this problem >>> for my >>> >> design by removing half a clock cycle latency from the interface, at >>> which >>> >> point the QDR would calibrate ok. What speed are you running your >>> design at? >>> >> To me, misordered 32-bit chunks of data sounds indicative of a latency >>> >> problem in the qdr_cal solution, since the interface is 32 bit DDR, >>> so half >>> >> a clock cycle mangles 32-bit chunks of data. >>> >> >>> >> Good luck! >>> >> >>> >> Jack >>> >> >>> >> >>> >> >>> >> On Fri, 6 Mar 2015 at 23:54 Matt Strader <mstra...@physics.ucsb.edu> >>> >> wrote: >>> >>> >>> >>> Hi all, >>> >>> >>> >>> I'm trying to use QDR on Roach2. I have a test design where I write >>> to >>> >>> QDR either in firmware or with katcp, then read it out in the >>> firmware and >>> >>> check the results with snapshot blocks. It mostly works, but with >>> some >>> >>> interesting quirks. >>> >>> >>> >>> I know that katcp can only write 64 bits of the 72 bit QDR word, so I >>> >>> changed the bitbasher blocks in qdr_mask.m to move the inaccessible >>> bits out >>> >>> of the way to the most significant bits. Now everything seems to >>> work well, >>> >>> except that in some builds (seems to depend on clock rate?), the >>> first 32 >>> >>> bits (of the 64 bits I'm writing in a cycle) get swapped with the >>> last 32 >>> >>> bits sometime between writing the word and reading it back. And >>> there is a >>> >>> sometimes a difference in whether I am writing in firmware or by >>> katcp to >>> >>> whether this swap happens. >>> >>> >>> >>> Also, when I write with katcp starting at address zero, I find that >>> when >>> >>> reading out, the write started at what seems to be address 1. Is the >>> >>> latency of QDR set to 10 cycles? >>> >>> >>> >>> I can live with all this, as long as it is repeatable for a given >>> design, >>> >>> but I thought I'd ask if anyone knows what is causing these quirks. >>> >>> >>> >>> Also, I understand that we should run a software calibration for qdr, >>> >>> which seems to be qdr_cal() in qdr.py in the ska-sa:casperfpga >>> repo. So, I >>> >>> run this first, which seems to work for my latest compiles, but I >>> haven't >>> >>> noticed a difference in the results. What exactly does this >>> calibration do? >>> >>> >>> >>> Thanks, >>> >>> Matt Strader >>> >>> UC Santa Barbara >>> > >>> > >>> >>> >> >
