Hi Jack,
Thanks for your reply. I think I understand better what's going on. I was
thinking in terms of 64-bit words that get written or read each cycle, but
looking in qdr_cal(), I should probably think of two 32-bit words read or
written each cycle. Then, perhaps I shouldn't be too surprised if the
order of the two words read out doesn't match my intuition.
Initially, I was compiling at 100 MHz, but the qdr_cal always failed. Then
I tried 200 and 250 MHz, and the qdr_cal succeeds.
My tests go like this. For the 200 MHz design,
I call qdr_cal, and it returns True.
Then I write this sequence of 64-bit values via the CPU:
[0, 1, 2, 3, 4, 5, ..., , 1048574, 1048575]
I read back the sequence from the CPU, and it agrees.
[0, 1, 2, 3, 4, 5, ..., , 1048574, 1048575]
I read it in fabric and get
[1048575<<32, 0<<32, 1<<32, 2<<32, 3<<32, 4<<32, ..., 1048574<<32, ]
Then I try the other direction, by writing a sequence of 64-bit values in
fabric:
[0,1,2,3,4,5, ... , 1048574, 1048575]
I read back the sequence in fabric and it agrees:
[0,1,2,3,4,5, ... , 1048574, 1048575]
I read it via the CPU:
nBytesPerSample=8
struct.unpack('>{}Q'.format(nSamples),
roach.read('qdr0_memory',nSamples*nBytesPerSample))
and get:
[1<<32, 2<<32, 3<<32, 4<<32, 5<<32, ..., 1048575<<32, 0<<32]
So reading and writing only by the CPU is consistent, and reading and
writing only in fabric is consistent, but when I do one by the CPU and the
other in fabric, the two 32 bit words get swapped, and the sequence is
offset by one value. I actually see the same offset in a Roach1 version of
the design as well. It all seems consistent, not intermittent.
Thanks,
Matt
On Mon, Mar 9, 2015 at 10:42 AM, Jack Hickish <jackhick...@gmail.com> wrote:
> Hi Matt,
>
> The qdr_cal routine calibrates the relationship of clock and data signals
> in the link between the FPGA and QDR signals. Basically it writes test
> patterns and reads them back looking for glitches. It then changes the
> delay of IODELAY blocks so that data read from the QDR is captured reliably.
> If this calibration script runs successfully, I'm surprised you still see
> glitches with 32 bits of data swapped between read and write (both from
> PPC). This would (surely?) count as a glitch and cause the calibration
> routine to fail(?). Do you only see strange behaviour when you read from
> the FPGA fabric, or do straight write/reads from the CPU also misbehave? Is
> the behaviour the same on every read / write (i.e., not intermittent)?
>
> My experience with the QDR is that at high clock speeds (not sure how
> high, but I see this at 312 MHz) the read latency of the QDR interface
> isn't 10 clock cycles as it should be even with the minimum IODELAY
> configuration, and there are no reliable qdr_cal solutions. I solved this
> problem for my design by removing half a clock cycle latency from the
> interface, at which point the QDR would calibrate ok. What speed are you
> running your design at? To me, misordered 32-bit chunks of data sounds
> indicative of a latency problem in the qdr_cal solution, since the
> interface is 32 bit DDR, so half a clock cycle mangles 32-bit chunks of
> data.
>
> Good luck!
>
> Jack
>
>
>
> On Fri, 6 Mar 2015 at 23:54 Matt Strader <mstra...@physics.ucsb.edu>
> wrote:
>
>> Hi all,
>>
>> I'm trying to use QDR on Roach2. I have a test design where I write to
>> QDR either in firmware or with katcp, then read it out in the firmware and
>> check the results with snapshot blocks. It mostly works, but with some
>> interesting quirks.
>>
>> I know that katcp can only write 64 bits of the 72 bit QDR word, so I
>> changed the bitbasher blocks in qdr_mask.m to move the inaccessible bits
>> out of the way to the most significant bits. Now everything seems to work
>> well, except that in some builds (seems to depend on clock rate?), the
>> first 32 bits (of the 64 bits I'm writing in a cycle) get swapped with the
>> last 32 bits sometime between writing the word and reading it back. And
>> there is a sometimes a difference in whether I am writing in firmware or by
>> katcp to whether this swap happens.
>>
>> Also, when I write with katcp starting at address zero, I find that when
>> reading out, the write started at what seems to be address 1. Is the
>> latency of QDR set to 10 cycles?
>>
>> I can live with all this, as long as it is repeatable for a given design,
>> but I thought I'd ask if anyone knows what is causing these quirks.
>>
>> Also, I understand that we should run a software calibration for qdr,
>> which seems to be qdr_cal() in qdr.py in the ska-sa:casperfpga repo. So, I
>> run this first, which seems to work for my latest compiles, but I haven't
>> noticed a difference in the results. What exactly does this calibration
>> do?
>>
>> Thanks,
>> Matt Strader
>> UC Santa Barbara
>>
>
