Re: [casper] yellow block for 2Gsps adc/dac

[Matt Strader]

To follow up and finish this thread:  I figured out that my main problem
was in how I was generating the ramp on the ADC/DAC board, which was
sensitive to timing problems.  I fixed this and now I get very nice looking
eye patterns, even when the data comes in at 500MHz DDR.

Also, after seeing a hint in the mailing list archive
, I
managed to use the mmcm in high bandwidth mode with a 125 MHz pfd freq.
When I instantiate the mmcm I tell it the pfd freq will be 135 MHz,
CLKIN1_PERIOD=7.407, and then give it 125 MHz instead.

Thanks for your help, Jack and John.

Matt

On Fri, Apr 29, 2016 at 5:56 AM, John Ford  wrote:

> Hi all.
>
> >From the work that Matt's done on the IODELAY and such, and the orginal
> description of the problem, i.e. a simultaneous switching of several bits
> at once, this sounds like a ground bounce problem or decoupling problems
> on the ADC/DAC board, and not a firmware or ROACH problem.
>
> John
>
>  > Matt,
> >
> > No good ones --
> >
> > Are the bits that fail hardware / boffile specific?
> > Are all the timing constraints correctly set up?
> > Are all the IODELAY blocks suitably configured with resets, etc?
> >
> > That's all I got :(
> >
> > The ADC5g does 1.25Gb/s per ZDOK pair perfectly happily (I have 18 boards
> > running them, and I don't remember the last time they failed to link up
> > properly), so I don't think there's any fundamental problem from the
> > ROACH2
> > side.
> >
> > Jack
> >
> >
> > On Thu, 28 Apr 2016 at 20:46 Matt Strader 
> > wrote:
> >
> >> Thanks, Jack.
> >>
> >> I added IODELAYs to all my data bits.  I used my ramp input to check for
> >> glitches in all the bits for each step in IODELAY value.  Also, I found
> >> that having the mmcm in low bandwidth mode is indeed a bad idea, and
> >> switched to high bandwidth mode and slightly slower clocks.
> >>
> >> I find a reasonable eye to calibrate with, except there are 4 bits that
> >> glitch no matter what value I use for their IODELAY.  I lowered the
> >> clock
> >> rates quite a bit more, with the same result.
> >>
> >> Any more recommendations?
> >>
> >> Thanks,
> >> Matt
> >>
> >> P.S. Here's what my eyes look like for 475 MHz DDR (1 indicates a bit
> >> glitches for the delay value).  Notice that data2 bits 1,5,8, and 9 are
> >> ones all the way down the column.
> >> dly : data0_bits   data1_bits   data2_bits   data3_bits
> >>   0 : 1011  101100100010 110110001000
> >>   1 : 0011  101110100010 110010001000
> >>   2 : 0011  101110100010 11001000
> >>   3 : 00010100 0100 10100010 11001000
> >>   4 : 0100 0100 11100010 1100
> >>   5 : 01001110 11001100 11101010 11000110
> >>   6 : 0100 11101100 1110 110001110010
> >>   7 : 0100 1110  110001110111
> >>   8 : 01001110 0100  110001110111
> >>   9 : 0100 0101 001100100010 111001110111
> >>  10 : 0100 1011 101100100010 001110011101
> >>  11 : 0100 1010 101100100010 00001000
> >>  12 : 1100 0010 101100100010 010110001000
> >>  13 : 1011  101100100010 110110001000
> >>  14 : 1011  101100100010 110110001000
> >>  15 : 0011  101110100010 11001000
> >>  16 : 00110100  101110100010 11001000
> >>  17 : 00010100 0100 10100010 1100
> >>  18 : 01001110 01000100 11100010 1100
> >>  19 : 0100 01001100 1010 110001100010
> >>  20 : 1100 1100 1110 110001110010
> >>  21 : 01001110   110001110111
> >>  22 : 0100   110001110111
> >>  23 : 10110011 0101 101100100011 111001110111
> >>  24 : 0011 1011 101100100010 
> >>  25 : 10110011 1010 101100100010 00001000
> >>  26 : 10110001 0010 101100100010 110110001000
> >>  27 : 1011  101100100010 110110001000
> >>  28 : 0011  101110100010 110110001000
> >>  29 : 00110100  101110100010 110110001000
> >>  30 : 00010100 0100 10100010 1100
> >>  31 : 0100 0100 11100010 1100
> >>
> >>
> >>
> >> On Tue, Apr 26, 2016 at 4:55 PM, Jack Hickish 
> >> wrote:
> >>
> >>> Hi Matt,
> >>>
> >>> The usual way to deal with this is IODELAY blocks as you suggest. The
> >>> adc5g core has an example of the correct instantiation of the IODELAY
> >>> primitive and some controller code to talk to them. IIRC, the delay
> >>> goes
> >>> straight after the input buffer, prior to the SERDES (or presumably
> >>> immediately before the output buffers for the DAC).

Re: [casper] yellow block for 2Gsps adc/dac

[Matt Strader]

Thanks, Jack.

I added IODELAYs to all my data bits.  I used my ramp input to check for
glitches in all the bits for each step in IODELAY value.  Also, I found
that having the mmcm in low bandwidth mode is indeed a bad idea, and
switched to high bandwidth mode and slightly slower clocks.

I find a reasonable eye to calibrate with, except there are 4 bits that
glitch no matter what value I use for their IODELAY.  I lowered the clock
rates quite a bit more, with the same result.

Any more recommendations?

Thanks,
Matt

P.S. Here's what my eyes look like for 475 MHz DDR (1 indicates a bit
glitches for the delay value).  Notice that data2 bits 1,5,8, and 9 are
ones all the way down the column.
dly : data0_bits   data1_bits   data2_bits   data3_bits
  0 : 1011  101100100010 110110001000
  1 : 0011  101110100010 110010001000
  2 : 0011  101110100010 11001000
  3 : 00010100 0100 10100010 11001000
  4 : 0100 0100 11100010 1100
  5 : 01001110 11001100 11101010 11000110
  6 : 0100 11101100 1110 110001110010
  7 : 0100 1110  110001110111
  8 : 01001110 0100  110001110111
  9 : 0100 0101 001100100010 111001110111
 10 : 0100 1011 101100100010 001110011101
 11 : 0100 1010 101100100010 00001000
 12 : 1100 0010 101100100010 010110001000
 13 : 1011  101100100010 110110001000
 14 : 1011  101100100010 110110001000
 15 : 0011  101110100010 11001000
 16 : 00110100  101110100010 11001000
 17 : 00010100 0100 10100010 1100
 18 : 01001110 01000100 11100010 1100
 19 : 0100 01001100 1010 110001100010
 20 : 1100 1100 1110 110001110010
 21 : 01001110   110001110111
 22 : 0100   110001110111
 23 : 10110011 0101 101100100011 111001110111
 24 : 0011 1011 101100100010 
 25 : 10110011 1010 101100100010 00001000
 26 : 10110001 0010 101100100010 110110001000
 27 : 1011  101100100010 110110001000
 28 : 0011  101110100010 110110001000
 29 : 00110100  101110100010 110110001000
 30 : 00010100 0100 10100010 1100
 31 : 0100 0100 11100010 1100



On Tue, Apr 26, 2016 at 4:55 PM, Jack Hickish  wrote:

> Hi Matt,
>
> The usual way to deal with this is IODELAY blocks as you suggest. The
> adc5g core has an example of the correct instantiation of the IODELAY
> primitive and some controller code to talk to them. IIRC, the delay goes
> straight after the input buffer, prior to the SERDES (or presumably
> immediately before the output buffers for the DAC).
>
> Cheers
> Jack
>
>
> On Tue, 26 Apr 2016 at 16:02 Matt Strader 
> wrote:
>
>> Hello again,
>>
>> Now that my clocks are working, I've run into the next problem in my
>> yellow block.  I have the adc/dac board sending a ramp, which I see on the
>> roach2 side, but there are periodic glitches (see attached).  ​
>>  zdok_ctr_ramp2.tiff
>> 
>>
>> The data is coming over four 14-bit buses aligned to a 500 MHz clock from
>> the adc/dac board, at DDR.  The glitches happen when the value coming
>> through is a round number in binary, i.e. whenever several bits flip at
>> once.  It seems that at each glitch, one or more bits fail to flip when
>> they should, though they usually arrive at the right value one cycle later.
>>
>> I saw in the mailing list archive that people have had problems
>> 
>> with glitches when using MMCMs in LOW bandwidth mode.  I've tried both LOW
>> and HIGH modes and see the same thing.  (For the HIGH mode I slowed my
>> clock from 500 MHz to 350 MHz.  To use exactly 500 MHz as I want, I'm stuck
>> with LOW mode.)
>>
>> I'm guessing that IODELAYs are the solution to this?  Do I put the delays
>> at the inputs of my OSERDESs?  Any recommendations?
>>
>> Thanks,
>> Matt​
>>
>> On Wed, Apr 20, 2016 at 6:47 PM, Matt Strader 
>> wrote:
>>
>>> I found my mistake.  I did not have feedback input and output ports of
>>> the mmcm (CLKFBOUT,CLKFBIN) connected correctly.
>>>
>>> Matt
>>>
>>> On Tue, Apr 19, 2016 at 12:10 PM, Matt Strader <
>>> mstra...@physics.ucsb.edu> wrote:
>>>
 Hello Casperites,

 I am working on a roach2 yellow block for a new 12bit,  2.0 Gsps
 ADC/DAC board designed at Fermilab.  I am having trouble with clocks.  The
 ADC/DAC board is providing 

Re: [casper] yellow block for 2Gsps adc/dac

[Jack Hickish]

Hi Matt,

The usual way to deal with this is IODELAY blocks as you suggest. The adc5g
core has an example of the correct instantiation of the IODELAY primitive
and some controller code to talk to them. IIRC, the delay goes straight
after the input buffer, prior to the SERDES (or presumably immediately
before the output buffers for the DAC).

Cheers
Jack


On Tue, 26 Apr 2016 at 16:02 Matt Strader  wrote:

> Hello again,
>
> Now that my clocks are working, I've run into the next problem in my
> yellow block.  I have the adc/dac board sending a ramp, which I see on the
> roach2 side, but there are periodic glitches (see attached).  ​
>  zdok_ctr_ramp2.tiff
> 
>
> The data is coming over four 14-bit buses aligned to a 500 MHz clock from
> the adc/dac board, at DDR.  The glitches happen when the value coming
> through is a round number in binary, i.e. whenever several bits flip at
> once.  It seems that at each glitch, one or more bits fail to flip when
> they should, though they usually arrive at the right value one cycle later.
>
> I saw in the mailing list archive that people have had problems
> 
> with glitches when using MMCMs in LOW bandwidth mode.  I've tried both LOW
> and HIGH modes and see the same thing.  (For the HIGH mode I slowed my
> clock from 500 MHz to 350 MHz.  To use exactly 500 MHz as I want, I'm stuck
> with LOW mode.)
>
> I'm guessing that IODELAYs are the solution to this?  Do I put the delays
> at the inputs of my OSERDESs?  Any recommendations?
>
> Thanks,
> Matt​
>
> On Wed, Apr 20, 2016 at 6:47 PM, Matt Strader 
> wrote:
>
>> I found my mistake.  I did not have feedback input and output ports of
>> the mmcm (CLKFBOUT,CLKFBIN) connected correctly.
>>
>> Matt
>>
>> On Tue, Apr 19, 2016 at 12:10 PM, Matt Strader > > wrote:
>>
>>> Hello Casperites,
>>>
>>> I am working on a roach2 yellow block for a new 12bit,  2.0 Gsps ADC/DAC
>>> board designed at Fermilab.  I am having trouble with clocks.  The ADC/DAC
>>> board is providing me with a 500 MHz clock over a particular zdok pair.  My
>>> yellow block takes this into an mmcm and divides it down to 250 MHz to be
>>> used as the fabric clock.
>>>
>>> When testing it, I set the adc/dac clock running, then program the
>>> roach2.  When I call KatcpFpga.estimate_fpga_clock() it returns 167 MHz
>>> instead of the expected 250 MHz.
>>>
>>> I have attached what I think are the relevant snippets from my yellow
>>> block.  The rest of it can be found at
>>> https://github.com/mstrader/mlib_devel
>>> Any ideas what could be the problem?
>>>
>>> Thanks,
>>> Matt Strader
>>>
>>
>>
>

Re: [casper] yellow block for 2Gsps adc/dac

[Matt Strader]

Hello again,

Now that my clocks are working, I've run into the next problem in my yellow
block.  I have the adc/dac board sending a ramp, which I see on the roach2
side, but there are periodic glitches (see attached).  ​
 zdok_ctr_ramp2.tiff


The data is coming over four 14-bit buses aligned to a 500 MHz clock from
the adc/dac board, at DDR.  The glitches happen when the value coming
through is a round number in binary, i.e. whenever several bits flip at
once.  It seems that at each glitch, one or more bits fail to flip when
they should, though they usually arrive at the right value one cycle later.

I saw in the mailing list archive that people have had problems

with glitches when using MMCMs in LOW bandwidth mode.  I've tried both LOW
and HIGH modes and see the same thing.  (For the HIGH mode I slowed my
clock from 500 MHz to 350 MHz.  To use exactly 500 MHz as I want, I'm stuck
with LOW mode.)

I'm guessing that IODELAYs are the solution to this?  Do I put the delays
at the inputs of my OSERDESs?  Any recommendations?

Thanks,
Matt​

On Wed, Apr 20, 2016 at 6:47 PM, Matt Strader 
wrote:

> I found my mistake.  I did not have feedback input and output ports of the
> mmcm (CLKFBOUT,CLKFBIN) connected correctly.
>
> Matt
>
> On Tue, Apr 19, 2016 at 12:10 PM, Matt Strader 
> wrote:
>
>> Hello Casperites,
>>
>> I am working on a roach2 yellow block for a new 12bit,  2.0 Gsps ADC/DAC
>> board designed at Fermilab.  I am having trouble with clocks.  The ADC/DAC
>> board is providing me with a 500 MHz clock over a particular zdok pair.  My
>> yellow block takes this into an mmcm and divides it down to 250 MHz to be
>> used as the fabric clock.
>>
>> When testing it, I set the adc/dac clock running, then program the
>> roach2.  When I call KatcpFpga.estimate_fpga_clock() it returns 167 MHz
>> instead of the expected 250 MHz.
>>
>> I have attached what I think are the relevant snippets from my yellow
>> block.  The rest of it can be found at
>> https://github.com/mstrader/mlib_devel
>> Any ideas what could be the problem?
>>
>> Thanks,
>> Matt Strader
>>
>
>