Re: [casper] Fwd: Problems with Speed Optimization toolflow

[Jack Hickish]

Hey Andres,

Just a quick follow up, since I've been playing some FFT timing issue
games myself --

I don't know which mlib_devel fork you are using, but in some you'll
find that rounding is implemented with the casper convert block, with
pipelined fabric adder cores. My experience is that whilst these
pipelined cores might help meet timing, the compiler does not optimize
them effectively. (E.g. there's one convert which essentially involves
an adder where one input is zero, but it's still compiled to a logic
chain if the adder is implemented in a core). I found that with a
couple of 4096pt FFTs, I saved ~6000 slices (on ROACH 2) by switching
back to the Xilinx cast blocks.

I've made a few mods to the FFT and convert blocks to allow:
1) Your own choice of cast block implementations, using either the
casper block in behavioural, Fabric core or DSP core modes, or the
Xilinx cast
2) Different choices of implementations in the twiddle blocks and
butterfly blocks (whose casts are different sizes).
3) Different choices of adder/convert latencies in the twiddle blocks
and butterflies. This allows you to tweak the latencies depending
which adders you want to implement in DSP.

I've also added a "UCF" yellow block, which allows you to add custom
UCF constraints (e.g. those generated by planahead) into a design, by
supplying an additional UCF file path within Simulink.

On the off chance any of this is useful to you or anyone else, it's in
https://github.com/jack-h/mlib_devel

Cheers,

Jack


On 27 November 2013 17:26, Andres Alvear  wrote:
> Hello guys,
>
> I'm so sorry to be confused to explain my project, right now have 2
> spectrometers implemented in the ROACH 1 with the FPGA Virtex-5
> xc5vsx95tff1136-1, with 2 ADC083000 boards. Now answering your question I'm
> thinking that i've a lot of failed reports, but all of them have the same
> pattern I mean the design have been compiled using logic extensively. The
> next are the results from ADC @ 500 MHz and the clock of the FPGA @ 125 MHz
> compilation:
>
>
> Device Utilization Summary:
>
> Number of BUFGs 8 out of 32 25%
> Number of DCM_ADVs 4 out of 12 33%
> Number of DSP48Es 384 out of 640 60%
> Number of ILOGICs 111 out of 800 13%
> Number of External IOBs 190 out of 640 29%
> Number of LOCed IOBs 190 out of 190 100%
> Number of OLOGICs 19 out of 800 2%
> Number of RAMB18X2s 103 out of 244 42%
> Number of RAMB36_EXPs 80 out of 244 32%
> Number of Slices 14584 out of 14720 99%
> Number of Slice Registers 53504 out of 58880 90%
> Number used as Flip Flops 53504
> Number used as Latches 0
> Number used as LatchThrus 0
> Number of Slice LUTS 44326 out of 58880 75%
> Number of Slice LUT-Flip Flop pairs 56278 out of 58880 95%
>
> Timing summary:
>
> ---
>
> Timing errors: 0 Score: 0 (Setup/Max: 0, Hold: 0)
> Constraints cover 801808 paths, 129 nets, and 158113 connections
>
> Design statistics:
>
> Minimum period: 8.332ns (Maximum frequency: 120.019MHz)
> Maximum net delay: 2.652ns
>
> I just have one way to get a successful compilation with two spectrometers
> using a 4096 points of pipeline-FFT each without check the option “DSP48E
> adders in butterfly” and low latencies like, “add latency=1”, “mult
> latency=2”, “BRAMs=2”, “convert latency=1”, “input latency=0” n “latency
> between biplexes and fft_direct=0”. The FIR filter work with their complete
> stuff.
>
>
> One of my specifics objectives of my project consist in increasing the
> bandwidth of each spectrometer from 500 MHz to 1500 MHz without losing
> spectral resolution, I mean increasing the number of channels proportionally
> to the increase of the bandwidth, so I need to increase from 2048 channels
> to 4096 channels too.
>
> For this reason I go ahead to use PlanAhead using the results showing above.
> After floorplaned my design like Ryan showed in his report using plan ahead
> to generate a ucf file, which then I placed in data/ system.ucf. and then re
> ran the tools for edk ise bitgen. This stage work successful for me.
>
> My constraints were ok because didn't get timing errors.
>
> I think that I've a conceptual error because I don't know why the
> compilation generate timing groups with this timing constraint. In this part
> I need your attention I don't know if I need to delete them or just modify
> them putting more timing groups or increasing their speed or something like
> that. Instead of configuring a clock pin to connect directly into an
> internal clock tree, that pin can be used to drive a special hard-wired
> function (block) called a clock manager that generates a number of daughter
> clocks. So the logic says us that we need to use the ADC's external signal
> clocks to propagate into the whole FPGA using the DCMs to generate daughter
> clocks used to drive internal clock trees or output pins.
>
> How can i do that? I'm going to attach my constraint file to check it out.
>
> Dan, indeed I want to get Simulink designs to run above 300MHz on a ROACH 1,
> so I'm thinking th

Re: [casper] Fwd: Problems with Speed Optimization toolflow

[Andrew Martens]

Hi Andres

Another way to save logic is to hard-code the shifting schedule for the 
FFT. (Go to the Implementation tab and choose 'Hardcode shift schedule). 
This removes quite a lot of logic. You must then decide what shift 
schedule to use to prevent overflows, starting with a '1' for every fft 
stage in the 'Shift Schedule' mask parameter is a start.


Regards
Andrew


Hey Andres,

Just a quick follow up, since I've been playing some FFT timing issue
games myself --

I don't know which mlib_devel fork you are using, but in some you'll
find that rounding is implemented with the casper convert block, with
pipelined fabric adder cores. My experience is that whilst these
pipelined cores might help meet timing, the compiler does not optimize
them effectively. (E.g. there's one convert which essentially involves
an adder where one input is zero, but it's still compiled to a logic
chain if the adder is implemented in a core). I found that with a
couple of 4096pt FFTs, I saved ~6000 slices (on ROACH 2) by switching
back to the Xilinx cast blocks.

I've made a few mods to the FFT and convert blocks to allow:
1) Your own choice of cast block implementations, using either the
casper block in behavioural, Fabric core or DSP core modes, or the
Xilinx cast
2) Different choices of implementations in the twiddle blocks and
butterfly blocks (whose casts are different sizes).
3) Different choices of adder/convert latencies in the twiddle blocks
and butterflies. This allows you to tweak the latencies depending
which adders you want to implement in DSP.

I've also added a "UCF" yellow block, which allows you to add custom
UCF constraints (e.g. those generated by planahead) into a design, by
supplying an additional UCF file path within Simulink.

On the off chance any of this is useful to you or anyone else, it's in
https://github.com/jack-h/mlib_devel

Cheers,

Jack


On 27 November 2013 17:26, Andres Alvear  wrote:

Hello guys,

I'm so sorry to be confused to explain my project, right now have 2
spectrometers implemented in the ROACH 1 with the FPGA Virtex-5
xc5vsx95tff1136-1, with 2 ADC083000 boards. Now answering your question I'm
thinking that i've a lot of failed reports, but all of them have the same
pattern I mean the design have been compiled using logic extensively. The
next are the results from ADC @ 500 MHz and the clock of the FPGA @ 125 MHz
compilation:


Device Utilization Summary:

Number of BUFGs 8 out of 32 25%
Number of DCM_ADVs 4 out of 12 33%
Number of DSP48Es 384 out of 640 60%
Number of ILOGICs 111 out of 800 13%
Number of External IOBs 190 out of 640 29%
Number of LOCed IOBs 190 out of 190 100%
Number of OLOGICs 19 out of 800 2%
Number of RAMB18X2s 103 out of 244 42%
Number of RAMB36_EXPs 80 out of 244 32%
Number of Slices 14584 out of 14720 99%
Number of Slice Registers 53504 out of 58880 90%
Number used as Flip Flops 53504
Number used as Latches 0
Number used as LatchThrus 0
Number of Slice LUTS 44326 out of 58880 75%
Number of Slice LUT-Flip Flop pairs 56278 out of 58880 95%

Timing summary:

---

Timing errors: 0 Score: 0 (Setup/Max: 0, Hold: 0)
Constraints cover 801808 paths, 129 nets, and 158113 connections

Design statistics:

Minimum period: 8.332ns (Maximum frequency: 120.019MHz)
Maximum net delay: 2.652ns

I just have one way to get a successful compilation with two spectrometers
using a 4096 points of pipeline-FFT each without check the option “DSP48E
adders in butterfly” and low latencies like, “add latency=1”, “mult
latency=2”, “BRAMs=2”, “convert latency=1”, “input latency=0” n “latency
between biplexes and fft_direct=0”. The FIR filter work with their complete
stuff.


One of my specifics objectives of my project consist in increasing the
bandwidth of each spectrometer from 500 MHz to 1500 MHz without losing
spectral resolution, I mean increasing the number of channels proportionally
to the increase of the bandwidth, so I need to increase from 2048 channels
to 4096 channels too.

For this reason I go ahead to use PlanAhead using the results showing above.
After floorplaned my design like Ryan showed in his report using plan ahead
to generate a ucf file, which then I placed in data/ system.ucf. and then re
ran the tools for edk ise bitgen. This stage work successful for me.

My constraints were ok because didn't get timing errors.

I think that I've a conceptual error because I don't know why the
compilation generate timing groups with this timing constraint. In this part
I need your attention I don't know if I need to delete them or just modify
them putting more timing groups or increasing their speed or something like
that. Instead of configuring a clock pin to connect directly into an
internal clock tree, that pin can be used to drive a special hard-wired
function (block) called a clock manager that generates a number of daughter
clocks. So the logic says us that we need to use the ADC's external signal
clocks to propagate into the whole FPGA using the DCMs to generate d

[casper] SFP+ board reliability

[Rich Lacasse]

Hi All,

We've had a failure of an SFP+ board for no apparent reason.  We've not 
put any work into trouble shooting except to isolate the failure to the 
SFP+ board, as opposed to a ROACH2 or optical transceiver.


Has anyone else experienced failures?

The two PHY chips on these boards run very warm whether or not they are 
transmitting data.  We are considering putting heatsinks on them.  Has 
anyone else been down this road already?  Any other advice?


Thanks,
Rich

[casper] SFP+ board reliability

[Lincoln Greenhill]

Hi Rich,

On a related topic, we lost one SFP+ port on a board after it had been running 
for O(1) month.  We replaced it w/ another board.  The same port was dead on the 
new mezz. board pointing to the problem being w/ the ROACH2 board.


Best,
Lincoln

--
Lincoln J. Greenhill   Harvard-Smithsonian CfA
Office: 1 617-495-7194 60 Garden St, Mail Stop 42
Cell:   1 650 722-7798 Cambridge, MA 02138
FAX:1 617-495-7345 greenh...@cfa.harvard.edu
Skype:  ljgreenhillwww.cfa.harvard.edu/~lincoln

Re: [casper] SFP+ board reliability

[Jason Ray]

Hi Rich,

So far we haven't had any trouble with the SFP+ boards (8 total) in 
our vegas machine.  I guess they've been powered up continuously and 
in use for around a year now.


We were concerned about how warm the PHY chips run so I spent a 
little time looking into it.  The datasheet says:


"For typical applications, the VSC8486 can be operated in an ambient 
temperature environment of +80 degrees C with no airflow and without 
a heatsink."


It also states that the maximum specification for case temperature is 
85C (95C in one case).  I used our IR viewer and noted that the case 
temperature reached ~60degC on the bench and a little less on the 
ones in the rack.


I should note that the datasheet was for the VSC8486 so I had to 
assume that the thermal specs are similar for the VSC8488, which is 
the actual chip on the SFP+ mezz boards.


Given all of that info, we elected not to put heat sinks on ours.

Hope this helps,
Jason



At 12:25 PM 12/6/2013, Rich Lacasse wrote:

Hi All,

We've had a failure of an SFP+ board for no apparent reason.  We've 
not put any work into trouble shooting except to isolate the failure 
to the SFP+ board, as opposed to a ROACH2 or optical transceiver.


Has anyone else experienced failures?

The two PHY chips on these boards run very warm whether or not they 
are transmitting data.  We are considering putting heatsinks on 
them.  Has anyone else been down this road already?  Any other advice?


Thanks,
Rich

Re: [casper] SFP+ board reliability

[Dan Werthimer]

hi jason,

thanks for the temperature data and measurements.

were your temperature measurements in the
standard roach2 enclosure?

i think the VEGAS SFP+ PHY's are mounted
differently and have different cooling than the
standard roach2 enclosure.

best wishes,

dan





On Fri, Dec 6, 2013 at 9:53 AM, Jason Ray  wrote:

> Hi Rich,
>
> So far we haven't had any trouble with the SFP+ boards (8 total) in our
> vegas machine.  I guess they've been powered up continuously and in use for
> around a year now.
>
> We were concerned about how warm the PHY chips run so I spent a little
> time looking into it.  The datasheet says:
>
> "For typical applications, the VSC8486 can be operated in an ambient
> temperature environment of +80 degrees C with no airflow and without a
> heatsink."
>
> It also states that the maximum specification for case temperature is 85C
> (95C in one case).  I used our IR viewer and noted that the case
> temperature reached ~60degC on the bench and a little less on the ones in
> the rack.
>
> I should note that the datasheet was for the VSC8486 so I had to assume
> that the thermal specs are similar for the VSC8488, which is the actual
> chip on the SFP+ mezz boards.
>
> Given all of that info, we elected not to put heat sinks on ours.
>
> Hope this helps,
> Jason
>
>
>
> At 12:25 PM 12/6/2013, Rich Lacasse wrote:
>
>> Hi All,
>>
>> We've had a failure of an SFP+ board for no apparent reason.  We've not
>> put any work into trouble shooting except to isolate the failure to the
>> SFP+ board, as opposed to a ROACH2 or optical transceiver.
>>
>> Has anyone else experienced failures?
>>
>> The two PHY chips on these boards run very warm whether or not they are
>> transmitting data.  We are considering putting heatsinks on them.  Has
>> anyone else been down this road already?  Any other advice?
>>
>> Thanks,
>> Rich
>>
>
>
>

Re: [casper] SFP+ board reliability

[Jason Ray]

Hi Dan,

One measurement was made with a bare roach2 board just sitting on the 
bench in the lab.


The other was made with the vertical mount roach2 arrangement that we 
use in the vegas rack.


Thanks,
Jason


At 01:19 PM 12/6/2013, Dan Werthimer wrote:



hi jason,

thanks for the temperature data and measurements.

were your temperature measurements in the
standard roach2 enclosure?

i think the VEGAS SFP+ PHY's are mounted
differently and have different cooling than the
standard roach2 enclosure.

best wishes,

dan





On Fri, Dec 6, 2013 at 9:53 AM, Jason Ray 
<j...@nrao.edu> wrote:

Hi Rich,

So far we haven't had any trouble with the SFP+ boards (8 total) in 
our vegas machine.  I guess they've been powered up continuously and 
in use for around a year now.


We were concerned about how warm the PHY chips run so I spent a 
little time looking into it.  The datasheet says:


"For typical applications, the VSC8486 can be operated in an ambient 
temperature environment of +80 degrees C with no airflow and without 
a heatsink."


It also states that the maximum specification for case temperature 
is 85C (95C in one case).  I used our IR viewer and noted that the 
case temperature reached ~60degC on the bench and a little less on 
the ones in the rack.


I should note that the datasheet was for the VSC8486 so I had to 
assume that the thermal specs are similar for the VSC8488, which is 
the actual chip on the SFP+ mezz boards.


Given all of that info, we elected not to put heat sinks on ours.

Hope this helps,
Jason



At 12:25 PM 12/6/2013, Rich Lacasse wrote:
Hi All,

We've had a failure of an SFP+ board for no apparent reason.  We've 
not put any work into trouble shooting except to isolate the failure 
to the SFP+ board, as opposed to a ROACH2 or optical transceiver.


Has anyone else experienced failures?

The two PHY chips on these boards run very warm whether or not they 
are transmitting data.  We are considering putting heatsinks on 
them.  Has anyone else been down this road already?  Any other advice?


Thanks,
Rich

Re: [casper] SFP+ board reliability

[Matt Dexter]

I've had no failures I can prove are related in anyway
to the operating temperature.

Even so and mainly just because I didn't like how warm the parts felt
and had no way to access nor control 
the Vitesse parts temperature monitoring functions

I've added  these heatsinks to my ~20 SFP+ boards

15.2 x 15.x 6.35 mm heat sink black anodize w/ adhesive thermal tape
http://www.aavid.com/products/bga/375424b00034g
40 units @ @2.994

On Fri, 6 Dec 2013, Rich Lacasse wrote:


Date: Fri, 06 Dec 2013 12:25:17 -0500
From: Rich Lacasse 
To: casper@lists.berkeley.edu
Subject: [casper] SFP+ board reliability

Hi All,

We've had a failure of an SFP+ board for no apparent reason.  We've not put 
any work into trouble shooting except to isolate the failure to the SFP+ 
board, as opposed to a ROACH2 or optical transceiver.


Has anyone else experienced failures?

The two PHY chips on these boards run very warm whether or not they are 
transmitting data.  We are considering putting heatsinks on them.  Has anyone 
else been down this road already?  Any other advice?


Thanks,
Rich