Re: [casper] spectrometer implementation using LX110T instead of SX95T
Hi Louis, You can grab the report from the terminal, but it's also at the top of the map report file, at compile-directory/XPS_ROACH_base/implementation/system_map.mrp Cheers, Jack On 29 October 2014 03:31, Louis Dartez louisdar...@gmail.com wrote: Hi Dan, I cut the number of frequency channels from 2k to 512 this afternoon. When I left the lab the design was still compiling so I won’t know how it fared until morning. I forgot to grab the resource utilization to post here. What’s the best way to get the utilization report anyway? Should I just cut it from the xps output in the terminal? I haven’t tried changing the number of PFB taps yet. It’s currently set to 4 in my design. The model I use now utilizes a single PFB block and a single FFT block, both configured for four simultaneous inputs. L Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas Rio Grande Valley (956) 372-5812 On Oct 28, 2014, at 10:26 PM, Dan Werthimer d...@ssl.berkeley.edu wrote: can you cut back the number of frequency channels? or the number of PFB FIR taps? or do away with the PFB FIR entirely? best wishes, dan On Mon, Oct 27, 2014 at 5:36 PM, Louis Dartez louisdar...@gmail.com wrote: Hi Dan, Slices is what seems to be problem from the error report (which I can send around tomorrow morning when I’m back in the lab). I seem to remember that that the compiler raised an error stating that I was trying to ~80k slices when only ~60k are available. I knew this would be a slippery slope when I started. But it would be great if we could salvage our LX110T. Any chance someone out there has a ROACHI SX95T that’s just collecting dust? L Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas Rio Grande Valley (956) 372-5812 On Oct 27, 2014, at 7:31 PM, Dan Werthimer d...@ssl.berkeley.edu wrote: hi louis, are you running out of memory? dsp48's? slices? if memory, the easiest thing to do is cut back on number of frequency channels. best, dan On Mon, Oct 27, 2014 at 4:59 PM, Louis Dartez louisdar...@gmail.com wrote: Hi all, I have implemented a 4 channel 200MHz correlating spectrometer on a ROACH 1 using the Virtex5 SX95T. Currently, I am trying to get this same design to compile and run on a LX110T chip instead. I know that the SX95T is much more DSP intensive and more suitable for this sort of thing. During compilation for the LX110T I ran into the expected issues with resources and trying to use more than were available on the LX110T. I was wondering if anyone had any tips/advice on how to go about this? Has anyone out there run into similar situations? What knobs should I be able to tweak to get the design to compile for a LX110T? Is it even possible? I’d be more than happy to share my mdl (slx) files if needed. :) Thanks in advance! L Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas at Brownsville (956) 372-5812
Re: [casper] more chipscope troubles
That's exactly what I needed, Thank you! Jay Date: Wed, 29 Oct 2014 06:33:18 +0200 Subject: Re: [casper] more chipscope troubles From: he...@ska.ac.za To: jay_br...@live.com CC: casper@lists.berkeley.edu Hi Jay Attached is a chipscope project file for ROACH2. You can open it and it will probably complain about dimension mismatches, but it will at least have each device name and run length. If this does not work here are the lengths (from the project file): Device run lengths: deviceChain.iRLength0=8 deviceChain.iRLength1=10 deviceChain.iRLength2=8 deviceChain.iRLength3=3 deviceChain.iRLength4=3 deviceChain.iRLength5=3 deviceChain.iRLength6=3 deviceChain.iRLength7=8 deviceChain.iRLength8=8 deviceChain.iRLength9=5 deviceChain.iRLength10=5 List of devices: (MyDevice1 = XC6VSX475T (Virtex 6 FPGA), MyDevice2 = XC2C256 (Max CPLD)) deviceChain.name0=PPC deviceChain.name1=MyDevice1 deviceChain.name2=MyDevice2 deviceChain.name3=QDR deviceChain.name4=QDR deviceChain.name5=QDR deviceChain.name6=QDR deviceChain.name7=MARVELL PHY deviceChain.name8=MARVELL PHY deviceChain.name9=MAX16071 deviceChain.name10=MAX16071 On Tue, Oct 28, 2014 at 11:27 PM, Jay Brady jay_br...@live.com wrote: Hey all - I'm trying to connect chipscope to a roach2 model. The model is super basic: all it has is a chipscope block being fed with a single counter. When I connect to the platform II cable in chipscope, I get a dialog box with a number of devices (see attached screenshot). Only two of these have a device name (XC6VSX475T and XC2C256). I don't know what values to put as the IR Length for the rest of the devices, and if I just guess, I am not able to find the actual chipscope core anywhere. All I see is the system monitor console with the on chip sensors (Die temperature, VCCINT supply VCCAUX supply). I have been using the release_jtag_port.py script as suggested in the mail archive. I feel like I am missing something, but I'm not sure what. Any ideas? Thanks, Jay Brady -- Kind regards,Henno Kriel DBE: Hardware Manager SKA South Africa Third Floor The Park Park Road (off Alexandra Road) Pinelands 7405 Western Cape South Africa Latitude: -33.94329 (South); Longitude: 18.48945 (East). (p) +27 (0)21 506 7300 (p) +27 (0)21 506 7374 (direct) (f) +27 (0)21 506 7375 (m) +27 (0)84 504 5050
Re: [casper] spectrometer implementation using LX110T instead of SX95T
Hi Jack, Dan, et al, I’ve linked to two .mrp files in this message. The first is here https://drive.google.com/file/d/0BwvomAqfDQ2DNFZJOEZnS2NUZHd0QV9GamN4S3F6dkl6MXpj/view?usp=sharing and contains the utilization report for the spectrometer logic on a SX95T chip. The second, this one https://drive.google.com/file/d/0BwvomAqfDQ2DcEdNUGpJWWNHQXIzeS1CanhKT1lxOFF1ZHRv/view?usp=sharing is the utilization report for essentially the same design but with a quarter of the FFT/PFB channels on a LX110T chip. The sad thing is that it looks like I’m way over the LX110T limits..even with only 512 channels (as opposed to the original 2k channels). The compile with a quarter of the channels failed, btw. What do you think? L links to utilization reports: LX110T: https://drive.google.com/file/d/0BwvomAqfDQ2DcEdNUGpJWWNHQXIzeS1CanhKT1lxOFF1ZHRv/view?usp=sharing SX95T: https://drive.google.com/file/d/0BwvomAqfDQ2DNFZJOEZnS2NUZHd0QV9GamN4S3F6dkl6MXpj/view?usp=sharing Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas Rio Grande Valley (956) 372-5812 On Oct 29, 2014, at 4:10 AM, Jack Hickish jackhick...@gmail.com wrote: Hi Louis, You can grab the report from the terminal, but it's also at the top of the map report file, at compile-directory/XPS_ROACH_base/implementation/system_map.mrp Cheers, Jack On 29 October 2014 03:31, Louis Dartez louisdar...@gmail.com wrote: Hi Dan, I cut the number of frequency channels from 2k to 512 this afternoon. When I left the lab the design was still compiling so I won’t know how it fared until morning. I forgot to grab the resource utilization to post here. What’s the best way to get the utilization report anyway? Should I just cut it from the xps output in the terminal? I haven’t tried changing the number of PFB taps yet. It’s currently set to 4 in my design. The model I use now utilizes a single PFB block and a single FFT block, both configured for four simultaneous inputs. L Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas Rio Grande Valley (956) 372-5812 On Oct 28, 2014, at 10:26 PM, Dan Werthimer d...@ssl.berkeley.edu wrote: can you cut back the number of frequency channels? or the number of PFB FIR taps? or do away with the PFB FIR entirely? best wishes, dan On Mon, Oct 27, 2014 at 5:36 PM, Louis Dartez louisdar...@gmail.com wrote: Hi Dan, Slices is what seems to be problem from the error report (which I can send around tomorrow morning when I’m back in the lab). I seem to remember that that the compiler raised an error stating that I was trying to ~80k slices when only ~60k are available. I knew this would be a slippery slope when I started. But it would be great if we could salvage our LX110T. Any chance someone out there has a ROACHI SX95T that’s just collecting dust? L Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas Rio Grande Valley (956) 372-5812 On Oct 27, 2014, at 7:31 PM, Dan Werthimer d...@ssl.berkeley.edu wrote: hi louis, are you running out of memory? dsp48's? slices? if memory, the easiest thing to do is cut back on number of frequency channels. best, dan On Mon, Oct 27, 2014 at 4:59 PM, Louis Dartez louisdar...@gmail.com wrote: Hi all, I have implemented a 4 channel 200MHz correlating spectrometer on a ROACH 1 using the Virtex5 SX95T. Currently, I am trying to get this same design to compile and run on a LX110T chip instead. I know that the SX95T is much more DSP intensive and more suitable for this sort of thing. During compilation for the LX110T I ran into the expected issues with resources and trying to use more than were available on the LX110T. I was wondering if anyone had any tips/advice on how to go about this? Has anyone out there run into similar situations? What knobs should I be able to tweak to get the design to compile for a LX110T? Is it even possible? I’d be more than happy to share my mdl (slx) files if needed. :) Thanks in advance! L Louis P. Dartez Graduate Research Assistant STARGATE Center for Advanced Radio Astronomy University of Texas at Brownsville (956) 372-5812
[casper] Starburst, an open-source 10gsps low-N correlator for ROACH2
Hey guys, The CASPER community has been a great help to me in the past few years. People have asked for my libraries and due to JPL policy, I've always had to turn them away. Thanks to help from Bob Jarnot, Jonathon Kocz and others, I'm now free to open-source some of my designs/libraries. For my PhD, I'm designing a 10gsps correlator. I'd really like for this to be an extremely versatile design, useful for radioastronomy and earth-observing-science, good for all broadband, low-N applications. *If there are any special features you'd like to see in this design, beyond what is listed below, tell me now!* I'm willing to add it, but I have to know before everything is finished up. Stats are: (note: N bits complex means N bits for each of real and imag) Mode A: Dual-polarization full-stokes, 2.5 GHz per pol 8192-channel (per pol) 8-tap hamming PFB Mode B: I/Q separating spectrometer 5 GHz total bandwidth 16384 channels across entire band 8-tap hamming PFB Features common to both: Time-domain delay tracking (sample resolution; 48k-sample range) Frequency domain delay tracking (linear interpolation, set two registers to update) Bandpass calibration (applied before I/Q separation): unique 16 bit complex gain applied to each signal= sideband rejection much greater than ADC SNR 10GBE full-duty cycle dump rate (4bits complex per sample) 1GBE accumulation dumps. accumulations supported [10ms - 100s for spectrometer only]; [10ms - 1s for correlator] Everything is synchronized off 1pps and the end of an FFT. Triggered accumulations via GPIO, software register or 1pps (accumulations can be one-off or continuous) In addition, the design will include a X-engine correlator (2 antennas, each 2-pol). The corner turn is performed simply by wiring 10gbe cables. The design can be used as a spectrometer though. The design requires an FPGA clock rate of 312.5 MHz, but I'm going to try for 375 MHz so that we can overclock if we want to (or if we get better ADCs later) I really want to make this a versatile, general purpose, broadband, spectrometer/low-N correlator. Features I could add if people want: DDR circular buffer (4bits of each adc sample, 1.6s of buffer@16 GB of ram) [requested by tom kuiper/majin walid] Larger x-engine (4 dual-pol antennas for charity, I could do 8 but it would be lots of work so we'll have to talk in that case) ADC core matching (if my old firmware for this still works!) *your feature request here* I look forward to your input! As a friendly reminder, my track record for designing FPGA firmware is extremely good, but this might not all pan out as expected. I'm making no promises quite yet. Timeline is currently to have simulated firmware which meets timing at 312.5 MHz (equals 5 GHz total bandwidth) by dec1. Fingers crossed! --Ryan
Re: [casper] Starburst, an open-source 10gsps low-N correlator for ROACH2
Hey Ryan, This sounds great. I've just got a 312mhz design for a project in Cambridge to meet timing (broadly similar to what you're describing, but 10 single pol antennas and only 4k channels over 5ghz bw). Whilst I don't have any particular requests, I would be very interested in hearing about how you end up reaching 375mhz. (What granularity you place pblocks/cunning code optimisations/etc). I found my experience to be educational, if a bit frustrating, and I'd be interested to know the gritty details used by others (equally, if anyone cares, I'm very happy to talk about my tactics). I certainly found that a vague high-level placement of pblocks with the standard mlibdevel libraries didn't work as well as I was hoping. Cheers, Jack On 29 Oct 2014 22:26, Ryan Monroe ryan.m.mon...@gmail.com wrote: Hey guys, The CASPER community has been a great help to me in the past few years. People have asked for my libraries and due to JPL policy, I've always had to turn them away. Thanks to help from Bob Jarnot, Jonathon Kocz and others, I'm now free to open-source some of my designs/libraries. For my PhD, I'm designing a 10gsps correlator. I'd really like for this to be an extremely versatile design, useful for radioastronomy and earth-observing-science, good for all broadband, low-N applications. *If there are any special features you'd like to see in this design, beyond what is listed below, tell me now!* I'm willing to add it, but I have to know before everything is finished up. Stats are: (note: N bits complex means N bits for each of real and imag) Mode A: Dual-polarization full-stokes, 2.5 GHz per pol 8192-channel (per pol) 8-tap hamming PFB Mode B: I/Q separating spectrometer 5 GHz total bandwidth 16384 channels across entire band 8-tap hamming PFB Features common to both: Time-domain delay tracking (sample resolution; 48k-sample range) Frequency domain delay tracking (linear interpolation, set two registers to update) Bandpass calibration (applied before I/Q separation): unique 16 bit complex gain applied to each signal= sideband rejection much greater than ADC SNR 10GBE full-duty cycle dump rate (4bits complex per sample) 1GBE accumulation dumps. accumulations supported [10ms - 100s for spectrometer only]; [10ms - 1s for correlator] Everything is synchronized off 1pps and the end of an FFT. Triggered accumulations via GPIO, software register or 1pps (accumulations can be one-off or continuous) In addition, the design will include a X-engine correlator (2 antennas, each 2-pol). The corner turn is performed simply by wiring 10gbe cables. The design can be used as a spectrometer though. The design requires an FPGA clock rate of 312.5 MHz, but I'm going to try for 375 MHz so that we can overclock if we want to (or if we get better ADCs later) I really want to make this a versatile, general purpose, broadband, spectrometer/low-N correlator. Features I could add if people want: DDR circular buffer (4bits of each adc sample, 1.6s of buffer@16 GB of ram) [requested by tom kuiper/majin walid] Larger x-engine (4 dual-pol antennas for charity, I could do 8 but it would be lots of work so we'll have to talk in that case) ADC core matching (if my old firmware for this still works!) *your feature request here* I look forward to your input! As a friendly reminder, my track record for designing FPGA firmware is extremely good, but this might not all pan out as expected. I'm making no promises quite yet. Timeline is currently to have simulated firmware which meets timing at 312.5 MHz (equals 5 GHz total bandwidth) by dec1. Fingers crossed! --Ryan
Re: [casper] Starburst, an open-source 10gsps low-N correlator for ROACH2
Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published here: http://www.worldscientific.com/doi/pdfplus/10.1142/S2251171714500019?src=recsys There is also a recent publication by Jiang et al on the ADC in PASP: Vol. 126, No. 942 (August 2014), pp. 761-768 At this point have the logic for this correlator reduced to a fully working V6 bit code with all features except the phased array (design in progress). In fact, we are routinely taking observational data at SMA, and plan to field it for science in mid-November. However it is not yet running at our eventual design speed goal of 286 MHz, corresponding to 4.6 Gsps at the ADC—a little more modest than your 5 Gsps. Our experience attempting to meet 286 MHz with this complex of a design has been sobering so far, though we have not given up. If you really are able to get a comparable design running at 375 MHz with -1 speed grade parts, honestly you’d deserve an attaboy or two. And we’d gladly learn from how you got there, so please keep us in the loop. By the way, assuming you are using the ADC referenced with the architecture you describe I’d suggest it is appropriate to cite all above referenced and other relevant prior work in your PhD. Best of luck with it. Jonathan and SMA / EHT team On Oct 29, 2014, at 6:25 PM, Ryan Monroe ryan.m.mon...@gmail.com wrote: Hey guys, The CASPER community has been a great help to me in the past few years. People have asked for my libraries and due to JPL policy, I've always had to turn them away. Thanks to help from Bob Jarnot, Jonathon Kocz and others, I'm now free to open-source some of my designs/libraries. For my PhD, I'm designing a 10gsps correlator. I'd really like for this to be an extremely versatile design, useful for radioastronomy and earth-observing-science, good for all broadband, low-N applications. If there are any special features you'd like to see in this design, beyond what is listed below, tell me now! I'm willing to add it, but I have to know before everything is finished up. Stats are: (note: N bits complex means N bits for each of real and imag) Mode A: Dual-polarization full-stokes, 2.5 GHz per pol 8192-channel (per pol) 8-tap hamming PFB Mode B: I/Q separating spectrometer 5 GHz total bandwidth 16384 channels across entire band 8-tap hamming PFB Features common to both: Time-domain delay tracking (sample resolution; 48k-sample range) Frequency domain delay tracking (linear interpolation, set two registers to update) Bandpass calibration (applied before I/Q separation): unique 16 bit complex gain applied to each signal= sideband rejection much greater than ADC SNR 10GBE full-duty cycle dump rate (4bits complex per sample) 1GBE accumulation dumps. accumulations supported [10ms - 100s for spectrometer only]; [10ms - 1s for correlator] Everything is synchronized off 1pps and the end of an FFT. Triggered accumulations via GPIO, software register or 1pps (accumulations can be one-off or continuous) In addition, the design will include a X-engine correlator (2 antennas, each 2-pol). The corner turn is performed simply by wiring 10gbe cables. The design can be used as a spectrometer though. The design requires an FPGA clock rate of 312.5 MHz, but I'm going to try for 375 MHz so that we can overclock if we want to (or if we get better ADCs later) I really want to make this a versatile, general purpose, broadband, spectrometer/low-N correlator. Features I could add if people want: DDR circular buffer (4bits of each adc sample, 1.6s of buffer@16 GB of ram) [requested by tom kuiper/majin walid] Larger x-engine (4 dual-pol antennas for charity, I could do 8 but it would be lots of work so we'll have to talk in that case) ADC core matching (if my old firmware for this still works!) your feature request here I look forward to your input! As a friendly reminder, my track record for designing FPGA firmware is extremely good, but this might not all pan out as expected. I'm making no promises quite yet. Timeline is currently to have simulated firmware which
Re: [casper] Starburst, an open-source 10gsps low-N correlator for ROACH2
Hi Jonathan! Reply is inline (in blue) Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 That's the one. Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published here: http://www.worldscientific.com/doi/pdfplus/10.1142/S2251171714500019?src=recsys I've seen your work here and it's going to be extremely helpful. Thanks! There is also a recent publication by Jiang et al on the ADC in PASP: Vol. 126, No. 942 (August 2014), pp. 761-768 At this point have the logic for this correlator reduced to a fully working V6 bit code with all features except the phased array (design in progress). In fact, we are routinely taking observational data at SMA, and plan to field it for science in mid-November. However it is not yet running at our eventual design speed goal of 286 MHz, corresponding to 4.6 Gsps at the ADC—a little more modest than your 5 Gsps. Our experience attempting to meet 286 MHz with this complex of a design has been sobering so far, though we have not given up. If you really are able to get a comparable design running at 375 MHz with -1 speed grade parts, honestly you’d deserve an attaboy or two. And we’d gladly learn from how you got there, so please keep us in the loop. I have custom FFT libraries I've written, which consume much fewer resources than stock CASPER stuff. I've used them to close timing to 400 MHz before, but I'm worried that bussing signals around the FPGA is going to be rough at 375. I can talk to one of you, or direct you to reference designs, if you want help closing timing. Is it 2^15 point FFT, or 2^15 channel FFT? Can you handle 2^14 points (equals 2^13 channels) per 2.5 GHz? You are 8 single-pol antennas, each processing 2.5 GHz of bandwidth right? I could build my design with you guys in mind, and close to 312.5 MHz. My design supports all of your features and should be more-or-less plug and play once I'm finished. My output format will be different from yours though. By the way, assuming you are using the ADC referenced with the architecture you describe I’d suggest it is appropriate to cite all above referenced and other relevant prior work in your PhD. For sure! The ADC work is extremely relevant and we couldn't do it without you. Best of luck with it. Jonathan and SMA / EHT team On Wed, Oct 29, 2014 at 5:34 PM, Jonathan Weintroub jweintr...@cfa.harvard.edu wrote: Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published here: http://www.worldscientific.com/doi/pdfplus/10.1142/S2251171714500019?src=recsys There is also a recent publication by Jiang et al on the ADC in PASP: Vol. 126, No. 942 (August 2014), pp. 761-768 At this point have the logic for this correlator reduced to a fully working V6 bit code with all features except the phased array (design in progress). In fact, we are routinely taking observational data at SMA, and plan to field it for science in mid-November. However it is not yet running at our eventual design speed goal of 286 MHz, corresponding to 4.6 Gsps at the ADC—a little more modest than your 5 Gsps. Our experience attempting to meet 286 MHz with this complex of a design has been sobering so far, though we have not given up. If you really are able to get a comparable design running at 375 MHz with -1 speed grade parts, honestly you’d deserve an attaboy or two. And we’d gladly learn from how you got there, so please keep us
Re: [casper] Starburst, an open-source 10gsps low-N correlator for ROACH2
Hi Ryan, Thanks for the response. To answer your question we use 2^15 = 32 k FFTs operating on 8 bit real time samples, to channelize our visibility spectrum to 2^14 = 16k complex points. There is a pair of these 2^15 point PFBs on each Virtex 6, one for each 5 Gsps ADC input. We’d certainly be interested in learning about your custom FFT libraries especially if these may be helpful in getting to timing closure. We do seem to be I/O bound in this design, by the way. I need to leave it there for tonight. Best wishes, Jonathan On Oct 29, 2014, at 9:00 PM, Ryan Monroe ryan.m.mon...@gmail.com wrote: Hi Jonathan! Reply is inline (in blue) Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 That's the one. Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published here: http://www.worldscientific.com/doi/pdfplus/10.1142/S2251171714500019?src=recsys I've seen your work here and it's going to be extremely helpful. Thanks! There is also a recent publication by Jiang et al on the ADC in PASP: Vol. 126, No. 942 (August 2014), pp. 761-768 At this point have the logic for this correlator reduced to a fully working V6 bit code with all features except the phased array (design in progress). In fact, we are routinely taking observational data at SMA, and plan to field it for science in mid-November. However it is not yet running at our eventual design speed goal of 286 MHz, corresponding to 4.6 Gsps at the ADC—a little more modest than your 5 Gsps. Our experience attempting to meet 286 MHz with this complex of a design has been sobering so far, though we have not given up. If you really are able to get a comparable design running at 375 MHz with -1 speed grade parts, honestly you’d deserve an attaboy or two. And we’d gladly learn from how you got there, so please keep us in the loop. I have custom FFT libraries I've written, which consume much fewer resources than stock CASPER stuff. I've used them to close timing to 400 MHz before, but I'm worried that bussing signals around the FPGA is going to be rough at 375. I can talk to one of you, or direct you to reference designs, if you want help closing timing. Is it 2^15 point FFT, or 2^15 channel FFT? Can you handle 2^14 points (equals 2^13 channels) per 2.5 GHz? You are 8 single-pol antennas, each processing 2.5 GHz of bandwidth right? I could build my design with you guys in mind, and close to 312.5 MHz. My design supports all of your features and should be more-or-less plug and play once I'm finished. My output format will be different from yours though. By the way, assuming you are using the ADC referenced with the architecture you describe I’d suggest it is appropriate to cite all above referenced and other relevant prior work in your PhD. For sure! The ADC work is extremely relevant and we couldn't do it without you. Best of luck with it. Jonathan and SMA / EHT team On Wed, Oct 29, 2014 at 5:34 PM, Jonathan Weintroub jweintr...@cfa.harvard.edu wrote: Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published here: http://www.worldscientific.com/doi/pdfplus/10.1142/S2251171714500019?src=recsys There is also a recent publication by Jiang et al on the ADC in PASP: Vol. 126, No. 942 (August 2014), pp. 761-768 At this point have
Re: [casper] Starburst, an open-source 10gsps low-N correlator for ROACH2
They're not released yet, I'm going to deal with that once I've gotten the design up and running :-) Thanks for all your help as well! On Wed, Oct 29, 2014 at 6:21 PM, Jonathan Weintroub jweintr...@cfa.harvard.edu wrote: Hi Ryan, Thanks for the response. To answer your question we use 2^15 = 32 k FFTs operating on 8 bit real time samples, to channelize our visibility spectrum to 2^14 = 16k complex points. There is a pair of these 2^15 point PFBs on each Virtex 6, one for each 5 Gsps ADC input. We’d certainly be interested in learning about your custom FFT libraries especially if these may be helpful in getting to timing closure. We do seem to be I/O bound in this design, by the way. I need to leave it there for tonight. Best wishes, Jonathan On Oct 29, 2014, at 9:00 PM, Ryan Monroe ryan.m.mon...@gmail.com wrote: Hi Jonathan! Reply is inline (in blue) Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 That's the one. Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published here: http://www.worldscientific.com/doi/pdfplus/10.1142/S2251171714500019?src=recsys I've seen your work here and it's going to be extremely helpful. Thanks! There is also a recent publication by Jiang et al on the ADC in PASP: Vol. 126, No. 942 (August 2014), pp. 761-768 At this point have the logic for this correlator reduced to a fully working V6 bit code with all features except the phased array (design in progress). In fact, we are routinely taking observational data at SMA, and plan to field it for science in mid-November. However it is not yet running at our eventual design speed goal of 286 MHz, corresponding to 4.6 Gsps at the ADC—a little more modest than your 5 Gsps. Our experience attempting to meet 286 MHz with this complex of a design has been sobering so far, though we have not given up. If you really are able to get a comparable design running at 375 MHz with -1 speed grade parts, honestly you’d deserve an attaboy or two. And we’d gladly learn from how you got there, so please keep us in the loop. I have custom FFT libraries I've written, which consume much fewer resources than stock CASPER stuff. I've used them to close timing to 400 MHz before, but I'm worried that bussing signals around the FPGA is going to be rough at 375. I can talk to one of you, or direct you to reference designs, if you want help closing timing. Is it 2^15 point FFT, or 2^15 channel FFT? Can you handle 2^14 points (equals 2^13 channels) per 2.5 GHz? You are 8 single-pol antennas, each processing 2.5 GHz of bandwidth right? I could build my design with you guys in mind, and close to 312.5 MHz. My design supports all of your features and should be more-or-less plug and play once I'm finished. My output format will be different from yours though. By the way, assuming you are using the ADC referenced with the architecture you describe I’d suggest it is appropriate to cite all above referenced and other relevant prior work in your PhD. For sure! The ADC work is extremely relevant and we couldn't do it without you. Best of luck with it. Jonathan and SMA / EHT team On Wed, Oct 29, 2014 at 5:34 PM, Jonathan Weintroub jweintr...@cfa.harvard.edu wrote: Hi Ryan, That does look cool! You don’t mention which ADC you plan to use. Is it this one? https://casper.berkeley.edu/wiki/ADC1x5000-8 Just to mention in case it proves useful that our group at Submillimeter Array (SMA) and Event Horizon Telescope (EHT) has been working on a correlator / phased array system with what appear to be rather similar features (low N, wideband, high spectral resolution 32 k PFB etc) using the above ADC (DMUX 1:1 version) and ROACH2. We view it as dual 5 Gsps, but I suppose one might interpret that as 10Gsps. There are specs, a little outdated, here: https://www.cfa.harvard.edu/twiki5/view/SMAwideband/DigitalBackEnd This page includes a link to our open source githup repo with all model files. We have done a fair amount of work on ADC core calibration too, also on the wiki, poke around. The key results were recently published
