Hi Neil, regarding using GPU's or FPGA's for the X engine of an FX correlator:
You probably appreciate GPU's are best used in correlators with a large number of antennas. The problem with using GPU's for a small number of antennas is the GPU is limited by input data rate. The correlator input data rate is proportional to Nantenna, and the correlator X engine computation rate is proportional to Nantenna^2. I think with modern GPUs you need about 1024 antennas before you aren't I/O bound and can take advantage of the GPU's computational power; otherwise the GPU will be idle waiting for data. One way out of this problem is to find an application which can use the GPU for additional tasks besides correlation - tasks that don't need much I/O. Another possibility is to use cheap GPU's that can't do much computation, but then the host computer dominates the cost and power comsumption of the Xengine. FPGA's have a lot more I/O capacity and are lower power than GPU's, but as you know, they are harder to program. I think most modern GPU's can do integer arithmetic with a selectable number of bits, similar to FPGA's. But because floating point is easier, and many correlator GPU applications are I/O bound, most people send their data as small fixed point numbers into the GPU for correlation (eg: 4 bit real, 4 bit imag), but then compute the correlation in floating point. Best Wishes, Dan On Fri, Aug 14, 2020 at 9:37 PM Neil Salmon <n.sal...@mmu.ac.uk> wrote: > Hi Danny, > > > > GPU’s may be starting to rival FPGA’s in processing power for correlators. > However, are GPU’s restricted to long word correlation, say 32-bit floating > point, whereas in FPGA I’m assuming the bit length is variable, so you may > choose say 4-bit integer correlation – which would match to a 4-bit ADC. > > > > If you can’t adapt the word length for GPU correlators, then the FPGA > technology still has an edge, because as far as correlation is concerned, > I’m not sure if going to word lengths longer than 4-bit is resourceful use > of silicon real estate? > > > > What is your view on that? > > > > I did include a section on correlators in the document on security > screening imaging https://ieeexplore.ieee.org/document/9154708 and some > of your selected publications in the references – thank you. > > > > Cheers, > > Neil > > > > *From:* Danny Price <dan...@berkeley.edu> > *Sent:* 20 July 2020 14:44 > *To:* Neil Salmon <n.sal...@mmu.ac.uk>; casper@lists.berkeley.edu > *Subject:* RE: [casper] references to recent cross-correlator technology > developments > > > > Hi Neil, > > > > The correlation is indeed done in real time using stream processing > frameworks for most interferometer telescopes. Conversion from (very > sparse) visibilities to images is generally done offline (this can be very > time consuming!). > > > > There are a few real-time imaging systems: the EPIC correlator that Jack > mentioned, and the realfast system on the VLA ( > https://science.nrao.edu/facilities/vla/observing/realfast) are good > examples. > > > > Cheers, > > Danny > > On 20 July 2020 at 9:55:06 pm, Neil Salmon (n.sal...@mmu.ac.uk) wrote: > > Hi Danny, > > > > Thank you for these references. > > > > For security screening systems the name of the game is real-time, ie an > image in less than 1 second. However, I see a great many references to GPU > based correlators. I was used to seeing these devices as off-line > correlators, as in software correlators. Are the GPUs being used by the > radio astronomy community as real-time correlators, or as software > correlators? > > > > Many thanks, > > Neil > > > > *From:* Danny Price <dan...@berkeley.edu> > *Sent:* 20 July 2020 12:21 > *To:* casper@lists.berkeley.edu > *Subject:* Re: [casper] references to recent cross-correlator technology > developments > > > > Hi Neil, > > > > To add to Jack's post, allow me to plug some overview articles that may be > of interest. The first, https://arxiv.org/abs/1702.00442, was for an > introduction for a special issue of JAI on DSP in radio astronomy in 2016. > Table 1 summarises some of the larger correlators: the references therein > may be of use. Jack (et al)'s CASPER article in said JAI special issue is > also a font of references: https://arxiv.org/abs/1611.01826. The full > special issue article listing is up here: > https://www.worldscientific.com/toc/jai/05/04. > > > > More recently, here's my book chapter on real-time stream processing in > radio astronomy, https://arxiv.org/abs/1912.09041, which delves a bit > deeper into technical details for common approaches. > > > > In terms of cutting edge, there are various groups working with the Xilinx > RFSoC components for next-gen systems -- you will no doubt have seen some > traffic on this list. The ASKAP telescope group have plans to use an Alveo > Xilinx U280 accelerator card for high time resolution imaging + > dedispersion, which is an alternative to the GPU correlator. > > > > GPU correlators are still the most widespread for O(100) antennas. There's > some discussion on GPU correlator performance in J. Kocz et al 2014 ( > https://arxiv.org/abs/1401.8288); for O(100) inputs a GPU correlator will > likely be memory bandwidth bound. > > > > Cheers, > > Danny > > On 18 July 2020 at 7:54:49 pm, Neil Salmon (n.sal...@mmu.ac.uk) wrote: > > I need references on recent developments in cross-correlator technology > for an IEEE paper on the subject of aperture synthesis imaging in the area > of security screening of people for concealed weapons. Typical requirements > for this application are cross-correlators that can process in real-time > signals from hundreds of receiver channels with around 1 GHz of RF > bandwidth. As none of this technology is commercially available > off-the-shelf I’m dependent on the radio astronomy community to get the > latest information of correlator development. This might be just technical > knowhow on the building of correlators, or communities who would be willing > to supply for a fee correlators to a security screening technology > development company. > > > > Could anyone provide me with any references of papers on recent correlator > development that I could include in this paper? > > > > Many thanks, > > Neil > > "Before acting on this email or opening any attachments you should read > the Manchester Metropolitan University email disclaimer available on its > website http://www.mmu.ac.uk/emaildisclaimer " > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/7aff8f2f6ed3482a8283e2994bbd9fc6%40ASEX03.ad.mmu.ac.uk > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/7aff8f2f6ed3482a8283e2994bbd9fc6%40ASEX03.ad.mmu.ac.uk?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAAtMgq%3D_nce72fbSUWBZ62EiCxW5ojUgy%2ByAbPXHMNRd%3DXbLDw%40mail.gmail.com > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAAtMgq%3D_nce72fbSUWBZ62EiCxW5ojUgy%2ByAbPXHMNRd%3DXbLDw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/46372fbec8c34c909df5b72793ec1d09%40ASEX01.ad.mmu.ac.uk > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/46372fbec8c34c909df5b72793ec1d09%40ASEX01.ad.mmu.ac.uk?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To view this discussion on the web visit > https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/f058e58f91134f8bacc604ae28c844d9%40ASEX01.ad.mmu.ac.uk > <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/f058e58f91134f8bacc604ae28c844d9%40ASEX01.ad.mmu.ac.uk?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAGHS_vGLW%3DsUTb3-tkm%2BLZ-3%3DUEGraF3%2BxDb5mfmZcF0-1HboA%40mail.gmail.com.
