hi mayukh, another option to consider:
some of the low power mini-PC computers in the spreadsheet have two 10Gbit/sec ethernet ports built in. (see ben's and my email below for the link to this spreadsheet, and look at the 10G ethernet column.) if you transmit your 12Gbit/sec FPGA data over a pair of 10Gbit ports, then you wouldn't need a NIC, and your mini-PC wouldn't need any PCIe slots. best wishes, dan On Fri, Sep 27, 2024 at 1:32 PM Dan Werthimer <dans...@berkeley.edu> wrote: > > hi mayukh, > > if might help people guide you if you can you say a bit more about your > experiment. > what instrument and signal processing is running on the FPGA ? > (spectrometer? correlator ? beamformer ? ) > what is the FPGA output 12Gbit/sec data stream look like (spectra? > complex channelized voltage data? ) > > what does the processor do with the 12Gbit/sec data stream after it > receives it? > store it on hard disks? SSD? (for how long?) > compress the data and then store it ? > compress the data and then transmit it to a ground station ? > send uncompressed 12Gbit/sec to the ground via a very high speed data link > ? > > can you compress the data on the FPGA ? > > instead of sending the data over 100/25/10 Gbit link to an external > computer, > can you instead process the data using the ARM core processors on the > RFSOC ? > there are six ARM cores on the RFSOC's FPGA. > for info on these ARM processors, see: > > https://www.amd.com/en/corporate/university-program/aup-boards/rfsoc4x2.html > > if you need to send the data from the RFSOC board to an external computer > over 100G, 25G, or a pair of 10G ethernet links, > berkeley casper postdoc ben godfrey sent me a spreadsheet that lists a lot > of low power mini-PC computers. > (please see email from ben below). > one of the column lists the number and type of PCIe slots, > another column lists the power consumption of the board (you need to add > the NIC power to this). > > another berkeley casper postdoc, wei liu, sent me an email about the PCIe > slot on the Raspberry Pi 5. > please see email from wei liu below ben's. > > best wishes, > > dan > > > > > Ben Godfrey > 1:07 PM (9 minutes ago) > Reply > to Wei, me > Not sure how useful this is, but here's a link to a Google Drive with a > lot of mini PC's on it. > > Link: > https://docs.google.com/spreadsheets/d/1SWqLJ6tGmYHzqGaa4RZs54iw7C1uLcTU_rLTRHTOzaA/edit?gid=239063037#gid=239063037 > - Ben > > > > On Fri, Sep 27, 2024 at 1:02 PM Wei Liu <liuwei_berke...@berkeley.edu> > wrote: > >> Hi Dan, Ben, >> >> Here is an interesting blog about pcie on rpi5: >> >> https://www.jeffgeerling.com/blog/2023/forcing-pci-express-gen-30-speeds-on-pi-5 >> >> It looks like the pcie gen can be forced from 2.0 to 3.0. >> Someone tested 10G NIC on it, and got 6Gbps bandwidth. >> >> Best, >> Wei >> > > On Mon, Sep 23, 2024 at 11:33 AM Mayukh Bagchi <mayukh.bag...@queensu.ca> > wrote: > >> Hey Dan, >> >> Thanks for getting back. >> >> My first concern is being able to find a compatible hardware/computer to >> plug the NIC into. Are these systems commercially available? Or are most of >> them custom-built? I could budget around 100 W for such a computer. I don't >> think the power draw of the NICs themselves is concerning for us, we could >> manage that. >> >> Also, I am not sure if the 100 GbE yellow block would be compatible with >> a dual 10 GbE NIC or a 40 GbE NIC, if it does then that opens up quite a >> few options. >> >> >> Regards, >> Mayukh >> >> ------------------------------ >> *From:* 'Dan Werthimer' via casper@lists.berkeley.edu < >> casper@lists.berkeley.edu> >> *Sent:* Monday, September 23, 2024 1:48 PM >> *To:* casper@lists.berkeley.edu <casper@lists.berkeley.edu> >> *Subject:* Re: [casper] 100 GbE Hardware recommendation >> >> >> >> Hi mayukh, >> >> You might consider two 10 gbe links and a dual 10 gbe nic. Or a 40 gbe >> nic. Are you concerned with the power consumption of the 100 gbe nic, or >> the consumption of the computer the nic is plugged into? How much >> computing power and storage do you need ? >> >> Best wishes, >> >> Dan >> >> On Mon, Sep 23, 2024, 1:18 PM Mayukh Bagchi <mayukh.bag...@queensu.ca> >> wrote: >> >> Hello Casperites, >> I hope you all have been doing well. >> >> I have been developing an RFSoC-based backend for our Balloon-borne VLBI >> Experiment (BVEX) project, which will be a mobile K-band VLBI station, >> launching in August next year. I am using the RFSoC 4x2 to digitize and >> stream data for our IF stage. The CASPER community has been very helpful >> with setup and troubleshooting. >> >> So far, I've successfully implemented a spectrometer and a power meter. >> While the design for the 100 GbE data streaming is almost ready, I am >> having trouble selecting a compatible NAS/server-side system to directly >> communicate with the 100 GbE Mellanox NIC >> <https://www.fs.com/products/119648.html?attribute=67745&id=3746425>. >> The reason why it is getting tricky is the size, thermal regulation, and >> power consumption constraints, as this will be on a balloon platform with >> limited resources. Our data rates should be around ~10-12 Gbps, depending >> on how much we end up re-quantizing, so we may not use the full >> capabilities of the 100 GbE network. However, at this point, any suggestion >> on hardware will be useful in getting us pointed in the right direction. >> >> Feel free to share your advice on working with 100 GbE Mellanox NICs. If >> this seems challenging, I could probably lower our data rates and try the >> 10GbE yellow block Ethernet core. >> >> Thanks in advance! >> >> Regards, >> Mayukh >> *__________* >> >> *Mayukh* *Bagchi* (He/Him), PhD Candidate >> Department of Physics, Engineering Physics, and Astronomy >> mayukhbagchi.com | mayukh.bag...@queensu.ca >> [image: Queen's University Logo] <https://www.queensu.ca/> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "casper@lists.berkeley.edu" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to casper+unsubscr...@lists.berkeley.edu. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB10632111BE97CE7C079D9E886916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB10632111BE97CE7C079D9E886916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "casper@lists.berkeley.edu" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to casper+unsubscr...@lists.berkeley.edu. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzJmp1E%3DZknwdB4V18WypD8ak0TewhDoqe8WsTSk1i1-YA%40mail.gmail.com >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzJmp1E%3DZknwdB4V18WypD8ak0TewhDoqe8WsTSk1i1-YA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "casper@lists.berkeley.edu" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to casper+unsubscr...@lists.berkeley.edu. >> To view this discussion on the web visit >> https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB1063206055042E9DCB07FFFD2916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM >> <https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB1063206055042E9DCB07FFFD2916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzKD%3DrkarJ_NdAae_QJNqpVdQny1EaKe-aiDq5XjUad2ZQ%40mail.gmail.com.
