Re: [casper] 100 GbE Hardware recommendation

Mon, 23 Sep 2024 22:48:48 -0700 

Hey Mayukh,
a solution over PCIe, directly from the RFSoC into the SSDs, could save a lot of power, compared to a additional computer, which saves the data from the network onto the SSDs. Unfortunately, the missing PCIe connector on the RFSoC 4x2 makes this a little bit harder, as a custom PCB is needed to fan out the PCIe bus from the QSPI connector. Also, I do not know if such a solution is supported by CASPER out of the box. 

Regards,
Gerrit

Am 23.09.24 um 20:33 schrieb Mayukh Bagchi:

Hey Dan,

Thanks for getting back.


My first concern is being able to find a compatible hardware/computer 
to plug the NIC into. Are these systems commercially available? Or are 
most of them custom-built? I could budget around 100 W for such a 
computer. I don't think the power draw of the NICs themselves is 
concerning for us, we could manage that.



Also, I am not sure if the 100 GbE yellow block would be compatible 
with a dual 10 GbE NIC or a 40 GbE NIC, if it does then that opens up 
quite a few options.



Regards,
Mayukh

------------------------------------------------------------------------

*From:* 'Dan Werthimer' via casper@lists.berkeley.edu 
<casper@lists.berkeley.edu>

*Sent:* Monday, September 23, 2024 1:48 PM
*To:* casper@lists.berkeley.edu <casper@lists.berkeley.edu>
*Subject:* Re: [casper] 100 GbE Hardware recommendation


Hi mayukh,


You might consider two 10 gbe links and a dual 10 gbe nic.   Or a 40 
gbe nic.   Are you concerned with the power consumption of the 100 gbe 
nic, or the consumption of the computer the nic is plugged into?   How 
much computing power and storage do you need ?


Best wishes,

Dan


On Mon, Sep 23, 2024, 1:18 PM Mayukh Bagchi <mayukh.bag...@queensu.ca> 
wrote:


    Hello Casperites,
    I hope you all have been doing well.

    I have been developing an RFSoC-based backend for our
    Balloon-borne VLBI Experiment (BVEX) project, which will be a
    mobile K-band VLBI station, launching in August next year. I am
    using the RFSoC 4x2 to digitize and stream data for our IF stage.
    The CASPER community has been very helpful with setup and
    troubleshooting.

    So far, I've successfully implemented a spectrometer and a power
    meter. While the design for the 100 GbE data streaming is almost
    ready, I am having trouble selecting a compatible NAS/server-side
    system to directly communicate with the 100 GbE Mellanox NIC
    <https://www.fs.com/products/119648.html?attribute=67745&id=3746425>.
    The reason why it is getting tricky is the size, thermal
    regulation, and power consumption constraints, as this will be on
    a balloon platform with limited resources. Our data rates should
    be around ~10-12 Gbps, depending on how much we end up
    re-quantizing, so we may not use the full capabilities of the 100
    GbE network. However, at this point, any suggestion on hardware
    will be useful in getting us pointed in the right direction.

    Feel free to share your advice on working with 100 GbE Mellanox
    NICs. If this seems challenging, I could probably lower our data
    rates and try the 10GbE yellow block Ethernet core.

    Thanks in advance!

    Regards,
    Mayukh
    *__________*
    *
    *
    *Mayukh**Bagchi*(He/Him), PhD Candidate
    Department of Physics, Engineering Physics, and Astronomy
    mayukhbagchi.com <http://mayukhbagchi.com/> | mayukh.bag...@queensu.ca
    Queen's University Logo <https://www.queensu.ca/>

    -- 
    You received this message because you are subscribed to the Google

    Groups "casper@lists.berkeley.edu" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to casper+unsubscr...@lists.berkeley.edu.
    To view this discussion on the web visit
    
https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB10632111BE97CE7C079D9E886916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM
    
<https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB10632111BE97CE7C079D9E886916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer>.

--

You received this message because you are subscribed to the Google 
Groups "casper@lists.berkeley.edu" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to casper+unsubscr...@lists.berkeley.edu.
To view this discussion on the web visit 
https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzJmp1E%3DZknwdB4V18WypD8ak0TewhDoqe8WsTSk1i1-YA%40mail.gmail.com 
<https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CAA9jdzJmp1E%3DZknwdB4V18WypD8ak0TewhDoqe8WsTSk1i1-YA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--

You received this message because you are subscribed to the Google 
Groups "casper@lists.berkeley.edu" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to casper+unsubscr...@lists.berkeley.edu.
To view this discussion on the web visit 
https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB1063206055042E9DCB07FFFD2916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM 
<https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/YT4PR01MB1063206055042E9DCB07FFFD2916F2%40YT4PR01MB10632.CANPRD01.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer>.


--
You received this message because you are subscribed to the Google Groups 
"casper@lists.berkeley.edu" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to casper+unsubscr...@lists.berkeley.edu.
To view this discussion on the web visit 
https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/9311a891-8cd3-466b-b7a5-355860483c5e%40mpifr-bonn.mpg.de.






















					Reply via email to