Hi Tom,
We have been working with udp.
Basic problem is that the low-cost routers have too little memory, common
wisdom is that they should have 100-200 milliseconds worth.
This is for aggregation of data.
Packet size will be noticeable only above 2-4 Gbps, Myrinet webpages are a
good starting point.
Best regards,
--
Jouko
"Life is pretty simple: You do some stuff. Most fails. Some works. You do
more of what works. If it works big, others quickly copy it. Then you do
something else. The trick is to do something else."
On Mon, 20 Jun 2011, Tom Downes wrote:
Casper folks:
I've finally started working in earnest on a PPC-side C server that
reads out FPGA/PPC shared memory and sends out UDP packets over the
1Gbe interface. The idea being that our data rate is so much below
10GBe (about 1 megabyte per second per board) that it did not merit
investing in the 10Gbe switches necessary to readout 10-20 boards at a
time. Also, the experiment is expected to be running 5 years from now
and these switches are already hard to find and expensive.
I've got this scheme basically working and want to ask what I hope are
a couple reasonably simple questions that people out there may have
experience with. What is the typical largest UDP packet size that will
be passed by standard network hardware? Does anyone out there have
experience buying a 1Gbe switch with ~20 ports that has a good
configuration interface to basically ensure 100% passage of ~6kB UDP
packets?
My impression from CASPER Memo 21 is that a typical router will pass
UDP packets up to 9kb (maybe 8192?). My experience while writing the C
server is that packets above this size sometimes get through the LAN
but most times not. I have managed to divide our data into packets of
6208 bytes in size and they seem to get through every time. But I want
to make sure that the UDP packets really, truly get through every
time, not 99.9%.
I understand that the UDP standard is not designed around ensuring
100% delivery of information. I am mostly concerned that routers can
prioritize certain kinds of traffic over others. I don't want the
wrong choice of router to introduce problems.
The data will be passing through a private LAN comprised of a standard
linux machine receiving UDP packets from approximately 16 ROACH
boards. So we can buy a router that is very appropriate to this task
rather than generic network conditions. i.e. turn off anti-virus
scanning, firewall, etc, etc.
My testing thus far has been across a LAN shared with many computers at Caltech.
Tom
