Re: Optimizing network throughput over Ethernet with TCP/IP
hi, i just read this since i have a similar problem, trying to transfer data across two computers at ~30mbyte/s rate. after doing some testings, i found the optimal package size turns out to be 4096 byte or 4K for tcp/ip tranmsission on LAN.
Re: Optimizing network throughput over Ethernet with TCP/IP
Ethernet packets are range from 46-1500bytes, or there abouts, so anything is being broken up. Well, anything over 1500 bytes. I'll give the even multiples of 32768 a try and see how that goes. (Well, there seems to be support for frame sizes up to 9000 bytes in Gigabit Ethernet.) I'm going to have to experiment a bit more to see whether the delays are caused. I may also try to create an app in C or find a test app that will pump data through, to test out LV's receive limitations. Adam
Re: Optimizing network throughput over Ethernet with TCP/IP
Have you tried even multiples of 32768? I seem to remeber that ethernet (or was it IP) that limited the size of packets in order to prevent a single transmitter from dominating the network. The performance drop-off's for the larger packets may be due the packet being broken up and one of those "lower levels" dragging it feet about transfering the rest. I am gald I was of service, Ben
Re: Optimizing network throughput over Ethernet with TCP/IP
I encourage people to answer their own questions! This way you get the "stars" you deserve. Your numbers look good. Ben
Re: Optimizing network throughput over Ethernet with TCP/IP
I posted my last response and mistakenly did not know that if I selected the "answer this question" option at the top, that it would not display with all the other text, you may have to hit the "Show all answers to this question" option, to see the response. I didn't want it to be posted in 1/3rd width because it was quite long. Oh well. ADam
Re: Optimizing network throughput over Ethernet with TCP/IP
Adding to my previous; I have gotten good results using VI server technology and using call by reference to get data. I have never attempted to push the trasfer rates you are looking at but I think it is worth a quick try. The beauty of this approach (IMHO) is ZERO TCP DETAILS (sorry) to code. Please briefly outline the solution once you crack this performance nut. I would be interested in hearing how you end solving this challenge. Ben
Re: Optimizing network throughput over Ethernet with TCP/IP
70mbps Cool! I think the next test I would do is to try out the VI server example and modify the example so that it reads a huge array constant. This would take advantage the TCP/IP functionality that is built in to LV. If that works, you may be done! Ben
Re: Optimizing network throughput over Ethernet with TCP/IP
Hi Ben, Thanks for taking the time to respond. From my understanding, on a dedicated connection, a network utilization of close to 60% can be reasonably expected for TCP/IP connections. That would give me 60mbps in which to work. (I usually like to work with around 100% overhead available.) The computers are going to be connected up directly, or at least through a switch, so the only collisions that I have to worry about are the ACKs with data packets. I can do file transfers at up to 70mpbs for large transfers between machines, so I think that it is possible. MS may do something funky to improve speed, so that may not be the best test. I will try to find some other software to perform this test with. Maybe I can use SCP (w & w/o encryption) to see what network utilization I can get. I've thought about the gigabit route, but I'd like to avoid it if possible as it adds additional expenses and makes laptop use more difficult. But I might end up getting some gigabit cards and giving it a try to see whether I get a noticable improvement. Thanks, Adam
Re: Optimizing network throughput over Ethernet with TCP/IP
Hi Adam, It has been a couple of days and no response on this Q so I will relpy. First, you have done your homework well. I have not used the TCP functions from Sheldon Instruments so I can not comment on that. I think the Naegel option would help if you were send alot of small packets that need individual resposnes. I am a bit concerned about your target of 30mbps. It has been years since I dug this low but I think 30mbps is going to be the upper limit of a 100Mbps LAN. This is based on the "old days" when ethernet ran over coax. The issue as I see it is how the hardware gets access to the LAN. Ethernet adapters have to do a number of things when transmitting a packet. First it has to check the network to see if anyone else is talking. If the network is free, it will then transmit the data while simultaneously read from the network. If another node is trying to do the same thing at the same time, they will corrupt each others transmission resulting in a bad CRC being detected by each transmitting node. This is a collision. The interfaces are then supposed to execute a random wait and then retry. You probably new this already. When the network utilization is low, collisions are less likely and data can fly. Once the network utilization starts to climb, the collisions become more frequent and over-all throughput will decline. In the "old days" we used to recomend trying to keep utilization below 30% of the theoretical max. This is why I am concerned about you trying to do 30mbps over a 100mbps LAN. You are right on the edge where performance starts to drop off. Remember that TCP packets are ack'd by the reciever! This ack'ing is going to introduce collisions because this has to happen to support the integrety of the transmision. Enough about me fears, how about some ideas. First; Keep LV out of the mix and just try your hardware and OS out using file transfers. Just try copying large files from one machine to the other on your dedicated LAN. If you can transfer files at 30mbps, then you may be able to do the same with an app in LV. (Note File sharing is generally implemented using TCP/IP). Second; I think you probably will have to go for the 1Gig route. Your 30mbps is only 3% of that bandwidth so provided your hardware can pump as fast as the net can suck, you should be able to pull this off. In closing I would like to clearly state that I am not an expert in this area and it has been years since I got into the low-level details, so please forgive me if I have got something wrong here. I will watch this Q to see how things work out for you and see if anyone else would like to jump in here and straighten me out. Trying to help, Ben http://exchange.ni.com/servlet/ProcessRequest?RHIVEID=101&RPAGEID=261&HRedirected=True&HUserId=101_3529&RFORMNUMBER=6";>Ben Rayner http://volt.ni.com/niwc/common.jsp?page=products_certification_cld&node=10638";> Certified LabVIEW Developer www.DSAutomation.com
Optimizing network throughput over Ethernet with TCP/IP
Hello, I am interested in optimizing TCP/IP throughput in LabVIEW. I am looking to send data between two systems at rates around 20-30mbps on a dedicated 100base-t ethernet connection. All data needs to reach the other side w/o loss. I have a few questions with regard to accomplishing this. 1. Proper use of the Built-in TCP Vis: When sending data between two systems, what is the best manner? I have 50KB arrays of data that I need to move from one system to the next and have been wondering whether or not I need to packetize this, making multiple calls to the TCP/IP =93write=94 function, or whether it's better to just make the call with the full 20-50KB buffer? Does LabVIEW handle this for the user? What is the most efficient way to do it? 2. Built-in Limits: I have seen references to LabVIEW's TCP buffer size being limited to 32KB, is this correct? Is it more efficient to make direct calls to windows DLLs? See this discussion thread: http://exchange.ni.com/servlet/ProcessRequest?RHIVEID=3D101&RPAGEID= =3D135&HOID=3D5065000800719C">transfering data with TCP Additionally, I've seen another thread where it was mentioned that there was an =93improved=94 set of TCP/IP VIs. http://www.sheldoninst.com/support.html As mentioned in this discussion thread: http://exchange.ni.com/servlet/ProcessRequest?RHIVEID=3D101&RPAGEID= =3D135&HOID=3D50650008001294">200ms delay in windows tcp ip - how to work around? Has any one had any experience with these? Again, has anyone had any experience using LabVIEW to send data over the network at these rates? 3. Protocols: Are there any protocols that will allow for transfer at these rates? OPC? Modbus TCP? I don't mind creating one, but I'd rather not reinvent the wheel. System information: Both systems will be running Windows XP SP1 on P4 processors with lots of RAM, so lots of processing power. The network will be going through either a crossover cable or a dedicated switch, with these computers being the only devices attached to it. One final question, would it make a difference if I went up to gigabit Ethernet? What throughput have people managed on gigabit connections? Thanks, Adam
