Re: Linux sockets binding issues
Hi, I wish to thank everyone that answered me. In deed the problem was TIME_WAIT, because the client did not close the connection when I closed mine. The SO_REUSEADDR seems to solve the problem. Thanks again, Ido On 8/25/07, ik [EMAIL PROTECTED] wrote: Hello List, I have this weird problem that I can't figure out with Linux's sockets. I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. Now this issue does not always exists, so I don't know exactly what to look for. I'm using a multi-platform socket library named lnet (that I'm adding some new features), and this issue only exists on Linux, while BSD, MacOSX and Windows does not have such problem. I'm using Kubuntu 7.04 with kernel 2.6.20 So I'm looking for pointers and tips on what should I be looking for in order to understand what is happening. Thanks, Ido -- http://ik.homelinux.org/ -- http://ik.homelinux.org/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
ik wrote: Hi, I wish to thank everyone that answered me. In deed the problem was TIME_WAIT, because the client did not close the connection when I closed mine. No, that is not what causes TIME_WAIT. TIME_WAIT is caused for whoever it is that closes the connection first, even when the shutdown is orderly and ok. If both sides close the connection simultaneously (as will happen in POP3, SMTP, FTP and other protocols), then BOTH sides will be in TIME_WAIT. Again, this is part of the RFC. The SO_REUSEADDR seems to solve the problem. That's fairly clean, and I have to wonder why it's not the default. Thanks again, Ido Shachar On 8/25/07, ik [EMAIL PROTECTED] wrote: Hello List, I have this weird problem that I can't figure out with Linux's sockets. I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. Now this issue does not always exists, so I don't know exactly what to look for. I'm using a multi-platform socket library named lnet (that I'm adding some new features), and this issue only exists on Linux, while BSD, MacOSX and Windows does not have such problem. I'm using Kubuntu 7.04 with kernel 2.6.20 So I'm looking for pointers and tips on what should I be looking for in order to understand what is happening. Thanks, Ido -- http://ik.homelinux.org/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On Sun, Aug 26, 2007 at 11:16:22AM +0300, Shachar Shemesh wrote: The SO_REUSEADDR seems to solve the problem. That's fairly clean, and I have to wonder why it's not the default. Under some theoretical conditions you may get unexpected data on your connection when using SO_REUSEADDE. Therefore the API defaults to safe and less useful rather than not so safe (but far more useful). Cheers, Muli = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On 8/26/07, Shachar Shemesh [EMAIL PROTECTED] wrote: ik wrote: Hi, I wish to thank everyone that answered me. In deed the problem was TIME_WAIT, because the client did not close the connection when I closed mine. No, that is not what causes TIME_WAIT. TIME_WAIT is caused for whoever it is that closes the connection first, even when the shutdown is orderly and ok. If both sides close the connection simultaneously (as will happen in POP3, SMTP, FTP and other protocols), then BOTH sides will be in TIME_WAIT. Again, this is part of the RFC. The SO_REUSEADDR seems to solve the problem. That's fairly clean, and I have to wonder why it's not the default. It will be the default, now that I found the problem from the help given by the people at this list. Because the library is cross-platform (that's one of the reasons I'm using it), not all OS react the same to this flag, and some OS such as Windows98 even have a security risk (that was never handled afaik) regarding SO_REUSEADDR, and on XP there is a different flag to be used instead... And other OS have different issues as well, such as FreeBSD, So that's the reason why the original developer reluctant to make it the default, but we started working on a runtime identify of the OS (for Windows) and to add a good documentation about the matter, including the security risk issues. Thanks again, Ido Shachar On 8/25/07, ik [EMAIL PROTECTED] wrote: Hello List, I have this weird problem that I can't figure out with Linux's sockets. I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. Now this issue does not always exists, so I don't know exactly what to look for. I'm using a multi-platform socket library named lnet (that I'm adding some new features), and this issue only exists on Linux, while BSD, MacOSX and Windows does not have such problem. I'm using Kubuntu 7.04 with kernel 2.6.20 So I'm looking for pointers and tips on what should I be looking for in order to understand what is happening. Thanks, Ido -- http://ik.homelinux.org/ Ido -- http://ik.homelinux.org/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
Muli Ben-Yehuda wrote: That's fairly clean, and I have to wonder why it's not the default. Under some theoretical conditions you may get unexpected data on your connection when using SO_REUSEADDE. I'm not so sure that statement is true (at least in any practical sense). First, let's understand TIME_WAIT. We closed our side of the connection (using shutdown), and got an acknowledgment of that shutdown. Next, the other side asked to close its side of the connection. It sent a FIN packet (indicating that it has no more data to send), and we sent an ACK. However, we are not sure that this ACK was received, so we cannot afford to erase all trace of the socket just yet. Instead, we remember that we had such a connection (identified by our IP and port, and the remote IP and port), and that all that is left to do on it is to ACK any repeat FINs received. We do this for about a minute before we allow ourselves to totally remove the socket and forget about the connection (at which point, should a FIN arrive, we will respond with a RST, which will also do the right thing). Now, let's see what needs to happen for spurious data to arrive on a new socket, that actually belongs to the old socket: 1. We need to rebind to the same port (well, that's why we are using SO_REUSEADDR, so no coincidence there) 2. The same IP needs to connect to us from the same port 3. The same IP needs to send data from the old socket old socket (presumably, some leftover data from the old socket that the client did not notice was already ACKed) 4. The new data has to arrive with sequence numbers that are meaningful to the new connection. Let's check out the odds. 1 is, more or less, certain. 2 is somewhat unlikely, 3 can happen and 4 is extremely unlikely. If these were the stats I'd say Muli was right. These are not, however, the stats. 2 and 3 happening together requires some pretty heavy setup of caching routers and extreme out of order packet arrival for it to make any sense (or malicious activity, but that is irrelevant for the sake of this discussion, as a malicious attacker that can wield this attack can also wield much more effective attacks). For all practical purposes, 2 and 3 are contradicting. No sane TCP stack will open a new socket that has the same parameters as an already existing socket. Add 4, and you have a situation that can NEVER happen. What makes the entire notion, more or less, redundant is that the EXACT SAME THING can happen without any use of SO_REUSEADDR at all. Just replace step 1 above with the server never exiting, and the situation is just as likely to happen (if not more likely). Therefore the API defaults to safe and less useful rather than not so safe (but far more useful). If it's a problem when I use SO_REUSEADDR, it's also a problem when I do not quit the server at all. If it's not a problem in the later case, it shouldn't be a problem in the former. Either way, I see no reason to have SO_REUSEADDR (or rather, I see no reason for having it disabled). Cheers, Muli Shachar = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On Sun, Aug 26, 2007 at 12:22:49PM +0300, Shachar Shemesh wrote: are contradicting. No sane TCP stack will open a new socket that has the same parameters as an already existing socket. Add 4, and you have a situation that can NEVER happen. You have an interesting definition of NEVER. Surely you are familiar with the difference between the odds approach zero and the odds are zero ;-) What makes the entire notion, more or less, redundant is that the EXACT SAME THING can happen without any use of SO_REUSEADDR at all. Just replace step 1 above with the server never exiting, and the situation is just as likely to happen (if not more likely). Not sure what you mean here - do you mean the server never closes the socket? Cheers, Muli = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
Muli Ben-Yehuda wrote: What makes the entire notion, more or less, redundant is that the EXACT SAME THING can happen without any use of SO_REUSEADDR at all. Just replace step 1 above with the server never exiting, and the situation is just as likely to happen (if not more likely). Not sure what you mean here - do you mean the server never closes the socket? Yes. The exact same circumstances will cause the exact same problem if the server never shuts down, which is a much more likely case than a case where it happens to happen when the server shuts down and then restarts. Cheers, Muli Shachar = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Linux sockets binding issues
Hello List, I have this weird problem that I can't figure out with Linux's sockets. I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. Now this issue does not always exists, so I don't know exactly what to look for. I'm using a multi-platform socket library named lnet (that I'm adding some new features), and this issue only exists on Linux, while BSD, MacOSX and Windows does not have such problem. I'm using Kubuntu 7.04 with kernel 2.6.20 So I'm looking for pointers and tips on what should I be looking for in order to understand what is happening. Thanks, Ido -- http://ik.homelinux.org/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On 8/25/07, ik [EMAIL PROTECTED] wrote: Hello List, I have this weird problem that I can't figure out with Linux's sockets. I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. Now this issue does not always exists, so I don't know exactly what to look for. I'm using a multi-platform socket library named lnet (that I'm adding some new features), and this issue only exists on Linux, while BSD, MacOSX and Windows does not have such problem. I'm using Kubuntu 7.04 with kernel 2.6.20 So I'm looking for pointers and tips on what should I be looking for in order to understand what is happening. Not much of a help but I've experienced the same issue with Python 2.5 on Ubuntu 6.10. I didn't bother to search for a solution as this behaviour was accepted for my needs. I did hear other people complaining about this issue, with no obvious solution. Thanks, Ido -- http://ik.homelinux.org/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED] -- Cheers, Maxim Veksler Free as in Freedom - Do u GNU ? = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On Sat, 2007-08-25 at 18:24 +0300, ik wrote: I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. I'm not sure, but I think this has something to do with the CLOSE_WAIT status. If this is indeed the problem, then netstat -a (not -l) would show your socket in a CLOSE_WAIT status. The problem occurs when one side (probably the client) didn't close the socket correctly after it was closed from the other side. There supposedly should be a setting for changing the CLOSE_WAIT timeout, but I can't find it. -- Oded = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On Sat, Aug 25, 2007, ik wrote about Linux sockets binding issues: I have this weird problem that I can't figure out with Linux's sockets. I have a server that I'm writing that bind a port to 0.0.0.0, now when I disconnect, and quit the server and then re-run the server, sometimes (not all of the time) the binding is failing as the port is still binded, however netstat -lnp does not show any indication for such binding to still be existed, and it takes few seconds until I'm able to restart the server without any issues. Did you try to use the SO_REUSEADDR option? The problem is that without this option, the port might be still look busy to the kernel because one of your old connections might be in the TIME_WAIT state (if any of these things sound unfamiliar, Google is your friend, and any good TCP/IP book covers this material as well). In a minute of Googling, I found this, which at first glance appears a good explanation of the situation: http://hea-www.harvard.edu/~fine/Tech/addrinuse.html -- Nadav Har'El| Saturday, Aug 25 2007, 12 Elul 5767 [EMAIL PROTECTED] |- Phone +972-523-790466, ICQ 13349191 |How could we possibly use sex to get http://nadav.harel.org.il |what we want? Sex IS what we want! Fraser = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Linux sockets binding issues
On Sat, Aug 25, 2007 at 06:24:32PM +0300, ik wrote: So I'm looking for pointers and tips on what should I be looking for in order to understand what is happening. http://www.unixguide.net/network/socketfaq/4.5.shtml Cheers, Muli = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
