Re: select.select and socket.setblocking
Laszlo Nagy wrote: [...] I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? There is, but it may not effect you. There are cases where a socket can select() as readable, but not be readable by the time of a following recv() or accept() call. All such cases with which I'm familiar call for a non-blocking socket. Where does this come up? Suppose that to take advantage of multi-core processors, our server runs as four processes, each with a single thread that responds to events via select(). Clients all connect to the same server port, so the socket listening on that port is shared by all four processes. A perfectly reasonable architecture (though with many more processes the simple implementation suffers the thundering herd problem). Two of our processors may be waiting on select() when a new connections comes in. The select() call returns in both processes, showing the socket ready for read, so both call accept() to complete the connection. The O.S. ensures that accept() [and recv()] are atomic, so one process gets the new connection; what happens in the other depends on whether we use a blocking or non-blocking socket, and clearly we want non-blocking. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Bryan Olson fakeaddr...@nowhere.org wrote: Where does this come up? Suppose that to take advantage of multi-core processors, our server runs as four processes, each with a single thread that responds to events via select(). Clients all connect to the same server port, so the socket listening on that port is shared by all four processes. A perfectly reasonable architecture (though with many more processes the simple implementation suffers the thundering herd problem). Which is why it is common for real world servers to serialize the select()/accept() code - usually via a file lock or a semaphore. -srp -- http://saju.net.in Two of our processors may be waiting on select() when a new connections comes in. The select() call returns in both processes, showing the socket ready for read, so both call accept() to complete the connection. The O.S. ensures that accept() [and recv()] are atomic, so one process gets the new connection; what happens in the other depends on whether we use a blocking or non-blocking socket, and clearly we want non-blocking. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Bryan Olson fakeaddr...@nowhere.org wrote: There are cases where a socket can select() as readable, but not be readable by the time of a following recv() or accept() call. All such cases with which I'm familiar call for a non-blocking socket. I used to believe that if select() said data was ready for reading, a subsequent read/recv/recvfrom() call could not block. It could return an error, but it could not block. I was confident of this until just a few months ago when reality blew up in my face. The specific incident involved a bug in the linux kernel. If you received an UDP packet with a checksum error, the select() would return when the packet arrived, *before* the checksum was checked. By the time you did the recv(), the packet had been discarded and the recv() would block. This led me on a big research quest (including some close readings of Stevens, which appeared to say that this couldn't happen). The more I read, the more I (re) discovered just how vague and poorly written the Berkeley Socket API docs are :-) The bottom line is that Bryan is correct -- regardless of what the various man pages and textbooks say, in the real world, it is possible for a read() to block after select() says the descriptor is ready. The right way to think about select() is to treat it as a heuristic which can make a polling loop more efficient, but should never be relied upon to predict the future. Neither the negative nor positive behavior is guaranteed. There's no guaranteed response time; just because select() hasn't returned yet doesn't mean a descriptor couldn't be read without blocking in another thread right now. And, just because it has returned, that doesn't mean by the time you get around to reading, there will still be anything there. -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Can you post an example program that exhibits the behavior you describe? I was forgetting about the MSG_WAITALL flag ... When I started programming with sockets, it was on a platform (IIRC Solaris) that by default behaved like MSG_WAITALL was set by default (actually, I don't remember it being mentioned at all in the man pages). This sort of biased my understanding of the matter. I actually used that flag recently - on Linux - to get the same behavior I was used to, but forgot about that. My bad :-) Ciao -- FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Grant Edwards ha scritto: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I always check the returned value of recv and signal an error if the read bytes are less than the expected ones, but this error is never occurred (and its about 20 years that I use sockets in various languages and various flavor of unix and occasionally on windows. Maybe have always been lucky ? :-) And, on some unices system call recv also returns when a signal interrupts the syscall, but I half-remember reading that python recv in such a case repeat the system call by itself ... although this might be only my desire ... In non-blocking mode, it will always return immediately, either with some data, no data (other end closed), or an EAGAIN or EWOULDBLOCK error (I forget which). [...] I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... That depends on whether you can tolerate blocking or not. In an event-loop, blocking is generally not allowed. What I usually do, when I cannot block is: - use socket in blocking mode - do a select with a very small timeout and do a recv only if the select returns with input events - (with TCP) do a recv for the exact amount of bytes that I expect ( this mean having a user protocol that carries the message size in the header, but this is usually the case ). This usually worked for me. If my process (or thread) has only to deal with socket I/O, I make a blocking select, and then make an 'exact' recv on whichever socket the select signals. Ciao FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Jean-Paul Calderone ha scritto: On Tue, 30 Dec 2008 19:19:08 +0100, Francesco Bochicchio bock...@virgilio.it wrote: [snip] If you are interested in socket errors, you should also fill the third 'fd-set' in the select call, and after select returns check that fd is not in it anymore: ready = select.select( [fd],[], [fd] ) if fd in ready[2]: # raise your error here The third argument to select() isn't for monitoring sockets for errors. Its behavior is also rather platform sensitive. In general, you don't need it at all on POSIX, but on Windows you should pass the same list for it as you pass for the write-set, merge the results, and treat them all as writeable. Or use a higher-level library that deals with all the asinine details for you. ;) Jean-Paul Yes, now that you mention it I remember having to do something like that on a socket library I wrote on windows ... IIRC, the send could not complete and then signal the readyness of the socket through the third argument of the select ... My experience is mostly on unices, and I usually don't use the third argument (and not often the second) but I remember having read on select manual page that it was for errors. Now both python manuals than select manual page say it is for 'exceptional conditions', without going into details ... Tx for the clarification, anyway ... Ciao FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Francesco Bochicchio ha scritto: No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I always check the returned value of recv and signal an error if the read bytes are less than the expected ones, but this error is never occurred (and its about 20 years that I use sockets in various languages and various flavor of unix and occasionally on windows. Maybe have always been lucky ? :-) BTW, this is not a rethorical or ironic question... my applications mostly run on LANs or dedicated WANs so maybe they never experienced the kind of network congestion that could cause recv to return less than the expected amount of bytes ... but then, IIRC TCP guarantees that the packet is fully received by hand-shaking at transport level between sender and receiver. Ad once the packet is fully in the receiver buffer, why should recv choose to give back to the application only a piece of it? Ciao - FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Dec 31, 2:01 pm, Francesco Bochicchio bock...@virgilio.it wrote: Grant Edwards ha scritto: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I always check the What if the sending end actually sent less than you asked for ? -srp returned value of recv and signal an error if the read bytes are less than the expected ones, but this error is never occurred (and its about 20 years that I use sockets in various languages and various flavor of unix and occasionally on windows. Maybe have always been lucky ? :-) And, on some unices system call recv also returns when a signal interrupts the syscall, but I half-remember reading that python recv in such a case repeat the system call by itself ... although this might be only my desire ... In non-blocking mode, it will always return immediately, either with some data, no data (other end closed), or an EAGAIN or EWOULDBLOCK error (I forget which). [...] I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... That depends on whether you can tolerate blocking or not. In an event-loop, blocking is generally not allowed. What I usually do, when I cannot block is: - use socket in blocking mode - do a select with a very small timeout and do a recv only if the select returns with input events - (with TCP) do a recv for the exact amount of bytes that I expect ( this mean having a user protocol that carries the message size in the header, but this is usually the case ). This usually worked for me. If my process (or thread) has only to deal with socket I/O, I make a blocking select, and then make an 'exact' recv on whichever socket the select signals. Ciao FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
... Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I always check the What if the sending end actually sent less than you asked for ? -srp In blocking mode and with TCP protocol, the recv waits until more bytes are received - mixing up the next message with the previous one and then loosing the 'sync' and being unable to interpretate the received data - or the remote end disconnects. Yes this is bad, and is a good reason why socket receive should be handled in non-blocking mode if you receive data from untrusted sources. But luckily for me, as I said in the other post, I used socket mostly to communicate between specific applications on a private LAN or WAN, so I could afford to ignore the problem. Ciao FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Dec 31, 7:48 pm, Francesco Bochicchio bock...@virgilio.it wrote: ... Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I always check the What if the sending end actually sent less than you asked for ? -srp In blocking mode and with TCP protocol, the recv waits until more bytes are received - mixing up the next message with the previous one and Is this correct ? IIRC even in blocking mode recv() can return with less bytes than requested, unless the MSG_WAITALL flag is supplied. Blocking mode only guarantees that recv() will wait for a message if none is available - but not that it *will* return the number of bytes requested. -srp then loosing the 'sync' and being unable to interpretate the received data - or the remote end disconnects. Yes this is bad, and is a good reason why socket receive should be handled in non-blocking mode if you receive data from untrusted sources. But luckily for me, as I said in the other post, I used socket mostly to communicate between specific applications on a private LAN or WAN, so I could afford to ignore the problem. Ciao FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Wed, Dec 31, 2008 at 5:39 AM, Francesco Bochicchio bock...@virgilio.itwrote: Francesco Bochicchio ha scritto: No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I always check the returned value of recv and signal an error if the read bytes are less than the expected ones, but this error is never occurred (and its about 20 years that I use sockets in various languages and various flavor of unix and occasionally on windows. Maybe have always been lucky ? :-) BTW, this is not a rethorical or ironic question... my applications mostly run on LANs or dedicated WANs so maybe they never experienced the kind of network congestion that could cause recv to return less than the expected amount of bytes ... but then, IIRC TCP guarantees that the packet is fully received by hand-shaking at transport level between sender and receiver. Ad once the packet is fully in the receiver buffer, why should recv choose to give back to the application only a piece of it? One way to get less data than you ask for: TCP internally buffers a finite amount of data for your connection. If you ever ask to receive in one recv() call more than TCP is willing to buffer for your connection, you will get back only what TCP has buffered, not the amount you ask for, even if the other side has sent the larger amount. The remaining data will likely be buffered in the TCP send buffers on the other side of the connection*. Your call to recv() will empty your side's receive buffers (it is likely TCP will return all the data it has buffered in a single call, though I don't think that is an API requirement), causing your side's TCP to send a window update to the other side indicating it is willing to receive more data on the connection, the data will be transfered, and a subsequent recv() call by your application will be able to retrieve more of the data. TCP provides a stream of bytes to its applications, not a datagram/packet interface. Assuming packets sent by one side will be received in their entirety on a single receive call on the other side is really asking for trouble. TCP does not maintain any sort of packet boundaries that you seem to think are implied by the sequence of sends issued on one side of the connection. Karen *This is assuming the other side's send buffers aren't filled up by the left-over data your TCP isn't willing to buffer. If they are, and the sending socket is in blocking mode, the sender will have been blocked during the send waiting for buffer space to free up. If the sender's socket is non-blocking, and both the send and receive buffers fill up, then the sending socket will start returning EWOULDBLOCK or EAGAIN (I forget which) on calls to send data, until some buffer space is freed up by your side's receiving some data. -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Wed, 31 Dec 2008 15:48:50 +0100, Francesco Bochicchio bock...@virgilio.it
wrote:
...
Uhm. In my experience, with TCP protocol recv only returned less than
the required bytes if the remote end disconnects. I always check the
What if the sending end actually sent less than you asked for ?
-srp
In blocking mode and with TCP protocol, the recv waits until more bytes are
received - mixing up the next message with the previous one and then
loosing the 'sync' and being unable to interpretate the received data - or
the remote end disconnects.
Nope, this isn't how TCP works, whether your sockets are blocking or non-
blocking. It's easy to see this:
s = socket.socket()
s.connect(('www.google.com', 80))
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection:
keep-alive\r\n\r\n')
len(s.recv(1024 * 1024))
6191
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection:
keep-alive\r\n\r\n')
len(s.recv(1024 * 1024))
6191
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection:
keep-alive\r\n\r\n')
len(s.recv(1024 * 1024))
6191
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection:
keep-alive\r\n\r\n')
len(s.recv(1024 * 1024))
6191
Numerous blocking recv calls which returned fewer bytes than asked for
without the connection having been closed.
The only guarantee you get with a blocking socket is that the recv will
return at least one byte and at most the number of bytes you asked for
(or raise an exception instead).
On a LAN, it's likely that you'll generally get the exact number of bytes
which the sender passed to one call of send (until the sender starts to
pass really huge strings to send, then it'll get split up) just because
the network has lots of capacity compared to the traffic you're putting
on it. However, even on a LAN it's not guaranteed, and on the internet,
it's extremely likely that this won't happen most of the time.
Jean-Paul
--
http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On 2008-12-31, Francesco Bochicchio bock...@virgilio.it wrote: Grant Edwards ha scritto: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Uhm. In my experience, with TCP protocol recv only returned less than the required bytes if the remote end disconnects. I've no idea how you got recv() to behave that way, because I've never seen it do that. Take a look at the echo client/server examples at http://docs.python.org/library/socket.html. If recv worked the way you claimed it did, those programs wouldn't work -- they would deadlock. But, that example does work. In that example, the server program calls recv(1024), and yet it returns 12 bytes. The manual page for recv() specifically states that the len parameter is a maximum: The maximum amount of data to be received at once is specified by bufsize. See the Unix manual page recv(2) for the meaning of the optional argument flags; it defaults to zero. Note: For best match with hardware and network realities, the value of bufsize should be a relatively small power of 2, for example, 4096. The Linux man page for recv() also says len is the length of the receive buffer and is an upper limit on the number of bytes to read. Can you post an example program that exhibits the behavior you describe? [...] What I usually do, when I cannot block is: - use socket in blocking mode - do a select with a very small timeout and do a recv only if the select returns with input events - (with TCP) do a recv for the exact amount of bytes that I expect ( this mean having a user protocol that carries the message size in the header, but this is usually the case ). This usually worked for me. Yes that will usually (almost always) work. But, IIRC, there are theoretically situraitons where something happens after select returns and before you call recv() that could cause recv() to block. -- Grant Edwards -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Jean-Paul Calderone wrote: ... On a LAN, it's likely that you'll generally get the exact number of bytes which the sender passed to one call of send (until the sender starts to pass really huge strings to send, then it'll get split up) just because the network has lots of capacity compared to the traffic you're putting on it. However, even on a LAN it's not guaranteed, and on the internet, it's extremely likely that this won't happen most of the time. I don't mean to rebut your point here,rather to pile on with a bit more information. One point I haven't seen mentioned in this discussion is that nodes in the internet that carry the TCP/IP traffic are allowed to, for their own purposes, break large packets up into several smaller packets, and combine several smaller packets into a single larger packet. So, no matter how your TCP/IP packets leave your machine, there is no guarantee they will reach the destination in the same clumps. It is the stream, and not the packets, that is provided by TCP/IP. --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Saju Pillai ha scritto: On Dec 31, 7:48 pm, Francesco Bochicchio bock...@virgilio.it wrote: Is this correct ? IIRC even in blocking mode recv() can return with less bytes than requested, unless the MSG_WAITALL flag is supplied. Blocking mode only guarantees that recv() will wait for a message if none is available - but not that it *will* return the number of bytes requested. -srp You are right ... most of my socket experience predates MSG_WAITALL, and I forgot that now the default behaviour is different ... oops ... Ciao -- FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Francesco Bochicchio b...@virgilio.it wrote: but then, IIRC TCP guarantees that the packet is fully received by hand-shaking at transport level between sender and receiver. Ad once the packet is fully in the receiver buffer, why should recv choose to give back to the application only a piece of it? This depends a lot on the definition of package - At the TCP/IP level, the protocol is quite complex - there are all sorts of info flowing back and forth, telling the transmitter how much space the receiver has available. So your record or package could be split up... But it gets worse, or better, depending on your point of view: At the ethernet level, a packet is less than 1.5k - so if your record is longer, it can also be split up - OTOH, if it all fits into one ethernet packet, there is every chance that it won't be split up, unless you send a lot of them in a row, without waiting for a response - if you are running something that sends a small request and listens for a small answer, then you will probably never see a record split - but if you run a kind of sliding window protocol that streams a lot of data (even in small packets) then sooner or later one of them will be partly delivered... - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
select.select and socket.setblocking
I'm using this method to read from a socket: def read_data(self,size): Read data from connection until a given size. res = fd = self.socket.fileno() while not self.stop_requested.isSet(): remaining = size - len(res) if remaining=0: break # Give one second for an incoming connection so we can stop the # server in seconds when needed ready = select.select([fd], [], [], 1) if fd in ready[0]: data = self.socket.recv(min(remaining,8192)) # 8192 is recommended by socket.socket manual. if not data: # select returns the fd but there is no data to read - connection closed! raise TransportError(Connection closed.) else: res += data else: pass if self.stop_requested.isSet(): raise SystemExit(0) return res This works: if I close the socket on the other side, then I see this in the traceback: File /usr/home/gandalf/Python/Projects/OrbToy/orb/endpoint.py, line 233, in read_data raise TransportError(Connection closed.) TransportError: Connection closed. Also when I call stop_requested.set() then the thread stops within one seconds. Then I switch to non blocking mode, my code works exactly the same way, or at least I see no difference. I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Laszlo Nagy wrote: I'm using this method to read from a socket: def read_data(self,size): Read data from connection until a given size. res = fd = self.socket.fileno() while not self.stop_requested.isSet(): remaining = size - len(res) if remaining=0: break # Give one second for an incoming connection so we can stop the # server in seconds when needed ready = select.select([fd], [], [], 1) if fd in ready[0]: data = self.socket.recv(min(remaining,8192)) # 8192 is recommended by socket.socket manual. if not data: # select returns the fd but there is no data to read - connection closed! raise TransportError(Connection closed.) else: res += data else: pass if self.stop_requested.isSet(): raise SystemExit(0) return res This works: if I close the socket on the other side, then I see this in the traceback: File /usr/home/gandalf/Python/Projects/OrbToy/orb/endpoint.py, line 233, in read_data raise TransportError(Connection closed.) TransportError: Connection closed. Also when I call stop_requested.set() then the thread stops within one seconds. Then I switch to non blocking mode, my code works exactly the same way, or at least I see no difference. I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? Well, ignoring your question for the moment, what's wrong with the following (untested) code? def read_data(self, size): data = while len(data) size: d = self.socket.read(size-len(data)) if not d: raise TransportError(Socket closed while reading data) data += d return data I feel the raw socket operations are easier to understand and less confusing. Since you don't want to return until you've read the data there really doesn't seem to be any point using select(), which is mainly useful in asynchronous operations (in which case you would have to use non-blocking sockets). It's not obvious from your code how self.stop_requested is supposed to change the result of its is_set() method. Maybe that's where the select() comes in? regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Laszlo Nagy ha scritto: I'm using this method to read from a socket: def read_data(self,size): Read data from connection until a given size. res = fd = self.socket.fileno() while not self.stop_requested.isSet(): remaining = size - len(res) if remaining=0: break # Give one second for an incoming connection so we can stop the # server in seconds when needed ready = select.select([fd], [], [], 1) if fd in ready[0]: data = self.socket.recv(min(remaining,8192)) # 8192 is recommended by socket.socket manual. if not data: # select returns the fd but there is no data to read - connection closed! raise TransportError(Connection closed.) else: res += data else: pass if self.stop_requested.isSet(): raise SystemExit(0) return res This works: if I close the socket on the other side, then I see this in the traceback: File /usr/home/gandalf/Python/Projects/OrbToy/orb/endpoint.py, line 233, in read_data raise TransportError(Connection closed.) TransportError: Connection closed. Also when I call stop_requested.set() then the thread stops within one seconds. Then I switch to non blocking mode, my code works exactly the same way, or at least I see no difference. I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? Thanks Couple of remarks: 1. AFAIK, select in python accepts also socket objects, or anything which has a fileno() method returning an integer. So you don't need to extract the fileno from the socket (python will do for you) although it does no harm. 2. IMO, the behaviour of your code is correct: with TCP protocol, when the remote ends disconnects, your end receives a 'read event' without data; you should just handle the fact that recv returns nothing as normal, not as error, and close your end of the connection. If you are interested in socket errors, you should also fill the third 'fd-set' in the select call, and after select returns check that fd is not in it anymore: ready = select.select( [fd],[], [fd] ) if fd in ready[2]: # raise your error here 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, or the disconnection, in the other case would return immediately (and you should check the number of returned bytes, and when you read the remaining bytes of the message put the pieces together). I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... HTH Ciao -- FB -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Tue, 30 Dec 2008 19:19:08 +0100, Francesco Bochicchio bock...@virgilio.it wrote: [snip] If you are interested in socket errors, you should also fill the third 'fd-set' in the select call, and after select returns check that fd is not in it anymore: ready = select.select( [fd],[], [fd] ) if fd in ready[2]: # raise your error here The third argument to select() isn't for monitoring sockets for errors. Its behavior is also rather platform sensitive. In general, you don't need it at all on POSIX, but on Windows you should pass the same list for it as you pass for the write-set, merge the results, and treat them all as writeable. Or use a higher-level library that deals with all the asinine details for you. ;) Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. In non-blocking mode, it will always return immediately, either with some data, no data (other end closed), or an EAGAIN or EWOULDBLOCK error (I forget which). [...] I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... That depends on whether you can tolerate blocking or not. In an event-loop, blocking is generally not allowed. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Tue, 30 Dec 2008 14:41:17 -0600, Grant Edwards gra...@visi.com wrote: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Hi Grant, I don't think you read Francesco's message carefully enough. :) He said: there is no difference in select between blocking and non-blocking mode. You're describing a difference in recv, not select. In non-blocking mode, it will always return immediately, either with some data, no data (other end closed), or an EAGAIN or EWOULDBLOCK error (I forget which). [...] I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... That depends on whether you can tolerate blocking or not. In an event-loop, blocking is generally not allowed. If you're careful, it's possible to avoid blocking, even when using a blocking socket, at least for AF_INET, SOCK_STREAM sockets. Of course, it's easier to avoid blocking by using a non-blocking socket. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Tue, 30 Dec 2008 15:55:51 -0500, Jean-Paul Calderone exar...@divmod.com wrote: On Tue, 30 Dec 2008 14:41:17 -0600, Grant Edwards gra...@visi.com wrote: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Hi Grant, I don't think you read Francesco's message carefully enough. :) Ah, no, it was I who didn't read carefully enough. Nevermind. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
