Re: send(2) does not block, send(2) man page wrong?
On Mon, 26 Jan 2004, Steve Watt wrote: > [EMAIL PROTECTED] wrote: > >do what ping does (ping -f) > >when you get an ENOBUFS do a usleep for 1 mSec. > >and then send it again. > > So how, exactly, do you actually sleep for 1mSec? I recently did some > experiments using nanosleep(), and it seems that the minimum sleep time > is 2 / HZ. I.e. ask for 100nS, get 20mS (on a 10mS-ticking system). > > Mind you, that behavior is precisely aligned with what POSIX says should > happen, since nanosleep() is not allowed to return success before the > specified amount of time has expired, and you might be calling it 5nS > before the clock tick. But it does make doing correct Tx pacing a bit > more challenging. For what it's worth, when I tried this I wound up using gettimeofday (IIRC) as a "macroscopic" clock and calculating nanosecond sleeps between transmits; drift due to HZ was correctable because I knew the average throughput I was after. -- jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/ Tel +44(0)117 9287088 Fax +44 (0)117 9287112 http://ioctl.org/jan/ That which does not kill us goes straight to our thighs. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On Mon, 26 Jan 2004, Steve Watt wrote:
> [EMAIL PROTECTED] wrote:
> >do what ping does (ping -f)
> >when you get an ENOBUFS do a usleep for 1 mSec.
> >and then send it again.
you are correct, but I just meant that it requested to sleep 1mSec, not
that the sleep actually WAS 1mSec.
Making udp sockets block would break so many things since
it was always this way since sockets were invented in BSD2.x
>
> So how, exactly, do you actually sleep for 1mSec? I recently did some
> experiments using nanosleep(), and it seems that the minimum sleep time
> is 2 / HZ. I.e. ask for 100nS, get 20mS (on a 10mS-ticking system).
>
> Mind you, that behavior is precisely aligned with what POSIX says should
> happen, since nanosleep() is not allowed to return success before the
> specified amount of time has expired, and you might be calling it 5nS
> before the clock tick. But it does make doing correct Tx pacing a bit
> more challenging.
>
> Tried the same thing with usleep(1), same result of ~20mS per
> sleep.
>
> Here's the program I tested that with. Same results on a 4.4-RELEASE
> and a 5.2-RELEASE machine.
>
> Numbers from one run:
> 4.4-REL: 1501 loops, 30.017931 elapsed, time per loop: 19998.622 us
> 5.2-REL: 1501 loops, 30.016053 elapsed, time per loop: 19997.371 us
>
> - - - 8< - - -
>
> #include
> #include
> #include
>
> /* Seconds to count loops */
> #define RUNTIME 30
>
> int
> main(int argc, char **argv)
> {
> struct timespec start, now, end, delay, remain;
> double ts, te;
> long loops = 0;
> int rv;
>
> clock_gettime(CLOCK_REALTIME, &start);
> end.tv_sec = start.tv_sec + RUNTIME;
> end.tv_nsec = start.tv_nsec;
>
> do {
> delay.tv_sec = 0;
> delay.tv_nsec = 1; /* 10uS */
>
> do {
> rv = nanosleep(&delay, &remain);
> delay = remain;
> } while (rv < 0 && errno == EINTR);
>
> ++loops;
> clock_gettime(CLOCK_REALTIME, &now);
> } while ((now.tv_sec == end.tv_sec) ?
> (now.tv_nsec < end.tv_nsec) :
> (now.tv_sec < end.tv_sec));
>
> te = now.tv_sec + (now.tv_nsec / 10.);
> ts = start.tv_sec + (start.tv_nsec / 10.);
>
> printf("%d loops, %f elapsed, ", loops, te - ts);
> printf("time per loop: %.3f us\n", ((te - ts) / loops) * 100.);
>
> return 0;
> }
>
>
> --
> Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8" / 37N 20' 14.9"
> Internet: steve @ Watt.COM Whois: SW32
>Free time? There's no such thing. It just comes in varying prices...
>
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
[EMAIL PROTECTED] wrote:
>do what ping does (ping -f)
>when you get an ENOBUFS do a usleep for 1 mSec.
>and then send it again.
So how, exactly, do you actually sleep for 1mSec? I recently did some
experiments using nanosleep(), and it seems that the minimum sleep time
is 2 / HZ. I.e. ask for 100nS, get 20mS (on a 10mS-ticking system).
Mind you, that behavior is precisely aligned with what POSIX says should
happen, since nanosleep() is not allowed to return success before the
specified amount of time has expired, and you might be calling it 5nS
before the clock tick. But it does make doing correct Tx pacing a bit
more challenging.
Tried the same thing with usleep(1), same result of ~20mS per
sleep.
Here's the program I tested that with. Same results on a 4.4-RELEASE
and a 5.2-RELEASE machine.
Numbers from one run:
4.4-REL: 1501 loops, 30.017931 elapsed, time per loop: 19998.622 us
5.2-REL: 1501 loops, 30.016053 elapsed, time per loop: 19997.371 us
- - - 8< - - -
#include
#include
#include
/* Seconds to count loops */
#define RUNTIME 30
int
main(int argc, char **argv)
{
struct timespec start, now, end, delay, remain;
double ts, te;
long loops = 0;
int rv;
clock_gettime(CLOCK_REALTIME, &start);
end.tv_sec = start.tv_sec + RUNTIME;
end.tv_nsec = start.tv_nsec;
do {
delay.tv_sec = 0;
delay.tv_nsec = 1; /* 10uS */
do {
rv = nanosleep(&delay, &remain);
delay = remain;
} while (rv < 0 && errno == EINTR);
++loops;
clock_gettime(CLOCK_REALTIME, &now);
} while ((now.tv_sec == end.tv_sec) ?
(now.tv_nsec < end.tv_nsec) :
(now.tv_sec < end.tv_sec));
te = now.tv_sec + (now.tv_nsec / 10.);
ts = start.tv_sec + (start.tv_nsec / 10.);
printf("%d loops, %f elapsed, ", loops, te - ts);
printf("time per loop: %.3f us\n", ((te - ts) / loops) * 100.);
return 0;
}
--
Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8" / 37N 20' 14.9"
Internet: steve @ Watt.COM Whois: SW32
Free time? There's no such thing. It just comes in varying prices...
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On Mon, Jan 26, 2004 at 10:53:54AM -0800, Julian Elischer wrote: ... > On Mon, 26 Jan 2004, Stuart Pook wrote: > > > > On 23 Jan 2004, Don Lewis wrote: > > > > the send does not give an error: the packet is just thrown away. > > > > > > Which is the same result as you would get if the bottleneck is just one > > > network hop away instead of at the local NIC. > > > > But it isn't. I'm broadcasting onto the local network. With Linux and > > Solaris (which implement what FreeBSD send(2) says), it is so easy: I just > > send(2) away, and because the send blocks when the kernel buffer space is I'd be really curious to know how Linux/Solaris actually implement this blocking send and if they really block or use some kind of timeout/retry loop in the kernel. To implement a blocking send() on UDP sockets, you need a different driver model from the one we have, one where sockets and other data sources trying to access a full interface queue should be queued into some kind of list hanging off the interface, so that when the interface is ready again you can wake up the pending clients in turn and process their requests. This would cause the output queue to become effectively unbounded (basically, it is like reserving at least one slot per socket -- more if you want to deal with fragments), and even if the slot can be allocated as part of the socket, the delay would become unbounded as well. Secondly, if the interface for some reason goes "temporarily" down (e.g. no-carrier or the like) the process would suddenly block unless you mark the socket as non blocking. cheers luigi ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
do what ping does (ping -f) when you get an ENOBUFS do a usleep for 1 mSec. and then send it again. On Mon, 26 Jan 2004, Stuart Pook wrote: > > On 23 Jan 2004, Don Lewis wrote: > > > the send does not give an error: the packet is just thrown away. > > > > Which is the same result as you would get if the bottleneck is just one > > network hop away instead of at the local NIC. > > But it isn't. I'm broadcasting onto the local network. With Linux and > Solaris (which implement what FreeBSD send(2) says), it is so easy: I just > send(2) away, and because the send blocks when the kernel buffer space is > full, I lose very few packets. With FreeBSD, I lose 60% of the packets. > (The aim is to broadcast onto a private 802.11b network.) > > If I don't want to saturate the network then I will use kernel level > traffic shaping to limit the outgoing bandwidth. I have already done this > on Linux when I was broadcasting onto my 802.11b network via an access > point connected via 100Mbits/s Ethernet. I just used traffic shaping > to limit the outgoing traffic on that Ethernet interface to 3Mbits/s. > I didn't have to change my program at all. (At one point I did try > to put the delays (with nanosleep) into my program but it worked very > badly because the scheduling delays were too big. The kernel does it > so much better.) Once again it is vital that send blocks. > > I guess that I'm out of luck with *BSD. I hope that someone will update > the send(2) man page so that the next person who wants to do what I'm > doing will know that it isn't possible with FreeBSD. > > Stuart > ___ > [EMAIL PROTECTED] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
> On 23 Jan 2004, Don Lewis wrote: > > the send does not give an error: the packet is just thrown away. > > Which is the same result as you would get if the bottleneck is just one > network hop away instead of at the local NIC. But it isn't. I'm broadcasting onto the local network. With Linux and Solaris (which implement what FreeBSD send(2) says), it is so easy: I just send(2) away, and because the send blocks when the kernel buffer space is full, I lose very few packets. With FreeBSD, I lose 60% of the packets. (The aim is to broadcast onto a private 802.11b network.) If I don't want to saturate the network then I will use kernel level traffic shaping to limit the outgoing bandwidth. I have already done this on Linux when I was broadcasting onto my 802.11b network via an access point connected via 100Mbits/s Ethernet. I just used traffic shaping to limit the outgoing traffic on that Ethernet interface to 3Mbits/s. I didn't have to change my program at all. (At one point I did try to put the delays (with nanosleep) into my program but it worked very badly because the scheduling delays were too big. The kernel does it so much better.) Once again it is vital that send blocks. I guess that I'm out of luck with *BSD. I hope that someone will update the send(2) man page so that the next person who wants to do what I'm doing will know that it isn't possible with FreeBSD. Stuart ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On 23 Jan, Stuart Pook wrote: >> send() for UDP should block if the socket is filled and the interface >> can't drain the data fast enough. > > It doesn't (at least I cannot make it block) > >> Good question. There is not feedback loop like in tcp, so handling this >> blocking and releasing would be a little bit harder to do for UDP. > > Send(2) indicates that it should do so. > >> > I have written a test program, >> > http://www.infres.enst.fr/~pook/send/server.c, that shows that send does >> > not block on FreeBSD. It does with Linux and Solaris. >> >> Do you know what the behaviour of Net- and/or OpenBSD is? > > NetBSD is the same as FreeBSD. I have not tested OpenBSD. > MacOS X is similiar to FreeBSD in that send doesn't block, howver > the send does not give an error: the packet is just thrown away. Which is the same result as you would get if the bottleneck is just one network hop away instead of at the local NIC. Even if you changed the network stack to block or return an error when it detected that it was tossing packets away, the application has no way of knowing that all, a majority of, or even any of its data was getting though even though it wasn't blocked by send() and didn't receive any error returns. Think about the case of a gigabit LAN connected to the Internet over a modem link. Even with a stack that blocked send() so that no packets were lost in the stack, the application would think it was sending data to a peer on the Internet at gigabit speeds, but in reality most of the traffic would be silently dropped. Even within the LAN, traffic could be dropped if the outgoing switch port was more congested than the link from the sending host to the NIC. If you want to send a lot of data as fast as possible using UDP, then you'll probably need to reinvent the TCP congestion avoidance algorithms in your application so that you don't overly impact the network. The application can't rely on send() blocking or returning errors, since you don't know that the local network interface is the bottleneck. Since the bottleneck could be anywhere, the application code is simpler if it relies on cues that are the same no matter where the bottleneck is located rather than adding extra code just to handle a local bottleneck. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On Fri, 23 Jan 2004, Andre Oppermann wrote: > Stuart Pook wrote: > >>send() for UDP should block if the socket is filled and the interface > >>can't drain the data fast enough. > > > > It doesn't (at least I cannot make it block) > > This stuff is rather complex. A send() on a UDP socket processes right > down to the if_output. If that fails because the ifqueue is full, the > packet will be free()d right away. No luck with blocking and retrying. > > >>Good question. There is not feedback loop like in tcp, so handling this > >>blocking and releasing would be a little bit harder to do for UDP. > > > > Send(2) indicates that it should do so. > > True. The only thing I can offer is that when one packet produces an > ENOBUFS, the socket will block on the next one for a couple of milliseconds. > Doing the full program requires significant changes to the current structure > of the BSD network code. > > You could do the same in userland with a call to nanosleep(2) when you get > a ENOBUFS. from memory that is what ping -f does.. > > >>>I have written a test program, > >>>http://www.infres.enst.fr/~pook/send/server.c, that shows that send does > >>>not block on FreeBSD. It does with Linux and Solaris. > >> > >>Do you know what the behaviour of Net- and/or OpenBSD is? > > > > NetBSD is the same as FreeBSD. I have not tested OpenBSD. > > MacOS X is similiar to FreeBSD in that send doesn't block, howver > > the send does not give an error: the packet is just thrown away. > > Browsing through the code I see that none of the BSDs are able to block > on an UDP send. > > -- > Andre > > ___ > [EMAIL PROTECTED] mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On Fri, Jan 23, 2004 at 06:09:20PM +0100, Andre Oppermann wrote: ... > >>send() for UDP should block if the socket is filled and the interface > >>can't drain the data fast enough. > > > > It doesn't (at least I cannot make it block) > > This stuff is rather complex. A send() on a UDP socket processes right > down to the if_output. If that fails because the ifqueue is full, the > packet will be free()d right away. No luck with blocking and retrying. and there would be no point in blocking given that the protocol (UDP) is unreliable and designed not to give any guarantee whatsoever. The most you can get is an error code on return from send()/write() and friends. Furthermore, send() and write() block on the socket buffer filling up, not on the interface queue. Because UDP has no output socket buffer, there is no way it can block. Finally, overflows in the interface queue are never handled by send() kernel code, not even for TCP: in this case, it is just TCP congestion control that acts and, either at the next incoming ACK, or upon a timeout, tries a retransmission. > > Send(2) indicates that it should do so. i admit the manpage should definitely be clarified -- it says 'if no message space is available at the socket...', but it does cover the UDP behaviour. Technically, in the UDP case there is always space at the sending socket, because that space is never used - by definition of the UDP protocol - and the packet goes straight to the ip layer and then down to the interface. cheers luigi ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
Stuart Pook wrote: send() for UDP should block if the socket is filled and the interface can't drain the data fast enough. It doesn't (at least I cannot make it block) This stuff is rather complex. A send() on a UDP socket processes right down to the if_output. If that fails because the ifqueue is full, the packet will be free()d right away. No luck with blocking and retrying. Good question. There is not feedback loop like in tcp, so handling this blocking and releasing would be a little bit harder to do for UDP. Send(2) indicates that it should do so. True. The only thing I can offer is that when one packet produces an ENOBUFS, the socket will block on the next one for a couple of milliseconds. Doing the full program requires significant changes to the current structure of the BSD network code. You could do the same in userland with a call to nanosleep(2) when you get a ENOBUFS. I have written a test program, http://www.infres.enst.fr/~pook/send/server.c, that shows that send does not block on FreeBSD. It does with Linux and Solaris. Do you know what the behaviour of Net- and/or OpenBSD is? NetBSD is the same as FreeBSD. I have not tested OpenBSD. MacOS X is similiar to FreeBSD in that send doesn't block, howver the send does not give an error: the packet is just thrown away. Browsing through the code I see that none of the BSDs are able to block on an UDP send. -- Andre ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On Fri, Jan 23, 2004 at 04:25:07PM +0100, Stuart Pook wrote:
> The documentation for send(2) says
>
> If no messages space is available at the socket to hold the message to be
> transmitted, then send() normally blocks, unless the socket has been
> placed in non-blocking I/O mode. The select(2) call may be used to
> determine when it is possible to send more data.
>
> I cannot get send (or sendto which is what really interests me)
> to block on FreeBSD 4.9. When I send as fast as I can to a socket,
> send rapidly fails with ENOBUFS. I am not surprised that the kernel is
> running out of mbufs but I am surprised that send does not block until
> more become available.
>
> Select does not block either. It always says that I can write to the
> socket and then send fails with ENOBUFS.
>
> The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
>
> /*
> * Calculate data length and get a mbuf
> * for UDP and IP headers.
> */
> M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
> if (m == 0) {
> error = ENOBUFS;
> if (addr)
> splx(s);
> goto release;
> }
>
> There is no sign of send blocking waiting for a mbuf or of it returning
> EAGAIN if the socket is non-blocking.
>
> Is the documentation for send(2) wrong or is there some way to make
> send and sendto block?
>
> I have used setsockopt(s, SOL_SOCKET, SO_SNDBUF) to reduce the size
> of the output queue for the socket but send still returns ENOBUFS and
> never blocks or returns EAGAIN.
>
UDP sockets don't use send buffer (like TCP does) for storing data,
only receive buffer, so SO_SNDBUF only affects the maximum size of
one write operation.
Cheers,
--
Ruslan Ermilov
FreeBSD committer
[EMAIL PROTECTED]
pgp0.pgp
Description: PGP signature
Re: send(2) does not block, send(2) man page wrong?
> send() for UDP should block if the socket is filled and the interface > can't drain the data fast enough. It doesn't (at least I cannot make it block) > Good question. There is not feedback loop like in tcp, so handling this > blocking and releasing would be a little bit harder to do for UDP. Send(2) indicates that it should do so. > > I have written a test program, > > http://www.infres.enst.fr/~pook/send/server.c, that shows that send does > > not block on FreeBSD. It does with Linux and Solaris. > > Do you know what the behaviour of Net- and/or OpenBSD is? NetBSD is the same as FreeBSD. I have not tested OpenBSD. MacOS X is similiar to FreeBSD in that send doesn't block, howver the send does not give an error: the packet is just thrown away. Stuart ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
Stuart Pook wrote:
The documentation for send(2) says
If no messages space is available at the socket to hold the message to be
transmitted, then send() normally blocks, unless the socket has been
placed in non-blocking I/O mode. The select(2) call may be used to
determine when it is possible to send more data.
I cannot get send (or sendto which is what really interests me)
to block on FreeBSD 4.9. When I send as fast as I can to a socket,
send rapidly fails with ENOBUFS. I am not surprised that the kernel is
running out of mbufs but I am surprised that send does not block until
more become available.
Select does not block either. It always says that I can write to the
socket and then send fails with ENOBUFS.
send() for UDP should block if the socket is filled and the interface
can't drain the data fast enough.
The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == 0) {
error = ENOBUFS;
if (addr)
splx(s);
goto release;
}
There is no sign of send blocking waiting for a mbuf or of it returning
EAGAIN if the socket is non-blocking.
Is the documentation for send(2) wrong or is there some way to make
send and sendto block?
Good question. There is not feedback loop like in tcp, so handling this
blocking and releasing would be a little bit harder to do for UDP.
I have used setsockopt(s, SOL_SOCKET, SO_SNDBUF) to reduce the size
of the output queue for the socket but send still returns ENOBUFS and
never blocks or returns EAGAIN.
I note that send on Linux and Solaris blocks and that on these systems
select can be used to wait until the send will not block.
I have written a test program,
http://www.infres.enst.fr/~pook/send/server.c, that shows that send does
not block on FreeBSD. It does with Linux and Solaris.
Do you know what the behaviour of Net- and/or OpenBSD is?
--
Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
send(2) does not block, send(2) man page wrong?
The documentation for send(2) says
If no messages space is available at the socket to hold the message to be
transmitted, then send() normally blocks, unless the socket has been
placed in non-blocking I/O mode. The select(2) call may be used to
determine when it is possible to send more data.
I cannot get send (or sendto which is what really interests me)
to block on FreeBSD 4.9. When I send as fast as I can to a socket,
send rapidly fails with ENOBUFS. I am not surprised that the kernel is
running out of mbufs but I am surprised that send does not block until
more become available.
Select does not block either. It always says that I can write to the
socket and then send fails with ENOBUFS.
The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == 0) {
error = ENOBUFS;
if (addr)
splx(s);
goto release;
}
There is no sign of send blocking waiting for a mbuf or of it returning
EAGAIN if the socket is non-blocking.
Is the documentation for send(2) wrong or is there some way to make
send and sendto block?
I have used setsockopt(s, SOL_SOCKET, SO_SNDBUF) to reduce the size
of the output queue for the socket but send still returns ENOBUFS and
never blocks or returns EAGAIN.
I note that send on Linux and Solaris blocks and that on these systems
select can be used to wait until the send will not block.
I have written a test program,
http://www.infres.enst.fr/~pook/send/server.c, that shows that send does
not block on FreeBSD. It does with Linux and Solaris.
thanks for your help
Stuart
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: send(2) does not block, send(2) man page wrong?
On Monday, January 19, 2004, at 08:53 AM, Stuart Pook wrote: The documentation for send(2) says If no messages space is available at the socket to hold the message to be transmitted, then send() normally blocks, unless the socket has been placed in non-blocking I/O mode. The select(2) call may be used to determine when it is possible to send more data. I cannot get send (or sendto which is what is really interests me) to block on FreeBSD 4.9. When I send as fast as I can to a socket, send rapidly fails with ENOBUFS. I am not surprised that the kernel is running out of mbufs but I am surprised that send does not block until more become available. Select does not block either. It always says that I can write to the socket and then send fails with ENOBUFS. The udp_output function in /sys/netinet/udp_usrreq.c, seems clear: UDP does not have a need to block, so it does not. The UDP delivery 'promise' is best effort, and datagrams will get spilled anywhere along the route to the destination (no user crying over ...). If you use TCP, which has a need to block, your sending process will be blocked until both local and remote buffering space is available. You could make a case that the man page for 'send(2)' is either inaccurate or misleading. The problem is that 'send()' actually doesn't handle this; as your post indicates, the decision to block or fail a transmission is taken at the protocol level, and 'send()' is generally blind to that. Cheers, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics| If you're not confused, | You're not paying attention *--*---* ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
send(2) does not block, send(2) man page wrong?
The documentation for send(2) says
If no messages space is available at the socket to hold the message to be
transmitted, then send() normally blocks, unless the socket has been
placed in non-blocking I/O mode. The select(2) call may be used to
determine when it is possible to send more data.
I cannot get send (or sendto which is what is really interests me)
to block on FreeBSD 4.9. When I send as fast as I can to a socket,
send rapidly fails with ENOBUFS. I am not surprised that the kernel is
running out of mbufs but I am surprised that send does not block until
more become available.
Select does not block either. It always says that I can write to the
socket and then send fails with ENOBUFS.
The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == 0) {
error = ENOBUFS;
if (addr)
splx(s);
goto release;
}
There is no sign of send blocking waiting for a mbuf or of it returning
EAGAIN if the socket is non-blocking.
Is the documentation for send(2) wrong or is there some way to make
send and sendto block?
I have used setsockopt(s, SOL_SOCKET, SO_SNDBUF) to reduce the size
of the output queue for the socket but send still returns ENOBUFS and
never blocks or returns EAGAIN.
I note that send on Linux and Solaris blocks and that on these systems
select can be used to wait until the send will not block.
I have written a test program,
http://www.infres.enst.fr/~pook/send/server.c, that shows that send does
not block on FreeBSD. It does with Linux and Solaris.
thanks for your help
Stuart
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
