Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-29 Thread Alan Burlison
[off list] On 29/10/2015 17:07, Al Viro wrote: Could the esteemed sir possibly be ars^H^H^Hprevailed upon to quote the exact place in POSIX that requires such behaviour? If that's the way the conversation is going to go, sorry, no. -- Alan Burlison -- -- To unsubscribe from this list:

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-29 Thread Alan Burlison
at while NetBSD apparently supports this behavior because someone copied it from Solaris, I'm about to go recommend it be removed. Which behaviour? The abort accept() on close() behaviour? -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in t

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-29 Thread Alan Burlison
ment it that isn't a trainwreck. close(2) as per specified by POSIX works just fine on Solaris, if that was the case everywhere then it wouldn't be an issue. And for cases where it is necessary to keep the FD assigned because of races, the dup2(2) trick works fine as well. -- Alan Burlison

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-28 Thread Alan Burlison
out at this point. Thanks again for all the helpful explanation. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-27 Thread Alan Burlison
On 27/10/2015 15:04, David Miller wrote: I said "run time". Like when your final code runs on a target system. Not at build tree configure time. Yes, you could do it then as well. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev&qu

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-27 Thread Alan Burlison
you need to persuade everyone else with a stake in Linux that your viewpoint is the only valid one, and you will have to also persuade them that Linux should no longer concern itself with POSIX compliance. I know *I* couldn't do that for Solaris, and with all due respect, I very mu

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-27 Thread Alan Burlison
On 27/10/2015 13:59, David Miller wrote: From: Alan Burlison Date: Tue, 27 Oct 2015 13:37:26 + If you took that argument to it's logical extreme they you'd never make any changes that made changes to existing behaviour, and that's patently not the case. You know exac

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-27 Thread Alan Burlison
our, and that's patently not the case. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-27 Thread Alan Burlison
it does an implicit shutdown() and all other children no longer can accept() ? No, of course not. I made it quite clear I was talking about MT programs. Surely you did not gave all details on how it is really working. In the case of Hadoop, it works the way I describe. -- Alan Burlison

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-27 Thread Alan Burlison
7;s a simple fix for this, as discussed earlier in the thread. Having said that, if close() on Linux also did an implicit shutdown() it would mean that well-written applications that handled the scoping, sharing and reuse of FDs properly could just call close() and have it work the same way ac

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-23 Thread Alan Burlison
even more unwieldy than it already is. And if what you are asserting is correct, why isn't the AF_INET behaviour the same as the AF_UNIX behaviour? -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord.

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-23 Thread Alan Burlison
simply not setting the output bits in the poll events mask, so it's not exactly high priority. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-23 Thread Alan Burlison
This list is about linux kernel development. Thank you for the clarification. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-23 Thread Alan Burlison
s on the subject but you can't simultaneously claim POSIX conformance. One or the other, not both. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
inux and Solaris happen to cover these gaps differently; FreeBSD and OpenBSD are probably closer to Linux variant, NetBSD - to Solaris one. Yes, true enough. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
aving the flag of POSIX compliance when you are actually talking about the ways your implementation plugs the holes in a badly incomplete spec... Personally I believe the spec is clear enough to allow an unambiguous interpretation of the required behavior in this area. If you think there are are

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
simply can't continue to claim that the relevant Linux APIs are fully POSIX conformant, so I believe at the minimum the Linux manpages need modifying. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord.

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
is a 'listener' or even a 'socket' without extra cache line misses. To force a close of an accept() or whatever blocking socket related system call a shutdown() makes a lot of sense. This would have zero additional overhead for the fast path. Yes, that would I believe be

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
don't quite agree with that) +1. If there's interest in doing that I'll ask our POSIX rep the best way of initiating such a conversation. -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
#x27;t think it's *that* obscure, when I found the original shutdown() problem google showed up another occurrence pretty quickly - https://lists.gnu.org/archive/html/libmicrohttpd/2011-09/msg00024.html. If a fd is closed then allowing other uses of it to continue in other threads seems incorrec

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-22 Thread Alan Burlison
akeup even if that's not POSIX-compliant - it may make it slightly easier for applications avoid fd recycling races. However the current situation is that shutdown is the *only* way to perform such a wakeup - simply closing the fd has no effect on any other threads. That seems incorrect. -- A

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-21 Thread Alan Burlison
;d be interested to hear if there's a better and more performant way of handling the situation that doesn't involve doing the sort of bookkeeping Casper described,. To quote one of my colleague's favourite sayings: Performance is a goal, correctness is a constraint. -- Alan Burli

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-21 Thread Alan Burlison
ering to that socket - fd2 remains alive and well. Behaviour you describe below matches the _third_ option. Ah, I wasn't 100% sure what the intended difference between #2 and #3 was TBH, it does sound like I meant #3, yes :-) -- Alan Burlison -- -- To unsubscribe from this list: sen

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-21 Thread Alan Burlison
the thread will return from its syscall and will in most cases return EBADF. A second thread trying to close this same file descriptor may return earlier with close() returning EBADF. -- -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-21 Thread Alan Burlison
On 21/10/2015 12:28, Eric Dumazet wrote: This works for me. Please double check your programs I have just done so, it works as you say for AF_INET sockets but if you switch to AF_UNIX sockets it does the wrong thing in the way I described. -- Alan Burlison -- -- To unsubscribe from this

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-21 Thread Alan Burlison
: polling #1 [waits for connection] main: poll #1: Success poll fd: 0 revents: POLLIN POLLRDNORM -- Alan Burlison -- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-20 Thread Alan Burlison
ions that may cause unintended side effects. You are in this grey zone. No, the race issue with file descriptor reuse and the close() behaviour are not the same thing. The manpage comment is correct, but not relevant. -- Alan Burlison -- -- To unsubscribe from this list: send the lin

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-20 Thread Alan Burlison
behaviour on sockets that are in the listen state. If you are interested, take a look at fs/* code, and try to implement your proposal and keep good performance. You might find a clever way, without infringing prior art. We did not yet. I might, but contractually I can't, unfortunately.

Re: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-20 Thread Alan Burlison
ld be impacted, i.e mainly threaded apps that act as network service providers of one form or another. But having said that, the behaviour of close() and poll() on sockets being used in accept() still looks incorrect. -- Alan Burlison -- -- To unsubscribe from this list: send the line "un

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-20 Thread Alan Burlison
On 20/10/2015 02:45, Eric Dumazet wrote: On Tue, 2015-10-20 at 02:12 +0100, Alan Burlison wrote: Another problem is that if I call close() on a Linux socket that's in accept() the accept call just sits there until there's an incoming connection, which succeeds even though the

Re: Fw: [Bug 106241] New: shutdown(3)/close(3) behaviour is incorrect for sockets in accept(3)

2015-10-19 Thread Alan Burlison
that's race-free and cross-platform involving the use of /dev/null and dup2(), see Listing Five on http://www.drdobbs.com/parallel/file-descriptors-and-multithreaded-progr/212001285 but I haven't confirmed it works yet. Thanks, -- Alan Burlison -- -- To unsubscribe from this list: s