Re: BIO_get_accept_socket weirdness
On 5 July 2014 12:37, Kurt Roeckx wrote: > But then I found some MSDN documentation that says that Windows > allows others to hijack your socket when you've set SO_REUSEADDR > and the results are non-deterministic. They also created an > SO_EXCLUSIVEADDRUSE and I'm getting confused what it really does, > but they say that server applications should set it. If you think that opening the socket first is a security measure, then you've got some pretty serious problems. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On Sat, Jul 05, 2014 at 02:55:36PM -0400, Tim Hudson wrote: > > Some google engineering (search) will show the the variety of > confusion that this causes in cross-platform code. > > Start here for some interesting reading - > http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t > You will find *many* exchanges of details about that. It doesn't tell me anything not mentioned in Stevens, except that Windows thing that I already found earlier. So I still fail to see why we shouldn't just set SO_REUSEADDR except on Windows. Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On Sat, Jul 5, 2014 at 2:37 PM, Tim Hudson wrote:
> On 5/07/2014 2:14 PM, Kurt Roeckx wrote:
> > On Sat, Jul 05, 2014 at 12:45:37PM -0400, Tim Hudson wrote:
> >> If you have SO_REUSEADDR set and a listener already in place you will
> >> start a new listener
> > No you won't. You will get a bind() error:
> > socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
> > setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> > bind(3, {sa_family=AF_INET, sin_port=htons(3344),
> sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EADDRINUSE (Address already in use)
> >
> > Except on windows it seems.
>
> All the world is not (yet) Linux :-) ... and those semantics were
> defined log ago - and evolved ... and there is also REUSEPORT (added
> later) and a variety of interpretations - but the base REUSEADDR can
> indeed behave that way depending on what platform you are on. Linux has
> its own slightly different interpretation. Some google engineering
> (search) will show the the variety of confusion that this causes in
> cross-platform code.
>
FWIW, Windows is the only platform for which Apache httpd does not enable
SO_REUSEADDR prior to bind. That itself doesn't make it right, but there
has been plenty of time to complain. And admins notice things like
multiple instances on the same port and confusion about which instance
handles which connection.
As for Windows, SO_EXCLUSIVEADDRUSE looks interesting:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms740621(v=vs.85).aspx
Incidentally, I don't readily see a Windows bind-while-TIME_WAIT
discussion. And I see that httpd enables SO_REUSEADDR *after* the bind on
Windows. The commit message for that doesn't lead me to believe that there
was definite rhyme or reason, but it was 12 years ago so it probably
doesn't hurt. I *guess* the TIME_WAIT issue doesn't hinder.
>
> Tim
>
> __
> OpenSSL Project http://www.openssl.org
> Development Mailing List openssl-dev@openssl.org
> Automated List Manager majord...@openssl.org
>
--
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/
Re: BIO_get_accept_socket weirdness
> Some google engineering (search) will show the the variety of confusion that this causes in cross-platform code. Start here for some interesting reading - http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t You will find *many* exchanges of details about that. And yes - originally this was (mostly) for being able to start a server again after stopping it when sockets were remaining in TIME_WAIT state - but it wasn't implemented as just that and the semantics have varied. Tim.
Re: BIO_get_accept_socket weirdness
On Sat, Jul 05, 2014 at 02:37:49PM -0400, Tim Hudson wrote:
> On 5/07/2014 2:14 PM, Kurt Roeckx wrote:
> > On Sat, Jul 05, 2014 at 12:45:37PM -0400, Tim Hudson wrote:
> >> If you have SO_REUSEADDR set and a listener already in place you will
> >> start a new listener
> > No you won't. You will get a bind() error:
> > socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
> > setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> > bind(3, {sa_family=AF_INET, sin_port=htons(3344),
> > sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EADDRINUSE (Address already in use)
> >
> > Except on windows it seems.
>
> All the world is not (yet) Linux :-) ... and those semantics were
> defined log ago - and evolved ... and there is also REUSEPORT (added
> later) and a variety of interpretations - but the base REUSEADDR can
> indeed behave that way depending on what platform you are on. Linux has
> its own slightly different interpretation. Some google engineering
> (search) will show the the variety of confusion that this causes in
> cross-platform code.
Are you saying BSD has different behavior than Linux, other than
Linux requiring that it's also set on the old socket?
Kurt
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
RE: BIO_get_accept_socket weirdness
Those who forget history are doomed to re-implement it, wrongly. SO_REUSEADDR was implemented in 4.2BSD so that a server could restart without waiting for the various FIN_WAIT timeouts to happen. :) /r$ -- Principal Security Engineer Akamai Technologies, Cambridge, MA IM: rs...@jabber.me; Twitter: RichSalz __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On 5/07/2014 2:14 PM, Kurt Roeckx wrote:
> On Sat, Jul 05, 2014 at 12:45:37PM -0400, Tim Hudson wrote:
>> If you have SO_REUSEADDR set and a listener already in place you will
>> start a new listener
> No you won't. You will get a bind() error:
> socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
> setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
> bind(3, {sa_family=AF_INET, sin_port=htons(3344),
> sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EADDRINUSE (Address already in use)
>
> Except on windows it seems.
All the world is not (yet) Linux :-) ... and those semantics were
defined log ago - and evolved ... and there is also REUSEPORT (added
later) and a variety of interpretations - but the base REUSEADDR can
indeed behave that way depending on what platform you are on. Linux has
its own slightly different interpretation. Some google engineering
(search) will show the the variety of confusion that this causes in
cross-platform code.
Tim
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On Sat, Jul 05, 2014 at 12:45:37PM -0400, Tim Hudson wrote:
> On 5/07/2014 9:12 AM, Kurt Roeckx wrote:
> > On Sat, Jul 05, 2014 at 08:13:04AM -0400, Eric Covener wrote:
> >> On Sat, Jul 5, 2014 at 7:37 AM, Kurt Roeckx wrote:
> >>> Does anybody have an idea why it's trying to do that, and why we
> >>> shouldn't just do SO_REUSEADDR the first time? Was there some
> >>> OS that maybe did strange things when trying to use SO_REUSEADDR
> >>> and it was already in use?
> >> FWLIW: I've seen this pattern in some other proprietary software,
> >> where they try hard to not set SO_REUSEADDR unless it appears needed
> >> due to a bind failure. But whatever they were working around, it is
> >> detrimental to modern Linux where the outgoing TIME_WAIT socket has to
> >> also have been opened with SO_REUSEADDR for the reuse to be allowed.
> > man socket(7) documents that behavior on Linux. They also say that
> > you ussually don't notice it because most things always set
> > SO_REUSEADDR. So it seems to me like the behavior of that piece
> > of code will at least don't do what you want it to do on Linux so
> > other than being weird it looks like an other reason to just drop
> > it.
>
> If you have SO_REUSEADDR set and a listener already in place you will
> start a new listener
No you won't. You will get a bind() error:
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(3344),
sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EADDRINUSE (Address already in use)
Except on windows it seems.
Kurt
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On 5/07/2014 9:12 AM, Kurt Roeckx wrote: > On Sat, Jul 05, 2014 at 08:13:04AM -0400, Eric Covener wrote: >> On Sat, Jul 5, 2014 at 7:37 AM, Kurt Roeckx wrote: >>> Does anybody have an idea why it's trying to do that, and why we >>> shouldn't just do SO_REUSEADDR the first time? Was there some >>> OS that maybe did strange things when trying to use SO_REUSEADDR >>> and it was already in use? >> FWLIW: I've seen this pattern in some other proprietary software, >> where they try hard to not set SO_REUSEADDR unless it appears needed >> due to a bind failure. But whatever they were working around, it is >> detrimental to modern Linux where the outgoing TIME_WAIT socket has to >> also have been opened with SO_REUSEADDR for the reuse to be allowed. > man socket(7) documents that behavior on Linux. They also say that > you ussually don't notice it because most things always set > SO_REUSEADDR. So it seems to me like the behavior of that piece > of code will at least don't do what you want it to do on Linux so > other than being weird it looks like an other reason to just drop > it. If you have SO_REUSEADDR set and a listener already in place you will start a new listener and who gets the incoming connections is not well defined. You need to determine if this is just a time-wait socket stopping the bind or there is a listener in place. That is the semantics being handled in the code. See crypto/bio/bss_acpt.c where the bind_mode is "documented" in a comment: /* If 0, it means normal, if 1, do a connect on bind failure, * and if there is no-one listening, bind with SO_REUSEADDR. * If 2, always use SO_REUSEADDR. */ Setting SO_REUSEADDR on always will not result in what you are expecting ... Tim. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On Sat, Jul 05, 2014 at 08:13:04AM -0400, Eric Covener wrote: > On Sat, Jul 5, 2014 at 7:37 AM, Kurt Roeckx wrote: > > Does anybody have an idea why it's trying to do that, and why we > > shouldn't just do SO_REUSEADDR the first time? Was there some > > OS that maybe did strange things when trying to use SO_REUSEADDR > > and it was already in use? > > FWLIW: I've seen this pattern in some other proprietary software, > where they try hard to not set SO_REUSEADDR unless it appears needed > due to a bind failure. But whatever they were working around, it is > detrimental to modern Linux where the outgoing TIME_WAIT socket has to > also have been opened with SO_REUSEADDR for the reuse to be allowed. man socket(7) documents that behavior on Linux. They also say that you ussually don't notice it because most things always set SO_REUSEADDR. So it seems to me like the behavior of that piece of code will at least don't do what you want it to do on Linux so other than being weird it looks like an other reason to just drop it. Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: BIO_get_accept_socket weirdness
On Sat, Jul 5, 2014 at 7:37 AM, Kurt Roeckx wrote: > Does anybody have an idea why it's trying to do that, and why we > shouldn't just do SO_REUSEADDR the first time? Was there some > OS that maybe did strange things when trying to use SO_REUSEADDR > and it was already in use? FWLIW: I've seen this pattern in some other proprietary software, where they try hard to not set SO_REUSEADDR unless it appears needed due to a bind failure. But whatever they were working around, it is detrimental to modern Linux where the outgoing TIME_WAIT socket has to also have been opened with SO_REUSEADDR for the reuse to be allowed. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
BIO_get_accept_socket weirdness
Hi, I've been working on IPv6 support, and one of the strangest things I find is BIO_get_accept_socket(). If bind_mode == BIO_BIND_REUSEADDR_IF_UNUSED, and bind() fails with EADDRINUSE it creates a new socket and tries to connect to the port it tried to bind() to, and if that fails tries to bind to it again but now with setting SO_REUSEADDR. Does anybody have an idea why it's trying to do that, and why we shouldn't just do SO_REUSEADDR the first time? Was there some OS that maybe did strange things when trying to use SO_REUSEADDR and it was already in use? The documentation also says: | BIO_set_bind_mode() and BIO_get_bind_mode() set and retrieve | the current bind mode. If BIO_BIND_NORMAL (the default) is set | then another socket cannot be bound to the same port. If | BIO_BIND_REUSEADDR is set then other sockets can bind to the | same port. If BIO_BIND_REUSEADDR_IF_UNUSED is set then and | attempt is first made to use BIO_BIN_NORMAL, if this fails | and the port is not in use then a second attempt is made | using BIO_BIND_REUSEADDR. The documentation is at least confusing, since it's about the address and port combination and not just the port. The main reason for using SO_REUSEADDR is that you can bind to an address / port combination that is in TIME_WAIT state, and it's recommended for server applications to always set this, and it's what I ended up doing in my new function. There are also some other cases you might want to use SO_REUSEADDR, and I suggest you read Stevens for that. In no case should it be possible to bind to the same TCP address / port from different applications, SO_REUSEADDR does not allow you to do that. But then I found some MSDN documentation that says that Windows allows others to hijack your socket when you've set SO_REUSEADDR and the results are non-deterministic. They also created an SO_EXCLUSIVEADDRUSE and I'm getting confused what it really does, but they say that server applications should set it. Anyway, is that whole logic with the connect() needed, or can I just always use SO_REUSEADDR (on non-windows hosts)? Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
