Hello, I know SO_REUSEPORT has been discussed here a few times and I am aware that haproxy uses it to make restarts less disruptive, as a new instance can bind() to the listen ports without the need to stop the old instance first.
But there is another aspect of SO_REUSEPORT that I have not yet seen discussed here (my apologies if I missed this, pointers welcome). The original patch for SO_REUSEPORT [1] explicitly mentions an uneven distribution across threads/processes for the "accept on a single listener socket from multiple threads" case, which the author intended to get rid of with SO_REUSEPORT. What I have so far never seen a detailed explanation of, though, is what the exact criteria are to get this to work. We always have and still are seeing this uneven distribution in our haproxy processes (haproxy 1.6, linux 3.16). My assumption is that this is because we use haproxy in daemon mode, which (I think) basically means that when a reload happens, one process is started, it binds all the sockets (using SO_REUSEPORT, which lets it take over sockets from the previous instance), but then fork()s the child processes, which thus inherit the already bound socket(s). My gut feeling - but I cannot point to any reliable reference - is that the even load distribution mecahnism only kicks in if all processes do the call to at least bind(), possibly listen() or even socket() themselves. Just setting SO_REUSEPORT but otherwise using the "accept on a single listener socket from multiple threads" might not actually reap the intended benefits here. So my question is basically: do I understand this situation correctly? Can someone who is more experienced with the kernel networking code comment on this aspect of SO_REUSEPORT? [1] https://lwn.net/Articles/542718/ Thanks a lot, Conrad -- Conrad Hoffmann Traffic Engineer SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany Managing Director: Alexander Ljung | Incorporated in England & Wales with Company No. 6343600 | Local Branch Office | AG Charlottenburg | HRB 110657B
