Re: Multi-threaded operation?

2015-10-05 Thread W.C.A. Wijngaards via Unbound-users
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hi Havard,

On 05/10/15 14:43, Havard Eidnes via Unbound-users wrote:
> Hi,
> 
> it looks like I'll have to answer my own question, which is a 
> little disappointing:
> 
>> I'm running unbound 1.5.4 on NetBSD/amd64 7.0, and I notice that 
>> despite me having configured
>> 
>> server: num-threads: 12 so-reuseport: yes
>> 
>> only one of the threads is handling all the queries, according
>> to the output from "unbound-control stats".  "Not what I
>> wanted."
> 
> It turns out that using the "so-reuseport" setting to distribute 
> the load over the threads is a fairly recent Linuxism, and relying
> on it causing the kernel to distribute the load over the different
> sockets is not portable.

Yes, I did not realise this.  If you disable reuseport on BSD I think
it will start to work.

> 
> The first answer in
> 
> http://stackoverflow.com/questions/14388706/socket-options-so-reuseadd
r-and-so-reuseport-how-do-they-differ-do-they-mean-t
>
>  says it quite clearly:
> 
> Linux 3.9 added the option SO_REUSEPORT to Linux as well. [...]
> Additionally the kernel performs some "special magic" for
> SO_REUSEPORT sockets that isn't found in any other operating system
> so far: For UDP sockets, it tries to distribute datagrams evenly,
> for TCP listening sockets, it tries to distribute incoming connect
> requests (those accepted by calling accept()) evenly across all the
> sockets that share the same address and port combination.  [...]
> 
> I'll try to turn off the so-reuseport option later today, and see 
> if that improves the situation.

The option provides nice benefits on Linux; somehow having one file
descriptor per thread/process that is tied to the same socket allows
for much faster operation in the network stack.

Best regards, Wouter

> 
> Regards,
> 
> - Håvard
> 

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCAAGBQJWEnK/AAoJEJ9vHC1+BF+N1v8P+gKLb2H6TDIQiV9ZFvEfg18Y
JQN6jdL1MaDQFQO07SRt3xTKP+GRVy8M3FrSIFC0GeFIR2IYku9FJz7cHFchU3Dp
TimQd6tP/D6qOoOT3W6zoGAFy+RREoc3ZJK66nLMOY5U6PWlgekJsHwgH+LN1whO
2WWSg7om7Fe0xpYdhKaTlBpliWHDXO/J/8jBd+v1+SCqWpg+EsH8NUQkzNEoHE8q
EGs6TjKv/UgnumzapPlKgA2PxGUQArdwCrY4V7/5AdLUa+9qxLjAvu+X17Sj7iHd
grXbygi/h4kXjZyPx9KbzTCtG1NNH1n/cFcfAoGi4n+UBFAfrSr3dF8r/n9CIXCw
M/OmEZ592sKMihtVk5qWi2uXMJD6OPZoCLC+7SEhSfkPTaI4P7phlczBEmt9i9Ro
fCOezRVRg75GU4t1A5vtVLV+SHlFV6wfsaxbLgOSi8OpTmlOs6HvfFlr7wnQb/Jd
OUu30ltwbV3kLVJQngSAtOAOByt6dviCJwcjk5S/LEqOOfTJ1iT809OMI4y3E0aY
t1N+tBXi3MtvQaeid9Drf9COVWIrmj6qFwZuLH+wdPbXKuRpeSdYOi+0HgYMBUBP
6N9DDF3CykVNBJeJMyEsmuIczRRsUZxIGv0Ls0wCLeeBpxmCF0wpicKAu8Z2n8xa
V4hZ0GAkNMOHNY031vRR
=EoJr
-END PGP SIGNATURE-


Re: Multi-threaded operation?

2015-10-05 Thread Havard Eidnes via Unbound-users
Hi,

it looks like I'll have to answer my own question, which is a
little disappointing:

> I'm running unbound 1.5.4 on NetBSD/amd64 7.0, and I notice that
> despite me having configured
>
> server:
>   num-threads: 12
>   so-reuseport: yes
>
> only one of the threads is handling all the queries, according to
> the output from "unbound-control stats".  "Not what I wanted."

It turns out that using the "so-reuseport" setting to distribute
the load over the threads is a fairly recent Linuxism, and
relying on it causing the kernel to distribute the load over the
different sockets is not portable.

The first answer in

  
http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t

says it quite clearly:

Linux 3.9 added the option SO_REUSEPORT to Linux as well.
[...] Additionally the kernel performs some "special magic"
for SO_REUSEPORT sockets that isn't found in any other
operating system so far: For UDP sockets, it tries to
distribute datagrams evenly, for TCP listening sockets, it
tries to distribute incoming connect requests (those accepted
by calling accept()) evenly across all the sockets that share
the same address and port combination.  [...]

I'll try to turn off the so-reuseport option later today, and see
if that improves the situation.

Regards,

- Håvard