Re: reached select() limit

[eugeny gladkih]

On 01/31/2017 07:28 PM, Zan Lynx wrote:

On 01/31/2017 06:41 AM, eugeny gladkih wrote:

no, that's not a truth. you may use any socket number on all modern UNIX
systems. the only thing you need - you have to allocate enough memory
for fd_set. one more thing you have to know is still here. on Solaris
you may define the FD_SETSIZE preprocessor constant to any numer which
is big enough.


I'd be careful of that in Linux. I have always been told that FD_SETSIZE
is fixed at the time of glibc compile. That may not be true of course,
and it may not be true for other C libraries.

I would certainly not assume that I can just redefine FD_SETSIZE without
double-checking.



that was a piece of production code. it works well.

--
Yours sincerely, Eugeny.
GM of Enterprise Solutions Department
Doctor Web, Ltd.
http://www.drweb.com, +79119997425

Re: reached select() limit

[David Guillen]

Yep, I think glibc related functions might be using some compile time
information (like sizeof or #defines) so I wouldn't date to try that :D

Thanks!

2017-01-31 17:28 GMT+01:00 Zan Lynx :

> On 01/31/2017 06:41 AM, eugeny gladkih wrote:
> > no, that's not a truth. you may use any socket number on all modern UNIX
> > systems. the only thing you need - you have to allocate enough memory
> > for fd_set. one more thing you have to know is still here. on Solaris
> > you may define the FD_SETSIZE preprocessor constant to any numer which
> > is big enough.
>
> I'd be careful of that in Linux. I have always been told that FD_SETSIZE
> is fixed at the time of glibc compile. That may not be true of course,
> and it may not be true for other C libraries.
>
> I would certainly not assume that I can just redefine FD_SETSIZE without
> double-checking.
>
> --
> Knowledge is Power -- Power Corrupts
> Study Hard -- Be Evil
>

Re: reached select() limit

[Zan Lynx]

On 01/31/2017 06:41 AM, eugeny gladkih wrote:
> no, that's not a truth. you may use any socket number on all modern UNIX
> systems. the only thing you need - you have to allocate enough memory
> for fd_set. one more thing you have to know is still here. on Solaris
> you may define the FD_SETSIZE preprocessor constant to any numer which
> is big enough.

I'd be careful of that in Linux. I have always been told that FD_SETSIZE
is fixed at the time of glibc compile. That may not be true of course,
and it may not be true for other C libraries.

I would certainly not assume that I can just redefine FD_SETSIZE without
double-checking.

-- 
Knowledge is Power -- Power Corrupts
Study Hard -- Be Evil

Re: reached select() limit

[Leif Thuresson]

Just remembered an quirk we used to do way back to over come a problem 
with old solaris versions where the

stdio struct used a char for the file descriptor.
If you have control over the file descriptors you application is 
creating you can push them to higher numbers with

  newfd = fcntl(orgfd, F_DUPFD, lowlimit);
leaving a range of low number file descriptors free to use for c-ares.
Yes it's ugly, but if nothing else helps...

/Leif


On 2017-01-26 14:32, dan...@poradnik-webmastera.com wrote:
You can also register callbacks using ares_options::sock_state_cb and 
ares_set_socket_callback(), which simplifies things a lot. I used them 
when I was integrating c-ares with our epoll-based code.


Daniel

W dniu 2017-01-25 18:48, David Guillen Fandos napisał(a):

Ha!

Sorry for that, I just realized that it returns a bitmap of sockets.
Why doesn't it return something like uint32_t?

Allright then, so there's really no way to workaround my problem
except, perhaps, to initialize c-ares at the begining of the process.
But c-ares doesn't guarantee that all fds are gonna get created at
start right? If some server doesn't respond it might try TCP and open
a new socket, am I right?

Thanks for your help!
David

On 25/01/17 15:17, Daniel Stenberg wrote:

On Wed, 25 Jan 2017, David Guillen Fandos wrote:


Yeah agreed, but how do you retrieve the fds to use poll? With getsock
you get up to 16 sockets which is insufficient for my needs (and also,
BTW, sounds like an arbitrary and ridiculous number to hardcode in
such a function)


Arbitrary? Not really, it happens to be the number that fits in an
unsigned 32bit variable as each socket needs two bits in the API. I 
just
considered it a dececently convenient API at the time (and happens 
to be

similar to a function interface we use internally in libcurl).

Ridiculous? Maybe, but I added that API a long time ago and I have yet
to end up in a case where the 16 limit has been reached and impacted us
(or anyone else). I added this API to allow libcurl to use c-ares
event-based and we've since used it with many thousand simultaneous
requests and DNS looks with success - over the period of many years.

Can it be improved? I'm sure.

Re: reached select() limit

[David Guillen Fandos]

Ha!

Sorry for that, I just realized that it returns a bitmap of sockets. Why 
doesn't it return something like uint32_t?


Allright then, so there's really no way to workaround my problem except, 
perhaps, to initialize c-ares at the begining of the process. But c-ares 
doesn't guarantee that all fds are gonna get created at start right? If 
some server doesn't respond it might try TCP and open a new socket, am I 
right?


Thanks for your help!
David

On 25/01/17 15:17, Daniel Stenberg wrote:

On Wed, 25 Jan 2017, David Guillen Fandos wrote:


Yeah agreed, but how do you retrieve the fds to use poll? With getsock
you get up to 16 sockets which is insufficient for my needs (and also,
BTW, sounds like an arbitrary and ridiculous number to hardcode in
such a function)


Arbitrary? Not really, it happens to be the number that fits in an
unsigned 32bit variable as each socket needs two bits in the API. I just
considered it a dececently convenient API at the time (and happens to be
similar to a function interface we use internally in libcurl).

Ridiculous? Maybe, but I added that API a long time ago and I have yet
to end up in a case where the 16 limit has been reached and impacted us
(or anyone else). I added this API to allow libcurl to use c-ares
event-based and we've since used it with many thousand simultaneous
requests and DNS looks with success - over the period of many years.

Can it be improved? I'm sure.

Re: reached select() limit

[Brad House via c-ares]

Why is 16 too few?  Do you have more than 8-16 name servers configured?
Remember, it is one fd per nameserver (or possibly 2 if it has to fall back
to TCP), NOT one fd per query, as my original reply stated.

The ares_getsock is what you'd use, and yes, it does have a limit of 16
fds.

-Brad

On 1/25/17 3:36 AM, David Guillen Fandos wrote:

Yeah agreed, but how do you retrieve the fds to use poll?
With getsock you get up to 16 sockets which is insufficient for my needs (and 
also, BTW, sounds like an arbitrary and
ridiculous number to hardcode in such a function)

Thanks!

On 25/01/17 04:05, Zan Lynx wrote:


That will not help because select has a hard coded limit of the number of bits 
in its read, write and error sets.

What could help us to open c-ares socket very early in the program to get a low 
FD number. Or switch to poll. I posted
a half done, good enough for me poll fix to this list some years ago. Or use 
one of the other APIs to get the socket
descriptors and poll.

Re: reached select() limit

[Daniel Hardman]

Thank you for reminding me about the select() limit's relationship to
FD_SET. I had forgotten that.

My quick answer about setrlimit() was based on a binary I worked in that
used c-ares plus numerous other socket-oriented libraries. I ran into the
FD_SET problem in my other socket-dependent layers, and converted them to
use epoll. Once I did so, I stopped seeing the limitation (after I also
called setrlimit()). This leads me to believe that David's theorized
workaround might be reasonably reliable, because otherwise the epoll
upgrade I did wouldn't have eliminated the problem for the c-ares layers. I
think my code had to do a DNS lookup before any other network I/O happened,
so c-ares got very low fd numbers...

--Daniel



On Tue, Jan 24, 2017 at 8:05 PM, Zan Lynx  wrote:

> On January 24, 2017 7:16:35 PM MST, Daniel Hardman <
> daniel.hard...@gmail.com> wrote:
> >Your program should call setrlimit on startup to change the number of
> >file
> >descriptors. See http://unix.stackexchange.com/a/29579.
> >
> >
>
>
> That will not help because select has a hard coded limit of the number of
> bits in its read, write and error sets.
>
> What could help us to open c-ares socket very early in the program to get
> a low FD number. Or switch to poll. I posted a half done, good enough for
> me poll fix to this list some years ago. Or use one of the other APIs to
> get the socket descriptors and poll.
>

Re: reached select() limit

[Zan Lynx]

On January 24, 2017 7:16:35 PM MST, Daniel Hardman  
wrote:
>Your program should call setrlimit on startup to change the number of
>file
>descriptors. See http://unix.stackexchange.com/a/29579.
>
>


That will not help because select has a hard coded limit of the number of bits 
in its read, write and error sets.

What could help us to open c-ares socket very early in the program to get a low 
FD number. Or switch to poll. I posted a half done, good enough for me poll fix 
to this list some years ago. Or use one of the other APIs to get the socket 
descriptors and poll.

Re: reached select() limit

[Brad House via c-ares]

David is correct, on many systems, like Linux, select() cannot handle
file descriptor *numbers*, higher than 1023 (most people assume it is
just the count of fds set via FD_SET cannot exceed 1024, which is also true).

Since he stated he is working with fds higher than that, it is presumed
his rlimits are already set correctly.

-Brad

On 1/24/17 9:16 PM, Daniel Hardman wrote:

Your program should call setrlimit on startup to change the number of file 
descriptors.
See http://unix.stackexchange.com/a/29579.

On Tue, Jan 24, 2017 at 5:14 PM, Brad House via c-ares > wrote:

I'm pretty sure c-ares just uses a single socket per nameserver, so to
have more than 16 socks, you'd have to have more than 8 DNS servers (I'm
assuming here that each server tries both UDP, then due to response
overflow has to retry via TCP).

I use c-ares with an event based system where we use epoll()/kqueue()
instead of select, we load test regularly with high connection counts,
can't say we've seen an issue.

-Brad


On 1/24/17 6:18 PM, David Guillen Fandos wrote:

Hello,

I wrote an app that was crashing in c-ares due to fds being bigger than 
1024. While c-ares might be using around
30 fds
it is unable to use fds above 1024.

I looked into using getsock but it is capped at 16 sockets (although 
could be worked around by building c-ares
myself
and tweaking the constant).

Is there any other way or suggestion? I thought of initializing c-ares 
at the startup phase so I always get
smallish fds
but it is a bit dodgy :D My program uses many many sockets

Thanks!

David

Re: reached select() limit

[Brad House via c-ares]

I'm pretty sure c-ares just uses a single socket per nameserver, so to
have more than 16 socks, you'd have to have more than 8 DNS servers (I'm
assuming here that each server tries both UDP, then due to response
overflow has to retry via TCP).

I use c-ares with an event based system where we use epoll()/kqueue()
instead of select, we load test regularly with high connection counts,
can't say we've seen an issue.

-Brad

On 1/24/17 6:18 PM, David Guillen Fandos wrote:

Hello,

I wrote an app that was crashing in c-ares due to fds being bigger than 1024. 
While c-ares might be using around 30 fds
it is unable to use fds above 1024.

I looked into using getsock but it is capped at 16 sockets (although could be 
worked around by building c-ares myself
and tweaking the constant).

Is there any other way or suggestion? I thought of initializing c-ares at the 
startup phase so I always get smallish fds
but it is a bit dodgy :D My program uses many many sockets

Thanks!

David

reached select() limit

[David Guillen Fandos]

Hello,

I wrote an app that was crashing in c-ares due to fds being bigger than 
1024. While c-ares might be using around 30 fds it is unable to use fds 
above 1024.


I looked into using getsock but it is capped at 16 sockets (although 
could be worked around by building c-ares myself and tweaking the 
constant).


Is there any other way or suggestion? I thought of initializing c-ares 
at the startup phase so I always get smallish fds but it is a bit dodgy 
:D My program uses many many sockets


Thanks!

David