FYI, the bug is submitted: https://bz.apache.org/bugzilla/show_bug.cgi?id=59897
Chris and Rainer, thanks for pointing me in the right direction! Michael On 19 July 2016 at 11:42, Michael Diener <mdie...@mdiener.de> wrote: > Chris, > > thanks a lot for explaining what could be overflowing the FD_SETSIZE of > 1024. > > Just for reference, I use mpm_event mostly with SSL connections, so it > should behave like mpm_worker. This is my configuration: > > StartServers 2 > ServerLimit 16 > > MinSpareThreads 256 > MaxSpareThreads 1280 > > ThreadLimit 1024 > ThreadsPerChild 1024 > > MaxRequestWorkers 16384 > MaxConnectionsPerChild 0 > > > One more thing, poll() seems to be used in jk_connect.c in other places > already, just not at the spot that matters in my case. > > Anyhow, I will submit a bug report later this week with all the > information and will post a link over here as well. > > Thank you, > Michael > > > > > On 18 July 2016 at 16:56, Christopher Schultz < > ch...@christopherschultz.net> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> Michael, >> >> On 7/18/16 10:10 AM, Christopher Schultz wrote: >> > Michael, >> > >> > On 7/18/16 8:53 AM, Michael Diener wrote: >> >> On 6 July 2016 at 00:09, Christopher Schultz >> >> <ch...@christopherschultz.net> wrote: >> > >> >>>> From what I understand a buffer overflow would only happen >> >>>> for FD_SET if the fd_set gets over 1024 descriptors. I made >> >>>> sure that my ulimit for open files is set and applied large >> >>>> enough, so that's not it. >> >>> >> >>> There's nothing magic about the ulimit. An fd_set should size >> >>> appropriately for your OS. On my Linux system, FD_SETSIZE >> >>> happens to be set to 1024. Reading through the byzantine >> >>> labyrinth of includes, it appears that FD_SET has zero >> >>> boundary-checking, so it's therefore possible that overflow >> >>> will occur. >> > >> > >> >> Regarding the FD_SETSIZE, it is also set for me to 1024 although >> >> the ulimit is set higher. >> > >> > Well, the FD_SETSIZE is just the maximum size of the fd set that >> > can be passed-into select() specifically. That doesn't limit the >> > number of file descriptors your processes can open. The ulimit >> > settings are the limits on those. >> > >> >> I'm a bit lost now on what I should do now. What makes me wonder >> >> is, that nobody else seems to hit this limitation of FD_SET and >> >> this makes me think something on my Linux machine is not right. >> > >> > Well, not everyone is using the same setup. For example, using NIO >> > through Java is likely to get you the poll() call under the hood, >> > so supporting more than e.g. 1024 file descriptors is not an issue >> > there. That's just a guess, but I think it's a reasonable guess. >> > >> > I think tcnative+APR is not widely deployed. I have no numbers to >> > back that up, but we don't get a huge volume of questions in the >> > list about it. >> >> Of course, the above statement makes no sense whatsoever. This is >> about mod_jk and not tcnative. Sorry for the confusion. >> >> But I was thinking, the case above would require a single httpd thread >> to accumulate more than 1024 connections, and that would require some >> very specific circumstances. >> >> First, you'd have to be using an MPM that allowed more than one >> connection per process/thread, so that means no pre-fork, leaving >> basically event or worker available. >> >> Then, you'd have to have enough traffic to cause one thread to grow to >> more than 1024 connections. The default for httpd's worker MPM has 64 >> threads per child and 16 server processes. >> >> For one process to exceed the 1024 fd limit, you'd have to be handling >> roughly 16 simultaneous connections per thread PER PROCESS. I'm not >> sure how httpd auto-scales-out its child processes, so you could >> conceivably get 1024 simultaneous connections in a burst of requests >> to a single child process, but it seems likely that load would be >> roughly-split between 16 child processes, keeping those 1024 >> connections down to a mild 64-jk-connections per-process. >> >> Assuming you have 16 child processes and a perfectly uniform >> load-distribution between them, you'd have to get a burst of 16k >> simultaneous requests in order to max-out that fd array on a single >> server. That's probably why it's quite rare. >> >> - -chris >> -----BEGIN PGP SIGNATURE----- >> Comment: GPGTools - http://gpgtools.org >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ >> >> iQIcBAEBCAAGBQJXjO4tAAoJEBzwKT+lPKRYb9wP/j2i6FW7gOGf/G4/CTundEMY >> WcKKPfNXAOU270KyiBkuSLw06Cjw8mk3vsjpg5i5JkPC4TUNy1jyhaI5cs0RvisV >> pzqFgZCuyz4gPtg4TRCXw5RrmLwe7unBldTETjK54P9/Nd6Vuj34mUV8OLcnYZh6 >> UMCe0ULbBV5IoGhLmGQ5yy7MfHGdq1vwmxR41i4A4rc4J9fOC1UI4pIOvJRM1cnT >> 5cfLEavT3tbqxaxLCs5V/pQkkuwQMKITSW+JGdDPxN3oXL7b1QCw8RGBqhpDgjE/ >> FIIIBGfbMGHDXstmSBRXGlxkASKKdYlK9qoYU3f7G0PK053zx5TD+2vOfb0u0Vi/ >> lwIkk3lhL7Azw9hYKFr1R+PW3ewUqXI7Nh05HldNWlJ48I91cTGLF2mC7uRo6uQ2 >> M9pUCuyCtL1ajgG6eUmBlo2soIAaHPkorCmCdUAiv/zWfHKSSEGTwr3l81DtSapE >> iORRoCyLVIhxKQprvBlTHp2uDIa7lzXOI83RcMb6ZqdxiNe3LscTRsVl/Ll8cVHj >> Fw7klJgbPrRPmMn02hANdalDE96CvagPlEmgCGhp9h3TPD7fV28a3vY154IYxLBW >> C2ksoMNv12ha+kiTvrYLc7j85drtZg7V0C/5TfRdBRSxxJ0KF/ye7ed2SN/8RRKC >> QgyoIkZ2oSBIoHc/ss26 >> =f33p >> -----END PGP SIGNATURE----- >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > > > -- > > ______________________________ > NEW GAME! http://www.dig-pig.com > > Michael Diener - Software e.K. > > mdie...@mdiener.de > +49 178 501 601 8 > www.mdiener.de > > @mdienersoftware > > Grünberger Str. 62, > 10245 Berlin, Germany > > Sitz Berlin, Amtsgericht Charlottenburg, HRA 46760 B > USt-IdNr. DE233968393 > -- ______________________________ NEW GAME! http://www.dig-pig.com Michael Diener - Software e.K. mdie...@mdiener.de +49 178 501 601 8 www.mdiener.de @mdienersoftware Grünberger Str. 62, 10245 Berlin, Germany Sitz Berlin, Amtsgericht Charlottenburg, HRA 46760 B USt-IdNr. DE233968393
