nsdispatch performance issue for large group files

[Anthony Bourov]

Regarding performance of: lib/libc/net/nsdispatch.c
When used from: lib/libc/net/getgrent.c (called by initgroups())

I don't normally post here but I wanted to make a suggestion on a performance 
issue that I spotted. I run a large number of high-volume web hosting servers 
and noticed on some of the servers a severe decrease in Apache's performance 
when the /etc/group file is large (over 100,000 entries in a group file as it 
is combined across servers).

I did a trace and found the following operation:
stat(/etc/nsswitch.conf, {st_mode=052, st_size=4503681233059861, ...}) = 0

repeating as many times as there is groups in the group file. I narrowed the 
problem down to where apache calls initgroups() before forking each process 
(nothing wrong here). And init groups goes through every entry in the group 
file using getgrent(), which in turn calls nsdispatch and which for every 
single call does stat on /etc/nsswitch.conf to see if it changed. 

This issue impacts different servers differently, on most of the SCSI servers 
this delays apache startup my maybe a minute, however, on a Dell SATA raid the 
stat command was significantly slower and caused everything to come to a halt 
for several minuted every time apache starts.

In my opinion this is a very significant performance issue when working with 
large servers. Most programs, including apache, will call initgroups() for 
every time they fork, and it the group file is large this means as many stat 
requests on the file system as there are entries in the group file for every 
single fork() that the server does.

For myself I just made it never test stat on /etc/nsswitch.conf after the 
first time since I know that file is never modified. However, a better solution 
would for nsdispatch realise in that case that it is being ran in batch mode 
and should not keep testing if the file has changed. This would effect both 
getgrent and getpwent. 

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Q: xl0: no memory for rx list -- packet dropped!

[Anthony Bourov]

Hi,

I am running a FreeBSD server, and I am running into this problem very 
often. The machine stops responding and instead outputs 1000s of

xl0: no memory for rx list -- packet dropped!
xl0: no memory for rx list -- packet dropped!
xl0: no memory for rx list -- packet dropped!
xl0: no memory for rx list -- packet dropped!
messages. The machine is usually pulling about 4-5 megabits but there is 
usually a traffic spike right before this happens so an attack is not out 
of the question, but I was wondering if there was any way I can raise the 
threshold  for this (would more BUFFERs do the trick?).


Thanks in advance,
Anthony


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message