> Dnia czwartek, 1 grudnia 2011 21:12:25 Roberto De Ioris pisze: >> > Dnia czwartek, 1 grudnia 2011 04:46:25 Roberto De Ioris pisze: >> >> No, workers are forked from the master(). >> > >> > I wanted to verify one thing: when I run with processes = 8 I have 10 >> > workers >> > running: >> > 1x running as root and eats very little memory >> >> this is the jail manager (the /sbin/init equivalent of your main system) >> >> > 1x running as www-data as set in uid/git option, eating less then >> other >> > workers but its memory usage grow over time >> >> this is the master, and if it grows in memory there is a problem/bug. > > I tested it with various options (ksm on/off, chroot on/off, stats > on/off, > diffrent request/memory limits and few more) and with another app > (redmine) to > verify that this is not a bug in my application, and it always happens. My > master process slowly grows under high load, I didn't noticed it when > vassal > is idle. > >>
I spent the whole day tracking this. The behaviour is a combination of 1) another micro-leak in ksm 2) log-socket being stuck on high load 3) the stats server being too "slow" and blocking the logging subsystem too much. The 1) has been fixed and committed. 3) will be fixed soon and 2) is a total PITA. Practically when you have a socketpair() and you async wait for on of its side, the kernel starts to allocate an UNLIMITED number of pages to have the data saved in memory 'til the process read them. I have not found documentation about this behaviour (i suspect it is linux/epoll specific) and i have tried an unlimited series of technics. The problem is practically non-existent as logs passed by the workers to the master are read fastly 99.999% of the times, but i suppose that an app running for months could grows in a dangerous way. Putting a memory check in the master looks too much for me (it is expected the master has 0 leak), so i will start another series of test (and heavy study on the matter) tomorrow. In the mean time you should note a better behaviour with the latest tip + ksm -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
