On Sun, 2011-05-01 at 16:19 -0400, Nate Aune wrote: > Every time there is a tick and memmon checks the memory of a Gunicorn > process, the following output is logged to the error log: > > $ tail -f > /var/log/supervisor/bundle_p00000017_2011-05-01-19.28.03-stderr---memmon.log > Checking programs bundle_p00000017_2011-05-01-19.28.03=26214400 > ERROR: Process ID out of range. > ********* simple selection ********* ********* selection by list ********* > -A all processes -C by command name > -N negate selection -G by real group ID (supports names) > -a all w/ tty except session leaders -U by real user ID (supports names) > -d all except session leaders -g by session OR by effective group name > -e all processes -p by process ID > T all processes on this terminal -s processes in the sessions given > a all w/ tty, including other users -t by tty > g OBSOLETE -- DO NOT USE -u by effective user ID (supports names) > r only running processes U processes for specified users > x processes w/o controlling ttys t by tty > *********** output format ********** *********** long options *********** > -o,o user-defined -f full --Group --User --pid --cols --ppid > -j,j job control s signal --group --user --sid --rows --info > -O,O preloaded -o v virtual memory --cumulative --format --deselect > -l,l long u user-oriented --sort --tty --forest --version > -F extra full X registers --heading --no-heading --context > ********* misc options ********* > -V,V show version L list format codes f ASCII art forest > -m,m,-L,-T,H threads S children in sum -y change -l format > -M,Z security data c true command name -c scheduling class > -w,w wide output n numeric WCHAN,UID -H process hierarchy > ERROR: Process ID out of range. > ********* simple selection ********* ********* selection by list ********* > -A all processes -C by command name > -N negate selection -G by real group ID (supports names) > -a all w/ tty except session leaders -U by real user ID (supports names) > -d all except session leaders -g by session OR by effective group name > -e all processes -p by process ID > T all processes on this terminal -s processes in the sessions given > a all w/ tty, including other users -t by tty > g OBSOLETE -- DO NOT USE -u by effective user ID (supports names) > r only running processes U processes for specified users > x processes w/o controlling ttys t by tty > *********** output format ********** *********** long options *********** > -o,o user-defined -f full --Group --User --pid --cols --ppid > -j,j job control s signal --group --user --sid --rows --info > -O,O preloaded -o v virtual memory --cumulative --format --deselect > -l,l long u user-oriented --sort --tty --forest --version > -F extra full X registers --heading --no-heading --context > ********* misc options ********* > -V,V show version L list format codes f ASCII art forest > -m,m,-L,-T,H threads S children in sum -y change -l format > -M,Z security data c true command name -c scheduling class > -w,w wide output n numeric WCHAN,UID -H process hierarchy > RSS of > bundle_p00000017_2011-05-01-19.28.03:bundle_p00000017_2011-05-01-19.28.03 > is 9297920 > > You can see that in the end it properly gets the RSS of the process, > but why does it report "ERROR: Process ID out of range." two times > once the process is running? > > Here is the relevant section in my supervisor conf file: > > [eventlistener:memmon] > command=/usr/local/bin/memmon -p bundle_p00000017_2011-05-01-19.28.03=25MB > events=TICK_60 > stdout_logfile=/var/log/supervisor/bundle_p00000017_2011-05-01-19.28.03-stdout---memmon.log > stderr_logfile=/var/log/supervisor/bundle_p00000017_2011-05-01-19.28.03-stderr---memmon.log > > The log file is going to get quite verbose with all of these messages > (consuming disk space), so I'm trying to find out where the problem is > occurring so I can get less verbose (and more accurate) log files. If > you have any ideas what the problem might be, I'd very much appreciate > your input.
memmon uses the system "ps" command under the hood. It seems as if the ps command it's sending produces output that is unexpected. This might be due to a change to "ps" that isn't anticipated by memmon, or it might be a bug in the data that supervisor sends back to memmon when it asks for "all process info". The actual command sent is: ps -orss= -p <pid> .. where "pid" is the pid of the process. It's not just a logfile problem, it probably means that things are not working as you expect them to. I don't know how to solve it without being able to reproduce it, and I don't know how to reproduce it, but I'd suggest editing memmon.py and writing the value of "pid" when it fails to some file somewhere (the same pid as in data = shell(self.pscommand % pid) within memmon.py) to see if the pid it's trying to check even makes any sense. - C _______________________________________________ Supervisor-users mailing list [email protected] http://lists.supervisord.org/mailman/listinfo/supervisor-users
