What is in the data of the PROCESS_LOG event payload? I'm wondering if its possible to write an event listener which will "tail -F" a process' STDOUT/STDERR and, if a particular message appears, take action by a) notifying the administrators and b) restarting the process. (I'm managing Apache Tomcat now, so the "SEVERE: All *threads* (125) are currently busy, waiting. Increase *maxThreads..." or Out of Memory messages from are initial examples)*
Next, I'm seeing a lot of the "event buffer overflowed" messages from my event listener. For the period of the log snippet below, the only configured/running event listener was "crashmail." crashmail is still working as expected (when I did a pkill -f auto-qa-tomcat, I received the email from crashmail) but I see the event buffer overflowed messages for crashmail. In addition, I see those messages for event listeners which aren't running, are removed, and are no longer configured. 2010-09-02 11:35:02,836 ERRO pool crashmail event buffer overflowed, discarding event 2738 2010-09-02 11:35:02,837 ERRO pool crashmail event buffer overflowed, discarding event 2738 2010-09-02 11:35:02,837 ERRO pool crashmail event buffer overflowed, discarding event 2738 2010-09-02 11:35:02,837 INFO exited: auto-qa-tomcat (exit status 143; not expected) 2010-09-02 11:35:03,838 ERRO pool crashmail event buffer overflowed, discarding event 2739 ... REMOVED DUPLICATE LINES 2010-09-02 11:35:03,841 ERRO pool crashmail event buffer overflowed, discarding event 2739 2010-09-02 11:35:03,905 INFO spawned: 'auto-qa-tomcat' with pid 13198 2010-09-02 11:35:04,922 ERRO pool logmail event buffer overflowed, discarding event 2079 2010-09-02 11:35:04,923 ERRO pool custom event buffer overflowed, discarding event 2079 2010-09-02 11:35:04,923 ERRO pool custom event buffer overflowed, discarding event 2079 2010-09-02 11:35:04,924 ERRO pool custom event buffer overflowed, discarding event 2079 2010-09-02 11:35:04,924 ERRO pool crashmail event buffer overflowed, discarding event 2742 2010-09-02 11:35:04,925 ERRO pool crashmail event buffer overflowed, discarding event 2742 2010-09-02 11:35:04,925 ERRO pool crashmail event buffer overflowed, discarding event 2742 ... REMOVED DUPLICATE LINES 2010-09-02 11:35:06,433 ERRO pool logmail event buffer overflowed, discarding event 2150 2010-09-02 11:35:06,434 ERRO pool custom event buffer overflowed, discarding event 2150 ... REMOVED DUPLICATE LINES 2010-09-02 11:35:06,435 ERRO pool custom event buffer overflowed, discarding event 2150 2010-09-02 11:35:06,436 ERRO pool crashmail event buffer overflowed, discarding event 2763 ... REMOVED DUPLICATE LINES 2010-09-02 11:35:14,449 ERRO pool crashmail event buffer overflowed, discarding event 2742 2010-09-02 11:35:14,449 INFO success: auto-qa-tomcat entered RUNNING state, process has stayed up for > than 10 seconds (startsecs) 2010-09-02 11:36:00,476 ERRO pool crashmail event buffer overflowed, discarding event 2748 Using versions superlance-0.5-py2.4.egg and supervisor-3.0a8-py2.4.egg as installed from easy_install. Here is my event listener configuration: [eventlistener:crashmail] command=/usr/bin/crashmail -a -m [email protected] -s '/usr/sbin/sendmail -t -i -f [email protected]' ; the program (relative uses PATH, can take args) events=PROCESS_STATE ; event notif. types to subscribe to (req'd) buffer_size=20 ; event buffer queue size (default 10) environment=SUPERVISOR_USERNAME='admin',SUPERVISOR_PASSWORD='password',SUPERVISOR_SERVER_URL=' http://localhost:9001' ; process environment additions I see that Chris replied to http://lists.supervisord.org/pipermail/supervisor-users/2008-April/000234.html a while back; but I'm not sure that response still applies to my situation. Thanks, -- Jason Koppe [email protected]
_______________________________________________ Supervisor-users mailing list [email protected] http://lists.supervisord.org/mailman/listinfo/supervisor-users
