Hi all,
we had an outage of Wikitech and Labsconsole / Labs.
Apache on virt0 was broken.
It was reported by S Page on IRC, the symptons were
wikitech and labsconsole being down via HTTP(S) but pingable.
We also had Icinga monitoring telling us about it.
I went to virt0 and saw Apache running, shortly after restarted it and that
fixed the issue.
The root cause i found to be a bug in Phusion Passenger module that is used
on virt0,
as opposed to cluster Apache.
from /usr/lib/phusion_passenger/passenger-spawn-server:61
*** Exception Errno::EPIPE in Passenger RequestHandler (Broken pipe)
(process 23592):
I made a copy of the full log and for now it is on virt0 in
/root/apache2_error_log_20131219.log if you would like to see it all.
So far i just blame https://www.phusionpassenger.com/ respectively
libapache2-mod-passenger and didn't take any further action. I did tell
the labs IRC channel but did not post on labs-l.
Daniel
--
Daniel Zahn <dz...@wikimedia.org>
Operations Engineer
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
