On 2013-12-28 06:13, Tracy Reed wrote:
On Fri, Dec 27, 2013 at 08:54:17PM PST, Jefferson Ogata spake thusly:
Log rotation tends to run around that time on Red Hat. Check your logrotate
configuration. Maybe something is rotating corosync logs and using the wrong
signal to start a new log file.
That was actually the first thing I looked at! I found
/etc/logrotate.d/shorewall and removed it. But that seems to have had no effect
on the problem. That file has been gone for 3 weeks, the machines rebooted (not
that it should matter), and the problem has happened several times since then.
I've searched all over and can't find anything. And it doesn't even happen
every morning, just every week or two. Hard to nail down a real pattern other
than usually (not always) 4am.
Is it possible that it's a coincidence of log rotation after patching?
In certain circumstances i've had library replacement or subsequent
prelink activity on libraries lead to a crash of some services during
log rotation. This hasn't happened to me with pacemaker/cman/corosync,
but it might conceivably explain why it only happens to you once in a while.
You might take a look at the pacct data in /var/account/ for the time of
the crash; it should indicate exit status for the dying process as well
as what other processes were started around the same time.
Or, if not that, could it be some other cronned task?
These firewall machines are standard CentOS boxes. The stock crons (logrotate
etc) and a 5 minute nagios passive check are the only things on them as far as
I can tell. Although I haven't quite figured out what causes logrotate to run
at 4am. I know it is in the /etc/cron.daily/logrotate but what runs this at
4am? Is 4am some special hard-coded time in crond?
I just noticed that there is an /etc/logrotate.d/cman which rotates
/var/log/cluster/*log Could this somehow be an issue? I'm running pacemaker and
corosync but I'm not running cman:
# /etc/init.d/cman status
cman is not running
Should I be? I don't think it is necessary for this particular kind of
cluster... But since it isn't running it shouldn't matter.
Yes, you're supposed to switch to cman. Not sure if it's related to your
problem, tho.
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
