Correct, the only thing changed was the rsyslog version. I know it's strange and understand it's controlled by init, but to me it's very odd that the hang is so deviant when there is literally no saveonshutdown, no queues, it's a vanilla install in at least 2 instances.
I figured it bared mentioning given the issues mentioned previously on a VM instance (in the 7.6.X stability thread.) Ultimately I can live with the "ok" messages and putting in a sleep between my init script 'restart' function, what REALLY bothers me though is the case of the missing exit messages. On my baremetal hosts I get the exit messages, etc. without incident, on my VM hosts however, that message seems to almost be non-existant (or in my case will RARELY generate the dodie message prior to seeing the 'start' message) So in summary, I can live with the init, but the lack of standard process exit information I'm used to seeing on all prior versions (and currently see on baremetal 8.2.X) concerns me that there is other information that just isn't being processed correctly. On Tue, Jul 8, 2014 at 5:57 PM, David Lang <da...@lang.hm> wrote: > I'm not sure who is maintaining the init scripts for RHEL/CentOS6 (isn't > that upstart?) > > We should get them to weigh in here. > > if you were to do killall rsyslogd; /usr/bin/rsyslogd I would expect that > you will see the saem thing. > > With shared storage between the VMs, I would expect that it would take > longer to flush the queues, so you would be more likely to run into this > problem than on bare metal. > > unfortunantly, when you run "service rsyslog start", teh "Ok" is being > generated by the initscript, not by rsyslog. > > Was the only thing changed in the upgrade the rsyslog version? > > David Lang > > > On Tue, 8 Jul 2014, Nick Syslog wrote: > > They are indeed VMs using a shared storage methodology in a state-lite >> configuration. That being said, up until this point I'd seen normal >> behavior out of 7.6.2 but as soon as we jumped to 8.2.1 I noticed >> inconsistent behavior on my VM servers. (On baremetal hosts I do not seem >> to encounter this same issue.) >> >> In addition I've gone through the rigor of removing all of the packages >> from the prior versions for a 'fresh start' approach and have observed the >> same behavior even after installation. >> >> In my case all of the systems observed are considered identical platforms >> (VMs on shared infrastructure) and in the 3 instances that were upgraded I >> experienced very similar issues regarding the exit status messages being >> omitted or the 'DoDie' message being printed instead. >> >> With regards to the 'service rsyslog restart' command, the only way that >> the init script was successfully able to start/stop was when there was a >> 'sleep .3' inserted between the start and stop stanza's under restart. If >> executing 'service rsyslog stop' and then start independently the service >> reported back the status of 'ok' each time (the exception being that >> process exit messages were still being omitted from the /var/log/messages >> file) >> >> >> >> On Mon, Jul 7, 2014 at 1:58 PM, David Lang <da...@lang.hm> wrote: >> >> The problem is that there's not much that rsyslog can do. >>> >>> It can't write to the files (they are owned by the old copy) >>> >>> It can't write to the old copy (it's shutdown it's input as it's trying >>> to >>> stop cleanly) >>> >>> It can't reliably write to stderr because most of the distro startup >>> scripts capture this output and don't show it to the user (this is >>> probably >>> what's happening when you find that it's waiting for user input but >>> doesn't >>> show any prompt and you need to control-c to get out) >>> >>> now, you say this is new behavior for you, so what changed? >>> >>> are you using a different version, different storage? are these VMs? >>> >>> David Lang >>> >>> >>> On Mon, 7 Jul 2014, Nick Syslog wrote: >>> >>> Either way completely hanging service restart on the start and not >>> >>>> emitting >>>> exit signals to log files as well as the dodie message in place of the >>>> exit >>>> message seems a bit more complex than just a simple hang... >>>> >>>> Especially since this behavior was previously un observed and reproduced >>>> on >>>> 3 different identical hosts including one with a 'base' install. >>>> >>>> >>>> On Sat, Jul 5, 2014 at 4:11 PM, David Lang <da...@lang.hm> wrote: >>>> >>>> On Thu, 3 Jul 2014, Nick Syslog wrote: >>>> >>>>> >>>>> I've upgraded my VMWare based hosts to 8.2.2 recently and have had >>>>> some >>>>> >>>>> weird phenomena that I can't seemingly explain from the previous >>>>>> installation...specifically: >>>>>> >>>>>> OS: RHEL 6.3 >>>>>> -Starting/stopping the service using "service rsyslog restart" hangs >>>>>> unless >>>>>> a sleep is inserted between the stop/start execution in the init. >>>>>> >>>>>> >>>>>> remember that the stop signal does not instantly kill rsyslog, it >>>>> just >>>>> sends it a message asking it to shutdown gracefully, not corrupting >>>>> anything in the meantime. This means that rsylog works to flush the >>>>> messages it has in queues, close files cleanly, etc. There is a timeout >>>>> in >>>>> the rsyslog config that tells it that if it hasn't finished after X >>>>> seconds, abandon the remaining queue messages and just close things. >>>>> >>>>> If the start happens while rsyslog is still in the process of shutting >>>>> down, it finds that there is already a pidfile in place (meaning that >>>>> the >>>>> process is either still running, or crashed) and it asks the user if it >>>>> should go ahead and start or not (because if it is already running, >>>>> having >>>>> two copies manipulating the same files will cause "interesting" >>>>> results) >>>>> >>>>> David Lang >>>>> >>>>> The >>>>> >>>>> service appears to still start but the message "OK" is never released >>>>>> back >>>>>> to the cli leaving the user to ctrl-c to exit. >>>>>> >>>>>> -Messages for "exit signal 15" (normal shutdown messages) have >>>>>> disappeared >>>>>> completely leaving only the 'start' messages in their place and in >>>>>> some >>>>>> cases the message "DoDie Called." appears immediately prior to a start >>>>>> event. >>>>>> >>>>>> _______________________________________________ >>>>> >>>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
