Re: [rsyslog] Rsyslog with queue files won't start
Not sure whether we will be able to upgrade in near future. It would be great to be possible to check the queue file consistency (at least). Is there any description of the format to check against? Or any tool available out there? On Thu, Mar 23, 2017 at 8:43 AM, Rainer Gerhards wrote: > You should move up to 8.25, chances are very good the issue is fixed > there. If not, we should see a better diagnostic (8.26 will have even > better diagnostics). > > Rainer > > 2017-03-23 8:40 GMT+01:00 Peter Viskup via rsyslog > : >> Just experienced the same issue again. >> Rsyslog version 8.15.0 crashed after the FS for queues got full. >> >> Mar 23 08:29:49 server kernel: [23586716.165832] rs:STRMEP2 >> queu[31413]: segfault at 1 ip 7f90dc66b34c sp 7f90d99daa40 >> error 4 in rsyslogd[7f90dc645000+9] >> >> Recovered most of the queue by removal of the oldest queue file. >> Removal of the queue file from the crash (the newest one) didn't help. >> >> -- >> Peter >> >> On Fri, Feb 17, 2017 at 9:02 AM, Peter Viskup wrote: >>> Sorry for not to mentioned. >>> The qi file was rebuilt. Once I moved the files out, the rsyslog >>> started just fine. I have a copy of the whole queue, thus can be used >>> for further investigation. >>> Are we able to check the consistency of queue files somehow? >>> >>> -- >>> Peter >>> >>> On Thu, Feb 16, 2017 at 3:44 PM, David Lang wrote: odds are that the queue files have been corrupted. you need to rebuild the .qi file and then it should be able to startup. David Lang On Thu, 16 Feb 2017, Peter Viskup via rsyslog wrote: > Date: Thu, 16 Feb 2017 13:50:37 +0100 > From: Peter Viskup via rsyslog > To: rsyslog-users > Cc: Peter Viskup > Subject: [rsyslog] Rsyslog with queue files won't start > > > Just experienced issue with rsyslog with DA queue files. > The process just died without any error. > We do run rsyslog version 8.15.0 > These are the last lines from debug output > > 8488.225603049:STRMEP2 queue[DA]:Reg/w0: omfwd: beginTransaction > 8488.225606660:STRMEP2 queue[DA]:Reg/w0: 10.1.25.181 > 8488.225610889:STRMEP2 queue[DA]:Reg/w0: Action 4 transitioned to state: > itx > 8488.225614575:STRMEP2 queue[DA]:Reg/w0: Action 4 transitioned to state: > rdy > 8488.225618011:STRMEP2 queue[DA]:Reg/w0: actionCommit, in retry loop, iRet > 0 > 8488.225622069:STRMEP2 queue[DA]:Reg/w0: regular consumer finished, > iret=0, szlog 128708 sz phys 128716 > 8488.225631031:STRMEP2 queue[DA]:Reg/w0: DeleteProcessedBatch: we > deleted 8 objects and enqueued 0 objects > 8488.225635210:STRMEP2 queue[DA]:Reg/w0: doDeleteBatch: delete batch > from store, new sizes: log 128708, phys 128708 > 8488.225672091:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 > read 4096 bytes > 8488.225760644:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 > read 4096 bytes > 8488.225864111:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 > read 4096 bytes > 8488.225966025:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 > read 4096 bytes > 8488.225977652:STRMEP2 queue[DA]:Reg/w0: error -2059 deserializing > property name, offset 208904, step 2 > 8488.225984716:STRMEP2 queue[DA]:Reg/w0: error property name: 'msg 8488.225991530:STRMEP2 queue[DA]:Reg/w0: error var type: '1' > > What's going wrong here? > > >> ___ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] liblognorm 2.0.3 released
The load rulebase from a string is nice! Will add that to my wrapper I'm using in normz ( https://github.com/taotetek/normz ). On Thu, Mar 23, 2017 at 1:04 PM Florian Riedl wrote: > Hi all, > > We have just released liblognorm 2.0.3. This new version provides some > fixes for the the annotate function and adds a test for it. A few > different issues have also been fixed. See the Changelog for details. > > Changelog:Version 2.0.3, 2017-03-22 > > - add ability to load rulebase from a string > introduces new API: > int ln_loadSamplesFromString(ln_ctx ctx, const char *string); > closes https://github.com/rsyslog/liblognorm/issues/239 > - bugfix: string parser did not correctly parse word at end of line > - bugfix: literal parser does not always store value if name is specified > if > rule=:%{"type":"literal", "text":"a", "name":"var"}% > is used and matching message is provided, variable var ist not persisted. > see also > http://lists.adiscon.net/pipermail/rsyslog/2016-December/043985.html > > Download: > http://www.liblognorm.com/download/liblognorm-2-0-3/ > > As always, feedback is appreciated. > > Best regards, > Florian Riedl > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] buildbot PR failures
Hi all, note that we are currently working on integrating back Solaris support into the build environment. For this reason, all PRs currently fail with the 4 Solaris tests. Please don't be concerned about this, I'll handle the situation. Typically a merge is without problems if just the Solaris ones fail. Rainer ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] REQUEST FOR COMMENTS: rsyslog code style
Hi all, I have finally made a shot at specifying a standardized code style for the rsyslog projects. It took quite a while, as I wanted to suggest something a) that is pretty common, b) where current code can be mostly automatic adjusted, c) can be automatically enforced in the future. I have finally found a way and am now looking for feedback. If there is no objection, I'll start to apply by April, 6th (or later). Note that once we made this big change, I will not accept style changes except of extremely important reasons. Note that I have not just taken my personal preference and turned that into a proposal. While there is a taste of it, I have also required some styles that I usually do not use. Style, samples, more info can be found in this PR: https://github.com/rsyslog/rsyslog/pull/1479 I would also appreciate if comments go there (and not the mailing list!) as I would like to keep this as single source of the decision process. Thanks, Rainer ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] $DirCreateMode ignored?
I'm running rsyslog 8.16.0-1ubuntu3 on Ubuntu 16.04. I have $DirCreateMode set to 0755: root@log01:/# grep -i dircre /etc/rsyslog.conf $DirCreateMode 0755 root@log01:/# grep -i dircre /etc/rsyslog.d/* /etc/rsyslog.d/05-accept-remote.conf:$DirCreateMode 0755 Unfortunately, directories automatically created (for data received from remote hosts) by rsyslogd are still 0700. Did I use "$DirCreateMode 0755" in a wrong way? If not - how do I debug this? If it matters - my template for accepting remote data is as below: template (name="RemoteMessage" type="string" string="%msg:2:$%\n") template (name="mypath" type="string" string="/var/log/remote/%$year%-%$month%-%$day%/%hostname%/%programname%") if ($inputname == 'imtcp' or $inputname == 'imudp') then { action(type="omfile" dynafile="mypath" template="RemoteMessage") stop } Also tried adding dirCreateMode="0755" to the template, but it didn't help and directories are still created with 0700 permissions: template (name="RemoteMessage" type="string" string="%msg:2:$%\n") template (name="mypath" type="string" dirCreateMode="0755" string="/var/log/remote/%$year%-%$month%-%$day%/%hostname%/%programname%") if ($inputname == 'imtcp' or $inputname == 'imudp') then { action(type="omfile" dynafile="mypath" template="RemoteMessage") stop } Tomasz Chmielewski https://lxadm.com ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] $DirCreateMode ignored?
2017-03-24 14:40 GMT+01:00 Tomasz Chmielewski via rsyslog : > I'm running rsyslog 8.16.0-1ubuntu3 on Ubuntu 16.04. > > I have $DirCreateMode set to 0755: > > root@log01:/# grep -i dircre /etc/rsyslog.conf > $DirCreateMode 0755 > > root@log01:/# grep -i dircre /etc/rsyslog.d/* > /etc/rsyslog.d/05-accept-remote.conf:$DirCreateMode 0755 > > > Unfortunately, directories automatically created (for data received from > remote hosts) by rsyslogd are still 0700. > > Did I use "$DirCreateMode 0755" in a wrong way? If not - how do I debug > this? Old-Style directives ($DirCreateMode) by design only affect old-style constructs (/path/to/file). So the action() statment needs to have the mode setting. > > > If it matters - my template for accepting remote data is as below: > > template (name="RemoteMessage" type="string" string="%msg:2:$%\n") > template (name="mypath" type="string" > string="/var/log/remote/%$year%-%$month%-%$day%/%hostname%/%programname%") > if ($inputname == 'imtcp' or $inputname == 'imudp') then { > action(type="omfile" dynafile="mypath" template="RemoteMessage") > stop > } > > > Also tried adding dirCreateMode="0755" to the template, but it didn't help > and directories are still created with 0700 permissions: > > template (name="RemoteMessage" type="string" string="%msg:2:$%\n") > template (name="mypath" type="string" dirCreateMode="0755" > string="/var/log/remote/%$year%-%$month%-%$day%/%hostname%/%programname%") You should also get error messages from rsyslog, because this is nothing you can do inside a template. A template just creates a string, and that's it. So you need to apply the setting to the action in question... > if ($inputname == 'imtcp' or $inputname == 'imudp') then { > action(type="omfile" dynafile="mypath" template="RemoteMessage") action(type="omfile" dynafile="mypath" template="RemoteMessage" dirCreateMode="0755") > stop > } > see the example in the doc: http://www.rsyslog.com/doc/v8-stable/configuration/modules/omfile.html HTH Rainer > > Tomasz Chmielewski > https://lxadm.com > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Rsyslog with queue files won't start
There is the tool that recreates the queue index, you can use that (I'm blanking on the name, something.qi) On Fri, 24 Mar 2017, Peter Viskup wrote: Not sure whether we will be able to upgrade in near future. It would be great to be possible to check the queue file consistency (at least). Is there any description of the format to check against? Or any tool available out there? On Thu, Mar 23, 2017 at 8:43 AM, Rainer Gerhards wrote: You should move up to 8.25, chances are very good the issue is fixed there. If not, we should see a better diagnostic (8.26 will have even better diagnostics). Rainer 2017-03-23 8:40 GMT+01:00 Peter Viskup via rsyslog : Just experienced the same issue again. Rsyslog version 8.15.0 crashed after the FS for queues got full. Mar 23 08:29:49 server kernel: [23586716.165832] rs:STRMEP2 queu[31413]: segfault at 1 ip 7f90dc66b34c sp 7f90d99daa40 error 4 in rsyslogd[7f90dc645000+9] Recovered most of the queue by removal of the oldest queue file. Removal of the queue file from the crash (the newest one) didn't help. -- Peter On Fri, Feb 17, 2017 at 9:02 AM, Peter Viskup wrote: Sorry for not to mentioned. The qi file was rebuilt. Once I moved the files out, the rsyslog started just fine. I have a copy of the whole queue, thus can be used for further investigation. Are we able to check the consistency of queue files somehow? -- Peter On Thu, Feb 16, 2017 at 3:44 PM, David Lang wrote: odds are that the queue files have been corrupted. you need to rebuild the .qi file and then it should be able to startup. David Lang On Thu, 16 Feb 2017, Peter Viskup via rsyslog wrote: Date: Thu, 16 Feb 2017 13:50:37 +0100 From: Peter Viskup via rsyslog To: rsyslog-users Cc: Peter Viskup Subject: [rsyslog] Rsyslog with queue files won't start Just experienced issue with rsyslog with DA queue files. The process just died without any error. We do run rsyslog version 8.15.0 These are the last lines from debug output 8488.225603049:STRMEP2 queue[DA]:Reg/w0: omfwd: beginTransaction 8488.225606660:STRMEP2 queue[DA]:Reg/w0: 10.1.25.181 8488.225610889:STRMEP2 queue[DA]:Reg/w0: Action 4 transitioned to state: itx 8488.225614575:STRMEP2 queue[DA]:Reg/w0: Action 4 transitioned to state: rdy 8488.225618011:STRMEP2 queue[DA]:Reg/w0: actionCommit, in retry loop, iRet 0 8488.225622069:STRMEP2 queue[DA]:Reg/w0: regular consumer finished, iret=0, szlog 128708 sz phys 128716 8488.225631031:STRMEP2 queue[DA]:Reg/w0: DeleteProcessedBatch: we deleted 8 objects and enqueued 0 objects 8488.225635210:STRMEP2 queue[DA]:Reg/w0: doDeleteBatch: delete batch from store, new sizes: log 128708, phys 128708 8488.225672091:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 read 4096 bytes 8488.225760644:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 read 4096 bytes 8488.225864111:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 read 4096 bytes 8488.225966025:STRMEP2 queue[DA]:Reg/w0: strm 0x7f0761c13b40: file 11 read 4096 bytes 8488.225977652:STRMEP2 queue[DA]:Reg/w0: error -2059 deserializing property name, offset 208904, step 2 8488.225984716:STRMEP2 queue[DA]:Reg/w0: error property name: 'msg ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] TCP Stops Local Logging
On RH 6 systems running rsyslog 5.8.10 we noticed that if we setup a client system to use TCP to log to a remote server: *.* @@192.168.1.2 If the remote log server is not reachable for some reason no logging takes place, not even local logging to the local system log files. When the log server is available and rsyslog is restarted both local logging and remote logging work. Is this a known issue or is there some way to ensure that local logging still occurs when the TCP remote server is down? ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] TCP Stops Local Logging
On Fri, 24 Mar 2017, Chris wrote: On RH 6 systems running rsyslog 5.8.10 we noticed that if we setup a client system to use TCP to log to a remote server: *.* @@192.168.1.2 If the remote log server is not reachable for some reason no logging takes place, not even local logging to the local system log files. When the log server is available and rsyslog is restarted both local logging and remote logging work. Is this a known issue or is there some way to ensure that local logging still occurs when the TCP remote server is down? This is working as designed (for the config you specified), if a message cannot be delivered to one destination, and you don't have rsyslog configured to throw it away, it is not able to finish processing that log message and start work on the next one. You can create an action queue for the delivery to a remote system, and until that queue fills up, other log processing will continue. You really should move to at least v7, if not v8, a lot of things have changed, especially the available syntax for specifying queues. David Lang ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.