Re: [rsyslog] rsyslog 8.17.0 (v8-stable) released

[Thomas D.]

Hi,

I am getting test failures for the new timegenerated* tests on x86 only:

https://github.com/rsyslog/rsyslog/issues/873

Something bad?


-Thomas
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] rsyslog 8.17 mmnormalizer problem and characters change

[holo]

Hello

I'm trying to use such .rb file to parse logs:

version=2

rule=:%Server:char-to:\t%\t%stamp:char-to:\t%\t%ip:ipv4%\t%Site:char-to:\t%\t%BID:char-to:\t%\t%SID:char-to:\t%\t%LD:char-to:\t%\t%UserID:char-to:\t%\t%logged:char-to:\t%\t%event:char-to:\t%\t%User_Agent:char-to:\t%\t%Parameters:rest%
Problem is such for strings is working only "char-to" any other like 
"string-to" or "rest" don't want to work. I figure out most of the things with 
"char-to" but still need to take everything till end of line from one point. 
How can i do it?
Second thing is that in rsyslog 8.17 bellow settings are not working working:
Here is my example log where my \t characters are changed:
Mar 10 10:57:35 servername log_tag 
servername.at.google.com#01120160310105735#01144.44.44.443#011app#011162040-1441908796007#011918408-1457625732031#0110#0110#011N#011pageview#011Mozilla/5.0
 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 
Safari/537.36#011deviceClass=desktop#011pageName=login#011referrer=#011clickId=#011mseg=

I set up in main configuration file such option to let known rsyslog i don't 
want such change: 
global (
parser.escapeControlCharactersCStyle="off"
)

But rsyslog is still changing \t characters to #011 like you can see in log 
example and cos of that lognoromalizer not doing its job. How to force him to 
not to do it?




Regards
Robert
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] rsyslog 8.17.0 (v8-stable) released

[Brian Knox]

Thanks Thomas!

On Thu, Mar 10, 2016 at 10:32 AM, Thomas D.  wrote:

> Hi,
>
> Brian Knox wrote:
> > Will this fix be appearing in 8.18 in this case?  I just need to know so
> I
> > can plan on patching my local builds of 8.17 as impstats is pretty
> critical
> > for us and I'd rather have the fix than block the malformed stats lines.
> > Not a huge deal either way, just wondering.
>
> Rainer merged the fix today. So it will be in rsyslog-8.18.
>
>
> -Thomas
>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] rsyslog 8.17.0 (v8-stable) released

[Thomas D.]

Hi,

Brian Knox wrote:
> Will this fix be appearing in 8.18 in this case?  I just need to know so I
> can plan on patching my local builds of 8.17 as impstats is pretty critical
> for us and I'd rather have the fix than block the malformed stats lines.
> Not a huge deal either way, just wondering.

Rainer merged the fix today. So it will be in rsyslog-8.18.


-Thomas

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] problem with leap year and unixtimestamp template option

[Rainer Gerhards]

2016-03-09 23:02 GMT+01:00 Rainer Gerhards :
> This looks like a too old version of liblognorm. I need to check with the
> package guys why this is not automatically updated. Andre? Florian?

Well... I assumed you installed from our packages. But let's verify.
How exactly did you update rsyslog and which platform is this on?

Rainer
>
> Rainer
>
> Sent from phone, thus brief.
>
> Am 09.03.2016 19:33 schrieb "Joe Blow" :
>>
>> Hey all,
>>
>> After the update to 8.17 to fix this, i'm now getting this when
>> mmnormalize
>> loads:
>>
>> rsyslogd-2066: could not load module '/lib64/rsyslog/mmnormalize.so',
>> dlopen: /lib64/rsyslog/mmnormalize.so: undefined symbol: ln_setErrMsgCB
>> [v8.17.0 try http://www.rsyslog.com/e/2066 ]
>>
>> Any fix?  This basically broke our elasticsearch parsing.
>>
>> Thanks in advance.
>>
>> Cheers,
>>
>> JB
>>
>> On Wed, Mar 2, 2016 at 12:12 PM, Rainer Gerhards
>> 
>> wrote:
>>
>> > 2016-03-02 16:51 GMT+01:00 Peter Portante :
>> > > And it appears to also affect the "$date-ordinal" property.  Can you
>> > > confirm?
>> >
>> > Yeah, right because it uses the same helper function. But so the same
>> > fix fixes that issue as well. But... having said that, it's probably
>> > better to add some tests for it to the testbench as well.
>> >
>> > Thanks for bringing it up.
>> >
>> > Rainer
>> > >
>> > > On Wed, Mar 2, 2016 at 10:47 AM, Peter Portante <
>> > peter.a.porta...@gmail.com>
>> > > wrote:
>> > >
>> > >> Hi Rainer,
>> > >>
>> > >> Does this also affect ommongodb use?
>> > >>
>> > >> -peter
>> > >>
>> > >>
>> > >> On Wed, Mar 2, 2016 at 6:11 AM, Rainer Gerhards <
>> > rgerha...@hq.adiscon.com>
>> > >> wrote:
>> > >>
>> > >>> Hi all,
>> > >>>
>> > >>> I just wanted to let you know that we have found an issue with the
>> > >>> unixtimestamp formatting option in leap years. The problem started
>> > >>> on
>> > >>> March, 1st and will persist till the end of the year. The timestamp
>> > >>> is
>> > >>> one day in the past.
>> > >>>
>> > >>> A patch is available here, it shall probably work with almost all
>> > >>> rsyslog versions:
>> > >>>
>> > >>>
>> >
>> > https://github.com/rgerhards/rsyslog/commit/ffb321f1698a971e0acda48cafa97bb344cf0829
>> > >>>
>> > >>> Full details can be found in this issue tracker:
>> > >>>https://github.com/rsyslog/rsyslog/issues/830
>> > >>>
>> > >>> Note that I will NOT release a 8.16.1 version for this fix. Usually,
>> > >>> I
>> > >>> would have done so, but the 8.17.0 release is due next Tuesday and
>> > >>> will probably available as RC on friday. Given the fact that I would
>> > >>> need at least one additional day to craft 8.16.1, I don't think this
>> > >>> makes much sense.
>> > >>>
>> > >>> Distro packages (and other users as well, of course) can apply
>> > >>> above-mentioned short patch. I actually suggest to do so, as this
>> > >>> problem exists, I think, in all versions supporting the
>> > >>> "unixtimestamp" formatting option (side-note: v5.8 does not have
>> > >>> this
>> > >>> option).
>> > >>>
>> > >>> Rainer
>> > >>> ___
>> > >>> rsyslog mailing list
>> > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > >>> http://www.rsyslog.com/professional-services/
>> > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> > myriad
>> > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
>> > >>> you
>> > >>> DON'T LIKE THAT.
>> > >>>
>> > >>
>> > >>
>> > > ___
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > http://www.rsyslog.com/professional-services/
>> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> > > myriad
>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> > DON'T LIKE THAT.
>> > ___
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> > DON'T LIKE THAT.
>> >
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
___
rsyslog mailing list

Re: [rsyslog] problem with leap year and unixtimestamp template option

[Andre Lorbach]

I have to correct myself, liblognorm Version 1.1.2 or higher is needed.

Regards,
Andre

> -Ursprüngliche Nachricht-
> Von: Andre Lorbach [mailto:alorb...@adiscon.com]
> Gesendet: Donnerstag, 10. März 2016 09:43
> An: rsyslog-users
> Betreff: AW: [rsyslog] problem with leap year and unixtimestamp template
> option
>
> Which Version of liblognorm is installed?
> For rsyslog 8.17 we need newer version V1.1.3.
>
> Packages for Ubuntu, Redhat and Debian are up to date in our repositories.
>
>
> Best regards,
> Andre Lorbach
>
> > -Ursprüngliche Nachricht-
> > Von: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
> > boun...@lists.adiscon.com] Im Auftrag von Rainer Gerhards
> > Gesendet: Mittwoch, 9. März 2016 23:02
> > An: rsyslog-users
> > Betreff: Re: [rsyslog] problem with leap year and unixtimestamp
> > template option
> >
> > This looks like a too old version of liblognorm. I need to check with
> the
> > package guys why this is not automatically updated. Andre? Florian?
> >
> > Rainer
> >
> > Sent from phone, thus brief.
> > Am 09.03.2016 19:33 schrieb "Joe Blow" :
> >
> > > Hey all,
> > >
> > > After the update to 8.17 to fix this, i'm now getting this when
> > > mmnormalize
> > > loads:
> > >
> > > rsyslogd-2066: could not load module
> > > '/lib64/rsyslog/mmnormalize.so',
> > > dlopen: /lib64/rsyslog/mmnormalize.so: undefined symbol:
> > > ln_setErrMsgCB
> > > [v8.17.0 try http://www.rsyslog.com/e/2066 ]
> > >
> > > Any fix?  This basically broke our elasticsearch parsing.
> > >
> > > Thanks in advance.
> > >
> > > Cheers,
> > >
> > > JB
> > >
> > > On Wed, Mar 2, 2016 at 12:12 PM, Rainer Gerhards
> > >  > > >
> > > wrote:
> > >
> > > > 2016-03-02 16:51 GMT+01:00 Peter Portante
> > :
> > > > > And it appears to also affect the "$date-ordinal" property.  Can
> > > > > you confirm?
> > > >
> > > > Yeah, right because it uses the same helper function. But so the
> > > > same fix fixes that issue as well. But... having said that, it's
> > > > probably better to add some tests for it to the testbench as well.
> > > >
> > > > Thanks for bringing it up.
> > > >
> > > > Rainer
> > > > >
> > > > > On Wed, Mar 2, 2016 at 10:47 AM, Peter Portante <
> > > > peter.a.porta...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Hi Rainer,
> > > > >>
> > > > >> Does this also affect ommongodb use?
> > > > >>
> > > > >> -peter
> > > > >>
> > > > >>
> > > > >> On Wed, Mar 2, 2016 at 6:11 AM, Rainer Gerhards <
> > > > rgerha...@hq.adiscon.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi all,
> > > > >>>
> > > > >>> I just wanted to let you know that we have found an issue with
> > > > >>> the unixtimestamp formatting option in leap years. The problem
> > > > >>> started on March, 1st and will persist till the end of the year.
> > > > >>> The timestamp
> > > is
> > > > >>> one day in the past.
> > > > >>>
> > > > >>> A patch is available here, it shall probably work with almost
> > > > >>> all rsyslog versions:
> > > > >>>
> > > > >>>
> > > >
> > >
> >
> https://github.com/rgerhards/rsyslog/commit/ffb321f1698a971e0acda48caf
> > > a97bb344cf0829
> > > > >>>
> > > > >>> Full details can be found in this issue tracker:
> > > > >>>https://github.com/rsyslog/rsyslog/issues/830
> > > > >>>
> > > > >>> Note that I will NOT release a 8.16.1 version for this fix.
> > > > >>> Usually,
> > > I
> > > > >>> would have done so, but the 8.17.0 release is due next Tuesday
> > > > >>> and will probably available as RC on friday. Given the fact
> > > > >>> that I would need at least one additional day to craft 8.16.1,
> > > > >>> I don't think this makes much sense.
> > > > >>>
> > > > >>> Distro packages (and other users as well, of course) can apply
> > > > >>> above-mentioned short patch. I actually suggest to do so, as
> > > > >>> this problem exists, I think, in all versions supporting the
> > > > >>> "unixtimestamp" formatting option (side-note: v5.8 does not
> > > > >>> have this option).
> > > > >>>
> > > > >>> Rainer
> > > > >>> ___
> > > > >>> rsyslog mailing list
> > > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >>> http://www.rsyslog.com/professional-services/
> > > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
> > > > >>> by a
> > > > myriad
> > > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
> > > > >>> POST if
> > > you
> > > > >>> DON'T LIKE THAT.
> > > > >>>
> > > > >>
> > > > >>
> > > > > ___
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com/professional-services/
> > > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > > > NOTE
> > > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > > myriad
> > 

Re: [rsyslog] problem with leap year and unixtimestamp template option

[Andre Lorbach]

Which Version of liblognorm is installed?
For rsyslog 8.17 we need newer version V1.1.3.

Packages for Ubuntu, Redhat and Debian are up to date in our repositories.


Best regards,
Andre Lorbach

> -Ursprüngliche Nachricht-
> Von: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
> boun...@lists.adiscon.com] Im Auftrag von Rainer Gerhards
> Gesendet: Mittwoch, 9. März 2016 23:02
> An: rsyslog-users
> Betreff: Re: [rsyslog] problem with leap year and unixtimestamp template
> option
>
> This looks like a too old version of liblognorm. I need to check with
the
> package guys why this is not automatically updated. Andre? Florian?
>
> Rainer
>
> Sent from phone, thus brief.
> Am 09.03.2016 19:33 schrieb "Joe Blow" :
>
> > Hey all,
> >
> > After the update to 8.17 to fix this, i'm now getting this when
> > mmnormalize
> > loads:
> >
> > rsyslogd-2066: could not load module '/lib64/rsyslog/mmnormalize.so',
> > dlopen: /lib64/rsyslog/mmnormalize.so: undefined symbol:
> > ln_setErrMsgCB
> > [v8.17.0 try http://www.rsyslog.com/e/2066 ]
> >
> > Any fix?  This basically broke our elasticsearch parsing.
> >
> > Thanks in advance.
> >
> > Cheers,
> >
> > JB
> >
> > On Wed, Mar 2, 2016 at 12:12 PM, Rainer Gerhards
> >  > >
> > wrote:
> >
> > > 2016-03-02 16:51 GMT+01:00 Peter Portante
> :
> > > > And it appears to also affect the "$date-ordinal" property.  Can
> > > > you confirm?
> > >
> > > Yeah, right because it uses the same helper function. But so the
> > > same fix fixes that issue as well. But... having said that, it's
> > > probably better to add some tests for it to the testbench as well.
> > >
> > > Thanks for bringing it up.
> > >
> > > Rainer
> > > >
> > > > On Wed, Mar 2, 2016 at 10:47 AM, Peter Portante <
> > > peter.a.porta...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi Rainer,
> > > >>
> > > >> Does this also affect ommongodb use?
> > > >>
> > > >> -peter
> > > >>
> > > >>
> > > >> On Wed, Mar 2, 2016 at 6:11 AM, Rainer Gerhards <
> > > rgerha...@hq.adiscon.com>
> > > >> wrote:
> > > >>
> > > >>> Hi all,
> > > >>>
> > > >>> I just wanted to let you know that we have found an issue with
> > > >>> the unixtimestamp formatting option in leap years. The problem
> > > >>> started on March, 1st and will persist till the end of the year.
> > > >>> The timestamp
> > is
> > > >>> one day in the past.
> > > >>>
> > > >>> A patch is available here, it shall probably work with almost
> > > >>> all rsyslog versions:
> > > >>>
> > > >>>
> > >
> >
> https://github.com/rgerhards/rsyslog/commit/ffb321f1698a971e0acda48caf
> > a97bb344cf0829
> > > >>>
> > > >>> Full details can be found in this issue tracker:
> > > >>>https://github.com/rsyslog/rsyslog/issues/830
> > > >>>
> > > >>> Note that I will NOT release a 8.16.1 version for this fix.
> > > >>> Usually,
> > I
> > > >>> would have done so, but the 8.17.0 release is due next Tuesday
> > > >>> and will probably available as RC on friday. Given the fact that
> > > >>> I would need at least one additional day to craft 8.16.1, I
> > > >>> don't think this makes much sense.
> > > >>>
> > > >>> Distro packages (and other users as well, of course) can apply
> > > >>> above-mentioned short patch. I actually suggest to do so, as
> > > >>> this problem exists, I think, in all versions supporting the
> > > >>> "unixtimestamp" formatting option (side-note: v5.8 does not have
> > > >>> this option).
> > > >>>
> > > >>> Rainer
> > > >>> ___
> > > >>> rsyslog mailing list
> > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >>> http://www.rsyslog.com/professional-services/
> > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
> > > >>> a
> > > myriad
> > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> > > >>> if
> > you
> > > >>> DON'T LIKE THAT.
> > > >>>
> > > >>
> > > >>
> > > > ___
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com/professional-services/
> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > myriad
> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> > > you DON'T LIKE THAT.
> > > ___
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> > > you DON'T LIKE THAT.
> > >
> > ___
> > rsyslog mailing