Re: [rsyslog] rsyslog next release and lookup-tables

[Rainer Gerhards]

2015-11-03 12:54 GMT+01:00  :
> Hello,
> As far as I see, today is the release data of the next rsyslog version.
> I did not see any changes about the lookup diffs, Janmejay promised, so I'm 
> quite nerverous that the new release will no longer contain the lookup-tables.

Please have a look here for status updates:
https://github.com/rsyslog/rsyslog/pull/544

In short: I won't remove it this release, as I have no longer been
tortured with CVEs and I think we can let it stand as is - NOT
officially existing - for a bit longer. I hope we can merge something
solid into the december or january relaese.

Rainer
>
> Please do not remove it, as it works fine (after the last patch) and I (and 
> possibly others) use it already in production.
> If it is needed I will help to document the functionality as it exists right 
> now.
>
> Best regards,
> Christopher
>
>
> 
>
>
>
>
>
>
>
>
> Hello,
> I have never heared such a nonsense.
> Actually the number of applications that does not include features that are 
> not official documented shoult be extremly limited.
>
> The functionality is really usefull and already in big landscapes productive.
> Please, please do NOT remove the lookup-table from the main branch.
> The functionaltiy works fine, I'm using this since march and I did not have 
> any issue since the latest patch of janmejay.
>
> Even the "concept" is not fully implemented (e.g. smaller things like 
> nomatch) the main part works fine.
>
>
> My suggestion would be to document everything which is currently implemented 
> and keep the "conceptual documentation" as it is.
> So the Maintainer should no longer have an issue with it.
>
>
> If the main issue it the time to document the already implemented features, I 
> can create a patch.
>
>
> Chris
>
>
>
>> Gesendet: Dienstag, 06. Oktober 2015 um 07:36 Uhr
>> Von: "David Lang" 
>> An: rsyslog-users 
>> Betreff: Re: [rsyslog] Separation of actions based on log source - with good 
>> performance
>>
>> a CVE for something that requires manually enabling an experimental 
>> feature???
>>
>> it would be one thing if a default config had the problem, or if it was
>> something entirely dependent on remote data.
>>
>> I would be very tempted to respond to the CVE with "don't enable this 
>> incomplete
>> feature" as the solution. It's very common for incomplete features to be
>> included in released versions
>>
>> grumble, we have enough real bugs to worry about.
>>
>> David Lang
>>
>> On Tue, 6 Oct 2015, Rainer Gerhards wrote:
>>
>> > Date: Tue, 6 Oct 2015 07:15:31 +0200
>> > From: Rainer Gerhards 
>> > Reply-To: rsyslog-users 
>> > To: rsyslog-users 
>> > Subject: Re: [rsyslog] Separation of actions based on log source - with 
>> > good
>> > performance
>> >
>> > Sorry, folks, good intent always seems to find someone who turns it
>> > into negative. I was yesterday contacted by a distro maintainer who
>> > wants to turn this bug in the officially non-existant lookup table
>> > feature into a CVE and insists that it is a vuln even after the
>> > argument that the feature never oficially existed.
>> >
>> > It looks like it was a bad idea to merge potentially useful yet
>> > incomplete code into the main branch (and documenting it to be not
>> > present). It looks like I need to re-think my stance on experimental
>> > features.
>> >
>> > Anyhow, I really don't want to support the argument that something
>> > non-existing can be a CVE. As such, I will create a new
>> > master-insecure branch, which will be a clone of the current master
>> > branch. Then I'll remove the lookup table code, so that the code base
>> > matches the documentation. I really don't want to create a general
>> > principle here that we need to create CVEs (and patched) for something
>> > that was just added as a convenience for a handful of folks who were
>> > ready to take a risk.
>> >
>> > If there is sufficient interest, we can consider officially adding
>> > this feature to the January 8.15 release iff it is ready by then.
>> > @janmejay: please let me know if you would like to continue with your
>> > work on lookup tables under this new situation.
>> >
>> > As soon as I have time, I'll check what else needs to be removed. Not
>> > sure about the ./contributed branch, because the project cannot
>> > guarantee at all this is bug-free. It's documented to be so, but if
>> > that is not sufficient, it should probably live only in the
>> > master-insecure branch.
>> >
>> > Rainer
>> >
>> > 2015-10-02 17:29 GMT+02:00 singh.janmejay :
>> >> As of now it returns empty string for no-match. I guess we should go ahead
>> >> with it in its current form. We can add default value any time later
>> >> without breaking 

Re: [rsyslog] mmpstrucdata doesn't seem to work

[Rainer Gerhards]

Mmm... There is no structured data in that message, hence nothing is
populated. The dash is the nilvalue.

Rainer

Sent from phone, thus brief.
Am 03.11.2015 17:47 schrieb "Radu Gheorghe" :

> Hi David,
>
> Here's how the debug template writes with a "server" config like the
> one I pasted in the first Email:
>
> Debug line with all properties:
> FROMHOST: 'localhost', fromhost-ip: '127.0.0.1', HOSTNAME:
> 'rgheorghe-ubuntu', PRI: 46,
> syslogtag 'rsyslogd', programname: 'rsyslogd', APP-NAME: 'rsyslogd',
> PROCID: '-', MSGID: '-',
> TIMESTAMP: 'Nov  3 18:38:09', STRUCTURED-DATA: '-',
> msg: ' [origin software="rsyslogd" swVersion="8.13.0" x-pid="1623"
> x-info="http://www.rsyslog.com;] start'
> escaped msg: ' [origin software="rsyslogd" swVersion="8.13.0"
> x-pid="1623" x-info="http://www.rsyslog.com;] start'
> inputname: imtcp rawmsg: '<46>1 2015-11-03T18:38:09.287115+02:00
> rgheorghe-ubuntu rsyslogd - - -  [origin software="rsyslogd"
> swVersion="8.13.0" x-pid="1623" x-info="http://www.rsyslog.com;]
> start'
> $!:
> $.:
> $/:
>
> So it sounds like rsyslog parses the RFC5424 message, but mmpstrucdata
> doesn't seem to put anything in that $! variable.
>
> Thanks and best regards,
> Radu
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Mon, Nov 2, 2015 at 7:45 PM, David Lang  wrote:
> > can you show us a same of the rawlog that you are receiving?
> >
> > among other things, it shows up with the template RSYSLOG_DebugFormat
> >
> > David Lang
> >
> > On Mon, 2 Nov 2015, Radu Gheorghe wrote:
> >
> >> Date: Mon, 2 Nov 2015 14:13:23 +0200
> >> From: Radu Gheorghe 
> >> Reply-To: rsyslog-users 
> >> To: rsyslog-users 
> >> Subject: [rsyslog] mmpstrucdata doesn't seem to work
> >>
> >>
> >> Hello rsysloggers :)
> >>
> >> I'm having trouble setting up mmpstrucdata (running on 64-bit Ubuntu
> >> 14.04 with rsyslog 8.13 installed from the official packages, if it
> >> matters).
> >>
> >> I've followed the docs
> >>
> >> (
> http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmpstrucdata.html
> )
> >> and I didn't get anything out of the $! or the $!rfc5424-sd variables.
> >> I've changed the jsonRoot path - still no go. Tried with $!all-json -
> >> nothing. And by "nothing", I mean "the contents of these variables are
> >> always empty". Except for $!all-json, which naturally shows an empty
> >> JSON.
> >>
> >> I see there is a test there on the testbench so I figured I must be
> >> doing something wrong, then I tried to emulate that. Here's my last
> >> (failed) attempt:
> >>
> >> Server config:
> >> -
> >> module(load="imtcp")
> >> module(load="mmpstrucdata")
> >>
> >> input(type="imtcp" port="514")
> >> action(type="mmpstrucdata")
> >>
> >> template(name="jsondump" type="string" string="%$!%\n")
> >>
> >> action(type="omfile"
> >>  file="/var/log/test"
> >>  template="jsondump")
> >> -
> >>
> >> Client config:
> >> 
> >> module(load="imuxsock")
> >>
> >> action(type="omfwd"
> >>  protocol="tcp"
> >>  target="127.0.0.1"
> >>  port="514"
> >>  template="RSYSLOG_SyslogProtocol23Format")
> >> 
> >>
> >> If I had to bet, I'd still go for me missing something (as I would
> >> expect the test to fail otherwise). Can someone confirm that
> >> mmpstrucdata still works on 8.13 and show an example config? Does
> >> anyone use this module at all? (I wouldn't blame anyone if they don't
> >> use it, I prefer JSON in the message anyway :p)
> >>
> >> Thanks and best regards,
> >> Radu
> >> --
> >> Performance Monitoring * Log Analytics * Search Analytics
> >> Solr & Elasticsearch Support * http://sematext.com/
> >> ___
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T
> >> LIKE THAT.
> >>
> > ___
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of
> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> > LIKE THAT.
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our 

Re: [rsyslog] rsyslog 8.14.0 (v8-stable) released

[Brian Knox]

Thanks for the heads up Florian!

On Tue, Nov 3, 2015 at 11:16 AM, Florian Riedl  wrote:

> Hi all,
>
> We have released rsyslog 8.14.0.
>
> This is primarily a bug-fixing release with a couple of fixes for imfile
> and Rainerscript. Also the property engine has now a new property:
> rawmsg-after-pri.
> For more details, please take a look at the Changelog.
>
> ChangeLog:
>
> http://www.rsyslog.com/changelog-for-8-14-0-v8-stable/
>
> Download:
>
> http://www.rsyslog.com/downloads/download-v8-stable/
>
> As always, feedback is appreciated.
>
> Best regards,
> Florian Riedl
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] mmpstrucdata doesn't seem to work

[David Lang]

I'm not seeing anything obviously wrong (and apologies for missing the sample in 
your first post)


have you checked for syntax errors in the config ( rsyslogd -N2 )?

David Lang
On Tue, 3 Nov 2015, Radu Gheorghe wrote:


Date: Tue, 3 Nov 2015 18:47:05 +0200
From: Radu Gheorghe 
Reply-To: rsyslog-users 
To: rsyslog-users 
Subject: Re: [rsyslog] mmpstrucdata doesn't seem to work

Hi David,

Here's how the debug template writes with a "server" config like the
one I pasted in the first Email:

Debug line with all properties:
FROMHOST: 'localhost', fromhost-ip: '127.0.0.1', HOSTNAME:
'rgheorghe-ubuntu', PRI: 46,
syslogtag 'rsyslogd', programname: 'rsyslogd', APP-NAME: 'rsyslogd',
PROCID: '-', MSGID: '-',
TIMESTAMP: 'Nov  3 18:38:09', STRUCTURED-DATA: '-',
msg: ' [origin software="rsyslogd" swVersion="8.13.0" x-pid="1623"
x-info="http://www.rsyslog.com;] start'
escaped msg: ' [origin software="rsyslogd" swVersion="8.13.0"
x-pid="1623" x-info="http://www.rsyslog.com;] start'
inputname: imtcp rawmsg: '<46>1 2015-11-03T18:38:09.287115+02:00
rgheorghe-ubuntu rsyslogd - - -  [origin software="rsyslogd"
swVersion="8.13.0" x-pid="1623" x-info="http://www.rsyslog.com;]
start'
$!:
$.:
$/:

So it sounds like rsyslog parses the RFC5424 message, but mmpstrucdata
doesn't seem to put anything in that $! variable.

Thanks and best regards,
Radu
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Nov 2, 2015 at 7:45 PM, David Lang  wrote:

can you show us a same of the rawlog that you are receiving?

among other things, it shows up with the template RSYSLOG_DebugFormat

David Lang

On Mon, 2 Nov 2015, Radu Gheorghe wrote:


Date: Mon, 2 Nov 2015 14:13:23 +0200
From: Radu Gheorghe 
Reply-To: rsyslog-users 
To: rsyslog-users 
Subject: [rsyslog] mmpstrucdata doesn't seem to work


Hello rsysloggers :)

I'm having trouble setting up mmpstrucdata (running on 64-bit Ubuntu
14.04 with rsyslog 8.13 installed from the official packages, if it
matters).

I've followed the docs

(http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmpstrucdata.html)
and I didn't get anything out of the $! or the $!rfc5424-sd variables.
I've changed the jsonRoot path - still no go. Tried with $!all-json -
nothing. And by "nothing", I mean "the contents of these variables are
always empty". Except for $!all-json, which naturally shows an empty
JSON.

I see there is a test there on the testbench so I figured I must be
doing something wrong, then I tried to emulate that. Here's my last
(failed) attempt:

Server config:
-
module(load="imtcp")
module(load="mmpstrucdata")

input(type="imtcp" port="514")
action(type="mmpstrucdata")

template(name="jsondump" type="string" string="%$!%\n")

action(type="omfile"
 file="/var/log/test"
 template="jsondump")
-

Client config:

module(load="imuxsock")

action(type="omfwd"
 protocol="tcp"
 target="127.0.0.1"
 port="514"
 template="RSYSLOG_SyslogProtocol23Format")


If I had to bet, I'd still go for me missing something (as I would
expect the test to fail otherwise). Can someone confirm that
mmpstrucdata still works on 8.13 and show an example config? Does
anyone use this module at all? (I wouldn't blame anyone if they don't
use it, I prefer JSON in the message anyway :p)

Thanks and best regards,
Radu
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with 

[rsyslog] gtls with rainerscript syntax?

[Jörgen Maas]

Hi guys,

I'm in the process of testing rsyslogd for a rather large logmanagement
environment.
So far i succeeded in accomplishing most tasks using the rainerscript
syntax, but for gnutls input/output the syntax is unclear, also cant seem
to find it in the docs.

I would appreciate it if someone could assist by supplying an example or
point me to the correct docs. I'm on rsyslog 7 as shipped with EL7.

Thanks!

Best regards,

Jörgen
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] mmpstrucdata doesn't seem to work

[Radu Gheorghe]

Hello and thanks for your replies!

@David: yes, I tried that and didn't see any config errors.

@Rainer: I thought this is strictured data: [origin
software="rsyslogd" swVersion="8.13.0"
x-pid="1623" x-info="http://www.rsyslog.com;]
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Tue, Nov 3, 2015 at 8:51 PM, Rainer Gerhards
 wrote:
> Mmm... There is no structured data in that message, hence nothing is
> populated. The dash is the nilvalue.
>
> Rainer
>
> Sent from phone, thus brief.
> Am 03.11.2015 17:47 schrieb "Radu Gheorghe" :
>
>> Hi David,
>>
>> Here's how the debug template writes with a "server" config like the
>> one I pasted in the first Email:
>>
>> Debug line with all properties:
>> FROMHOST: 'localhost', fromhost-ip: '127.0.0.1', HOSTNAME:
>> 'rgheorghe-ubuntu', PRI: 46,
>> syslogtag 'rsyslogd', programname: 'rsyslogd', APP-NAME: 'rsyslogd',
>> PROCID: '-', MSGID: '-',
>> TIMESTAMP: 'Nov  3 18:38:09', STRUCTURED-DATA: '-',
>> msg: ' [origin software="rsyslogd" swVersion="8.13.0" x-pid="1623"
>> x-info="http://www.rsyslog.com;] start'
>> escaped msg: ' [origin software="rsyslogd" swVersion="8.13.0"
>> x-pid="1623" x-info="http://www.rsyslog.com;] start'
>> inputname: imtcp rawmsg: '<46>1 2015-11-03T18:38:09.287115+02:00
>> rgheorghe-ubuntu rsyslogd - - -  [origin software="rsyslogd"
>> swVersion="8.13.0" x-pid="1623" x-info="http://www.rsyslog.com;]
>> start'
>> $!:
>> $.:
>> $/:
>>
>> So it sounds like rsyslog parses the RFC5424 message, but mmpstrucdata
>> doesn't seem to put anything in that $! variable.
>>
>> Thanks and best regards,
>> Radu
>> --
>> Performance Monitoring * Log Analytics * Search Analytics
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>> On Mon, Nov 2, 2015 at 7:45 PM, David Lang  wrote:
>> > can you show us a same of the rawlog that you are receiving?
>> >
>> > among other things, it shows up with the template RSYSLOG_DebugFormat
>> >
>> > David Lang
>> >
>> > On Mon, 2 Nov 2015, Radu Gheorghe wrote:
>> >
>> >> Date: Mon, 2 Nov 2015 14:13:23 +0200
>> >> From: Radu Gheorghe 
>> >> Reply-To: rsyslog-users 
>> >> To: rsyslog-users 
>> >> Subject: [rsyslog] mmpstrucdata doesn't seem to work
>> >>
>> >>
>> >> Hello rsysloggers :)
>> >>
>> >> I'm having trouble setting up mmpstrucdata (running on 64-bit Ubuntu
>> >> 14.04 with rsyslog 8.13 installed from the official packages, if it
>> >> matters).
>> >>
>> >> I've followed the docs
>> >>
>> >> (
>> http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmpstrucdata.html
>> )
>> >> and I didn't get anything out of the $! or the $!rfc5424-sd variables.
>> >> I've changed the jsonRoot path - still no go. Tried with $!all-json -
>> >> nothing. And by "nothing", I mean "the contents of these variables are
>> >> always empty". Except for $!all-json, which naturally shows an empty
>> >> JSON.
>> >>
>> >> I see there is a test there on the testbench so I figured I must be
>> >> doing something wrong, then I tried to emulate that. Here's my last
>> >> (failed) attempt:
>> >>
>> >> Server config:
>> >> -
>> >> module(load="imtcp")
>> >> module(load="mmpstrucdata")
>> >>
>> >> input(type="imtcp" port="514")
>> >> action(type="mmpstrucdata")
>> >>
>> >> template(name="jsondump" type="string" string="%$!%\n")
>> >>
>> >> action(type="omfile"
>> >>  file="/var/log/test"
>> >>  template="jsondump")
>> >> -
>> >>
>> >> Client config:
>> >> 
>> >> module(load="imuxsock")
>> >>
>> >> action(type="omfwd"
>> >>  protocol="tcp"
>> >>  target="127.0.0.1"
>> >>  port="514"
>> >>  template="RSYSLOG_SyslogProtocol23Format")
>> >> 
>> >>
>> >> If I had to bet, I'd still go for me missing something (as I would
>> >> expect the test to fail otherwise). Can someone confirm that
>> >> mmpstrucdata still works on 8.13 and show an example config? Does
>> >> anyone use this module at all? (I wouldn't blame anyone if they don't
>> >> use it, I prefer JSON in the message anyway :p)
>> >>
>> >> Thanks and best regards,
>> >> Radu
>> >> --
>> >> Performance Monitoring * Log Analytics * Search Analytics
>> >> Solr & Elasticsearch Support * http://sematext.com/
>> >> ___
>> >> rsyslog mailing list
>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> http://www.rsyslog.com/professional-services/
>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T
>> >> LIKE THAT.
>> >>
>> > ___
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up 

Re: [rsyslog] gtls with rainerscript syntax?

[Radu Gheorghe]

Hi Jörgen,

You can find the client config in this blog post:
http://blog.sematext.com/2014/03/25/encrypting-logs-on-their-way-to-elasticsearch-part-2-tls-syslog/

I suppose you can deduce the server config from that and the linked
howtos (which are old-style). If you can't, please let me know and
I'll dig for a server example.

Best regards,
Radu
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Tue, Nov 3, 2015 at 9:41 PM, Jörgen Maas  wrote:
> Hi guys,
>
> I'm in the process of testing rsyslogd for a rather large logmanagement
> environment.
> So far i succeeded in accomplishing most tasks using the rainerscript
> syntax, but for gnutls input/output the syntax is unclear, also cant seem
> to find it in the docs.
>
> I would appreciate it if someone could assist by supplying an example or
> point me to the correct docs. I'm on rsyslog 7 as shipped with EL7.
>
> Thanks!
>
> Best regards,
>
> Jörgen
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
> LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] rsyslog next release and lookup-tables

[singh.janmejay]

Sorry for the delay. Here is the PR: https://github.com/rsyslog/rsyslog/pull/578

On Tue, Nov 3, 2015 at 6:02 PM, Rainer Gerhards
 wrote:
> 2015-11-03 12:54 GMT+01:00  :
>> Hello,
>> As far as I see, today is the release data of the next rsyslog version.
>> I did not see any changes about the lookup diffs, Janmejay promised, so I'm 
>> quite nerverous that the new release will no longer contain the 
>> lookup-tables.
>
> Please have a look here for status updates:
> https://github.com/rsyslog/rsyslog/pull/544
>
> In short: I won't remove it this release, as I have no longer been
> tortured with CVEs and I think we can let it stand as is - NOT
> officially existing - for a bit longer. I hope we can merge something
> solid into the december or january relaese.
>
> Rainer
>>
>> Please do not remove it, as it works fine (after the last patch) and I (and 
>> possibly others) use it already in production.
>> If it is needed I will help to document the functionality as it exists right 
>> now.
>>
>> Best regards,
>> Christopher
>>
>>
>> 
>>
>>
>>
>>
>>
>>
>>
>>
>> Hello,
>> I have never heared such a nonsense.
>> Actually the number of applications that does not include features that are 
>> not official documented shoult be extremly limited.
>>
>> The functionality is really usefull and already in big landscapes productive.
>> Please, please do NOT remove the lookup-table from the main branch.
>> The functionaltiy works fine, I'm using this since march and I did not have 
>> any issue since the latest patch of janmejay.
>>
>> Even the "concept" is not fully implemented (e.g. smaller things like 
>> nomatch) the main part works fine.
>>
>>
>> My suggestion would be to document everything which is currently implemented 
>> and keep the "conceptual documentation" as it is.
>> So the Maintainer should no longer have an issue with it.
>>
>>
>> If the main issue it the time to document the already implemented features, 
>> I can create a patch.
>>
>>
>> Chris
>>
>>
>>
>>> Gesendet: Dienstag, 06. Oktober 2015 um 07:36 Uhr
>>> Von: "David Lang" 
>>> An: rsyslog-users 
>>> Betreff: Re: [rsyslog] Separation of actions based on log source - with 
>>> good performance
>>>
>>> a CVE for something that requires manually enabling an experimental 
>>> feature???
>>>
>>> it would be one thing if a default config had the problem, or if it was
>>> something entirely dependent on remote data.
>>>
>>> I would be very tempted to respond to the CVE with "don't enable this 
>>> incomplete
>>> feature" as the solution. It's very common for incomplete features to be
>>> included in released versions
>>>
>>> grumble, we have enough real bugs to worry about.
>>>
>>> David Lang
>>>
>>> On Tue, 6 Oct 2015, Rainer Gerhards wrote:
>>>
>>> > Date: Tue, 6 Oct 2015 07:15:31 +0200
>>> > From: Rainer Gerhards 
>>> > Reply-To: rsyslog-users 
>>> > To: rsyslog-users 
>>> > Subject: Re: [rsyslog] Separation of actions based on log source - with 
>>> > good
>>> > performance
>>> >
>>> > Sorry, folks, good intent always seems to find someone who turns it
>>> > into negative. I was yesterday contacted by a distro maintainer who
>>> > wants to turn this bug in the officially non-existant lookup table
>>> > feature into a CVE and insists that it is a vuln even after the
>>> > argument that the feature never oficially existed.
>>> >
>>> > It looks like it was a bad idea to merge potentially useful yet
>>> > incomplete code into the main branch (and documenting it to be not
>>> > present). It looks like I need to re-think my stance on experimental
>>> > features.
>>> >
>>> > Anyhow, I really don't want to support the argument that something
>>> > non-existing can be a CVE. As such, I will create a new
>>> > master-insecure branch, which will be a clone of the current master
>>> > branch. Then I'll remove the lookup table code, so that the code base
>>> > matches the documentation. I really don't want to create a general
>>> > principle here that we need to create CVEs (and patched) for something
>>> > that was just added as a convenience for a handful of folks who were
>>> > ready to take a risk.
>>> >
>>> > If there is sufficient interest, we can consider officially adding
>>> > this feature to the January 8.15 release iff it is ready by then.
>>> > @janmejay: please let me know if you would like to continue with your
>>> > work on lookup tables under this new situation.
>>> >
>>> > As soon as I have time, I'll check what else needs to be removed. Not
>>> > sure about the ./contributed branch, because the project cannot
>>> > guarantee at all this is bug-free. It's documented to be so, but if
>>> > that is not sufficient, it should probably live only in the
>>> > 

Re: [rsyslog] rsyslog next release and lookup-tables

[singh.janmejay]

Will cross-reference in the kill-feature PR.

On Tue, Nov 3, 2015 at 7:37 PM, singh.janmejay  wrote:
> Sorry for the delay. Here is the PR: 
> https://github.com/rsyslog/rsyslog/pull/578
>
> On Tue, Nov 3, 2015 at 6:02 PM, Rainer Gerhards
>  wrote:
>> 2015-11-03 12:54 GMT+01:00  :
>>> Hello,
>>> As far as I see, today is the release data of the next rsyslog version.
>>> I did not see any changes about the lookup diffs, Janmejay promised, so I'm 
>>> quite nerverous that the new release will no longer contain the 
>>> lookup-tables.
>>
>> Please have a look here for status updates:
>> https://github.com/rsyslog/rsyslog/pull/544
>>
>> In short: I won't remove it this release, as I have no longer been
>> tortured with CVEs and I think we can let it stand as is - NOT
>> officially existing - for a bit longer. I hope we can merge something
>> solid into the december or january relaese.
>>
>> Rainer
>>>
>>> Please do not remove it, as it works fine (after the last patch) and I (and 
>>> possibly others) use it already in production.
>>> If it is needed I will help to document the functionality as it exists 
>>> right now.
>>>
>>> Best regards,
>>> Christopher
>>>
>>>
>>> 
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Hello,
>>> I have never heared such a nonsense.
>>> Actually the number of applications that does not include features that are 
>>> not official documented shoult be extremly limited.
>>>
>>> The functionality is really usefull and already in big landscapes 
>>> productive.
>>> Please, please do NOT remove the lookup-table from the main branch.
>>> The functionaltiy works fine, I'm using this since march and I did not have 
>>> any issue since the latest patch of janmejay.
>>>
>>> Even the "concept" is not fully implemented (e.g. smaller things like 
>>> nomatch) the main part works fine.
>>>
>>>
>>> My suggestion would be to document everything which is currently 
>>> implemented and keep the "conceptual documentation" as it is.
>>> So the Maintainer should no longer have an issue with it.
>>>
>>>
>>> If the main issue it the time to document the already implemented features, 
>>> I can create a patch.
>>>
>>>
>>> Chris
>>>
>>>
>>>
 Gesendet: Dienstag, 06. Oktober 2015 um 07:36 Uhr
 Von: "David Lang" 
 An: rsyslog-users 
 Betreff: Re: [rsyslog] Separation of actions based on log source - with 
 good performance

 a CVE for something that requires manually enabling an experimental 
 feature???

 it would be one thing if a default config had the problem, or if it was
 something entirely dependent on remote data.

 I would be very tempted to respond to the CVE with "don't enable this 
 incomplete
 feature" as the solution. It's very common for incomplete features to be
 included in released versions

 grumble, we have enough real bugs to worry about.

 David Lang

 On Tue, 6 Oct 2015, Rainer Gerhards wrote:

 > Date: Tue, 6 Oct 2015 07:15:31 +0200
 > From: Rainer Gerhards 
 > Reply-To: rsyslog-users 
 > To: rsyslog-users 
 > Subject: Re: [rsyslog] Separation of actions based on log source - with 
 > good
 > performance
 >
 > Sorry, folks, good intent always seems to find someone who turns it
 > into negative. I was yesterday contacted by a distro maintainer who
 > wants to turn this bug in the officially non-existant lookup table
 > feature into a CVE and insists that it is a vuln even after the
 > argument that the feature never oficially existed.
 >
 > It looks like it was a bad idea to merge potentially useful yet
 > incomplete code into the main branch (and documenting it to be not
 > present). It looks like I need to re-think my stance on experimental
 > features.
 >
 > Anyhow, I really don't want to support the argument that something
 > non-existing can be a CVE. As such, I will create a new
 > master-insecure branch, which will be a clone of the current master
 > branch. Then I'll remove the lookup table code, so that the code base
 > matches the documentation. I really don't want to create a general
 > principle here that we need to create CVEs (and patched) for something
 > that was just added as a convenience for a handful of folks who were
 > ready to take a risk.
 >
 > If there is sufficient interest, we can consider officially adding
 > this feature to the January 8.15 release iff it is ready by then.
 > @janmejay: please let me know if you would like to continue with your
 > work on lookup tables under this new situation.
 >
 > As soon as I have time, I'll check 

[rsyslog] rsyslog 8.14.0 (v8-stable) released

[Florian Riedl]

Hi all,

We have released rsyslog 8.14.0.

This is primarily a bug-fixing release with a couple of fixes for imfile
and Rainerscript. Also the property engine has now a new property:
rawmsg-after-pri.
For more details, please take a look at the Changelog.

ChangeLog:

http://www.rsyslog.com/changelog-for-8-14-0-v8-stable/

Download:

http://www.rsyslog.com/downloads/download-v8-stable/

As always, feedback is appreciated.

Best regards,
Florian Riedl
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] mmpstrucdata doesn't seem to work

[Radu Gheorghe]

Hi David,

Here's how the debug template writes with a "server" config like the
one I pasted in the first Email:

Debug line with all properties:
FROMHOST: 'localhost', fromhost-ip: '127.0.0.1', HOSTNAME:
'rgheorghe-ubuntu', PRI: 46,
syslogtag 'rsyslogd', programname: 'rsyslogd', APP-NAME: 'rsyslogd',
PROCID: '-', MSGID: '-',
TIMESTAMP: 'Nov  3 18:38:09', STRUCTURED-DATA: '-',
msg: ' [origin software="rsyslogd" swVersion="8.13.0" x-pid="1623"
x-info="http://www.rsyslog.com;] start'
escaped msg: ' [origin software="rsyslogd" swVersion="8.13.0"
x-pid="1623" x-info="http://www.rsyslog.com;] start'
inputname: imtcp rawmsg: '<46>1 2015-11-03T18:38:09.287115+02:00
rgheorghe-ubuntu rsyslogd - - -  [origin software="rsyslogd"
swVersion="8.13.0" x-pid="1623" x-info="http://www.rsyslog.com;]
start'
$!:
$.:
$/:

So it sounds like rsyslog parses the RFC5424 message, but mmpstrucdata
doesn't seem to put anything in that $! variable.

Thanks and best regards,
Radu
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Nov 2, 2015 at 7:45 PM, David Lang  wrote:
> can you show us a same of the rawlog that you are receiving?
>
> among other things, it shows up with the template RSYSLOG_DebugFormat
>
> David Lang
>
> On Mon, 2 Nov 2015, Radu Gheorghe wrote:
>
>> Date: Mon, 2 Nov 2015 14:13:23 +0200
>> From: Radu Gheorghe 
>> Reply-To: rsyslog-users 
>> To: rsyslog-users 
>> Subject: [rsyslog] mmpstrucdata doesn't seem to work
>>
>>
>> Hello rsysloggers :)
>>
>> I'm having trouble setting up mmpstrucdata (running on 64-bit Ubuntu
>> 14.04 with rsyslog 8.13 installed from the official packages, if it
>> matters).
>>
>> I've followed the docs
>>
>> (http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmpstrucdata.html)
>> and I didn't get anything out of the $! or the $!rfc5424-sd variables.
>> I've changed the jsonRoot path - still no go. Tried with $!all-json -
>> nothing. And by "nothing", I mean "the contents of these variables are
>> always empty". Except for $!all-json, which naturally shows an empty
>> JSON.
>>
>> I see there is a test there on the testbench so I figured I must be
>> doing something wrong, then I tried to emulate that. Here's my last
>> (failed) attempt:
>>
>> Server config:
>> -
>> module(load="imtcp")
>> module(load="mmpstrucdata")
>>
>> input(type="imtcp" port="514")
>> action(type="mmpstrucdata")
>>
>> template(name="jsondump" type="string" string="%$!%\n")
>>
>> action(type="omfile"
>>  file="/var/log/test"
>>  template="jsondump")
>> -
>>
>> Client config:
>> 
>> module(load="imuxsock")
>>
>> action(type="omfwd"
>>  protocol="tcp"
>>  target="127.0.0.1"
>>  port="514"
>>  template="RSYSLOG_SyslogProtocol23Format")
>> 
>>
>> If I had to bet, I'd still go for me missing something (as I would
>> expect the test to fail otherwise). Can someone confirm that
>> mmpstrucdata still works on 8.13 and show an example config? Does
>> anyone use this module at all? (I wouldn't blame anyone if they don't
>> use it, I prefer JSON in the message anyway :p)
>>
>> Thanks and best regards,
>> Radu
>> --
>> Performance Monitoring * Log Analytics * Search Analytics
>> Solr & Elasticsearch Support * http://sematext.com/
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.