Re: [rsyslog] mmnormalize thoughts

2015-03-11 Thread singh.janmejay
On Thu, Mar 12, 2015 at 9:19 AM, David Lang wrote: > On Thu, 12 Mar 2015, singh.janmejay wrote: > >> Tried re-ordering it? Put the one with /port first? > > > no, lognorm rules are not supposed to be order dependent, so I didn't try > that (especially after finding things failing to parse with rsy

Re: [rsyslog] mmnormalize thoughts

2015-03-11 Thread David Lang
On Thu, 12 Mar 2015, singh.janmejay wrote: Tried re-ordering it? Put the one with /port first? no, lognorm rules are not supposed to be order dependent, so I didn't try that (especially after finding things failing to parse with rsyslog that worked manually) Yes, rest must get atleast one

Re: [rsyslog] mmnormalize thoughts

2015-03-11 Thread singh.janmejay
Tried re-ordering it? Put the one with /port first? Yes, rest must get atleast one char to succeed. I'll create some new tests without rest-capture (and see what fails). On Thu, Mar 12, 2015 at 1:09 AM, David Lang wrote: > I just upgraded to liblognorm 1.1.1 (unfortunantly I didn't get a chance

Re: [rsyslog] liblognorm 1.1.1 released

2015-03-11 Thread singh.janmejay
Well, we can fix it to allow -j. In this case its easy because no global resources are involved. But in case of rsyslog tests it will be slightly harder (as listening port is involved). But that is where it matters more because this test-suite is fairly small :-). On Thu, Mar 12, 2015 at 2:15 AM,

[rsyslog] how do I tell if a variable is defined?

2015-03-11 Thread David Lang
how can I tell if a variable is defined. Where this is tripping me up On some of my relay boxes I set $!msg = $msg on my central box I do a json parse and then I want to detect if there is a $!msg variable (some things are sending me data that has always been in JSON format, so $!msg was neve

Re: [rsyslog] Replacing newlines in incoming messages

2015-03-11 Thread Troels Arvin
Hello Rainer, You wrote: > Pls read my previous message carefully: I think the LF *is* the message > delimiter. I've read it again, but I probably just don't understand your point. I should an example from /var/log/messages: == 201

Re: [rsyslog] liblognorm 1.1.1 released

2015-03-11 Thread Thomas D.
Hi, yes, Gentoo. And yes, parallel make issue. I should have checked that before. Sorry. All tests passed with "make -j1". Thanks! -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/profession

[rsyslog] liblognorm 1.1.1 doesn't work with rsyslog

2015-03-11 Thread David Lang
I'm running rsyslog 8.8.0.ad1 and upgraded from liblognorm 1.1.0 to 1.1.1 and discovered that parsing is not working If I run the parse manually with grep ASA-6-302016 /var/log/cisco |head -1|/usr/lib/lognorm/lognormalizer -r /root/rsyslog.rulebase -T it shows everything getting parsed proper

Re: [rsyslog] Spooling server per datacenter

2015-03-11 Thread David Lang
On Wed, 11 Mar 2015, smain...@free.fr wrote: Hi David, Thanks for your answer. Waiting to hear you again :) Actually my main issue is to avoid to spool on the source server and send all my logs to the spooling server. The key question you need to think about is: When things go badly wrong

Re: [rsyslog] Spooling server per datacenter

2015-03-11 Thread smainklh
Furthermore, i noticed some data loss when i use tcp without relp. In my use case we would like to avoid that if it's possible. Regards, Smana - Mail original - De: smain...@free.fr À: "rsyslog-users" Envoyé: Mercredi 11 Mars 2015 17:54:57 Objet: Re: [rsyslog] Spooling server per datace

Re: [rsyslog] Spooling server per datacenter

2015-03-11 Thread smainklh
Hi David, Thanks for your answer. Waiting to hear you again :) Actually my main issue is to avoid to spool on the source server and send all my logs to the spooling server. Regards, Smana - Mail original - De: "David Lang" À: "rsyslog-users" Envoyé: Mercredi 11 Mars 2015 15:45:22 Ob

Re: [rsyslog] Spooling server per datacenter

2015-03-11 Thread David Lang
here are some things to get you started. When I get to work today I can give you examples of my live configs. https://www.usenix.org/publications/login/august-2013-volume-38-number-4/enterprise-logging https://www.usenix.org/publications/login/october-2013-volume-38-number-5/log-filtering-rsyslo

Re: [rsyslog] Spooling server per datacenter

2015-03-11 Thread smainklh
Please let me know i you need more info. OS : debian wheezy rsyslog version : 8.8.0.ad1-1 Regards, Smana - Mail original - De: smain...@free.fr À: "rsyslog-users" Envoyé: Mercredi 11 Mars 2015 09:44:45 Objet: [rsyslog] Spooling server per datacenter Hi guys, Could you please help me

Re: [rsyslog] Use GnuTLS instead libgcrypt per default

2015-03-11 Thread Rainer Gerhards
I think if you disable it the encryption features (e.g for disk queues ) are disabled. That's not something I want by default. Sent from phone, thus brief. Am 11.03.2015 15:07 schrieb "Thomas D." : > Hi, > > currently libgcrypt is enabled per default. > > While reading the Debian build logs fro

Re: [rsyslog] failing test-suite for 8.8.8

2015-03-11 Thread Michael Biebl
2015-03-11 12:11 GMT+01:00 Michael Biebl : > I'll make another upload, which dumps the tests/test-suite.log to > stdout in case the test suite fails. New results are in If you go to https://buildd.debian.org/status/package.php?p=rsyslog&suite=experimental, and click on "Build-Attempted" in the St

Re: [rsyslog] Question about maximum queue.workerthreads count

2015-03-11 Thread paulo bruck
unsubscribe 2015-03-11 4:15 GMT-03:00 David Lang : > How fat a pipe do you have? that sounds like it's pushing the limit for > 10G interfaces. > > Also, remember that using multiple threads on the input module only helps > if you have multiple connections to you. A given connection can only reall

Re: [rsyslog] failing test-suite for 8.8.8

2015-03-11 Thread Michael Biebl
2015-03-11 7:51 GMT+01:00 Rainer Gerhards : > 2015-03-11 0:48 GMT+01:00 Michael Biebl : > >> FYI: After addressing the .rulesbase files and -lgcrypt linkage issue, >> I've uploaded the package to the Debian buildd network. >> The current state can be seen at >> https://buildd.debian.org/status/pack

[rsyslog] imptcp vs imtcp - TCP Syslog Interface binds vs. performance

2015-03-11 Thread Christopher . Racky
Hallo list, I have to setup a solution using rsyslog and TCP syslog sources. Therefore I made some tests using tmptcp and imtcp especially focusing on "dying" tcp connections. I realized, that imtcp does much more efficient handle death connections compared to imptcp. With imtcp they are dyin

[rsyslog] imptcp vs imtcp - TCP Syslog Interface binds vs. Performance

2015-03-11 Thread Christopher . Racky
___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad

[rsyslog] Spooling server per datacenter

2015-03-11 Thread smainklh
Hi guys, Could you please help me to find out the proper configuration for the following use case ? * We have multiple datacenters * All our logs are sent to a central analytic platform * In each dc we'd like to have a spooling server which will keep to logs in case of network failure. * All th

Re: [rsyslog] liblognorm 1.1.1 released

2015-03-11 Thread Florian Riedl
2015-03-11 0:49 GMT+01:00 Kendall Green : > When will rpm packages be available for latest release updates that provide > fixed liblognorm-1-1-1 and json-c-0.11-11.el6.x86_64 ??? > > Packages are already built and available. > On Tue, Mar 10, 2015 at 10:32 AM, Florian Riedl > wrote: > > > Hi a

Re: [rsyslog] Replacing newlines in incoming messages

2015-03-11 Thread Rainer Gerhards
2015-03-10 21:10 GMT+01:00 Troels Arvin : > Hello, > > Rainer Gerhards wrote: > > I think what happens here is that the message > > is sent via TCP syslog, and there LF is the *frame delimitor*. So we > > actually have a protocol error in this case - the LF indicates that a > > new message begins.

Re: [rsyslog] Replacing newlines in incoming messages

2015-03-11 Thread Rainer Gerhards
2015-03-10 19:59 GMT+01:00 Nathan Brown : > As a side note there is a feature in mlogger that will try and collapse > multiline log messages into a single message The -I option. This is > conjunction with the parser options I posted earlier will get you what you > want. > > https://github.com/nbro

Re: [rsyslog] Question about maximum queue.workerthreads count

2015-03-11 Thread David Lang
How fat a pipe do you have? that sounds like it's pushing the limit for 10G interfaces. Also, remember that using multiple threads on the input module only helps if you have multiple connections to you. A given connection can only really be handled by one thread (if you think about it, the pos