Re: [rsyslog] FrosCon 2018

2018-07-30 Thread Brian Knox via rsyslog
Hello Rainer! That sounds like it would be great.. I'm already traveling in October and November this year, and I probably couldn't squeeze in another trip in 2018. I usually need quite a bit of notice to figure out my travel agenda - I'd be totally up for getting together with people in 2019 somet

Re: [rsyslog] omelasticsearch - failed operation handling

2018-05-17 Thread Brian Knox via rsyslog
To my knowledge, Rich is correct. This also would explain a case we hit maybe every couple of months, where rsyslog very quickly duplicates some messages it is sending to elasticsearch. I would assume this would be a case where a batch is submitted, only some of the messages are rejected, and rsysl

Re: [rsyslog] rsyslog 8.28.0 (v8-stable) released

2017-06-27 Thread Brian Knox via rsyslog
Just finished upgrading us to 8.27 *shakes fist* haha. Thanks for the update Florian! Definitely interested in the kafka output changes. Cheers, Brian On Tue, Jun 27, 2017 at 10:54 AM Florian Riedl wrote: > Hi all, > > We have released rsyslog 8.28.0. > > This release features a lot of changes.

Re: [rsyslog] configure / make flags for directory prefix

2017-05-08 Thread Brian Knox via rsyslog
Thank you so much Michael! Sometimes you just need another pair of eyes on something. On Sun, May 7, 2017 at 10:39 PM Michael Biebl wrote: > 2017-05-05 16:18 GMT+02:00 Brian Knox via rsyslog < > rsyslog@lists.adiscon.com>: > > I'm working on an fpm-cookery recipe ( >

[rsyslog] configure / make flags for directory prefix

2017-05-05 Thread Brian Knox via rsyslog
I'm working on an fpm-cookery recipe ( https://github.com/bernd/fpm-cookery ) for building rsyslog, and running into issues getting rsyslog to honor prefix variables. I'm stuck in that any combination of --prefix vars I use, rsyslog is trying to place rsyslog.service in /lib/systemd/rsyslog.servi

Re: [rsyslog] liblognorm 2.0.3 released

2017-03-24 Thread Brian Knox via rsyslog
The load rulebase from a string is nice! Will add that to my wrapper I'm using in normz ( https://github.com/taotetek/normz ). On Thu, Mar 23, 2017 at 1:04 PM Florian Riedl wrote: > Hi all, > > We have just released liblognorm 2.0.3. This new version provides some > fixes for the the annotate fu

Re: [rsyslog] rsyslog 8.24.0 (v8-stable) released

2017-01-10 Thread Brian Knox via rsyslog
Yay! Thanks Florian! It will probably be a week or two before I get this out into production, will provide feedback when it happens. On Tue, Jan 10, 2017 at 10:03 AM Florian Riedl wrote: > Hi everybody, > > we have released rsyslog 8.24.0. > > This first release for 2017 brings a lot of changes.

[rsyslog] omelasticsearch outstanding pr

2017-01-06 Thread Brian Knox via rsyslog
Wanted to double check on thoughts concerning this: https://github.com/rsyslog/rsyslog/pull/1331 It would be super useful to me if i could get it in the next release but I didn't want to just merge it without checking. Cheers, Brian ___ rsyslog mailing

[rsyslog] Merge to Master?

2017-01-03 Thread Brian Knox via rsyslog
I have a PR for omczmq and for omelasticsearch for the next release - I just wanted to double check that we merge to master now in the absence of master-candidate. Happy New Year! Brian ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/r

Re: [rsyslog] Are we building an ERK stack?

2016-12-15 Thread Brian Knox via rsyslog
I noticed looking through the code that it looks like the error file routine in omelasticsearch is not tied into the stats system - we use impstats to monitor our rsyslog pipelines, and having a counter for write errors would be super useful. I've submitted a PR to add the counter: https://github

[rsyslog] New omczmq features

2016-12-13 Thread Brian Knox via rsyslog
Just a heads up that I'm working on new features for the omczmq zeromq output. The WIP PR is here: https://github.com/rsyslog/rsyslog/pull/1325 The README updates cover the gist of it: https://github.com/taotetek/rsyslog/blob/020184ea2553e287f4e977d9245f9abe154567a2/contrib/omczmq/README Summary:

Re: [rsyslog] REK stack

2016-12-02 Thread Brian Knox
Hahaha. I agree. "erk" sounds like the noise I make when my elasticsearch cluster goes red on friday night when I was just headed out the door. ;) On Fri, Dec 2, 2016 at 5:57 AM Bob Gregory wrote: > Big +1, because "erk" sounds like the noise you make when somebody stands > on your toe at a form

Re: [rsyslog] Are we building an ERK stack?

2016-11-26 Thread Brian Knox
Seeing people referring to "REK" made me smile this morning. We've been referring to our setup as REK for quit awhile - http://www.meetup.com/RVA-Data-Hackers/events/214996202/ I don't really see a reason for redis in the equation unless it's already part of your infrastructure though. imkafka is

Re: [rsyslog] liblognorm vs grok

2016-10-19 Thread Brian Knox
Getting some ideas from reading this. Thank you! On Tue, Oct 18, 2016 at 3:22 AM Radu Gheorghe wrote: > It look very very very very nice, Rainer! Thanks for publishing! > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > On

Re: [rsyslog] Fwd: Re: rsyslog kills entire system => force reboot

2016-09-28 Thread Brian Knox
For what it is worth, I am running rsyslog 8.21 on around 12,000 servers and have not run into any issues with it. Cheers, Brian On Wed, Sep 28, 2016 at 3:20 AM Raffael Sahli wrote: > > > On 09/27/2016 01:02 PM, Andre Lorbach wrote: > > So far it seems to be very difficult to reproduce this pro

Re: [rsyslog] rsyslog dying if redis output fails

2016-08-31 Thread Brian Knox
Hello! I can try to reproduce and take a look in a day or too, thanks for the bug report! Brian On Wed, Aug 31, 2016 at 6:40 AM Angel L. Mateo wrote: > Hello, > > I'm trying to configure my rsyslog server (8.21.0) to send logs to > a > redis server. > > I'm using the configurati

Re: [rsyslog] mmnormalize rule database Re: mmgrok packages

2016-06-24 Thread Brian Knox
I am very much looking forward to the custom data type support! Safe travels Rainer! Brian On Fri, Jun 24, 2016 at 2:07 AM Rainer Gerhards wrote: > Thanks all for the great discussion and effort going forward! I am in > preparation for a trip next week and so unfortunately had limited time > t

Re: [rsyslog] mmnormalize rule database Re: mmgrok packages

2016-06-23 Thread Brian Knox
ter than collaboration on mailing lists so would prefer to just get something moving and we can take it from there. Cheers, Brian On Thu, Jun 23, 2016 at 1:09 PM David Lang wrote: > On Thu, 23 Jun 2016, Brian Knox wrote: > > > David - I'm sure I could get some time to devote to sh

Re: [rsyslog] mmnormalize rule database Re: mmgrok packages

2016-06-23 Thread Brian Knox
you. > > > > - Original Message - > From: "Ryan Ward" > To: "rsyslog-users" > Sent: Thursday, June 23, 2016 8:51:48 AM > Subject: Re: [rsyslog] mmnormalize rule database Re: mmgrok packages > > All as a newbie to rsyslog I think this is

Re: [rsyslog] mmnormalize rule database Re: mmgrok packages

2016-06-23 Thread Brian Knox
David - I'm sure I could get some time to devote to shepherding this, and I could get some time and resources from our community team to write some articles / tutorials about rsyslog + mmnormalize and generate some publicity for the project. Additionally I have access to a decently large sampling

Re: [rsyslog] rsyslog's batch api documentation inconsistency

2016-04-20 Thread Brian Knox
I think at least once is the best choice as well. On Wed, Apr 20, 2016 at 2:18 PM Kane Kim wrote: > Yes, we would strongly prefer at-least-once semantics. If everyone agrees > that it's desired behavior for rsyslog I'll try to contribute something in > that direction. > > On Wed, Apr 20, 2016 at

Re: [rsyslog] segfault with rsyslog Version > 8.16.0

2016-04-20 Thread Brian Knox
I'm in them middle of upgrading some high load servers to rsyslog 8.18 - and have some fairly complex configs on them. I'll keep an eye on on things today and report back how it goes. On Wed, Apr 20, 2016 at 12:31 PM David Lang wrote: > If you can get a coredump, a gdb analysis of the core fi

Re: [rsyslog] pre-releease of rsyslog 08.18.0 available

2016-04-18 Thread Brian Knox
Yay! I'll try doing a local build. Brian On Mon, Apr 18, 2016 at 8:59 AM, Rainer Gerhards wrote: > Hi all, > > I have just placed pre-release tarballs for tomorrow's upcoming > rsyslog 8.18.0 release on the server. > > * http://www.rsyslog.com/files/download/rsyslog/rc/rsyslog-8.18.0.tar.gz >

Re: [rsyslog] Timeline for 8.18 release

2016-04-13 Thread Brian Knox
that reminds me I need to get in a small patch! will do it this morning if it's not too late! On Wed, Apr 13, 2016 at 2:44 AM, Rainer Gerhards wrote: > 2016-04-12 18:31 GMT+02:00 Alec Swan : > > Thanks, Rainer, I was mostly looking at the github repo and missed the > > version field on http://w

Re: [rsyslog] rsyslog 8.17.0 (v8-stable) released

2016-03-10 Thread Brian Knox
Thanks Thomas! On Thu, Mar 10, 2016 at 10:32 AM, Thomas D. wrote: > Hi, > > Brian Knox wrote: > > Will this fix be appearing in 8.18 in this case? I just need to know so > I > > can plan on patching my local builds of 8.17 as impstats is pretty > critical > > f

Re: [rsyslog] rsyslog 8.17.0 (v8-stable) released

2016-03-09 Thread Brian Knox
Will this fix be appearing in 8.18 in this case? I just need to know so I can plan on patching my local builds of 8.17 as impstats is pretty critical for us and I'd rather have the fix than block the malformed stats lines. Not a huge deal either way, just wondering. Cheers, Brian On Wed, Mar 9,

Re: [rsyslog] rsyslog 8.17.0 (v8-stable) released

2016-03-09 Thread Brian Knox
Andrew - I'm setting up to do an 8.17 build today - will see if I can replicate. On Wed, Mar 9, 2016 at 11:04 AM, Andrew Davidoff wrote: > On Tue, Mar 8, 2016 at 9:54 AM, Florian Riedl wrote: > > Hi all, > > > > We have released rsyslog 8.17.0. > > I just started testing this release and with t

[rsyslog] rsyslog 8.17 release

2016-03-09 Thread Brian Knox
First: Yay! Can't wait to get this into production :) Second: There was a feature added to allow adding @cee tags to pure JSON files that are being consumed by imfile that I think was missed in the change log. Cheers, Brian ___ rsyslog mailing list

Re: [rsyslog] pre-releease of rsyslog 08.17.0 available

2016-03-07 Thread Brian Knox
Rainer - question - is it possible to build rsyslog without libjson-c now? I've built libfastjson - but I notice that liblognorm's latest stable release is still 1.1.2, which I believe still requires libjson-c. Brian On Mon, Mar 7, 2016 at 2:54 AM, Rainer Gerhards wrote: > The doc tarball is no

Re: [rsyslog] pre-releease of rsyslog 08.17.0 available

2016-03-07 Thread Brian Knox
I'll try our standard build with this today, thanks Rainer! On Mon, Mar 7, 2016 at 2:54 AM, Rainer Gerhards wrote: > The doc tarball is now also online: > > http://www.rsyslog.com/files/download/rsyslog/rc/rsyslog-doc-8.17.0.tar.gz > > Rainer > > 2016-03-07 7:47 GMT+01:00 Rainer Gerhards : > > H

Re: [rsyslog] imfile buffer overflow master-candidate

2016-03-03 Thread Brian Knox
Here we go - https://github.com/rsyslog/rsyslog/pull/840 On Thu, Mar 3, 2016 at 9:15 AM, Brian Knox wrote: > > https://github.com/rsyslog/rsyslog/blob/b5649a98107a8e6b7042e103f17bb16e907504f2/plugins/imfile/imfile.c#L686 > > Looks like getBasename should perhaps return a -1 if it

Re: [rsyslog] imfile buffer overflow master-candidate

2016-03-03 Thread Brian Knox
e for a fix today or tomorrow. Cheers, Brian On Thu, Mar 3, 2016 at 9:04 AM, Brian Knox wrote: > line 727 in imfile.c : > > memcpy(dirn, inst->pszFileName, i); /* do not copy slash */ > > > On Thu, Mar 3, 2016 at 8:53 AM, Brian Knox wrote: > >> I've fo

Re: [rsyslog] imfile buffer overflow master-candidate

2016-03-03 Thread Brian Knox
line 727 in imfile.c : memcpy(dirn, inst->pszFileName, i); /* do not copy slash */ On Thu, Mar 3, 2016 at 8:53 AM, Brian Knox wrote: > I've found a buffer overflow in imfile in the master-candidate branch. To > reproduce, make an imfile config that uses a relative pa

Re: [rsyslog] imfile buffer overflow master-candidate

2016-03-03 Thread Brian Knox
rgc=1, argv=0x7fffe688) at rsyslogd.c:1640 (gdb) frame 13 #13 0x0040dfe0 in main (argc=1, argv=0x7fffe688) at rsyslogd.c:1640 1640initAll(argc, argv); (gdb) print argc $1 = 1 (gdb) print argv $2 = (char **) 0x7fffe688 On Thu, Mar 3, 2016 at 8:53 AM, Brian Knox wrote: >

[rsyslog] imfile buffer overflow master-candidate

2016-03-03 Thread Brian Knox
I've found a buffer overflow in imfile in the master-candidate branch. To reproduce, make an imfile config that uses a relative path rather than absolute to a file: ``` module(load="imfile" PollingInterval="10") input( type="imfile" tag="crash" File="crashme" ) *.* /var/

Re: [rsyslog] feedback request on format-changing bugfix

2016-02-18 Thread Brian Knox
None from me. On Feb 18, 2016 3:28 AM, "Rainer Gerhards" wrote: > Hi all, > > I received a bug fix for impstats that changes the json-based formats > slighly. Please see: > > https://github.com/rsyslog/rsyslog/pull/798 > > Any concerns? > > Thanks, > Rainer > _

Re: [rsyslog] Transaction support for omkafka

2016-01-15 Thread Brian Knox
Hello Ralph! I'm the initial author of the omhiredis output. I haven't looked at the omkafka source code yet so I can't speak to it's complexity - but I can say that writing the transaction support for omhiredis was fairly simple. It was just implementing the beginTransaction / endTransaction macr

Re: [rsyslog] elasticsearch 2.0 and field names

2015-12-08 Thread Brian Knox
As a short term solution I'm working on a small service (in golang) that accepts logs over tcp, can replace characters in JSON field names in a @cee syslog line, and then forward the line to another syslog destination. In tests on my laptop it handles modifying ~ 50,000 reasonably sized log lines

Re: [rsyslog] elasticsearch 2.0 and field names

2015-12-05 Thread Brian Knox
problem that works with existing syslog capability. Cheers, Brian On Fri, Dec 4, 2015 at 3:28 PM, Peter Portante wrote: > On Fri, Dec 4, 2015 at 3:00 PM, David Lang wrote: > > > On Fri, 4 Dec 2015, Peter Portante wrote: > > > > On Fri, Dec 4, 2015 at 12:40 PM, Brian

Re: [rsyslog] elasticsearch 2.0 and field names

2015-12-04 Thread Brian Knox
> constant(value="\",\"hostname\":\"") > property(name="$.hostname") > constant(value="\",\"level\":\"") > property(name="$.level") > constant(value="\",\"pid\":\"&q

[rsyslog] elasticsearch 2.0 and field names

2015-12-04 Thread Brian Knox
I found out today that elasticsearch 2.x does not allow field names to have the period character in them. This is making my life interesting as I use elasticsearch with rsyslog end to end (no logstash), and a lot of our field names have "." as a delimiter in them. In a perfect world, I'd like an

Re: [rsyslog] rsyslog 8.14.0 (v8-stable) released

2015-11-03 Thread Brian Knox
Thanks for the heads up Florian! On Tue, Nov 3, 2015 at 11:16 AM, Florian Riedl wrote: > Hi all, > > We have released rsyslog 8.14.0. > > This is primarily a bug-fixing release with a couple of fixes for imfile > and Rainerscript. Also the property engine has now a new property: > rawmsg-after-p

Re: [rsyslog] RFC: dynamic-stats support

2015-10-06 Thread Brian Knox
I personally would argue that stats around the actual content of syslog messages is outside of the domain that rsyslog should be responsible for. impstats makes sense to me as it provides statistics around rsyslogs operation itself. Once I start wanting stats and counters around message content,

Re: [rsyslog] Log Shipping Format Poll Results + Docker Logging Webinar

2015-09-30 Thread Brian Knox
Hi Otis! Nice work on gathering the data! I tried to attend the webinar but the video conferencing software didn't work with my microphone or camera on linux :/ Brian On Wed, Sep 30, 2015 at 10:25 AM, Otis Gospodnetić < otis.gospodne...@gmail.com> wrote: > Hi, > > Thanks to anyone who voted! >

Re: [rsyslog] rsyslog 8.13.0 (v8-stable) released

2015-09-22 Thread Brian Knox
Yay! Thanks to everyone involved! On Tue, Sep 22, 2015 at 11:58 AM, Florian Riedl wrote: > Hi all, > > We have released rsyslog 8.13.0. > > This release sports a big number of changes. While most are bugfixes, there > are also some additions to existing functionality, most notably the > enhance

[rsyslog] last minute omhiredis fix

2015-09-21 Thread Brian Knox
Of course I found a bug in omhiredis after the merge - here's a fix! https://github.com/rsyslog/rsyslog/pull/531 ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsysl

Re: [rsyslog] Can we have a minimum bulk size for omelasticsearch?

2015-08-30 Thread Brian Knox
Limiting the number of batches within a given interval is conceptually similar to the index refresh rate setting in elasticsearch itself. At first blush I like this idea, as it is simple to understand the impact on when a given log line will be available within elasticsearch if I know this interva

Re: [rsyslog] Can we have a minimum bulk size for omelasticsearch?

2015-08-21 Thread Brian Knox
I'll see if I can find spare capacity to set up a test cluster. We use omelasticsearch heavily. On Aug 21, 2015 7:44 AM, "Radu Gheorghe" wrote: > On Fri, Aug 21, 2015 at 1:22 PM, Rainer Gerhards > > wrote: > > > 2015-08-21 12:19 GMT+02:00 Otis Gospodnetić >: > > > Hi, > > > > > > This sounds l

Re: [rsyslog] Filter on if json property exists

2015-08-13 Thread Brian Knox
Thanks David - that looks reasonable. I'll give it a try! On Thu, Aug 13, 2015 at 4:54 PM, David Lang wrote: > On Thu, 13 Aug 2015, Brian Knox wrote: > > Hello! I was working on an rsyslog configuration today and realized I've >> never run into a situation where I

[rsyslog] Filter on if json property exists

2015-08-13 Thread Brian Knox
Hello! I was working on an rsyslog configuration today and realized I've never run into a situation where I wanted to filter based on whether or not a property derived from a call to mmjsonparse exists or not. E.g., some logs will have "myprop" in their json, I want to route all logs that do one

Re: [rsyslog] Load balancing issue

2015-07-23 Thread Brian Knox
Aha! David - to summarize, is the problem then that: a) the parameter did not exist previously, and b) was only added for the new style configs? Brian On Thu, Jul 23, 2015 at 7:59 AM, David Lang wrote: > On Thu, 23 Jul 2015, Brian Knox wrote: > > From your diagram, it looks lik

Re: [rsyslog] Load balancing issue

2015-07-23 Thread Brian Knox
From your diagram, it looks like you are trying to load balance RELP. As far as I know, RELP does not suppot ActionTCPRebindInterval. I believe this has been discussed on the mailing list: http://lists.adiscon.net/pipermail/rsyslog/2013-May/032549.html Unless something has changed, you need to u

Re: [rsyslog] field extraction with v8 format syntax

2015-07-07 Thread Brian Knox
Aha found it - - field.number - obtain this field match - field.delimiter - decimal value of delimiter character for field extraction Thanks anyway! Brian On Tue, Jul 7, 2015 at 11:29 AM, Brian Knox wrote: > I wish to extract the second part of a hostname as a property. The &qu

[rsyslog] field extraction with v8 format syntax

2015-07-07 Thread Brian Knox
I wish to extract the second part of a hostname as a property. The "field replacer" ( where you set fromChar to "F" and toChar as the number of the field you wish to extract) looks to fit the bill for what I want. I realized I don't know how to specify fromChar / toChar in v8 format in a list tem

Re: [rsyslog] build system update

2015-06-23 Thread Brian Knox
Thanks for the update! Guess it's time to work on some packages :) On Tue, Jun 23, 2015 at 4:46 AM, Rainer Gerhards wrote: > Hi all, > > just a quick update: we have successfully migrated to the new build > system and are ready for team contributions. > > I plan to do a little bit of restructur

Re: [rsyslog] templates with overrides

2015-06-18 Thread Brian Knox
Interesting. I didn't know about this technique with exec_template. On Thu, Jun 18, 2015 at 1:33 PM, David Lang wrote: > On Thu, 18 Jun 2015, Rainer Gerhards wrote: > > 2015-06-18 8:31 GMT+02:00 David Lang : >> >>> There are so many cases where someone needs to override something in the >>> de

Re: [rsyslog] templates with overrides

2015-06-18 Thread Brian Knox
Yeah - I think a performance decrease would have to be incurred only when using this feature. There would need to be a way to still use static templates as well. I know I'd personally whether work around things the way I do now than take a performance hit across all templates. On Thu, Jun 18, 20

Re: [rsyslog] templates with overrides

2015-06-18 Thread Brian Knox
On the subject of templates - I have cases where being able to use different templates on the same output action would be advantageous. If an override variable as proposed here would accomplish that, it would solve a problem for me. It's something I work around in various ways now so it's not cri

Re: [rsyslog] packaging project

2015-06-14 Thread Brian Knox
Rainer - Thanks for the update! Enjoy your family time! Brian On Sun, Jun 14, 2015 at 8:14 AM, Rainer Gerhards wrote: > I have setup a new github repro: > > https://github.com/rsyslog/rsyslog-infrastructure > > As it's readme says, it shall cover all the task that are not specific > to the pa

Re: [rsyslog] rsyslog packaging project - was: rsyslog adiscon packages

2015-06-12 Thread Brian Knox
un 12, 2015 at 8:25 AM, Rainer Gerhards wrote: > 2015-06-11 23:06 GMT+02:00 Brian Knox : > > Florian - plan will probably depend on what we decide we're deploying. > > Andre has set up an account already ( Rainer may have the details as well > > ). We gave them access t

Re: [rsyslog] packaging project mission statement

2015-06-12 Thread Brian Knox
Excellent! I'll start in on the omczmq and imczmq packages when I get a little time! Cheers, Brian On Fri, Jun 12, 2015 at 8:17 AM, Rainer Gerhards wrote: > Hi all, > > I have amended the README.md with a mission statement plus some more > details: > > https://github.com/rsyslog/rsyslog-pkg-ub

Re: [rsyslog] rsyslog packaging project - was: rsyslog adiscon packages

2015-06-11 Thread Brian Knox
sible to determine who actually > built the packages, whereas this is possible with option 2, but there every > "builder" needs to have a key. This even applies to logins. > > Opinions and comments anyone? > > Florian > > 2015-06-09 19:32 GMT+02:00 Brian Knox :

Re: [rsyslog] rsyslog packaging project - was: rsyslog adiscon packages

2015-06-09 Thread Brian Knox
Rsyslog is very useful and appreciated :) On Tue, Jun 9, 2015 at 1:28 PM, Rainer Gerhards wrote: > Sent from phone, thus brief. > Am 09.06.2015 19:01 schrieb "Brian Knox" : > > > > Coordinating on the mailing list is fine with me. My employer > > (DigitalOcea

Re: [rsyslog] rsyslog packaging project - was: rsyslog adiscon packages

2015-06-09 Thread Brian Knox
t updates and compile test packages >>> >>> find . -name .git |sed s/.git// |while read file >>> do >>> echo "$file" >>> cd $file >>> /usr/bin/git fetch >>> /usr/bin/git pull >>> /usr/bin/git fetch --tags >>> #

Re: [rsyslog] Using Buildbot for rsyslog?

2015-06-08 Thread Brian Knox
The i3 window manager developers have a pretty good article explaining how they use buildbot that might be a good reference - http://i3wm.org/docs/buildbot.html I have no personal experience with buildbot, but I'd be glad to start reading and join in the fun. Brian On Mon, Jun 8, 2015 at 12:08 P

Re: [rsyslog] [RFC] Log-forward destination-cluster support

2015-06-04 Thread Brian Knox
David - I agree that RELP would be the right place for it. For TCP load balancing with rsyslog currently, I find using an external load balancer such as haproxy works nicely. Brian On Thu, Jun 4, 2015 at 1:40 PM, David Lang wrote: > If we do decide to do this, it would be better to base the w

Re: [rsyslog] [RFC] Log-forward destination-cluster support

2015-06-04 Thread Brian Knox
You might want to take a look at the omczmq and imczmq (the new ZeroMQ input and output plugins I've been working on). See - https://github.com/rsyslog/rsyslog/tree/master/contrib/omczmq and https://github.com/rsyslog/rsyslog/tree/master/contrib/imczmq "Out of the box" they currently support fan

Re: [rsyslog] rsyslog adiscon packages

2015-06-03 Thread Brian Knox
ently for the ubuntu repo. If there's build scripts for the current repo now, I'd be happy to work through them and do the work. Brian On Wed, Jun 3, 2015 at 11:38 AM, Rainer Gerhards wrote: > Would it be a good idea to start with Ubuntu? > > Sent from phone, thus brie

Re: [rsyslog] rsyslog adiscon packages

2015-06-03 Thread Brian Knox
I'm on board! Cheers, Brian On Wed, Jun 3, 2015 at 10:07 AM, Rainer Gerhards wrote: > Sent from phone, thus brief. > Am 03.06.2015 15:58 schrieb "Brian Knox" : > > > > I'm a member of the zeromq team :) > > I know ;) > > > What would I need

Re: [rsyslog] rsyslog adiscon packages

2015-06-03 Thread Brian Knox
I'm a member of the zeromq team :) What would I need to do? Brian On Wed, Jun 3, 2015 at 9:56 AM, Rainer Gerhards wrote: > 2015-06-03 14:50 GMT+02:00 Brian Knox : > > I've been working on the new zeromq plugins ( contrib/omczmq and > > contrib/imczmq) for a bit, an

[rsyslog] rsyslog adiscon packages

2015-06-03 Thread Brian Knox
I've been working on the new zeromq plugins ( contrib/omczmq and contrib/imczmq) for a bit, and have been using them in production myself for quite awhile. Yesterday, the zeromq organization released zeromq 4.1 ( http://zeromq.org/intro:get-the-software ) and czmq 3.0 ( http://czmq.zeromq.org/page

Re: [rsyslog] looking for an example of using rsyslog-pkg-ubuntu scripts

2015-04-22 Thread Brian Knox
If it's helpful to know at all - I've been building our rsyslog packages with a fork of brew2deb - which is a rather odd beast that combines homebrew, and fpm to build packages. In my case, I just build one rsyslog package that contains everything we need since I'm not building it for general cons

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-15 Thread Brian Knox
We keep our logs in JSON format and don't find it to be a drawback. We have logs searchable in elasticsearch - and for working with logs on disk, have a small program that logs can be piped through that strips out everything but the json which makes it very easy to pipe logs to jq (a command line

Re: [rsyslog] omkafka module for rsyslog

2015-03-12 Thread Brian Knox
pkg-config. > See the pkg-config man page for more details. > > I cannot find a liblogging-stdlog rpm in the official yum repositories. > > Regards Hans-Peter > -Original Message- > From: rsyslog-boun...@lists.adiscon.com [mailto: > rsyslog-boun...@lists.adiscon.c

Re: [rsyslog] omkafka module for rsyslog

2015-03-12 Thread Brian Knox
I can't speak to whether the module is packaged or not for your OS as I build my own rsyslog packages. If you are building rsyslog yourself, you need to make sure you have this library: https://github.com/edenhill/librdkafka And then ./configure --enable-omkafka should work fine. On Thu, Mar 12

Re: [rsyslog] upcoming log normalization work

2015-02-05 Thread Brian Knox
Congratulations, Rainer! This sounds like a great opportunity. Cheers, Brian On Thu, Feb 5, 2015 at 11:39 AM, Rainer Gerhards wrote: > Hi all, > > finally, I can dispense some word on upcoming work for liblognorm. > > The short story is that I will have ample time in the next months to > serio

Re: [rsyslog] plans for rsyslog 8.8

2015-01-16 Thread Brian Knox
Rainer - the pull model is something I want to add to the zeromq plugins as well. The idea being, if I have multiple downstream zeromq destinations, they can then request more logs as they are able to perform work on them - which of course allows you to load balance across downstream workers that

Re: [rsyslog] omkafka: Output Module for Kafka (v8 version)

2014-12-15 Thread Brian Knox
So 8.7 is aiming for Jan 13th? I'll try to get some zeromq input and output improvements done for that release then as well. Cheers, and enjoy your vacation! Brian On Mon, Dec 15, 2014 at 6:54 AM, Rainer Gerhards wrote: > > 2014-12-15 12:29 GMT+01:00 Brian Knox : > > > This

Re: [rsyslog] omkafka: Output Module for Kafka (v8 version)

2014-12-15 Thread Brian Knox
This is great, because I was thinking I should start on an output plugin for kafka, and now I can be lazy ;) I'll be glad to test this module against our kafka infrastructure at work! Brian On Mon, Dec 15, 2014 at 5:04 AM, Rainer Gerhards wrote: > > So here it is: > > https://github.com/rsyslog

Re: [rsyslog] mmnormalize improvements

2014-12-10 Thread Brian Knox
Rainer - do you have a link to a quick summary of the changes? Would love to know about them and haven't been paying attention due to other work priorities. Thanks! Brian On Wed, Dec 10, 2014 at 10:01 AM, Rainer Gerhards wrote: > Hi all, > > I have now merged Janmejay's awesome mmnormalize imp

Re: [rsyslog] cee json + mmsequence

2014-12-03 Thread Brian Knox
er, three questions ;) On Wed, Dec 3, 2014 at 6:04 AM, Brian Knox wrote: > So two questions then: > > 1. In the case of using mmjsonparse on messages where we do not know > ahead of time what all the. json attributes will be, is there a var for a > place in the tree where

Re: [rsyslog] cee json + mmsequence

2014-12-03 Thread Brian Knox
pointer to how to increment global variables to provide a sequence? 3. This morning I remembered the "mmcount" module - is this also being deprecated in favor of global variables? Brian On Wed, Dec 3, 2014 at 3:52 AM, Rainer Gerhards wrote: > 2014-12-02 21:59 GMT+01:00 Brian Kn

Re: [rsyslog] cee json + mmsequence

2014-12-02 Thread Brian Knox
ut we wanted to see if there was a cleaner way of doing it first! Brian On Tue, Dec 2, 2014 at 3:59 PM, Brian Knox wrote: > We have a case where we are receiving messages in @cee: format, and would > like to add a sequence variable generated via mmsequence to the message. > > Before I sp

[rsyslog] cee json + mmsequence

2014-12-02 Thread Brian Knox
We have a case where we are receiving messages in @cee: format, and would like to add a sequence variable generated via mmsequence to the message. Before I spend too much time digging into this I wanted to ask if, using some template hackery and json subtrees, this is something that sounds possibl

Re: [rsyslog] ruleset with only stop

2014-11-20 Thread Brian Knox
gt; I have now extended the -N option to support -N3, which can be used for > checking include files: > > > https://github.com/rsyslog/rsyslog/commit/a75eb98c058665d82f168f834b392683b30e001d > > It does not require the existence of actions. > > Rainer > > > Rai

Re: [rsyslog] ruleset with only stop

2014-11-19 Thread Brian Knox
> > > 2014-11-19 16:40 GMT+01:00 Brian Knox : > >> > >> Ok - perhaps we have accidently conflated two problems: > >>> > >>> 1) An empty ruleset > >>> 2) A ruleset with only "stop" > >>> >

Re: [rsyslog] ruleset with only stop

2014-11-19 Thread Brian Knox
rsyslog.com/e/2207 to learn what that number means) Brian On Wed, Nov 19, 2014 at 10:35 AM, Brian Knox wrote: > For verifying the problem I ran rsyslog -N1 -f against just the subset of > the config, if I recall correctly. I believe my coworker had the same > issue with the full co

Re: [rsyslog] ruleset with only stop

2014-11-19 Thread Brian Knox
gt; nothing else in the config? If not, can you send me the config, so that I > can try to see what's going on. > > I assume we agree that a totally action-less config is an error ;) > > Rainer > > 2014-11-11 22:49 GMT+01:00 Brian Knox : > > > If was able to use

Re: [rsyslog] Monitor rsyslog performance

2014-11-18 Thread Brian Knox
I have a service I wrote that we use, that accepts impstats output from rsyslog, calculates various metrics from them and can forward those metrics to opentsdb, graphite, etc - I could check into open sourcing it if anyone else might have a use for such a thing. Brian On Tue, Nov 18, 2014 at 1:17

Re: [rsyslog] tls + tcp input

2014-11-12 Thread Brian Knox
Sneaky - that makes sense. :) On Wed, Nov 12, 2014 at 9:17 AM, Rainer Gerhards wrote: > 2014-11-12 14:25 GMT+01:00 Brian Knox : > > > It looks like the parameters that control the auth mode are global: > > > > > ah, you are right. I just checked the code. I

Re: [rsyslog] liblognorm

2014-11-12 Thread Brian Knox
disable doc generation. > > Rainer > > 2014-11-12 15:14 GMT+01:00 Brian Knox : > > > Interesting - 1.0.0 builds fine - and 1.0.1 builds fine if I use the > > configure script from 1.0.0. > > > > Brian > > > > On Wed, Nov 12, 2014 at 9:06 AM, Brian Kn

Re: [rsyslog] liblognorm

2014-11-12 Thread Brian Knox
Interesting - 1.0.0 builds fine - and 1.0.1 builds fine if I use the configure script from 1.0.0. Brian On Wed, Nov 12, 2014 at 9:06 AM, Brian Knox wrote: > I'm trying to build the latest (1.0.1) version of liblognorm and getting > an error during configure - I'm wondering if

[rsyslog] liblognorm

2014-11-12 Thread Brian Knox
I'm trying to build the latest (1.0.1) version of liblognorm and getting an error during configure - I'm wondering if anyone else has run into this. I'm building on Ubuntu 14.04LTS: checking for JSON_C... yes checking for sphinx-build... no checking for sphinx-build3... no checking for sphinx-buil

Re: [rsyslog] tls + tcp input

2014-11-12 Thread Brian Knox
an input parameter? > > Sent from phone, thus brief. > Am 11.11.2014 23:11 schrieb "Brian Knox" : > > > Is it possible to specify TLS on a per input basis for the TCP input, or > is > > it all or nothing? > > > > Brian > > __

[rsyslog] tls + tcp input

2014-11-11 Thread Brian Knox
Is it possible to specify TLS on a per input basis for the TCP input, or is it all or nothing? Brian ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow h

Re: [rsyslog] ruleset with only stop

2014-11-11 Thread Brian Knox
If was able to use an empty ruleset, a warning resulting from that wouldn't bother me at all. Brian On Tue, Nov 11, 2014 at 4:25 PM, David Lang wrote: > On Tue, 11 Nov 2014, Rainer Gerhards wrote: > > 2014-11-11 17:22 GMT+01:00 David Lang : >> >> On Tue, 11

Re: [rsyslog] ruleset with only stop

2014-11-11 Thread Brian Knox
eld would > help significantly with. > > -- James > --- Sent from my mobile phone --- > > - Reply message - > From: "Rainer Gerhards" > To: "rsyslog-users" > Subject: [rsyslog] ruleset with only stop > Date: Tue, Nov 11, 2014 10:29 AM > > 2014-

Re: [rsyslog] ruleset with only stop

2014-11-11 Thread Brian Knox
On Tue, Nov 11, 2014 at 4:06 AM, Rainer Gerhards wrote: > 2014-11-10 16:23 GMT+01:00 Brian Knox : > > > Today I noticed a ruleset with only "stop" as it's action will fail to > > parse with rsyslog 8.4, but the same rule with a "~&qu

[rsyslog] ruleset with only stop

2014-11-10 Thread Brian Knox
Today I noticed a ruleset with only "stop" as it's action will fail to parse with rsyslog 8.4, but the same rule with a "~" will pass. ruleset(name="testme") { *.* ~ } bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf rsyslogd: version 8.5.0, config validation run (level 1), master config ./test.

Re: [rsyslog] can someone lend me a hand on git procedure

2014-11-07 Thread Brian Knox
I'm in favor of the process being as simple as possible. The more work and thought that has to be put into dealing with pull requests, the more likely either work will get slowed down, or someone will make a mistake. I feel like if I want to squash, etc I can just do that on branches on my own fo

  1   2   3   >