Re: [rsyslog] ommongodb feedback

2012-03-17 Thread Brian Knox
I look > at (before doing the config options). > > Rainer > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Saturday, March 17, 2012 5:11 PM > > To

Re: [rsyslog] ommongodb feedback

2012-03-17 Thread Brian Knox
on.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Saturday, March 17, 2012 5:45 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] ommongodb feedback > > > > Just let me know if there's anything I can help test after you di

[rsyslog] rsyslog feature request - unix time

2012-03-20 Thread Brian Knox
Rainer - Any chance of getting a formatting property for the timestamp related properties that would allow spitting them out as a high resolution (milliseconds) and / or low resolution (seconds) 64 bit int (unix time)? %timestamp%:::unixmilliseconds %timestamp%:::unixseconds Something like that?

Re: [rsyslog] rsyslog feature request - unix time

2012-03-20 Thread Brian Knox
quot;the stuff" (you know what I mean ;)) undone. > > Will give it a shot asap ;) > Rainer > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Tuesday

[rsyslog] rsyslog plugin questions - template passing

2012-03-20 Thread Brian Knox
I'm working on an rsyslog output plugin and spent some time this morning reading the code for other plugins. I've figured out that rsyslog supports passing vars from the template as an array. In array passing mode, this seems to be the idiom to get at the passed values: szParams = (char**)(void*

Re: [rsyslog] rsyslog plugin questions - template passing

2012-03-20 Thread Brian Knox
Ok, now what I'm digging for is the equivalent of the below stanza, if I wish to use a template as a string and not parse it as an array: 189 if(pData->tplName == NULL) { 190 CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) strdup(" StdDBFmt"), 191 OMSR_TPL_AS_ARRAY)); 192 } e

[rsyslog] libee, libestr, liblognorm

2012-03-26 Thread Brian Knox
Rainer - I'm working on automating creating nightly rsyslog head builds for arch linux for my dev environment this week - I'm having a little trouble building some of the prerequisites (libee and libestr specifically) from the git repos. Could you fill me in on what combination of evil magic I ne

Re: [rsyslog] libee, libestr, liblognorm

2012-03-26 Thread Brian Knox
Aha! After speaking with Rainer we figured out I wasn't calling the right invocations - to build libestr from a git clone: autoreconf -fvi ./configure make On Mon, Mar 26, 2012 at 9:46 AM, Rainer Gerhards wrote: > This order: libestr, libee, liblognirm > > Brian Knox hat gesc

Re: [rsyslog] rsyslog feature request - unix time

2012-03-28 Thread Brian Knox
Rainer - I think you mentioned you had checked in this change now so wanted to check and also bother you for the property format option for unix time. Brian On Tue, Mar 20, 2012 at 6:45 AM, Brian Knox wrote: > I saw you had to do "boring" stuff today ;) > > On Tue, Mar

[rsyslog] json output from impstats

2012-03-30 Thread Brian Knox
If I wanted to throw together a cheap hack to have impstats log with cee enhanced rfc3164, I take it I could just hack up getStatsLine / getallStatsLines? Brian ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog

Re: [rsyslog] json output from impstats

2012-03-30 Thread Brian Knox
quot;name":"action 9","processed":4,"failed":0} 2012-03-30T17:29:19.972718-04:00 aenea rsyslogd-pstats: @cee: {"name":"action 10","processed":312,"failed":0} 2012-03-30T17:29:19.972739-04:00 aenea rsyslogd-pstats: @cee: {"name

Re: [rsyslog] json output from impstats

2012-03-31 Thread Brian Knox
tive. > > Rainer > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Friday, March 30, 2012 11:30 PM > > To: rsyslog-users > > Subject: Re: [rsyslog]

Re: [rsyslog] rsyslog build error

2012-04-04 Thread Brian Knox
Just did a fresh build of rsyslog head on archlinux, no issues. gcc 4.7.0 autconf 2.68 flex 2.5.35 bison 2.5 ./autogen.sh --enable-impstats \ --enable-omhiredis \ --enable-ommongodb \ --enable-debug \ --enable-imttcp \

Re: [rsyslog] Rsyslog splitting long-lines into multiple smaller one

2012-04-20 Thread Brian Knox
Have you tried this: http://syslog4j.org/ ? It at the least support TCP. On Fri, Apr 20, 2012 at 3:47 AM, Florian Crouzat wrote: > Le 19/04/2012 21:05, da...@lang.hm a écrit : > > there is a max message size parameter that you can set in rsyslog, make >> sure it's set long enough for your mes

Re: [rsyslog] error on app-name contains

2012-04-30 Thread Brian Knox
The files are read in inode order. At the last place I worked we instituted a policy that all rsyslog conf files in the .d directory needed to be completely self contained and not rely on any other part of the conf because of this. It'll save you some headaches. Brian On Fri, Apr 27, 2012 at 12

[rsyslog] action names

2012-05-02 Thread Brian Knox
I have an issue I'd like to solve concerning action names in the stats module. While I could solve it in a local fork, if it can be solved in a way that would be useful to others and incorporated back into rsyslog that's my preference. Here's some current output from a test configuration, with JS

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
.com] On Behalf Of Brian Knox > > Sent: Wednesday, May 02, 2012 12:20 PM > > To: rsyslog-users > > Subject: [rsyslog] action names > > > > I have an issue I'd like to solve concerning action names in the stats > > module. While I could solve it in a local for

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
Hmm just tried that and not seeing it take effect, let me poke at it for a few. Brian On Wed, May 2, 2012 at 7:00 AM, Rainer Gerhards wrote: > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Be

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
un...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Wednesday, May 02, 2012 1:02 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] action names > > > > Hmm just tried that and not seeing it take effect, let me poke at

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
le: either the "file" or "dynfile" parameter must be given [try http://www.rsyslog.com/e/2211 ] rsyslogd: errors occured in file '/etc/rsyslog.conf' around line 28 rsyslogd: warning: selector line without actions will be discarded rsyslogd: End of config va

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
t;:0} 2012-05-02T07:22:31.236303-04:00 localhost rsyslogd-pstats: @cee: {"name":"main Q","size":7,"enqueued":193,"full":0,"discarded.full":0," discarded.nf":0,"maxqsize":8} Although I've still got a bug somewhere

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
Aha! PStatSeverity instead of PStatsSeverity. Thanks Rainer! Brian On Wed, May 2, 2012 at 7:23 AM, Brian Knox wrote: > So I changed to this: > > > $ModLoad immark # provides -MARK- message capability > $ModLoad imuxsock # provides support for local system logg

[rsyslog] omfile output with templates

2012-05-02 Thread Brian Knox
Last question of the day, hopefully. I'm playing around with v6 config format and I'm having trouble passing a template to an omfile output action. Here's my sample config: $ModLoad immark # provides -MARK- message capability $ModLoad imuxsock # provides support for local system lo

Re: [rsyslog] omfile output with templates

2012-05-02 Thread Brian Knox
sses this action. 6541.656200401:7f4befcc4700: Called LogError, msg: Could not find template '??' - action disabled On Wed, May 2, 2012 at 9:45 AM, Rainer Gerhards wrote: > Can you post the relevant part oft he debug log? > > > -Original Message- > > From: rsy

Re: [rsyslog] omfile output with templates

2012-05-02 Thread Brian Knox
sap... > > Rainer > > > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Wednesday, May 02, 2012 3:52 PM > > To: rsyslog-users > > Subject: Re: [rsy

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
e. > > Rainer > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Wednesday, May 02, 2012 1:20 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] act

Re: [rsyslog] action names

2012-05-02 Thread Brian Knox
Found it and changed it locally (from "main Q" to mainQ) in the source for now. Brian On Wed, May 2, 2012 at 1:35 PM, Brian Knox wrote: > So - action names are working wonderfully. My only problem at this point > is that the main queue has a set name that I can't find a

Re: [rsyslog] rsyslog 5.8.11 (v5-stable) released

2012-05-03 Thread Brian Knox
Congratulations on another release! Brian On Thu, May 3, 2012 at 9:43 AM, Florian Riedl wrote: > Hi all, > > We have just released rsyslog 5.8.11 (v5-stable). > > This is primarily a maintenance release. It includes several bugfixes for > ommysql, imptcp or da-queues as well as other fixes. For

Re: [rsyslog] rsyslog 6.2.1 (v6-stable) released

2012-05-10 Thread Brian Knox
Thanks for the fixes! Brian On Thu, May 10, 2012 at 6:02 AM, Tim Eifler wrote: > Hi all, > > we have just release rsyslog 6.2.1 (v6-stable) > > This is a bug-fixing release. Updating is recommended for users of the > v6-stable branch. New users please note that in order to build this > release,

Re: [rsyslog] Advice on developing module to rsyslog 6

2012-05-10 Thread Brian Knox
Pablo: Just a heads up - I'm one of the people from Aggregate Knowledge who worked on the ZeroMQ modules. I'm currently at a new startup, and we internally have a ground up rewrite of the modules using the v6 configuration, and using the czmq api (rather than wrapping libzmq directly). We've jus

Re: [rsyslog] Advice on developing module to rsyslog 6

2012-05-10 Thread Brian Knox
; piece. > Please let me known when you have a tentative release schedule. Thanks > > > > > On 05/10/2012 12:39 PM, Brian Knox wrote: > >> Pablo: >> >> Just a heads up - I'm one of the people from Aggregate Knowledge who >> worked >> on the

[rsyslog] new zeromq plugins for rsyslog

2012-06-01 Thread Brian Knox
I just wanted to give everyone a heads up that our new zeromq input and output plugins for rsyslog are now in the official rsyslog repo, on head in the master branch ( http://www.rsyslog.com/doc/build_from_repo.html). There's a little info on our company blog about them ( http://www.talksum.com/blo

[rsyslog] rule blocks and mmjsonparse

2012-06-12 Thread Brian Knox
I have a situation on rsyslog 6 (dev branch!) where, using the new config format, I'd like to run only messages that match a filter condition through mmjsonparse. First of all, is this possible, and secondly, what would the syntax be? Essentially, in pseudoish code I want: if ($programname == 'r

Re: [rsyslog] rule blocks and mmjsonparse

2012-06-13 Thread Brian Knox
s_json.log" template="parse_test") } *.* :mmjsonparse: *.* /data/rsyslog/logs/json_parse.log;parse_test Given the above, json_parse.log contains empty lines, while json_parse.log is correct. On Tue, Jun 12, 2012 at 4:44 PM, Brian Knox wrote: > I have a situation on rsyslog

Re: [rsyslog] rule blocks and mmjsonparse

2012-06-13 Thread Brian Knox
uot;processed": "127", "failed": "127"} some json should be here: {"name": "main Q", "size": "13", "enqueued": "140", "full": "0", "discarded.full": "0", "

[rsyslog] statsobj.c

2012-06-13 Thread Brian Knox
Rainer - I'm working on getting imstats data emitted by the module into a "flattened" cee-like json list. (This is part of what I was doing earlier today when I emailed about the mmjsonparse issue). So (of course speaking of after mmjsonparse in a rsyslog 6 style block is working properly) if I

Re: [rsyslog] statsobj.c

2012-06-13 Thread Brian Knox
plate like... "{ \"foo\":\"bar\", \"baz\":\"bat\", %msg% }\n" On Wed, Jun 13, 2012 at 3:31 PM, Brian Knox wrote: > Rainer - > > I'm working on getting imstats data emitted by the module into a > "flattened" cee-like j

Re: [rsyslog] statsobj.c

2012-06-14 Thread Brian Knox
.. Need to think a bit about > that. > > Rainer > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Wednesday, June 13, 2012 9:32 PM > > To: rsyslog-us

Re: [rsyslog] %msg% is missing a part of the message received in %rawmsg%

2012-06-14 Thread Brian Knox
You also might want to give Syslog4j a look - http://syslog4j.org/ I don't know specifically if it will solve your issue but it can be used as a log4j appender. Brian On Thu, Jun 14, 2012 at 8:00 AM, Florian Crouzat wrote: > Le 14/06/2012 12:32, Rainer Gerhards a écrit : > >> >> >> -Origin

Re: [rsyslog] statsobj.c

2012-06-14 Thread Brian Knox
le","size":0,"enqueued":0,"full":0,"discarded.full":0," discarded.nf":0,"maxqsize":0 } which solves my problem for the moment. Brian On Thu, Jun 14, 2012 at 8:23 AM, Brian Knox wrote: > If you have large plans for improve

Re: [rsyslog] How to extract the value from a specific JSON field in a syslog message?

2012-07-10 Thread Brian Knox
Instead of using regex, it might be time for you to take a look at rsyslog 6. Rsysog 6 includes mmjsonparse, which is a module for parsing json messages and extracting their fields. In rsyslog 6.2, it's as simple as: *.* :mmjsonparse: $template mytemplate, "my field: %$!some_json_field%\n" On

[rsyslog] rsyslog 6 config format omfile output with templates

2012-07-11 Thread Brian Knox
Is it possible that applying a template to an omfile output using rsyslog 6 config format is currently bugged? Here's my conf: --- $ModLoad immark # provides --MARK-- messag

Re: [rsyslog] rsyslog 6 config format omfile output with templates

2012-07-11 Thread Brian Knox
template") } > > and it seems to work fine. > > Thanks > > -Original Message- > From: rsyslog-boun...@lists.adiscon.com [mailto: > rsyslog-boun...@lists.adiscon.com] On Behalf Of Brian Knox > Sent: Wednesday, July 11, 2012 10:07 AM > To: Rainer Gerhards; rsyslog-users >

Re: [rsyslog] rsyslog 6 config format omfile output with templates

2012-07-11 Thread Brian Knox
en a bug > > tracker so that I don't forget about this (it's currently rather busy). > > > > Rainer > > > > > -Original Message- > > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > > boun...@lists.adiscon.com] On Behalf Of Bri

[rsyslog] rsyslog 6 config format and omruleset

2012-07-13 Thread Brian Knox
I was wondering if there were any examples of how to properly use the omruleset module with the new rsyslog 6 config format. Thanks! Brian ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-serv

Re: [rsyslog] rsyslog 6 config format and omruleset

2012-07-13 Thread Brian Knox
I like "better capabilities" :) Thanks for the info, Brian On Fri, Jul 13, 2012 at 12:13 PM, Rainer Gerhards wrote: > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox >

Re: [rsyslog] adding a tag to log messages

2012-08-08 Thread Brian Knox
Off the top of my head, I'd say the mmnormalize plugin is probably a good one to look at to get some ideas. It definitely has access to the entire message as it's passing the message to liblognorm. mmjsonparse also works with the entire message to pass it to a json parser. Neither of these modul

Re: [rsyslog] adding a tag to log messages

2012-08-08 Thread Brian Knox
hash in? Brian On Wed, Aug 8, 2012 at 9:09 AM, Brian Knox wrote: > Off the top of my head, I'd say the mmnormalize plugin is probably a good > one to look at to get some ideas. It definitely has access to the entire > message as it's passing the message to liblognorm. mmj

Re: [rsyslog] new module interface needed for structured data?

2012-08-23 Thread Brian Knox
David - I agree that this would (questions about implementation aside) be immensely useful. Last time I looked at ommongo I wanted to, in the cases of rsyslog receiving a cee enhanced rfc3164 message, have the message embedded properly as a BSON subdocument in the msg field. However at the time

Re: [rsyslog] rsyslog versions

2012-08-24 Thread Brian Knox
I think that bumping the version to 7 is reasonable if you're making core engine changes. +1 from me. On Fri, Aug 24, 2012 at 5:27 AM, Rainer Gerhards wrote: > Hi all, > > quick question, full facts and reasoning are in my other posts from today: > > we are heading toward major core engine chang

Re: [rsyslog] new module interface needed for structured data?

2012-08-24 Thread Brian Knox
Rainer - I would have all sorts of immediate wonderful uses for this. I don't know what your projected time frame is - just know we'll be glad to take this feature for a spin in our lab this year when it's approaching usability. Brian On Fri, Aug 24, 2012 at 9:27 AM, Rainer Gerhards wrote: > >

Re: [rsyslog] rsyslog + zmq

2012-08-27 Thread Brian Knox
There is a new version of the zeromq rsyslog plugin that is currently included in the rsyslog source itself. It's on the head of the master branch - I don't know if it's been included in any 6.x beta releases yet. I highly recommend it over the code you linked, as there are a) a lot of improvement

Re: [rsyslog] rsyslog + zmq

2012-08-27 Thread Brian Knox
Oh nice! I can't wait to get 6.5 up in the lab. Brian On Mon, Aug 27, 2012 at 9:21 AM, Rainer Gerhards wrote: > > > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox >

Re: [rsyslog] JSON template option

2012-08-29 Thread Brian Knox
I say go for it. I use it in one place but I can work around any changes you make in the name of improving things. Brian On Wed, Aug 29, 2012 at 1:09 PM, Rainer Gerhards wrote: > For quite a while, there is the JSON template option, which came in via a > contribution and was obviously useful a

Re: [rsyslog] omzmq3 plugin again

2012-09-17 Thread Brian Knox
no clue ;)). Usually, you should receive a >> copy of mails you sent. >> >> I know - but i did not received it, that's because i was worried. >> Thanks for reply Rainer, do you know how i can contact omzmq3 plugin >> authors? > > Is there any email address

Re: [rsyslog] Sending logs message to redis does not work ?

2012-09-20 Thread Brian Knox
Hello! I'm the author of the redis output plugin - I'm sorry for the slow response; I've been rather busy lately. I've used the plugin for incrementing key counters based on rsyslog fields, and never actually tried using it for an output message queue this way before. Until I can do more work on

Re: [rsyslog] rsyslog 7.1.6 (v7-devel) released

2012-09-28 Thread Brian Knox
Thanks for the update Florian. I can't wait to get this set up to play with it! Brian On Fri, Sep 28, 2012 at 8:58 AM, Florian Riedl wrote: > Hi all, > > This version implements the input() and ruleset() statements, which finally > permit a config file to be written in new style only. A sample

Re: [rsyslog] Using RSYSLOG within a script

2012-10-02 Thread Brian Knox
http://php.net/manual/en/function.syslog.php On Tue, Oct 2, 2012 at 4:23 PM, Peeran, Syed wrote: > What about PHP is opening of rsyslog supported in PHP scripts such as the one > for syslog to read the logs > > openlog("myScriptLog", LOG_PID | LOG_PERROR, LOG_LOCAL0); > > OR > > Is there an API

Re: [rsyslog] rsyslog 7.1.9 (v7-beta) released

2012-10-09 Thread Brian Knox
Keep 'em coming! Thanks Florian On Tue, Oct 9, 2012 at 10:35 AM, Florian Riedl wrote: > Hi all, > > We have just released 7.1.9 of the v7-beta branch. This is a pure bugfixing > release. More information on the changes can be found in the ChangeLog. > > ChangeLog: > > http://www.rsyslog.com/cha

Re: [rsyslog] rsyslog 7.1.10 segfaults when using imzmq3

2012-10-16 Thread Brian Knox
Martin - We have not done testing with rsyslog 7 yet to any extent - just got it set up in our lab. I'll definitely keep an eye out for this issue and let you know (we're on CentOS as well). Thanks for the feedback! Brian On Tue, Oct 16, 2012 at 9:05 AM, Martin Nilsson wrote: > Hi, > > I'm tr

Re: [rsyslog] rsyslog 7.1.10 segfaults when using imzmq3

2012-10-16 Thread Brian Knox
ich rsyslog version would be the recommended one > right now if you want to use the zeromq plugins? > > Regards, > /Martin > > > > > From: Brian Knox > To: rsyslog-users , > Date: 2012-10-16 15:15 > Martin - > > We have not done testing with rsyslog

Re: [rsyslog] rsyslog 7.1.10 segfaults when using imzmq3

2012-10-17 Thread Brian Knox
log 6.5.0 and 6.5.1 now with the same results, so > I think it might be related to some of the other libraries (czmq, > libzmq3). Could you tell me the versions you are using in a working setup? > > Regards, > /Martin > > > > > From: Brian Knox > To: rsyslog

Re: [rsyslog] rsyslog 7.1.10 segfaults when using imzmq3

2012-10-17 Thread Brian Knox
eported:1:3:date-subseconds%\",\"tag\":\"%syslogtag:::json%\"}" > > *.* action(name="test_zmq_to_zmq" type="omzmq3" > description="tcp://127.0.0.1:" socktype="PUSH" action="CONNECT" > template="js

[rsyslog] ruleset() {} and call

2012-10-26 Thread Brian Knox
I'm trying to test out rsyslog 7 and I cannot find any documentation for the proper syntax for using ruleset and call directives. Are there any examples of the new syntax available anywhere? Thanks! Brian ___ rsyslog mailing list http://lists.adiscon.ne

Re: [rsyslog] ruleset() {} and call

2012-10-26 Thread Brian Knox
So, now I just need to figure out "call" Brian On Fri, Oct 26, 2012 at 4:03 PM, Brian Knox wrote: > I'm trying to test out rsyslog 7 and I cannot find any documentation > for the proper syntax for using ruleset and call directives. A

Re: [rsyslog] ruleset() {} and call

2012-10-26 Thread Brian Knox
input(type="imptcp" port="13514" ruleset="test") - I get an error on this one - I'm guessing I'm almost but not quite right. Brian On Fri, Oct 26, 2012 at 4:41 PM, Rainer Gerhards wrote: > call rulesetname > > &

Re: [rsyslog] ruleset() {} and call

2012-10-26 Thread Brian Knox
log.com/e/2207 to learn what that number means) On Fri, Oct 26, 2012 at 4:54 PM, Brian Knox wrote: > Should this work? > - > module(load="imptcp" keepalive="on") > > ruleset(name="test1") { &g

Re: [rsyslog] ruleset() {} and call

2012-10-27 Thread Brian Knox
03ed622b9b61 > > The git v7-stable branch now also has a -D option, which enabled Bison > grammar debugging (may be useful for some of you, and for me inside a debug > log ;)). > > I will release all this on Monday as 7.2.1. > > Rainer > >> -Original

Re: [rsyslog] ruleset() {} and call

2012-10-30 Thread Brian Knox
oke around and see what I can find, I know I remember reading some stuff about it. Brian On Sat, Oct 27, 2012 at 1:57 PM, Brian Knox wrote: > Nice, thanks! > > Brian > > On Sat, Oct 27, 2012 at 9:07 AM, Rainer Gerhards > wrote: >> That was a dumb mistake in the gr

Re: [rsyslog] ruleset() {} and call

2012-10-30 Thread Brian Knox
Oh fantastic! I didn't realize the template documentation had been updated - thank you. Today will be fun. Brian On Tue, Oct 30, 2012 at 7:38 AM, Rainer Gerhards wrote: >> -Original Message- >> From: Brian Knox [mailto:bri...@talksum.com] >> Sent: Tuesday, Oc

Re: [rsyslog] Patch for imzmq3

2012-10-30 Thread Brian Knox
Eyeballing it it looks good to me as well. Let me run it by Dave real quick and I'll get back to the list! Brian On Tue, Oct 30, 2012 at 10:27 AM, Rainer Gerhards wrote: > Brian, > > any thought's. From brief look (and without zmq knowledge), this looks pretty > good to me ;) > > Rainer > >> -

Re: [rsyslog] Patch for imzmq3

2012-10-30 Thread Brian Knox
Patch looks good to us - Thank you! Brian On Tue, Oct 30, 2012 at 11:14 AM, Brian Knox wrote: > Eyeballing it it looks good to me as well. Let me run it by Dave real > quick and I'll get back to the list! > > Brian > > On Tue, Oct 30, 2012 at 10:27 AM, Rainer Gerh

[rsyslog] subtree templates

2012-10-30 Thread Brian Knox
I'm playing with the new template syntax today and for the most part am having a lot of success. The one thing I find a bit confusing still is working with the new subtree template types. Given a subtree that simply includes all properties: template(name="normalizeTest" type="subtree" subtree="$

Re: [rsyslog] subtree templates

2012-10-31 Thread Brian Knox
-- >> From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- >> boun...@lists.adiscon.com] On Behalf Of Brian Knox >> Sent: Tuesday, October 30, 2012 6:33 PM >> To: rsyslog-users >> Subject: [rsyslog] subtree templates >> >> I'm playing with the new te

[rsyslog] new "call" syntax and templates

2012-10-31 Thread Brian Knox
Is there a way to apply a template to a message that is being passed to another ruleset via call? It may be best to describe my actual desired result: Say I have rule A, and in that rule I make a call to mmjsonparse. After mmjsonparse, I call rule B In rule B, are the properties derived from mmjs

Re: [rsyslog] rsyslog queue subsystem - refactor or redesign?

2012-10-31 Thread Brian Knox
I have two initial thoughts from experience concerning heavy use of the queue system in high traffic (6+ billion log lines a day) environments - specifically concerning DA queues. 1. The queues for the most part were very reliable 2. The performance unspooling from a "fail" state (defined as "we h

Re: [rsyslog] new "call" syntax and templates

2012-10-31 Thread Brian Knox
Aha! This does clarify and it works like I hoped it would! Thank you, Brian On Wed, Oct 31, 2012 at 1:06 PM, Rainer Gerhards wrote: > > >> -Original Message- >> From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- >> boun...@lists.adiscon.com] On Beha

Re: [rsyslog] rsyslog queue subsystem - refactor or redesign?

2012-10-31 Thread Brian Knox
ource of the issues had to have been specifically the queue code and not something about rsyslog in general. Brian On Wed, Oct 31, 2012 at 5:27 PM, wrote: > On Wed, 31 Oct 2012, Brian Knox wrote: > >> I have two initial thoughts from experience concerning heavy use of >> t

Re: [rsyslog] rsyslog queue subsystem - refactor or redesign?

2012-11-01 Thread Brian Knox
This sounds similar to jsmn, a json tokenizer I've been playing with this week ( http://zserge.bitbucket.org/jsmn.html ). It gives you a struct per token, and each struct contains the type of the token (string, primitive, object or array) and offsets to the start and end of the token in the origina

Re: [rsyslog] rsyslog queue subsystem - refactor or redesign?

2012-11-01 Thread Brian Knox
..@lists.adiscon.com] On Behalf Of Brian Knox >> Sent: Wednesday, October 31, 2012 6:12 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog queue subsystem - refactor or redesign? >> >> I have two initial thoughts from experience concerning heavy use of >>

Re: [rsyslog] new "call" syntax and templates

2012-11-01 Thread Brian Knox
I'll do a build off of head and give it a try today Brian On Thu, Nov 1, 2012 at 1:31 PM, Rainer Gerhards wrote: >> > FYI: I have just committed changes to master which make mmjsonparse >> > compatible with the new syntax. >> >> ...now also mmjsonparse. I'd appreciate if you (or someone else ;))

Re: [rsyslog] rsyslog queue subsystem - refactor or redesign?

2012-11-02 Thread Brian Knox
Rainer - Unfortunately I won't be able to do any rsyslog testing in the lab this week - I'm a bit under the gun until I finish some code for a presentation. Once I'm clear of that I can probably get in some rsyslog tests again. Brian On Fri, Nov 2, 2012 at 1:54 PM, Rainer Gerhards wrote: > > >

Re: [rsyslog] rsyslog queue subsystem - refactor or redesign?

2012-11-05 Thread Brian Knox
Rainer - I can try to make some space to build from git head tomorrow and test out the queue changes on one of our lab servers. Brian On Mon, Nov 5, 2012 at 2:50 AM, Rainer Gerhards wrote: >> I have looked further at the code. I think I see part of the >> inefficiency that happens in the regula

Re: [rsyslog] Multiple Apache vhosts, keep individual log files AND send to remote logstash

2012-12-10 Thread Brian Knox
I'm going to chime in here to say that between logstash and rsyslog, logstash is the more experimental option. Rsyslog has been around for a long time, is heavily used, and is the default logger in many distributions. Note like Radu, I am not slighting logstash with this statement, nor am I sayi

[rsyslog] version 7 syntax with queue settings

2012-12-27 Thread Brian Knox
I'm currently working on some improvements to the omhiredis plugin to allow it to work with batching. I have the basics finished in a local build and would like to start testing. What's the proper syntax in v7 stable release for setting queue settings? My action config is: action(name="rediste

Re: [rsyslog] version 7 syntax with queue settings

2012-12-27 Thread Brian Knox
Never mind, figured it out! Brian On Thu, Dec 27, 2012 at 12:45 PM, Brian Knox wrote: > I'm currently working on some improvements to the omhiredis plugin to > allow it to work with batching. I have the basics finished in a local > build and would like to start testing.

[rsyslog] working with list templates

2013-01-03 Thread Brian Knox
I've been working on some improvements for the omhiredis output plugin this week. I think I've come up with a better way of combining templates with redis command formatting. In order to do this, I need to, given a list style template, get the number of elements in the list and the length of each

Re: [rsyslog] working with list templates

2013-01-03 Thread Brian Knox
wrote: Hello Brian, Just to make sure we're not missing the obvious: http://www.rsyslog.com/doc/rsyslog_conf_templates.html Besides that, there are a couple of examples here: 2013/1/3 Brian Knox I've been working on some improvements for the omhiredis output plugin this week. I think I&#

Re: [rsyslog] working with list templates

2013-01-03 Thread Brian Knox
I suppose Rainer can be bugged next week, or maybe someone else knows... Best regards, Radu 2013/1/3 Brian Knox Hi Radu! Thanks for the link, but I already know how to work with list templates from a conf perspective. I'm looking for good examples of accessing that data from code in a

Re: [rsyslog] working with list templates

2013-01-04 Thread Brian Knox
passing mode (or maybe array passing). Ommongodb should help you, else ping me early next week. Rainer -Original Message- From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- boun...@lists.adiscon.com] On Behalf Of Brian Knox Sent: Thursday, January 03, 2013 2:22 PM To: rsyslog

Re: [rsyslog] working with list templates

2013-01-05 Thread Brian Knox
is some talk of phasing it out rather than needing to > support multiple ways of passing groups of messages. > > David Lang > > > On Fri, 4 Jan 2013, Brian Knox wrote: > > Aha! I think in this case array passing may be what I need. Thanks! >> >> Brian >> On

Re: [rsyslog] working with list templates

2013-01-07 Thread Brian Knox
y(name="msg") } And here's the function I'm passing to: void redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen); So if I could pass this as essentially a JSON array ["LPUSH", "queue_key", "msg"] that would

Re: [rsyslog] working with list templates

2013-01-07 Thread Brian Knox
On Mon, Jan 7, 2013 at 10:59 AM, Rainer Gerhards wrote: > > -Original Message- > > From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- > > boun...@lists.adiscon.com] On Behalf Of Brian Knox > > Sent: Monday, January 07, 2013 2:11 PM > > To: rsyslog-u

Re: [rsyslog] working with list templates

2013-01-07 Thread Brian Knox
sure I did get this straight: This means you always include one field only? Rainer -Original Message- From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog- boun...@lists.adiscon.com] On Behalf Of Brian Knox Sent: Monday, January 07, 2013 5:18 PM To: rsyslog-users Subject: Re: [rsyslog]

Re: [rsyslog] working with list templates

2013-01-07 Thread Brian Knox
David - no, I have batch processing working already as part of the patch I'm working on. That part was actually pretty easy using rewriting the plugin to properly use dequeuebatchsize, etc. The issue is that the way the redis protocol constructs commands means that I can't just put a message payl

Re: [rsyslog] working with list templates

2013-01-08 Thread Brian Knox
ogname_key" is a constant, "nginx" is %programname%, and 1 is a constant, how would I go about that? Thanks! Brian On Tue, Jan 8, 2013 at 2:37 AM, Rainer Gerhards wrote: > > -Original Message- > > From: Brian Knox [mailto:bri...@talksum.com] > > Se

Re: [rsyslog] ACT_ARRAY_PASSING output plugin mode

2013-01-08 Thread Brian Knox
As per our discussion over the last couple of days - I plan on using this mode! I'll let you know how it goes and if it ends up fitting me use case as I think it will. Brian On Mon, Sep 17, 2012 at 1:10 PM, Rainer Gerhards wrote: > > is anybody actively using this mode? I have strong plans to

[rsyslog] create main queue with v7 ruleset definitions

2013-04-02 Thread Brian Knox
Is there a way to specify that a ruleset should have it's own main queue with the new ruleset() {...} v7 definition format? Brian ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ Wha

Re: [rsyslog] create main queue with v7 ruleset definitions

2013-04-02 Thread Brian Knox
do it by specifying the parameters inside ruleset(); for > example: > > ruleset(name="collectors" queue.type="linkedlist" queue.size="5") { ... > } > > Philippe Muller > > > On Tue, Apr 2, 2013 at 4:09 PM, Brian Knox wrote: > > > Is the

Re: [rsyslog] CSV Parsing

2013-06-03 Thread Brian Knox
Rainer - We have some patches for the zeromq input and output plugins that have just come out of our own internal testing. The soonest I could get the patches to you would be tomorrow morning (US EST). Since you're in the middle of trying to cut 7.4 stable, would you like those now, or should I

<    1   2   3   >