Re: [rsyslog] Plugin Questions

[Rainer Gerhards]

Folks, with Brian's permission I reply to our conversation on the rsyslog
mailing list. Brian is updating the 0mq plugins to v8 and we are discussing
some design issues. I think this is probably of interest for some others,
so I asked that we carry on on the list. Context is provided in the mail
text below...

On Mon, Feb 10, 2014 at 12:52 PM, Brian Knox  wrote:

> In ZeroMQ, the main thing binding on the output plugin would allow is
> easier load balancing in some cases.
>
> Let's say you have an omzmq output on tcp://127.0.0.1:, that you bind
> it, and that it is a ZMQ_PUSH socket.
>
> If I now connect say.. 4 ZMQ_PULL sockets to the bound output socket, then
> ZeroMQ will round robin load balance across the four connections.
>
> So, if you have some process that does some work that has a significant
> cost to rsyslog output.. you bind an output, and connect multiple workers
> to it to load balance.
>
> However from what you are saying, this will now keep us from having
> multiple worker instances on the output.
>

OK, I think I finally understand. Let's see if we can nail this down with a
simple solution. First let me repeat how I see the situation:

If you bind an output, than you expect that the output does not have a
performance problem. The performance problem is on the "receivers". So in
this case a single instance of omzmq is considered fast enough to push
messages into the queue. If so, we could effectively work with a single
action "worker instance", as described on slide 10 of this presentation:

http://www.slideshare.net/rainergerhards1/the-rsyslog-v8-engine-developers-view

It would just be that we do it because we want it that way and not because
of we are unable to do it any other way. Even further, we could have a
config parameter for the bind port. If given, we'd use a single "instance",
if not, then multiple. IN ANY CASE this will only affect the omzmq code.
Even in "single instance mode", the rsyslog core engine would do message
filtering, formating, etc on multiple workers -- just like it did in v7.
It's even a bit faster than in v7 because we have these optimized interface
callbacks.

If I am right with my assumption, I'd say that's the way to go. I would be
happy to help with the plumbing for this "single instance mode", if you
have you code on github or somewhere else I can pull it from, I'd happy to
simply add it (so that you can review).

Rainer



>
> It's not a huge deal.. it just means to do that sort of pattern I would
> need an external zmq proxy that rsyslog connects to, and then the proxy
> would be responsible for work distribution outside of zeromq.  I could
> write such a thing and release it open source (I'm the original author and
> maintainer of the zproxy proxy code in czmq so I'm very familiar with it).
>
> I have another idea for a solution that would work within the output
> plugin itself, but it is more complex and I'd rather keep things simple for
> the initial port to 8.
>
> My proposal at this point is:
>
> 1) I finish the new omzmq output for 8.  It supports the latest release of
> czmq.  I will only allow "connects" to keep things simple.  I will add in
> support for CURVE authentication and encryption (
> http://rfc.zeromq.org/spec:26).
>
> 2) I create a new version of imzmq.  The refactor will include code
> cleanup, and reducing the complexity of the options so that imzmq only
> supports "binds".  I add in support on the input side for CURVE as well.
>  This lets use connect together rsyslog instances over zeromq, with
> certificate based authentication and encryption.
>
> 3) The new plugins will be "imzmq" and "omzmq" rather than "imzmq3" and
> "omzmq3".   We can deprecate the 3 versions of the plugins.
>
> How's that sound?
> Brian
>
>
>
>
> On Mon, Feb 10, 2014 at 5:56 AM, Rainer Gerhards  > wrote:
>
>>
>> On Sun, Feb 9, 2014 at 9:23 PM, Brian Knox  wrote:
>>
>>> The reason I ask is - with zeromq, connections are stateless and either
>>> side may "bind" or "connect".  A transport address (tcp://127.0.0.1:5556,
>>> ipc:///tmp/myipsock, etc) can be -connected- to many times but only -bound-
>>> one time.  This effects how things like fan out an fan in patterns work.
>>>
>>> I would like to allow -binds- on the omzmq side.  However, if each
>>> thread spins up, creates a zeromq socket and tries to bind to the same
>>> address, this won't work.
>>>
>>>
>> I thought I was too tired yesterday to fully understand. Unfortunately, I
>> still don't grasp in full what that means. What is the advantage of being
>> "bound" as an output. I have sockets on my mind, maybe that's the problem.
>> If I bind an output socket to a specific port, I can obviously have only
>> one instance of that socket in the system. Is that what you want?
>>
>> We can of course ensure that each action will "activate" only one worker
>> instance at a time, but is that what we want? That would mean we could not
>> really us the multi-instance feature we introduduced. And so I conclude I

[rsyslog] LibLogging 1.0.1 released

[Florian Riedl]

Hi all,

we have just released v1.0.1 of liblogging. This version primarily
clarifies the licensing and fixes some packaging issues. Otherwise, it is
identical to 1.0.0.

Download and Changelog:
http://www.liblogging.org/2014/02/liblogging-101-released.html

Best regards,
Florian Riedl
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] rsyslog-7.4.9 ompgsql libpq threads problem

[Rainer Gerhards]

Unfortunately, I was too busy to merge before today. It's now merged into
the v7-stable branch. I would appreciate if you could check that it works
for you. Once this is OK, I think I can do a little tweak which may be
useful to prevent unnecessary startup work.

Patch as integrated can be viewed here:

https://github.com/rsyslog/rsyslog/commit/ff091b7ea963b63130cfb77af57f137e632bf765

Thanks again for the patch!


On Thu, Feb 6, 2014 at 11:47 AM, Rainer Gerhards
wrote:

> I am on my way to the Fedora Developer's Conference. I'll see that I
> review ASAP.
>
> Rainer
>
>
> On Thu, Feb 6, 2014 at 11:14 AM, ecze  wrote:
>
>> Ok, the patch works perfectly.
>> Is there anyone in charge to  check, comment  and commit it ?
>> I think this is quite critical because the *PGConn point to an invalid
>> address after the fork in the current versions ( 5/7/8 )
>>
>> Alain
>>
>>
>>
>>
>>
>> ecze wrote
>> > Another patch ( not yet tested ) without useless PG connect..
>> >
>> -patch--
>> > diff --git a/plugins/ompgsql/ompgsql.c b/plugins/ompgsql/ompgsql.c
>> > index 11f346f..e236234 100644
>> > --- a/plugins/ompgsql/ompgsql.c
>> > +++ b/plugins/ompgsql/ompgsql.c
>> > @@ -261,6 +261,8 @@ ENDtryResume
>> >
>> >
>> >  BEGINbeginTransaction
>> > +if(pData->f_hpgsql == NULL)
>> > +   initPgSQL(pData, 0);
>> >  CODESTARTbeginTransaction
>> >  dbgprintf("ompgsql: beginTransaction\n");
>> > iRet = writePgSQL((uchar*) "begin", pData); /* TODO: make
>> > user-configurable */
>> > @@ -346,8 +348,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1)
>> > if (iPgSQLPropErr) {
>> > errmsg.LogError(0, RS_RET_INVALID_PARAMS, "Trouble with
>> > PgSQL connection properties. -PgSQL logging disabled");
>> > ABORT_FINALIZE(RS_RET_INVALID_PARAMS);
>> > -   } else {
>> > -   CHKiRet(initPgSQL(pData, 0));
>> > }
>> >
>> >  CODE_STD_FINALIZERparseSelectorAct
>> >
>> -patch--
>> >
>> >
>> > Alain
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://rsyslog-users.1305293.n2.nabble.com/rsyslog-7-4-9-ompgsql-libpq-threads-problem-tp7584204p7584233.html
>> Sent from the rsyslog-users mailing list archive at Nabble.com.
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] maximum message-size / line-length with relp?

[Assaf Gordon]

Hello,

Is there a line-length limit for logging when using libRELP?
Or, is there a line-length limit when a "client" rsyslogd receives 
messages from a unix-socket and sends them to another rsyslogd with libRelp?


My server is configured like so:
===
module(load="imrelp" ruleset="relp")
input(type="imrelp" port="12345" tls="on")
ruleset (name="relp") { action(type="omfile" file="my.log") }
===

My client is configured like so:
===
module(load="imuxsock" SysSock.Use="off")
module(load="omrelp")
input(type="imuxsock" Socket="my.sock" CreatePath="on")
action(type="omrelp" target="1.2.3.4" port="12345" tls="on"
action.resumeRetryCount="-1" action.resumeInterval="5")
===

To send messages (on the client machine), I use:
===
echo "Hello World | logger -u my.sock -d
===

My question:
Does any of the components in this chain imposes a line-length limit on 
the textual messages?


On few occasions, I see truncated lines stored in the log file on the 
server side, and I'm not sure which component causes them.


Thanks!
 -gordon

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Plugin Questions

[Brian Knox]

Rainer - so we are saying that if the plugin is configured to "connect" the 
zeromq socket, we will allow multiple instance mode, and if it is configured to 
"bind" the zeromq socket, we will restrict to single instance mode?  This 
sounds reasonable to me.

This evening (GMT-5 evening) I'll add the configuration parameter for choosing 
between binding and connecting the socket to the new plugin code and get it up 
on github and we can go from there!

BRian

-Original Message-
From: rsyslog-boun...@lists.adiscon.com 
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards
Sent: Monday, February 10, 2014 10:00 AM
To: rsyslog-users
Subject: Re: [rsyslog] Plugin Questions

Folks, with Brian's permission I reply to our conversation on the rsyslog 
mailing list. Brian is updating the 0mq plugins to v8 and we are discussing 
some design issues. I think this is probably of interest for some others, so I 
asked that we carry on on the list. Context is provided in the mail text 
below...

On Mon, Feb 10, 2014 at 12:52 PM, Brian Knox  wrote:

> In ZeroMQ, the main thing binding on the output plugin would allow is 
> easier load balancing in some cases.
>
> Let's say you have an omzmq output on tcp://127.0.0.1:, that you 
> bind it, and that it is a ZMQ_PUSH socket.
>
> If I now connect say.. 4 ZMQ_PULL sockets to the bound output socket, 
> then ZeroMQ will round robin load balance across the four connections.
>
> So, if you have some process that does some work that has a 
> significant cost to rsyslog output.. you bind an output, and connect 
> multiple workers to it to load balance.
>
> However from what you are saying, this will now keep us from having 
> multiple worker instances on the output.
>

OK, I think I finally understand. Let's see if we can nail this down with a 
simple solution. First let me repeat how I see the situation:

If you bind an output, than you expect that the output does not have a 
performance problem. The performance problem is on the "receivers". So in this 
case a single instance of omzmq is considered fast enough to push messages into 
the queue. If so, we could effectively work with a single action "worker 
instance", as described on slide 10 of this presentation:

http://www.slideshare.net/rainergerhards1/the-rsyslog-v8-engine-developers-view

It would just be that we do it because we want it that way and not because of 
we are unable to do it any other way. Even further, we could have a config 
parameter for the bind port. If given, we'd use a single "instance", if not, 
then multiple. IN ANY CASE this will only affect the omzmq code.
Even in "single instance mode", the rsyslog core engine would do message 
filtering, formating, etc on multiple workers -- just like it did in v7.
It's even a bit faster than in v7 because we have these optimized interface 
callbacks.

If I am right with my assumption, I'd say that's the way to go. I would be 
happy to help with the plumbing for this "single instance mode", if you have 
you code on github or somewhere else I can pull it from, I'd happy to simply 
add it (so that you can review).

Rainer



>
> It's not a huge deal.. it just means to do that sort of pattern I 
> would need an external zmq proxy that rsyslog connects to, and then 
> the proxy would be responsible for work distribution outside of 
> zeromq.  I could write such a thing and release it open source (I'm 
> the original author and maintainer of the zproxy proxy code in czmq so I'm 
> very familiar with it).
>
> I have another idea for a solution that would work within the output 
> plugin itself, but it is more complex and I'd rather keep things 
> simple for the initial port to 8.
>
> My proposal at this point is:
>
> 1) I finish the new omzmq output for 8.  It supports the latest 
> release of czmq.  I will only allow "connects" to keep things simple.  
> I will add in support for CURVE authentication and encryption ( 
> http://rfc.zeromq.org/spec:26).
>
> 2) I create a new version of imzmq.  The refactor will include code 
> cleanup, and reducing the complexity of the options so that imzmq only 
> supports "binds".  I add in support on the input side for CURVE as well.
>  This lets use connect together rsyslog instances over zeromq, with 
> certificate based authentication and encryption.
>
> 3) The new plugins will be "imzmq" and "omzmq" rather than "imzmq3" and
> "omzmq3".   We can deprecate the 3 versions of the plugins.
>
> How's that sound?
> Brian
>
>
>
>
> On Mon, Feb 10, 2014 at 5:56 AM, Rainer Gerhards 
>  > wrote:
>
>>
>> On Sun, Feb 9, 2014 at 9:23 PM, Brian Knox  wrote:
>>
>>> The reason I ask is - with zeromq, connections are stateless and 
>>> either side may "bind" or "connect".  A transport address 
>>> (tcp://127.0.0.1:5556, ipc:///tmp/myipsock, etc) can be -connected- 
>>> to many times but only -bound- one time.  This effects how things like fan 
>>> out an fan in patterns work.
>>>
>>> I would like to allow -binds- on the om

Re: [rsyslog] Plugin Questions

[Rainer Gerhards]

On Mon, Feb 10, 2014 at 4:50 PM, Brian Knox  wrote:

> Rainer - so we are saying that if the plugin is configured to "connect"
> the zeromq socket, we will allow multiple instance mode, and if it is
> configured to "bind" the zeromq socket, we will restrict to single instance
> mode?  This sounds reasonable to me.
>

Yup, if you look at the presentation that I sent before:

http://www.slideshare.net/rainergerhards1/the-rsyslog-v8-engine-developers-view

Look at slide 11: it's actually that mutex in red that we need. It's a
little bit more logic, because we need to do it conditionally, but sure no
rocket science.


>
> This evening (GMT-5 evening) I'll add the configuration parameter for
> choosing between binding and connecting the socket to the new plugin code
> and get it up on github and we can go from there!
>

Oh great, I will than fork and add a proposal how to deal with that. Just
note that I have meetings tomorrow, so you'll probably not hear from me
before Wednesday.

Rainer

>
> BRian
>
> -Original Message-
> From: rsyslog-boun...@lists.adiscon.com [mailto:
> rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, February 10, 2014 10:00 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Plugin Questions
>
> Folks, with Brian's permission I reply to our conversation on the rsyslog
> mailing list. Brian is updating the 0mq plugins to v8 and we are discussing
> some design issues. I think this is probably of interest for some others,
> so I asked that we carry on on the list. Context is provided in the mail
> text below...
>
> On Mon, Feb 10, 2014 at 12:52 PM, Brian Knox  wrote:
>
> > In ZeroMQ, the main thing binding on the output plugin would allow is
> > easier load balancing in some cases.
> >
> > Let's say you have an omzmq output on tcp://127.0.0.1:, that you
> > bind it, and that it is a ZMQ_PUSH socket.
> >
> > If I now connect say.. 4 ZMQ_PULL sockets to the bound output socket,
> > then ZeroMQ will round robin load balance across the four connections.
> >
> > So, if you have some process that does some work that has a
> > significant cost to rsyslog output.. you bind an output, and connect
> > multiple workers to it to load balance.
> >
> > However from what you are saying, this will now keep us from having
> > multiple worker instances on the output.
> >
>
> OK, I think I finally understand. Let's see if we can nail this down with
> a simple solution. First let me repeat how I see the situation:
>
> If you bind an output, than you expect that the output does not have a
> performance problem. The performance problem is on the "receivers". So in
> this case a single instance of omzmq is considered fast enough to push
> messages into the queue. If so, we could effectively work with a single
> action "worker instance", as described on slide 10 of this presentation:
>
>
> http://www.slideshare.net/rainergerhards1/the-rsyslog-v8-engine-developers-view
>
> It would just be that we do it because we want it that way and not because
> of we are unable to do it any other way. Even further, we could have a
> config parameter for the bind port. If given, we'd use a single "instance",
> if not, then multiple. IN ANY CASE this will only affect the omzmq code.
> Even in "single instance mode", the rsyslog core engine would do message
> filtering, formating, etc on multiple workers -- just like it did in v7.
> It's even a bit faster than in v7 because we have these optimized
> interface callbacks.
>
> If I am right with my assumption, I'd say that's the way to go. I would be
> happy to help with the plumbing for this "single instance mode", if you
> have you code on github or somewhere else I can pull it from, I'd happy to
> simply add it (so that you can review).
>
> Rainer
>
>
>
> >
> > It's not a huge deal.. it just means to do that sort of pattern I
> > would need an external zmq proxy that rsyslog connects to, and then
> > the proxy would be responsible for work distribution outside of
> > zeromq.  I could write such a thing and release it open source (I'm
> > the original author and maintainer of the zproxy proxy code in czmq so
> I'm very familiar with it).
> >
> > I have another idea for a solution that would work within the output
> > plugin itself, but it is more complex and I'd rather keep things
> > simple for the initial port to 8.
> >
> > My proposal at this point is:
> >
> > 1) I finish the new omzmq output for 8.  It supports the latest
> > release of czmq.  I will only allow "connects" to keep things simple.
> > I will add in support for CURVE authentication and encryption (
> > http://rfc.zeromq.org/spec:26).
> >
> > 2) I create a new version of imzmq.  The refactor will include code
> > cleanup, and reducing the complexity of the options so that imzmq only
> > supports "binds".  I add in support on the input side for CURVE as well.
> >  This lets use connect together rsyslog instances over zeromq, with
> > certificate based authentication and encryption

Re: [rsyslog] maximum message-size / line-length with relp?

[Edmonds, Alan]

/bin/logger has a 1024 byte line length limit

> -Original Message-
> From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-
> boun...@lists.adiscon.com] On Behalf Of Assaf Gordon
> Sent: 10 February 2014 15:47
> To: rsyslog-users
> Subject: [rsyslog] maximum message-size / line-length with relp?
> 
> Hello,
> 
> Is there a line-length limit for logging when using libRELP?
> Or, is there a line-length limit when a "client" rsyslogd receives
> messages from a unix-socket and sends them to another rsyslogd with
> libRelp?
> 
> My server is configured like so:
> ===
> module(load="imrelp" ruleset="relp")
> input(type="imrelp" port="12345" tls="on")
> ruleset (name="relp") { action(type="omfile" file="my.log") }
> ===
> 
> My client is configured like so:
> ===
> module(load="imuxsock" SysSock.Use="off")
> module(load="omrelp")
> input(type="imuxsock" Socket="my.sock" CreatePath="on")
> action(type="omrelp" target="1.2.3.4" port="12345" tls="on"
>  action.resumeRetryCount="-1" action.resumeInterval="5")
> ===
> 
> To send messages (on the client machine), I use:
> ===
> echo "Hello World | logger -u my.sock -d
> ===
> 
> My question:
> Does any of the components in this chain imposes a line-length limit on
> the textual messages?
> 
> On few occasions, I see truncated lines stored in the log file on the
> server side, and I'm not sure which component causes them.
> 
> Thanks!
>   -gordon
> 
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> if you DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] advise on github issue tracker best use sought

[Rainer Gerhards]

Hi especially to those that use github for a while,

as you know, we have begun to utilize the github issue tracker. I admit
it's a fine tool. But I've a question how it is to be used in the best
possible way. I have a couple of "soft TODO" items, things that have low
priority or are at the idea stage. It would be tempting to add these as
issues, however,

a) the issue tracker will probably look somehwat clutered, with the
important things hard to see
b) at least some of these issues will probably stay open for quite
looong. Would that make folks think that the project is dead?

So any advise on how to best use the trackers would be appreciated. I
could, for example, also place these low priority things into a Google Doc
think tank or something along those lines. On the other hand, if it's up on
github we may find someone else who thinks it's a good idea and implements
it...

Comments appreciated!

Rainer
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] advise on github issue tracker best use sought

[Assaf Gordon]

Hi,

On 02/10/2014 11:10 AM, Rainer Gerhards wrote:
<...>


So any advise on how to best use the trackers would be appreciated. I
could, for example, also place these low priority things into a Google Doc
think tank or something along those lines. On the other hand, if it's up on
github we may find someone else who thinks it's a good idea and implements
it...



For collecting ideas, TODOs, issues, etc. with social/sharing features - 
"Trello" is a nice option to consider:


  https://trello.com/tour

-gordon


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] maximum message-size / line-length with relp?

[Assaf Gordon]

Is there a line-length limit for logging when using libRELP?
Or, is there a line-length limit when a "client" rsyslogd receives
messages from a unix-socket and sends them to another rsyslogd with
libRelp?



On 02/10/2014 11:01 AM, Edmonds, Alan wrote:
> /bin/logger has a 1024 byte line length limit

Thanks!
Any solid alternatives to it?

And just to verify:
Besides "/bin/logger", rsyslog+libRelp do not impose any limit on text 
line length, correct?


-gordon




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Question on features and version.

[Boylan, James]

Which version of rsyslog was the ability for 'if' to parse 
(field($timegenerated,':',3) % 2 == '0') introduced? 7.5.x+? I'm working on 
troubleshooting some config errors and I need to rule out some possibilities.



Thanks!



-- James




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] maximum message-size / line-length with relp?

[Rainer Gerhards]

Search fir rlogger on ml Archive.
You need to set size limit if larger 4k
Relp may have some 64k+ limit.  If that's an issue tell me and I'll check.
Not sure if i manage that today.
Rainer

Sent from phone, thus brief.
Am 10.02.2014 17:20 schrieb "Assaf Gordon" :

>
>>> Is there a line-length limit for logging when using libRELP?
>>> Or, is there a line-length limit when a "client" rsyslogd receives
>>> messages from a unix-socket and sends them to another rsyslogd with
>>> libRelp?
>>>
>>>
> On 02/10/2014 11:01 AM, Edmonds, Alan wrote:
> > /bin/logger has a 1024 byte line length limit
>
> Thanks!
> Any solid alternatives to it?
>
> And just to verify:
> Besides "/bin/logger", rsyslog+libRelp do not impose any limit on text
> line length, correct?
>
> -gordon
>
>
>
>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] RE : Re: maximum message-size / line-length with relp?

[rmkml]

Hi Gordon,

Modified logger limit 1000 to 8192 without apparently pb, but I'm not C 
developper. 

Regards
@Rmkml



 Message d'origine 
De : Assaf Gordon  
Date :  
A : rsyslog-users  
Objet : Re: [rsyslog] maximum message-size / line-length with relp? 
 
>>
>> Is there a line-length limit for logging when using libRELP?
>> Or, is there a line-length limit when a "client" rsyslogd receives
>> messages from a unix-socket and sends them to another rsyslogd with
>> libRelp?
>>

On 02/10/2014 11:01 AM, Edmonds, Alan wrote:
> /bin/logger has a 1024 byte line length limit

Thanks!
Any solid alternatives to it?

And just to verify:
Besides "/bin/logger", rsyslog+libRelp do not impose any limit on text 
line length, correct?

-gordon




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] LibLogging 1.0.1 released

[Thomas D.]

Hi,

Florian Riedl wrote:
> we have just released v1.0.1 of liblogging. This version primarily
> clarifies the licensing and fixes some packaging issues. Otherwise, it is
> identical to 1.0.0.

Should v1.0.1 represent the implementation of the new name scheme
announced in
?

I am wondering because the SO name is still "liblogging" and not
"liblogging1" and the README.md sounds like the package maintainer
should take care for the renaming -- that's not what I (and hopefully
others) expected from the "Feedback request: liblogging component names"
thread.


-Thomas

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] LibLogging 1.0.1 released

[Rainer Gerhards]

So let me fix this! Thanks for the note. Quite honestly, I am far from
being an expert on build system issues and this is an oversight.  Anything
else please let me know. I promise I'll do my best to fix it. Pull requests
are also happily accepted.

Thanks again,
Rainer

Sent from phone, thus brief.
Am 10.02.2014 21:05 schrieb "Thomas D." :

> Hi,
>
> Florian Riedl wrote:
> > we have just released v1.0.1 of liblogging. This version primarily
> > clarifies the licensing and fixes some packaging issues. Otherwise, it is
> > identical to 1.0.0.
>
> Should v1.0.1 represent the implementation of the new name scheme
> announced in
> ?
>
> I am wondering because the SO name is still "liblogging" and not
> "liblogging1" and the README.md sounds like the package maintainer
> should take care for the renaming -- that's not what I (and hopefully
> others) expected from the "Feedback request: liblogging component names"
> thread.
>
>
> -Thomas
>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[Assaf Gordon]

Hello,

A quick follow-up to the line-length limitations:

1. Most loggers are indeed limited (hard-coded) to 1024 bytes, but only 
when using the syslog(3) call.


2. When using UDP servers or Unix sockets, the limitation is worse: 400 
bytes for the message (This is related, I assume, to the inherit limit 
of 576 bytes per atomic UDP datagram).


Nathan Brown published a modified "mlogger" with bigger limits:
  http://lists.adiscon.net/pipermail/rsyslog/2014-February/036208.html
  https://github.com/nbrownus/mlogger

But this program is also limited when using unix-sockets (which is what 
I'm using).


I've sent PR ( https://github.com/nbrownus/mlogger/pull/2 ) which 
enlarges the buffer size for the unix-socket case.


With this patch, I can send messages (logger->unix 
socket->rsyslogd->relp) of 4KB and above as a single line.


Hope this helps,
 -gordon


On 02/10/2014 11:42 AM, rmkml wrote:

Hi Gordon,

Modified logger limit 1000 to 8192 without apparently pb, but I'm not C 
developper.

Regards
@Rmkml



 Message d'origine 
De : Assaf Gordon 
Date :
A : rsyslog-users 
Objet : Re: [rsyslog] maximum message-size / line-length with relp?



Is there a line-length limit for logging when using libRELP?
Or, is there a line-length limit when a "client" rsyslogd receives
messages from a unix-socket and sends them to another rsyslogd with
libRelp?



On 02/10/2014 11:01 AM, Edmonds, Alan wrote:

/bin/logger has a 1024 byte line length limit


Thanks!
Any solid alternatives to it?

And just to verify:
Besides "/bin/logger", rsyslog+libRelp do not impose any limit on text
line length, correct?

-gordon




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.



___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[David Lang]

On Mon, 10 Feb 2014, Assaf Gordon wrote:


A quick follow-up to the line-length limitations:

1. Most loggers are indeed limited (hard-coded) to 1024 bytes, but only when 
using the syslog(3) call.


2. When using UDP servers or Unix sockets, the limitation is worse: 400 bytes 
for the message (This is related, I assume, to the inherit limit of 576 bytes 
per atomic UDP datagram).


huh??? I've sent much larger logs via UDP without a problem, and /dev/log is a 
Unix socket, so if this was true then logger would be limited to 400 bytes.


David Lang



Nathan Brown published a modified "mlogger" with bigger limits:
 http://lists.adiscon.net/pipermail/rsyslog/2014-February/036208.html
 https://github.com/nbrownus/mlogger

But this program is also limited when using unix-sockets (which is what I'm 
using).


I've sent PR ( https://github.com/nbrownus/mlogger/pull/2 ) which enlarges 
the buffer size for the unix-socket case.


With this patch, I can send messages (logger->unix socket->rsyslogd->relp) of 
4KB and above as a single line.


Hope this helps,
-gordon


On 02/10/2014 11:42 AM, rmkml wrote:

Hi Gordon,

Modified logger limit 1000 to 8192 without apparently pb, but I'm not C 
developper.


Regards
@Rmkml



 Message d'origine 
De : Assaf Gordon 
Date :
A : rsyslog-users 
Objet : Re: [rsyslog] maximum message-size / line-length with relp?



Is there a line-length limit for logging when using libRELP?
Or, is there a line-length limit when a "client" rsyslogd receives
messages from a unix-socket and sends them to another rsyslogd with
libRelp?



On 02/10/2014 11:01 AM, Edmonds, Alan wrote:

/bin/logger has a 1024 byte line length limit


Thanks!
Any solid alternatives to it?

And just to verify:
Besides "/bin/logger", rsyslog+libRelp do not impose any limit on text
line length, correct?

-gordon




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.



___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[Assaf Gordon]

On 02/10/2014 04:12 PM, David Lang wrote:

On Mon, 10 Feb 2014, Assaf Gordon wrote:


2. When using UDP servers or Unix sockets, the limitation is worse:
400 bytes for the message (This is related, I assume, to the inherit
limit of 576 bytes per atomic UDP datagram).


huh??? I've sent much larger logs via UDP without a problem, and
/dev/log is a Unix socket, so if this was true then logger would be
limited to 400 bytes.



Perhaps there are different implementation issues (or perhaps I missed 
something), but this is what I've observed (on Ubuntu and Debian).



1. Ubuntu and Debian use "logger" from "bsdutils", which has the 
following line when logging through Unix/UDP sockets:

snprintf(buf, sizeof(buf), "<%d>%.15s %.200s%s: %.400s",
 pri, tp, cp, pid, msg);

Where "msg" is truncated to "%.400s" .
Same is in "mlogger" ( https://github.com/nbrownus/mlogger ).


2. "/dev/log" is indeed a unix-socket, but the default invocation of 
"logger" doesn't write to it directly, but instead uses "syslog(3)" call.


Do a "sudo tail -f /var/log/messages", and compare the output of the 
followings:


# Send this many characters
$ seq 300 | paste -s -d, | wc -c
1092

# Send with syslog(3)
$ seq 300 | paste -s -d, | logger

# Send directly to Unix-Socket
$ seq 300 | paste -s -d, | logger -u /dev/log -d

When sending with syslog(3), I observed that the numbers written to the 
log go all the way up to 283 (totaling in 1024 characters), then the 
next message starts with 284.


When sending to "/dev/log", the first lines goes from "1" to "127" 
(totaling in 400 characters), then the next message starts with "284".
The reason that the numbers between "128" and "283" are lost with this 
method, is because "logger" used a 1024 bytes buffer to read the input, 
but truncated it to 400 characters when sending the message.



3. When using a unix-socket, it should be possible to reliably send 
larger messages, as the kernel guarantees some atomicity.
I could not find an authoritative number, but to the best of my 
understanding, opening a unix socket with 
"socket(AF_LOCAL,SOCK_DGRAM,...)" you can send messages that are upto 
"SO_SNDBUF" in size, commonly ~120KB on linux.

(if someone knows differently, please correct me).

4. When using a UDP socket, the maximum size you can use and still 
guarantee that there will be no IP fragmentation is about 576 bytes 
(which, counting IP header and syslog headers, brings it down to a safe 
"ball-park" value of 400 bytes, which is what the limit is in the C 
source files).




regards,
-gordon






___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[David Lang]

On Mon, 10 Feb 2014, Assaf Gordon wrote:


On 02/10/2014 04:12 PM, David Lang wrote:

On Mon, 10 Feb 2014, Assaf Gordon wrote:


2. When using UDP servers or Unix sockets, the limitation is worse:
400 bytes for the message (This is related, I assume, to the inherit
limit of 576 bytes per atomic UDP datagram).


huh??? I've sent much larger logs via UDP without a problem, and
/dev/log is a Unix socket, so if this was true then logger would be
limited to 400 bytes.



Perhaps there are different implementation issues (or perhaps I missed 
something), but this is what I've observed (on Ubuntu and Debian).



1. Ubuntu and Debian use "logger" from "bsdutils", which has the following 
line when logging through Unix/UDP sockets:

   snprintf(buf, sizeof(buf), "<%d>%.15s %.200s%s: %.400s",
pri, tp, cp, pid, msg);

Where "msg" is truncated to "%.400s" .
Same is in "mlogger" ( https://github.com/nbrownus/mlogger ).


I can easily believe that logger does silly things

3. When using a unix-socket, it should be possible to reliably send larger 
messages, as the kernel guarantees some atomicity.
I could not find an authoritative number, but to the best of my 
understanding, opening a unix socket with "socket(AF_LOCAL,SOCK_DGRAM,...)" 
you can send messages that are upto "SO_SNDBUF" in size, commonly ~120KB on 
linux.

(if someone knows differently, please correct me).

4. When using a UDP socket, the maximum size you can use and still guarantee 
that there will be no IP fragmentation is about 576 bytes (which, counting IP 
header and syslog headers, brings it down to a safe "ball-park" value of 400 
bytes, which is what the limit is in the C source files).


fragmentation works on UDP messages into packets, and UDP packets can be any 
size up to the MTU of your network. so you can send a 4k message over UDP 
without using jumbo packets.


now, if your network is busy and there is a risk of loosing packets, a message 
that gets fragmented into multiple packets will be lost if any of the packets 
are lost.


You can (usually) set your network MTU up to ~8k and send 8k messages via UDP 
with very little chance of you loosing any logs (and in practice, UDP over a 
local switch is going to be pretty darn reliable anyway, up to the wirespeed 
limits of your switch)


all you need to do to send large messages over UDP is to set the message size on 
sender and receiver (up the max size compiled into rsyslog) and make sure that 
you don't use a template that trims the message to a given size.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[Assaf Gordon]

On 02/10/2014 04:56 PM, David Lang wrote:


fragmentation works on UDP messages into packets, and UDP packets can be
any size up to the MTU of your network. so you can send a 4k message
over UDP without using jumbo packets.



I agree with everything you wrote about UDP, but it was not the point of 
my original post.


My point was, after some testing and experimentation, I found that using 
"logger" with "-u" parameter, the messages will be truncated to 400 
bytes with most implementations, regardless of 'rsyslogd' .


The reason for that limitation is likely UDP, but this reason is mostly 
off-topic.


The take-home message is that if one wants longer messages, AND is using 
"logger -u", he/she will need a patched version of "logger".


-gordon




___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[David Lang]

On Mon, 10 Feb 2014, Assaf Gordon wrote:


On 02/10/2014 04:56 PM, David Lang wrote:


fragmentation works on UDP messages into packets, and UDP packets can be
any size up to the MTU of your network. so you can send a 4k message
over UDP without using jumbo packets.



I agree with everything you wrote about UDP, but it was not the point of my 
original post.


My point was, after some testing and experimentation, I found that using 
"logger" with "-u" parameter, the messages will be truncated to 400 bytes 
with most implementations, regardless of 'rsyslogd' .


The reason for that limitation is likely UDP, but this reason is mostly 
off-topic.


The take-home message is that if one wants longer messages, AND is using 
"logger -u", he/she will need a patched version of "logger".


Ok, identifying limits in logger is very good to do. It sounded like you were 
saying that those were the limits of using UDP for logs perion, not the limits 
of using UDP with logger.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] templates using local variables as a property

[Micah Yoder]

So is something like this possible? I am writing a template with a few
parts - a base directory name, the hostname of the originating server,
and then the log file name.  After defining the template, some
conditions for the different files.  I just want a different file based
on different conditions, without redefining the template.  So, something
like:

template(name="tpltest" type="list") {
constant(value="/var/log/")
property(name="hostname")
constant(value="/")
property(name="$.filename")
}

if $programname == "micah" and $syslogfacility-text == 'local1' then {
set $.filename = "loggerstuff"
action(type="omfile" Template="tpltest" DirOwner="root"
DirGroup="root" FileOwner="root" FileGroup="root" DirCreateMode="0755"
FileCreateMode="0644")
}

if $programname == "puppet-agent" and $syslogfacility-text ==
'local1' then {
set $.filename = "puppet"
action(type="omfile" Template="tpltest" DirOwner="root"
DirGroup="root" FileOwner="root" FileGroup="root" DirCreateMode="0755"
FileCreateMode="0644")
}


It doesn't seem to be parsing in 7.4.4 which I have installed, but I
might be doing something wrong.  If it isn't possible in this version,
is it in 7.5.x or 8.1.x?

Thanks much!
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] templates using local variables as a property

[David Lang]

On Mon, 10 Feb 2014, Micah Yoder wrote:


So is something like this possible? I am writing a template with a few
parts - a base directory name, the hostname of the originating server,
and then the log file name.  After defining the template, some
conditions for the different files.  I just want a different file based
on different conditions, without redefining the template.  So, something
like:

template(name="tpltest" type="list") {
   constant(value="/var/log/")
   property(name="hostname")
   constant(value="/")
   property(name="$.filename")
}

   if $programname == "micah" and $syslogfacility-text == 'local1' then {
   set $.filename = "loggerstuff"
   action(type="omfile" Template="tpltest" DirOwner="root"
DirGroup="root" FileOwner="root" FileGroup="root" DirCreateMode="0755"
FileCreateMode="0644")
   }

   if $programname == "puppet-agent" and $syslogfacility-text ==
'local1' then {
   set $.filename = "puppet"
   action(type="omfile" Template="tpltest" DirOwner="root"
DirGroup="root" FileOwner="root" FileGroup="root" DirCreateMode="0755"
FileCreateMode="0644")
   }


It doesn't seem to be parsing in 7.4.4 which I have installed, but I
might be doing something wrong.  If it isn't possible in this version,
is it in 7.5.x or 8.1.x?


This is exactly the type of thing that local variables were designed for. But 
just to check, try doing $! instead of $. the variable will show up as part of a 
JSON output of $! if you do this, but it will let you test if this is the 
problem


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] LibLogging 1.0.1 released

[Thomas D.]

Hi Rainer,

you wrote:
> So let me fix this! Thanks for the note. Quite honestly, I am far from
> being an expert on build system issues and this is an oversight.  Anything
> else please let me know. I promise I'll do my best to fix it. Pull requests
> are also happily accepted.

Sorry, I was wrong.

After re-reading I think we have misunderstood us. In the linked thread
I showed you the *package content* where I renamed the library's basename.

Seems like you didn't notice (which is good, because there's no need to
do that) and were referring to the *package name*.

Now I also understand the advice for package maintainers in the README.md...

Again: Everything is fine with liblogging-1.0.1! You don't have to fix
anything. I was wrong.


Final question: Can you tell us an E.T.A. for the new rsyslog 7.4.x
release, which uses the new sonames from liblogging-1.0.1? Thanks!


-Thomas

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] templates using local variables as a property

[Rainer Gerhards]

On Tue, Feb 11, 2014 at 1:10 AM, David Lang  wrote:

> On Mon, 10 Feb 2014, Micah Yoder wrote:
>
>  So is something like this possible? I am writing a template with a few
>> parts - a base directory name, the hostname of the originating server,
>> and then the log file name.  After defining the template, some
>> conditions for the different files.  I just want a different file based
>> on different conditions, without redefining the template.  So, something
>> like:
>>
>> template(name="tpltest" type="list") {
>>constant(value="/var/log/")
>>property(name="hostname")
>>constant(value="/")
>>property(name="$.filename")
>> }
>>
>>if $programname == "micah" and $syslogfacility-text == 'local1' then {
>>set $.filename = "loggerstuff"
>>action(type="omfile" Template="tpltest" DirOwner="root"
>> DirGroup="root" FileOwner="root" FileGroup="root" DirCreateMode="0755"
>> FileCreateMode="0644")
>>}
>>
>>if $programname == "puppet-agent" and $syslogfacility-text ==
>> 'local1' then {
>>set $.filename = "puppet"
>>action(type="omfile" Template="tpltest" DirOwner="root"
>> DirGroup="root" FileOwner="root" FileGroup="root" DirCreateMode="0755"
>> FileCreateMode="0644")
>>}
>>
>>
>> It doesn't seem to be parsing in 7.4.4 which I have installed, but I
>> might be doing something wrong.  If it isn't possible in this version,
>> is it in 7.5.x or 8.1.x?
>>
>
> This is exactly the type of thing that local variables were designed for.
> But just to check, try doing $! instead of $. the variable will show up as
> part of a JSON output of $! if you do this, but it will let you test if
> this is the problem
>
>
local vars are not in 7.4., but I think the actual problem is a different
one. In the config is:

set $.filename = "loggerstuff"

but it must be

set $.filename = "loggerstuff";

Note the semicolon at the end! It's a quirck, but we couldn't get away
without adding the need for it.

Rainer
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] LibLogging 1.0.1 released

[Rainer Gerhards]

On Tue, Feb 11, 2014 at 2:46 AM, Thomas D.  wrote:

> Hi Rainer,
>
> you wrote:
> > So let me fix this! Thanks for the note. Quite honestly, I am far from
> > being an expert on build system issues and this is an oversight.
>  Anything
> > else please let me know. I promise I'll do my best to fix it. Pull
> requests
> > are also happily accepted.
>
> Sorry, I was wrong.
>
>
no problem. I am never sure if I screw up on these matters. So all comments
are highly appreciated.


> After re-reading I think we have misunderstood us. In the linked thread
> I showed you the *package content* where I renamed the library's basename.
>
> Seems like you didn't notice (which is good, because there's no need to
> do that) and were referring to the *package name*.
>
> Now I also understand the advice for package maintainers in the
> README.md...
>
> Again: Everything is fine with liblogging-1.0.1! You don't have to fix
> anything. I was wrong.
>
>
> Final question: Can you tell us an E.T.A. for the new rsyslog 7.4.x
> release, which uses the new sonames from liblogging-1.0.1? Thanks!
>
>
I got a patch over the weekend (regarding pgsql), which I merged in
yesterday. I'd like to get feedback from the contributor but will time out
on that by the end of the week. So I'd say early next week at latest.

Just FYI: this will be 7.4.10, and it will be definitely the last in the
7.4 branch. We'll most probably release 7.6.0, the next stable,
concurrently with 7.4.10.

Once again, many thanks for your comments. If you have more advise, I am
all ears. You'll see liblogging-stdlog evolve quickly in the next couple of
weeks and I'd really appreciate if you could package the newer versions as
well. I talked to a lot of folks on the Fedora DeveloperConference last
weekend and there seemed to be ineeded quite a bit interest in a better,
just still very simple to use syslog lib.

Thanks!
Rainer


>
> -Thomas
>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] RE : Re: maximum message-size / line-length with relp?

[Rainer Gerhards]

Actually, the UDP size limitation is mentioned as the lowest "must be
supported" message size in the syslog UDP transport mapping RFC5426 for
exactly the reasons Assaf pointed out. But the RFC strongly recommends
using higher sizes. HOWEVER, if you are doing *network management* and use
UDP, you better make sure that everything fits in ~400 bytes because the
likelyhood of loosing fragmented messages in a broken network (or one under
attack) is considerably higher than for non-fragmented ones. Maybe (I am
guessing here) that was the motivation for logger developers.

In practice, much larger  UDP messages are no problem, but again keep the
broken network use case on your mind. When we wrote these RFCs we got
sufficient feedback that this could become a problem in practice.

Rainer


On Mon, Feb 10, 2014 at 11:08 PM, David Lang  wrote:

> On Mon, 10 Feb 2014, Assaf Gordon wrote:
>
>  On 02/10/2014 04:56 PM, David Lang wrote:
>>
>>>
>>> fragmentation works on UDP messages into packets, and UDP packets can be
>>> any size up to the MTU of your network. so you can send a 4k message
>>> over UDP without using jumbo packets.
>>>
>>>
>> I agree with everything you wrote about UDP, but it was not the point of
>> my original post.
>>
>> My point was, after some testing and experimentation, I found that using
>> "logger" with "-u" parameter, the messages will be truncated to 400 bytes
>> with most implementations, regardless of 'rsyslogd' .
>>
>> The reason for that limitation is likely UDP, but this reason is mostly
>> off-topic.
>>
>> The take-home message is that if one wants longer messages, AND is using
>> "logger -u", he/she will need a patched version of "logger".
>>
>
> Ok, identifying limits in logger is very good to do. It sounded like you
> were saying that those were the limits of using UDP for logs perion, not
> the limits of using UDP with logger.
>
> David Lang
>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.