Re: [rsyslog] ruleset with only stop
2014-11-10 16:23 GMT+01:00 Brian Knox :
> Today I noticed a ruleset with only "stop" as it's action will fail to
> parse with rsyslog 8.4, but the same rule with a "~" will pass.
>
> ruleset(name="testme") {
> *.* ~
> }
>
> bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
> rsyslogd: version 8.5.0, config validation run (level 1), master config
> ./test.conf
> rsyslogd: warning: ~ action is deprecated, consider using the 'stop'
> statement instead [try http://www.rsyslog.com/e/2307 ]
>
>
> Changing to "stop" :
> ruleset(name="testme") {
> stop
> }
>
> bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
> rsyslogd: version 8.5.0, config validation run (level 1), master config
> ./test.conf
> rsyslogd: CONFIG ERROR: there are no active actions configured. Inputs will
> run, but no output whatsoever is created. [try
> http://www.rsyslog.com/e/2103
> ]
> rsyslogd: run failed with error -2103 (see rsyslog.h or try
> http://www.rsyslog.com/e/2103 to learn what that number means)
>
> I have a situation where rules are being generated via templates in chef,
> and having a rule that just discards messages would actually be a useful
> thing to have, and ran into this.
>
> So my question is, should a rule that only calls a discard action be
> valid? If so, is this a bug in the config parser?
>
>
It's a little bit complex. The thing is that "~" actually *is* an action,
whereas "stop" is a statement. When I wrote that checking code, I never
envisioned that an empty ruleset could be useful for any case (if there is
just a "stop" inside it, it's practically empty, in that it simply does
nothing). I think in most cases this really is a config error. Maybe I
could add an "permitEmpty" parameter to the ruleset, which will then not
emit that error message.
To understand the whole picture: why do you need these empty rulesets?
Rainer
Brian
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] can someone lend me a hand on git procedure
2014-11-11 3:03 GMT+01:00 David Lang : > On Mon, 10 Nov 2014, Thomas D. wrote: > > Hi, >> >> On 2014-11-08 00:19, David Lang wrote: >> >>> Then when the feature is believed to be complete, it gets merged into a >>> 'pending' tree. This tree exists for the sole purpose of having the >>> testbench >>> run against it daily. This tree is not stable, it's recreated for each >>> day's >>> test run by copying the master and then merging in all the pending >>> features that >>> are beleived to be ready to be tested. >>> >> >> Mh... that's sounds like you don't always want to do QA. Then I disagree >> with you. >> >> QA should always be there. You shouldn't think about QA at all, because >> if we have CI, everything will happen automatically in the background. >> You don't need to care but if there is a QA problem, you will be notified. >> >> >> As a note on CI testing, no large entitiy does a full test run against >>> each >>> commit individually, this just doesn't scale. At some point the tests >>> just take >>> too long to run. So they fall back to doing the tests at a larer >>> granularity, >>> possibly with unit tests being run on each change (if you can define >>> unit tests >>> in a way that lets you decide easily what tests are relevant to the code >>> that >>> was changed) >>> >> >> Unit tests should *always* run. >> >> Some functional tests maybe skipped and only run when you are preparing >> a release. >> >> But the discussion is very theoretical at the moment. Github's Travis is >> very powerful. Let's wait until we really know how much time rsyslog's >> test suite would actual take. >> > > it all depends on how long the tests take to run. If it takes a couple > hours to run the tests, you aren't going to want to do them for every > commit. Doing them frequently is important, but it's not so important that > you need to throttle development to the speed that the tests can run. > > >> As for the history being dominated by merges, this is the result of >>> maintaining >>> so many different versions. As John noted when he did the documenation >>> tree, >>> merging changes into the oldest supported version and then merging that >>> into the >>> newer versions lets git do a LOT of work for you in how it remembers what >>> changes have been made and integrating the new changes into it. >>> >> >> Merging down instead of merging up would also help: >> > > I don't think so. Git already knows about the changes in the one > direction, so layering some additional changes on isn't that hard, but > moving the other way requires that git figure out where the change came > from in order to 'do the right thing', and that's a much harder thing to do. > ... especially as (to the best of my knowledge) git does not do that. So on the way down, you need to manually do the merging. Rainer ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
2014-11-11 4:00 GMT+01:00 singh.janmejay : > On Tue, Nov 11, 2014 at 7:43 AM, David Lang wrote: > > > On Mon, 10 Nov 2014, Thomas D. wrote: > > > > Hi, > >> > >> On 2014-11-06 08:10, Rainer Gerhards wrote: > >> > >>> Well, you can run the mysql test suite against any mysql server. It is > independent from source. So that's not the best example at all... > > >>> > >>> Don't you then need root (or similar) credentials to do some very > >>> delicate > >>> database operations like creating databases and tables? I wouldn't > permit > >>> this to be done to my production server. > >>> > >> > >> That's why I asked for running the test suite in a 'sandbox'. > >> > > > > you still need those credentials for your sandbox, right? > > > > If MySQL isn't installed on the system you will need to install the > > package (requiring root + distro specific knowledge), if it is installed, > > you need to configure users and permissions for the test user (requiring > DB > > root equivalent + distro specific knowlege) > > > > > > > > Well, finally I installed valgrind and had to re-install glibc with > debugging symbols (another error the test suite logged but did not > detect). > > >>> > >>> why did you need the debugging symbols? > >>> > >> > >> In Gentoo imdiag is currently only available when compiling with "debug" > >> USE flag. This will also enable "--enable-valgrind".. > >> > >> The error message logged in the "udp-msgreduc-vg" log file was something > >> like > >> > >> valgrind: Fatal error at startup: a function redirection > >>> valgrind: which is mandatory for this platform-tool combination > >>> valgrind: cannot be set up. Details of the redirection are: > >>> valgrind: > >>> valgrind: [...] > >>> valgrind: > >>> valgrind: Possible fixes: (1, short term): install glibc's debuginfo > >>> valgrind: package on this machine. (2, longer term): ask the > packagers > >>> valgrind: for your Linux distribution to please in future ship a non- > >>> valgrind: stripped ld.so (or whatever the dynamic linker .so is > called) > >>> valgrind: that exports the above-named function using the standard > >>> valgrind: calling conventions for this platform. > >>> valgrind: > >>> valgrind: Cannot continue -- exiting now. Sorry. > >>> > >> > >> > >> > >> But let me be clear. From this thread, it looks like these should be > our > >>> project priorities: > >>> > >>> 1. make the testbench fully automatic and self-contained > >>> 2. create unit tests > >>> 3. create more and better integration tests > >>> 4. document > >>> 5. fix bugs > >>> 6. develop new features > >>> > >>> Given the fact that I am I and not a big team like at MySQL, I don't > >>> believe this is the right order of priorities. > >>> > >>> In reality, it is me alone doing >95% of all work, and funding for this > >>> work is almost solely by Adiscon (and only partially backed by paying > >>> customers). > >>> > >> > >> Asking this question shows me that you understand my point. Great! ;) > >> > >> Yes, I am voting for changing the priority order like you said. > >> > >> Don't get me wrong. I am not saying that you MUST change anything. I am > >> only answering the question "If you could change something in the > >> project, what would you change?" ... and I would change the priority to > >> improve the current project quality using automatic tests. > >> > > > > I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 > lives > > is subject to a lot of debate) > > > > you could completely halt all documentation, bugfixing, and new features > > for a year while creating tests to cover everything. > > > > Testing is useful, but the end purpose of the software is not to pass > > tests, but rather to provide capabilities to the user. > > > > As an extreme example, I know of a company that lost a month of > > development and missed having the product ready for a national trade show > > because they got so caught up in writing tests for everything that a test > > that didn't work right when the software was being shutdown (after which > it > > would be reset for the next test) HAD to be written before anything more > > could be done, and there was an upstream bug in one component that caused > > this exit to never work cleanly. Don't fall into a similar trap. > > > > +1 tests are just means to an end, we should be disciplined to write them > as we go, but every project has corners(and sometimes large portions) that > are ill-tested or worse. Paralysing development completely to test-cover > them in one go doesn't sound like the best plan. > > FWIW, here is something that has worked for teams I have been on in the > past. We were aware of which areas were not test-covered, bug-prone, > hard-to-read etc. When we were to touch anything in those areas (like > adding new features that touched those parts of code, or fix a bug etc), we > used to test-cover directly and indirectly affected portions of that code, > and then refacto
Re: [rsyslog] Feedback Request: do we still need -devel versions?
On Tue, 11 Nov 2014, Rainer Gerhards wrote: 2014-11-11 4:00 GMT+01:00 singh.janmejay : On Tue, Nov 11, 2014 at 7:43 AM, David Lang wrote: On Mon, 10 Nov 2014, Thomas D. wrote: Hi, On 2014-11-06 08:10, Rainer Gerhards wrote: Well, you can run the mysql test suite against any mysql server. It is independent from source. So that's not the best example at all... Don't you then need root (or similar) credentials to do some very delicate database operations like creating databases and tables? I wouldn't permit this to be done to my production server. That's why I asked for running the test suite in a 'sandbox'. you still need those credentials for your sandbox, right? If MySQL isn't installed on the system you will need to install the package (requiring root + distro specific knowledge), if it is installed, you need to configure users and permissions for the test user (requiring DB root equivalent + distro specific knowlege) Well, finally I installed valgrind and had to re-install glibc with debugging symbols (another error the test suite logged but did not detect). why did you need the debugging symbols? In Gentoo imdiag is currently only available when compiling with "debug" USE flag. This will also enable "--enable-valgrind".. The error message logged in the "udp-msgreduc-vg" log file was something like valgrind: Fatal error at startup: a function redirection valgrind: which is mandatory for this platform-tool combination valgrind: cannot be set up. Details of the redirection are: valgrind: valgrind: [...] valgrind: valgrind: Possible fixes: (1, short term): install glibc's debuginfo valgrind: package on this machine. (2, longer term): ask the packagers valgrind: for your Linux distribution to please in future ship a non- valgrind: stripped ld.so (or whatever the dynamic linker .so is called) valgrind: that exports the above-named function using the standard valgrind: calling conventions for this platform. valgrind: valgrind: Cannot continue -- exiting now. Sorry. But let me be clear. From this thread, it looks like these should be our project priorities: 1. make the testbench fully automatic and self-contained 2. create unit tests 3. create more and better integration tests 4. document 5. fix bugs 6. develop new features Given the fact that I am I and not a big team like at MySQL, I don't believe this is the right order of priorities. In reality, it is me alone doing >95% of all work, and funding for this work is almost solely by Adiscon (and only partially backed by paying customers). Asking this question shows me that you understand my point. Great! ;) Yes, I am voting for changing the priority order like you said. Don't get me wrong. I am not saying that you MUST change anything. I am only answering the question "If you could change something in the project, what would you change?" ... and I would change the priority to improve the current project quality using automatic tests. I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 lives is subject to a lot of debate) you could completely halt all documentation, bugfixing, and new features for a year while creating tests to cover everything. Testing is useful, but the end purpose of the software is not to pass tests, but rather to provide capabilities to the user. As an extreme example, I know of a company that lost a month of development and missed having the product ready for a national trade show because they got so caught up in writing tests for everything that a test that didn't work right when the software was being shutdown (after which it would be reset for the next test) HAD to be written before anything more could be done, and there was an upstream bug in one component that caused this exit to never work cleanly. Don't fall into a similar trap. +1 tests are just means to an end, we should be disciplined to write them as we go, but every project has corners(and sometimes large portions) that are ill-tested or worse. Paralysing development completely to test-cover them in one go doesn't sound like the best plan. FWIW, here is something that has worked for teams I have been on in the past. We were aware of which areas were not test-covered, bug-prone, hard-to-read etc. When we were to touch anything in those areas (like adding new features that touched those parts of code, or fix a bug etc), we used to test-cover directly and indirectly affected portions of that code, and then refactor them heavily to bring it up to the mark. I try to follow that same paradigm since quite a while. I have to admit it depends on workload, and some test are really *extremely* hard to create. For example, when you want to test out thing with the journal or imuxsock, you need to create a total mock environment and be extremely precise in when, how and where messages are injected and processed. Plus you need to make sure no other component on the running system in
[rsyslog] how to craft a test if the target behaves differently?
Hi experienced QA folks, I have one libdbi test that runs under valgrind (as do many other tests). The goal here is to automatically detect misadressings and memory leaks. The problem now is that different versions of libdbi have different memory "leaks"*. So on most current platforms, valgrind will report a memory. But that's a memory leak outside of my control. The usual approach is to use a suppression file. HOWEVER, the suppression file would need to be different for each platform/libdbi/dbserver combination. So this is not something I can ship as part of the testbench. Does anyone have an idea of how this could be solved? I will try now just to disable leak checking, so that the misaddressing "protection" is at least in place, but that's obviously a degradation of functionality. Help and feedback appreciated, Rainer * It's not a real leak in the case I am talking about, it has to do with when or when not libdbi calls mysql's thread shutdown functions. So this may even a problem that's solely related to the dbi driver layer. This is also nothing that needs to be reported to the libdbi project, because it definitely is fixed -- but as usual distros do not carry the fixed version. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
Hi, On 2014-11-11 03:13, David Lang wrote: > you still need those credentials for your sandbox, right? > > If MySQL isn't installed on the system you will need to install the package > (requiring root + distro specific knowledge), if it is installed, you need to > configure users and permissions for the test user (requiring DB root > equivalent > + distro specific knowlege) Yes, running tests against any third-party application would require access to those applications while testing. Theses tests are called "integration tests". Because you should *not* depend on those third-party applications you often cannot run your integration tests every time you run the normal test suite. Therefore you would split your tests. And this is already done for rsyslog: The mysql tests for example are separated. But this doesn't mean you cannot test the ommysql at all without mysql: You can write a mock so you can test your code at least. BTW: Travis will allow us to test against a running mysql each time... >>> But let me be clear. From this thread, it looks like these should be our >>> project priorities: >>> >>> 1. make the testbench fully automatic and self-contained >>> 2. create unit tests >>> 3. create more and better integration tests >>> 4. document >>> 5. fix bugs >>> 6. develop new features >>> >>> [...] >> >> [...] >> >> Yes, I am voting for changing the priority order like you said. > > I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 lives is > subject to a lot of debate) Wait, this order is not set in stone. For example, to write tests, you have to know what a component should do and how it should be used. So you need a documentation before you can write tests. > you could completely halt all documentation, bugfixing, and new features for > a > year while creating tests to cover everything. > > Testing is useful, but the end purpose of the software is not to pass tests, > but > rather to provide capabilities to the user. > > As an extreme example, I know of a company that lost a month of development > and > missed having the product ready for a national trade show because they got so > caught up in writing tests for everything that a test that didn't work right > when the software was being shutdown (after which it would be reset for the > next > test) HAD to be written before anything more could be done, and there was an > upstream bug in one component that caused this exit to never work cleanly. > Don't > fall into a similar trap. Our views are not so far apart. To be clear: I don't want to establish a process above the product. But the process should be part of the product. Change the way you currently work: When developing a new feature, start with the documentation. If you know how your new feature should look like, should be used, start writing tests so you can be sure your new feature is actual doing what you are are expecting. Now you can start the actual coding... In the end you have a new feature, fully documented and tested. If you don't follow this work flow, you will end up with the current situation: You maybe have great features -- but nobody knows due to missing/hard to use documentation. The tests will help you to keep your hard work working. Imagine you spend days for a new feature but another change will break it. Without tests you will only notice if somebody will actual try to use your feature and fill a bug. But we all know that this don't happen that often. Also, if you start looking for a new product which maybe solve your problem you are currently facing and experience a bug which is a show stopper for you in this moment, you will skip this product. So if Rainer should decide to switch the work flow like described, this would only help new features. But we still have to deal with the past. That's why I recommend to change priority for a moment. Like a "sprint"... like a bug hunt day/week/month where everybody focus on squashing bugs. If there is an upcoming "national trade show" where you want to show something new, yes... do that (but please follow the new work flow). But I am not aware of something like that in the next 4 months and I am not aware of a missing feature which must be implemented ASAP or rsyslog will go away... -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] how to craft a test if the target behaves differently?
Hi,
can't you write something like
> # libdbi
> {
>Leak from dbi/dlopen
>Memcheck:Leak
>fun:calloc
>obj:/lib/libdl-*.so
>fun:dlopen
>fun:dbi_initialize
> }
which should match multiple libdbi versions?
-Thomas
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
On Tue, 11 Nov 2014, Thomas D. wrote: Hi, On 2014-11-11 03:13, David Lang wrote: you still need those credentials for your sandbox, right? If MySQL isn't installed on the system you will need to install the package (requiring root + distro specific knowledge), if it is installed, you need to configure users and permissions for the test user (requiring DB root equivalent + distro specific knowlege) Yes, running tests against any third-party application would require access to those applications while testing. Theses tests are called "integration tests". Because you should *not* depend on those third-party applications you often cannot run your integration tests every time you run the normal test suite. Therefore you would split your tests. And this is already done for rsyslog: The mysql tests for example are separated. But this doesn't mean you cannot test the ommysql at all without mysql: You can write a mock so you can test your code at least. BTW: Travis will allow us to test against a running mysql each time... Add in Oracle, Postgres, RabbitMQ, 0MQ, journald, ElasticSearch, etc. The vast majority of the problems that we run into with rsyslog are in the interfaces to these third party destinations. But let me be clear. From this thread, it looks like these should be our project priorities: 1. make the testbench fully automatic and self-contained 2. create unit tests 3. create more and better integration tests 4. document 5. fix bugs 6. develop new features [...] [...] Yes, I am voting for changing the priority order like you said. I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 lives is subject to a lot of debate) Wait, this order is not set in stone. For example, to write tests, you have to know what a component should do and how it should be used. So you need a documentation before you can write tests. Our views are not so far apart. To be clear: I don't want to establish a process above the product. But the process should be part of the product. Change the way you currently work: When developing a new feature, start with the documentation. If you know how your new feature should look like, should be used, start writing tests so you can be sure your new feature is actual doing what you are are expecting. Now you can start the actual coding... In the end you have a new feature, fully documented and tested. If you don't follow this work flow, you will end up with the current situation: You maybe have great features -- but nobody knows due to missing/hard to use documentation. The tests will help you to keep your hard work working. Imagine you spend days for a new feature but another change will break it. Without tests you will only notice if somebody will actual try to use your feature and fill a bug. But we all know that this don't happen that often. Also, if you start looking for a new product which maybe solve your problem you are currently facing and experience a bug which is a show stopper for you in this moment, you will skip this product. you _are_ argueing for process over product. You are pushing for strict test-driven-development, and while there are a lot of academics who really believe in such development, there are almost no development teams in the real world (commercial or opensource) that work that way for very long. The problem with documentation isn't that no documents get written (with the exception of some contributed modules), but rather that the documentation has been written by someone so close to the problem that there is a huge amount of implied context that isn't obvious to others, and that the organization of the documentation is based on how the person writing it thinks about the problems, not how the people who don't know about the software think about the problems. There are also cases that the author (both software and documentation) didn't think of when programming/wriring, and so those may not be documented. So if Rainer should decide to switch the work flow like described, this would only help new features. But we still have to deal with the past. That's why I recommend to change priority for a moment. Like a "sprint"... like a bug hunt day/week/month where everybody focus on squashing bugs. You keep missing that there isn't a lot of "everybody" in rsyslog development. Rainer is the author of >90% of both the code and the documentation. He is very willing to accept contributions, but we need to get other people involved. The split of the documentation into a different repository was to help ease the process of contributing documentation, but after a brief initial flurry, the outside contributions have tapered off. Please contribute tests, contribute documentation, contribute new features with any process that you want. And feel free to discuss development models, but recognize that there are a lot of them out there and that there is no "one true way". Writing good tests is a lot
Re: [rsyslog] how to craft a test if the target behaves differently?
2014-11-11 14:26 GMT+01:00 Thomas D. :
> Hi,
>
> can't you write something like
>
> > # libdbi
> > {
> >Leak from dbi/dlopen
> >Memcheck:Leak
> >fun:calloc
> >obj:/lib/libdl-*.so
> >fun:dlopen
> >fun:dbi_initialize
> > }
>
> which should match multiple libdbi versions?
>
>
I gave it a try, doesn't seem to be so simple.
I let valgrind generate the suppressions to get started. This is for
Ubuntu14.04:
{
Memcheck:Leak
match-leak-kinds: definite
fun:calloc
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
fun:actionPrepare
fun:processMsgMain
}
This is for CentOS 7:
{
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:*
obj:*
obj:*
obj:*
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
fun:actionPrepare
fun:processMsgMain
fun:doSubmitToActionQ
}
{
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
fun:actionPrepare
fun:processMsgMain
}{
Memcheck:Leak
match-leak-kinds: definite
fun:calloc
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
fun:actionPrepare
fun:processMsgMain
}
{
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:*
obj:*
obj:*
fun:pthread_once
obj:*
obj:*
obj:*
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
}
[I missed one violation here, but that doesn't really matter for the
context].
So it's quite different. I now thought I check if I misunderstood the
format and I can skip some of the contents. So on CentOS I changed
{
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:*
obj:*
obj:*
fun:pthread_once
obj:*
obj:*
obj:*
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
}
to
{
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
obj:*
obj:*
obj:*
fun:pthread_once
obj:*
fun:dbi_conn_connect
fun:initConn
fun:beginTransaction
}
... and the error shows up again.
Anything that I should change?
Rainer
> -Thomas
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
2014-11-11 14:43 GMT+01:00 David Lang : > On Tue, 11 Nov 2014, Thomas D. wrote: > > Hi, >> >> On 2014-11-11 03:13, David Lang wrote: >> >>> you still need those credentials for your sandbox, right? >>> >>> If MySQL isn't installed on the system you will need to install the >>> package >>> (requiring root + distro specific knowledge), if it is installed, you >>> need to >>> configure users and permissions for the test user (requiring DB root >>> equivalent >>> + distro specific knowlege) >>> >> >> Yes, running tests against any third-party application would require >> access to those applications while testing. Theses tests are called >> "integration tests". >> >> Because you should *not* depend on those third-party applications you >> often cannot run your integration tests every time you run the normal >> test suite. >> >> Therefore you would split your tests. And this is already done for >> rsyslog: The mysql tests for example are separated. >> >> But this doesn't mean you cannot test the ommysql at all without mysql: >> You can write a mock so you can test your code at least. >> >> BTW: Travis will allow us to test against a running mysql each time... >> > > Add in Oracle, Postgres, RabbitMQ, 0MQ, journald, ElasticSearch, etc. > > The vast majority of the problems that we run into with rsyslog are in the > interfaces to these third party destinations. > I wouldn't go that far to say it's the "vast majority", but its a big bunch in any case. A common case, however, is that the most useful tests are those that are very hard to create - most notably those things that deal with races. I judge "most useful" by actual bug reports which could have been avoided by a test. > > But let me be clear. From this thread, it looks like these should be our > project priorities: > > 1. make the testbench fully automatic and self-contained > 2. create unit tests > 3. create more and better integration tests > 4. document > 5. fix bugs > 6. develop new features > > [...] > [...] Yes, I am voting for changing the priority order like you said. >>> >>> I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 >>> lives is >>> subject to a lot of debate) >>> >> >> Wait, this order is not set in stone. >> >> For example, to write tests, you have to know what a component should do >> and how it should be used. So you need a documentation before you can >> write tests. >> >> > >> Our views are not so far apart. >> >> To be clear: I don't want to establish a process above the product. >> >> But the process should be part of the product. Change the way you >> currently work: >> >> When developing a new feature, start with the documentation. >> If you know how your new feature should look like, should be used, start >> writing tests so you can be sure your new feature is actual doing what >> you are are expecting. >> Now you can start the actual coding... >> >> In the end you have a new feature, fully documented and tested. >> >> >> If you don't follow this work flow, you will end up with the current >> situation: You maybe have great features -- but nobody knows due to >> missing/hard to use documentation. >> The tests will help you to keep your hard work working. Imagine you >> spend days for a new feature but another change will break it. Without >> tests you will only notice if somebody will actual try to use your >> feature and fill a bug. But we all know that this don't happen that >> often. Also, if you start looking for a new product which maybe solve >> your problem you are currently facing and experience a bug which is a >> show stopper for you in this moment, you will skip this product. >> > > you _are_ argueing for process over product. You are pushing for strict > test-driven-development, and while there are a lot of academics who really > believe in such development, there are almost no development teams in the > real world (commercial or opensource) that work that way for very long. > > The problem with documentation isn't that no documents get written (with > the exception of some contributed modules), but rather that the > documentation has been written by someone so close to the problem that > there is a huge amount of implied context that isn't obvious to others, and > that the organization of the documentation is based on how the person > writing it thinks about the problems, not how the people who don't know > about the software think about the problems. There are also cases that the > author (both software and documentation) didn't think of when > programming/wriring, and so those may not be documented. > +1 > > So if Rainer should decide to switch the work flow like described, this >> would only help new features. But we still have to deal with the past. >> >> That's why I recommend to change priority for a moment. Like a >> "sprint"... like a bug hunt day/week/month where everybody focus on >> squashing bugs. >> > > Yo
Re: [rsyslog] Feedback Request: do we still need -devel versions?
side-note: It looks that with my last commit, the testbench in master branch should now work. I couldn't test all possible option/platform combinations. So it would be good if someone (Thomas? Anyone else?) could try to break it and report back. Thanks, Rainer 2014-11-11 15:20 GMT+01:00 Rainer Gerhards : > > 2014-11-11 14:43 GMT+01:00 David Lang : > >> On Tue, 11 Nov 2014, Thomas D. wrote: >> >> Hi, >>> >>> On 2014-11-11 03:13, David Lang wrote: >>> you still need those credentials for your sandbox, right? If MySQL isn't installed on the system you will need to install the package (requiring root + distro specific knowledge), if it is installed, you need to configure users and permissions for the test user (requiring DB root equivalent + distro specific knowlege) >>> >>> Yes, running tests against any third-party application would require >>> access to those applications while testing. Theses tests are called >>> "integration tests". >>> >>> Because you should *not* depend on those third-party applications you >>> often cannot run your integration tests every time you run the normal >>> test suite. >>> >>> Therefore you would split your tests. And this is already done for >>> rsyslog: The mysql tests for example are separated. >>> >>> But this doesn't mean you cannot test the ommysql at all without mysql: >>> You can write a mock so you can test your code at least. >>> >>> BTW: Travis will allow us to test against a running mysql each time... >>> >> >> Add in Oracle, Postgres, RabbitMQ, 0MQ, journald, ElasticSearch, etc. >> >> The vast majority of the problems that we run into with rsyslog are in >> the interfaces to these third party destinations. >> > > I wouldn't go that far to say it's the "vast majority", but its a big > bunch in any case. A common case, however, is that the most useful tests > are those that are very hard to create - most notably those things that > deal with races. I judge "most useful" by actual bug reports which could > have been avoided by a test. > > >> >> But let me be clear. From this thread, it looks like these should be our >> project priorities: >> >> 1. make the testbench fully automatic and self-contained >> 2. create unit tests >> 3. create more and better integration tests >> 4. document >> 5. fix bugs >> 6. develop new features >> >> [...] >> > > [...] > > Yes, I am voting for changing the priority order like you said. > I disagree that 1, 2, and 3 are more important than 4 and 5 (where 6 lives is subject to a lot of debate) >>> >>> Wait, this order is not set in stone. >>> >>> For example, to write tests, you have to know what a component should do >>> and how it should be used. So you need a documentation before you can >>> write tests. >>> >>> >> >>> Our views are not so far apart. >>> >>> To be clear: I don't want to establish a process above the product. >>> >>> But the process should be part of the product. Change the way you >>> currently work: >>> >>> When developing a new feature, start with the documentation. >>> If you know how your new feature should look like, should be used, start >>> writing tests so you can be sure your new feature is actual doing what >>> you are are expecting. >>> Now you can start the actual coding... >>> >>> In the end you have a new feature, fully documented and tested. >>> >>> >>> If you don't follow this work flow, you will end up with the current >>> situation: You maybe have great features -- but nobody knows due to >>> missing/hard to use documentation. >>> The tests will help you to keep your hard work working. Imagine you >>> spend days for a new feature but another change will break it. Without >>> tests you will only notice if somebody will actual try to use your >>> feature and fill a bug. But we all know that this don't happen that >>> often. Also, if you start looking for a new product which maybe solve >>> your problem you are currently facing and experience a bug which is a >>> show stopper for you in this moment, you will skip this product. >>> >> >> you _are_ argueing for process over product. You are pushing for strict >> test-driven-development, and while there are a lot of academics who really >> believe in such development, there are almost no development teams in the >> real world (commercial or opensource) that work that way for very long. >> >> The problem with documentation isn't that no documents get written (with >> the exception of some contributed modules), but rather that the >> documentation has been written by someone so close to the problem that >> there is a huge amount of implied context that isn't obvious to others, and >> that the organization of the documentation is based on how the person >> writing it thinks about the problems, not how the people who don't know >> about the software think about the problems. There are also cases that the >> author (b
Re: [rsyslog] ruleset with only stop
Rainer,
I agree that an empty ruleset is an oddity. In our case, the short answer
is that we are generating configurations from templates using chef, and the
ability to have a ruleset that simply discards would make part of that
process much simpler for us.
It is admittedly an edge case.
Brian
On Tue, Nov 11, 2014 at 4:06 AM, Rainer Gerhards
wrote:
> 2014-11-10 16:23 GMT+01:00 Brian Knox :
>
> > Today I noticed a ruleset with only "stop" as it's action will fail to
> > parse with rsyslog 8.4, but the same rule with a "~" will pass.
> >
> > ruleset(name="testme") {
> > *.* ~
> > }
> >
> > bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
> > rsyslogd: version 8.5.0, config validation run (level 1), master config
> > ./test.conf
> > rsyslogd: warning: ~ action is deprecated, consider using the 'stop'
> > statement instead [try http://www.rsyslog.com/e/2307 ]
> >
> >
> > Changing to "stop" :
> > ruleset(name="testme") {
> > stop
> > }
> >
> > bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
> > rsyslogd: version 8.5.0, config validation run (level 1), master config
> > ./test.conf
> > rsyslogd: CONFIG ERROR: there are no active actions configured. Inputs
> will
> > run, but no output whatsoever is created. [try
> > http://www.rsyslog.com/e/2103
> > ]
> > rsyslogd: run failed with error -2103 (see rsyslog.h or try
> > http://www.rsyslog.com/e/2103 to learn what that number means)
> >
> > I have a situation where rules are being generated via templates in chef,
> > and having a rule that just discards messages would actually be a useful
> > thing to have, and ran into this.
> >
> > So my question is, should a rule that only calls a discard action be
> > valid? If so, is this a bug in the config parser?
> >
> >
> It's a little bit complex. The thing is that "~" actually *is* an action,
> whereas "stop" is a statement. When I wrote that checking code, I never
> envisioned that an empty ruleset could be useful for any case (if there is
> just a "stop" inside it, it's practically empty, in that it simply does
> nothing). I think in most cases this really is a config error. Maybe I
> could add an "permitEmpty" parameter to the ruleset, which will then not
> emit that error message.
>
> To understand the whole picture: why do you need these empty rulesets?
>
> Rainer
>
> Brian
> > ___
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> >
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] ruleset with only stop
On Tue, 11 Nov 2014, Brian Knox wrote:
Rainer,
I agree that an empty ruleset is an oddity. In our case, the short answer
is that we are generating configurations from templates using chef, and the
ability to have a ruleset that simply discards would make part of that
process much simpler for us.
It is admittedly an edge case.
It's an edge case, but I think it's one that should be supported if possible.
automated config generation is very useful, and being able to group rules into
rulesets and call them with the calling function not having any idea of what is
going to happen with the logs at that point is very useful, being able to have a
high level config split the logs up and call different rulesets on different
logs without having to worry if the ruleset does something with the logs or just
throws them away is _very_ useful.
So it is a corner case, but I think it's a valuable one to support.
I would suggest that at config load time, that this is an odd enough corner case
that it's worth logging a "ruleset X can't do anything with the logs" message,
not just for the case where the only action is to throw it away, but also for
the cases where the conditions in a ruleset cannot possibly match any log
message (if priority = info then *.crit also cannot match anything for example)
David Lang
Brian
On Tue, Nov 11, 2014 at 4:06 AM, Rainer Gerhards
wrote:
2014-11-10 16:23 GMT+01:00 Brian Knox :
Today I noticed a ruleset with only "stop" as it's action will fail to
parse with rsyslog 8.4, but the same rule with a "~" will pass.
ruleset(name="testme") {
*.* ~
}
bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
rsyslogd: version 8.5.0, config validation run (level 1), master config
./test.conf
rsyslogd: warning: ~ action is deprecated, consider using the 'stop'
statement instead [try http://www.rsyslog.com/e/2307 ]
Changing to "stop" :
ruleset(name="testme") {
stop
}
bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
rsyslogd: version 8.5.0, config validation run (level 1), master config
./test.conf
rsyslogd: CONFIG ERROR: there are no active actions configured. Inputs
will
run, but no output whatsoever is created. [try
http://www.rsyslog.com/e/2103
]
rsyslogd: run failed with error -2103 (see rsyslog.h or try
http://www.rsyslog.com/e/2103 to learn what that number means)
I have a situation where rules are being generated via templates in chef,
and having a rule that just discards messages would actually be a useful
thing to have, and ran into this.
So my question is, should a rule that only calls a discard action be
valid? If so, is this a bug in the config parser?
It's a little bit complex. The thing is that "~" actually *is* an action,
whereas "stop" is a statement. When I wrote that checking code, I never
envisioned that an empty ruleset could be useful for any case (if there is
just a "stop" inside it, it's practically empty, in that it simply does
nothing). I think in most cases this really is a config error. Maybe I
could add an "permitEmpty" parameter to the ruleset, which will then not
emit that error message.
To understand the whole picture: why do you need these empty rulesets?
Rainer
Brian
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
Re: [rsyslog] ruleset with only stop
2014-11-11 17:22 GMT+01:00 David Lang :
> On Tue, 11 Nov 2014, Brian Knox wrote:
>
> Rainer,
>>
>> I agree that an empty ruleset is an oddity. In our case, the short answer
>> is that we are generating configurations from templates using chef, and
>> the
>> ability to have a ruleset that simply discards would make part of that
>> process much simpler for us.
>>
>> It is admittedly an edge case.
>>
>
> It's an edge case, but I think it's one that should be supported if
> possible.
>
> automated config generation is very useful, and being able to group rules
> into rulesets and call them with the calling function not having any idea
> of what is going to happen with the logs at that point is very useful,
> being able to have a high level config split the logs up and call different
> rulesets on different logs without having to worry if the ruleset does
> something with the logs or just throws them away is _very_ useful.
>
> So it is a corner case, but I think it's a valuable one to support.
>
>
ack
> I would suggest that at config load time, that this is an odd enough
> corner case that it's worth logging a "ruleset X can't do anything with the
> logs" message, not just for the case where the only action is to throw it
> away, but also for the cases where the conditions in a ruleset cannot
> possibly match any log message (if priority = info then *.crit also cannot
> match anything for example)
>
>
Let's start with what we have on the table. I think it is best to add a
ruleset parameter "permitEmpty=on". That way, we don't generate
error/warning messages when the user is aware of what he is doing. In any
manual case (without config gen), I'd still say that's an error indication.
On the topic of no filter matches. That's quite complex, as you would need
to evaluate all the filters and possible conditions. Not sure if it can
even reliably done. Am I overlooking something?
Rainer
> David Lang
>
>
> Brian
>>
>>
>>
>> On Tue, Nov 11, 2014 at 4:06 AM, Rainer Gerhards <
>> rgerha...@hq.adiscon.com>
>> wrote:
>>
>> 2014-11-10 16:23 GMT+01:00 Brian Knox :
>>>
>>> Today I noticed a ruleset with only "stop" as it's action will fail to
parse with rsyslog 8.4, but the same rule with a "~" will pass.
ruleset(name="testme") {
*.* ~
}
bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
rsyslogd: version 8.5.0, config validation run (level 1), master config
./test.conf
rsyslogd: warning: ~ action is deprecated, consider using the 'stop'
statement instead [try http://www.rsyslog.com/e/2307 ]
Changing to "stop" :
ruleset(name="testme") {
stop
}
bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
rsyslogd: version 8.5.0, config validation run (level 1), master config
./test.conf
rsyslogd: CONFIG ERROR: there are no active actions configured. Inputs
>>> will
>>>
run, but no output whatsoever is created. [try
http://www.rsyslog.com/e/2103
]
rsyslogd: run failed with error -2103 (see rsyslog.h or try
http://www.rsyslog.com/e/2103 to learn what that number means)
I have a situation where rules are being generated via templates in
chef,
and having a rule that just discards messages would actually be a useful
thing to have, and ran into this.
So my question is, should a rule that only calls a discard action be
valid? If so, is this a bug in the config parser?
It's a little bit complex. The thing is that "~" actually *is* an
>>> action,
>>> whereas "stop" is a statement. When I wrote that checking code, I never
>>> envisioned that an empty ruleset could be useful for any case (if there
>>> is
>>> just a "stop" inside it, it's practically empty, in that it simply does
>>> nothing). I think in most cases this really is a config error. Maybe I
>>> could add an "permitEmpty" parameter to the ruleset, which will then not
>>> emit that error message.
>>>
>>> To understand the whole picture: why do you need these empty rulesets?
>>>
>>> Rainer
>>>
>>> Brian
>>>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
___
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>> __
Re: [rsyslog] ruleset with only stop
I think that the "permitEmpty=on" field is a reasonable starting place. I have
a config management app that I use with rsyslog that this field would help
significantly with.
-- James
--- Sent from my mobile phone ---
- Reply message -
From: "Rainer Gerhards"
To: "rsyslog-users"
Subject: [rsyslog] ruleset with only stop
Date: Tue, Nov 11, 2014 10:29 AM
2014-11-11 17:22 GMT+01:00 David Lang :
> On Tue, 11 Nov 2014, Brian Knox wrote:
>
> Rainer,
>>
>> I agree that an empty ruleset is an oddity. In our case, the short answer
>> is that we are generating configurations from templates using chef, and
>> the
>> ability to have a ruleset that simply discards would make part of that
>> process much simpler for us.
>>
>> It is admittedly an edge case.
>>
>
> It's an edge case, but I think it's one that should be supported if
> possible.
>
> automated config generation is very useful, and being able to group rules
> into rulesets and call them with the calling function not having any idea
> of what is going to happen with the logs at that point is very useful,
> being able to have a high level config split the logs up and call different
> rulesets on different logs without having to worry if the ruleset does
> something with the logs or just throws them away is _very_ useful.
>
> So it is a corner case, but I think it's a valuable one to support.
>
>
ack
> I would suggest that at config load time, that this is an odd enough
> corner case that it's worth logging a "ruleset X can't do anything with the
> logs" message, not just for the case where the only action is to throw it
> away, but also for the cases where the conditions in a ruleset cannot
> possibly match any log message (if priority = info then *.crit also cannot
> match anything for example)
>
>
Let's start with what we have on the table. I think it is best to add a
ruleset parameter "permitEmpty=on". That way, we don't generate
error/warning messages when the user is aware of what he is doing. In any
manual case (without config gen), I'd still say that's an error indication.
On the topic of no filter matches. That's quite complex, as you would need
to evaluate all the filters and possible conditions. Not sure if it can
even reliably done. Am I overlooking something?
Rainer
> David Lang
>
>
> Brian
>>
>>
>>
>> On Tue, Nov 11, 2014 at 4:06 AM, Rainer Gerhards <
>> rgerha...@hq.adiscon.com>
>> wrote:
>>
>> 2014-11-10 16:23 GMT+01:00 Brian Knox :
>>>
>>> Today I noticed a ruleset with only "stop" as it's action will fail to
parse with rsyslog 8.4, but the same rule with a "~" will pass.
ruleset(name="testme") {
*.* ~
}
bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
rsyslogd: version 8.5.0, config validation run (level 1), master config
./test.conf
rsyslogd: warning: ~ action is deprecated, consider using the 'stop'
statement instead [try http://www.rsyslog.com/e/2307 ]
Changing to "stop" :
ruleset(name="testme") {
stop
}
bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
rsyslogd: version 8.5.0, config validation run (level 1), master config
./test.conf
rsyslogd: CONFIG ERROR: there are no active actions configured. Inputs
>>> will
>>>
run, but no output whatsoever is created. [try
http://www.rsyslog.com/e/2103
]
rsyslogd: run failed with error -2103 (see rsyslog.h or try
http://www.rsyslog.com/e/2103 to learn what that number means)
I have a situation where rules are being generated via templates in
chef,
and having a rule that just discards messages would actually be a useful
thing to have, and ran into this.
So my question is, should a rule that only calls a discard action be
valid? If so, is this a bug in the config parser?
It's a little bit complex. The thing is that "~" actually *is* an
>>> action,
>>> whereas "stop" is a statement. When I wrote that checking code, I never
>>> envisioned that an empty ruleset could be useful for any case (if there
>>> is
>>> just a "stop" inside it, it's practically empty, in that it simply does
>>> nothing). I think in most cases this really is a config error. Maybe I
>>> could add an "permitEmpty" parameter to the ruleset, which will then not
>>> emit that error message.
>>>
>>> To understand the whole picture: why do you need these empty rulesets?
>>>
>>> Rainer
>>>
>>> Brian
>>>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
___
>>> rsyslog mailing list
>>> http://lists
Re: [rsyslog] ruleset with only stop
+1 for permitEmpty=on - it would definitely simplify our lives.
Brian
On Tue, Nov 11, 2014 at 11:40 AM, Boylan, James
wrote:
> I think that the "permitEmpty=on" field is a reasonable starting place. I
> have a config management app that I use with rsyslog that this field would
> help significantly with.
>
> -- James
> --- Sent from my mobile phone ---
>
> - Reply message -
> From: "Rainer Gerhards"
> To: "rsyslog-users"
> Subject: [rsyslog] ruleset with only stop
> Date: Tue, Nov 11, 2014 10:29 AM
>
> 2014-11-11 17:22 GMT+01:00 David Lang :
>
> > On Tue, 11 Nov 2014, Brian Knox wrote:
> >
> > Rainer,
> >>
> >> I agree that an empty ruleset is an oddity. In our case, the short
> answer
> >> is that we are generating configurations from templates using chef, and
> >> the
> >> ability to have a ruleset that simply discards would make part of that
> >> process much simpler for us.
> >>
> >> It is admittedly an edge case.
> >>
> >
> > It's an edge case, but I think it's one that should be supported if
> > possible.
> >
> > automated config generation is very useful, and being able to group rules
> > into rulesets and call them with the calling function not having any idea
> > of what is going to happen with the logs at that point is very useful,
> > being able to have a high level config split the logs up and call
> different
> > rulesets on different logs without having to worry if the ruleset does
> > something with the logs or just throws them away is _very_ useful.
> >
> > So it is a corner case, but I think it's a valuable one to support.
> >
> >
> ack
>
>
> > I would suggest that at config load time, that this is an odd enough
> > corner case that it's worth logging a "ruleset X can't do anything with
> the
> > logs" message, not just for the case where the only action is to throw it
> > away, but also for the cases where the conditions in a ruleset cannot
> > possibly match any log message (if priority = info then *.crit also
> cannot
> > match anything for example)
> >
> >
> Let's start with what we have on the table. I think it is best to add a
> ruleset parameter "permitEmpty=on". That way, we don't generate
> error/warning messages when the user is aware of what he is doing. In any
> manual case (without config gen), I'd still say that's an error indication.
>
> On the topic of no filter matches. That's quite complex, as you would need
> to evaluate all the filters and possible conditions. Not sure if it can
> even reliably done. Am I overlooking something?
>
> Rainer
>
>
> > David Lang
> >
> >
> > Brian
> >>
> >>
> >>
> >> On Tue, Nov 11, 2014 at 4:06 AM, Rainer Gerhards <
> >> rgerha...@hq.adiscon.com>
> >> wrote:
> >>
> >> 2014-11-10 16:23 GMT+01:00 Brian Knox :
> >>>
> >>> Today I noticed a ruleset with only "stop" as it's action will fail to
> parse with rsyslog 8.4, but the same rule with a "~" will pass.
>
> ruleset(name="testme") {
> *.* ~
> }
>
> bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
> rsyslogd: version 8.5.0, config validation run (level 1), master
> config
> ./test.conf
> rsyslogd: warning: ~ action is deprecated, consider using the 'stop'
> statement instead [try http://www.rsyslog.com/e/2307 ]
>
>
> Changing to "stop" :
> ruleset(name="testme") {
> stop
> }
>
> bknox@seriamau:~$ rsyslogd -N1 -f ./test.conf
> rsyslogd: version 8.5.0, config validation run (level 1), master
> config
> ./test.conf
> rsyslogd: CONFIG ERROR: there are no active actions configured. Inputs
>
> >>> will
> >>>
> run, but no output whatsoever is created. [try
> http://www.rsyslog.com/e/2103
> ]
> rsyslogd: run failed with error -2103 (see rsyslog.h or try
> http://www.rsyslog.com/e/2103 to learn what that number means)
>
> I have a situation where rules are being generated via templates in
> chef,
> and having a rule that just discards messages would actually be a
> useful
> thing to have, and ran into this.
>
> So my question is, should a rule that only calls a discard action be
> valid? If so, is this a bug in the config parser?
>
>
> It's a little bit complex. The thing is that "~" actually *is* an
> >>> action,
> >>> whereas "stop" is a statement. When I wrote that checking code, I never
> >>> envisioned that an empty ruleset could be useful for any case (if there
> >>> is
> >>> just a "stop" inside it, it's practically empty, in that it simply does
> >>> nothing). I think in most cases this really is a config error. Maybe I
> >>> could add an "permitEmpty" parameter to the ruleset, which will then
> not
> >>> emit that error message.
> >>>
> >>> To understand the whole picture: why do you need these empty rulesets?
> >>>
> >>> Rainer
> >>>
> >>> Brian
> >>>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/lis
Re: [rsyslog] Feedback Request: do we still need -devel versions?
Hi, On 2014-11-11 15:24, Rainer Gerhards wrote: > side-note: It looks that with my last commit, the testbench in master > branch should now work. I couldn't test all possible option/platform > combinations. So it would be good if someone (Thomas? Anyone else?) could > try to break it and report back. "imptcp_conndrop.sh" is failing, but this time the test suite won't wait forever, nice! ;) > [...] > PASS: failover-no-rptd-vg.sh > PASS: udp-msgreduc-vg.sh > PASS: udp-msgreduc-orgmsg-vg.sh > PASS: tcp-msgreduc-vg.sh > PASS: manyptcp.sh > PASS: imptcp_large.sh > PASS: imptcp_addtlframedelim.sh > libgcc_s.so.1 must be installed for pthread_cancel to work > FAIL: imptcp_conndrop.sh > PASS: rscript_replace.sh > PASS: rscript_replace_complex.sh > PASS: rscript_wrap2.sh > PASS: rscript_wrap3.sh > [...] > > Testsuite summary for rsyslog 8.5.0 > > # TOTAL: 124 > # PASS: 118 > # SKIP: 5 > # XFAIL: 0 > # FAIL: 1 > # XPASS: 0 > # ERROR: 0 > > See tests/test-suite.log > Please report to rsyslog@lists.adiscon.com > tests/test-suite.log: https://bpaste.net/show/a488b224cfd9 rsyslog was build with: > /var/tmp/portage/app-admin/rsyslog-8./work/rsyslog-8./configure > --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu > --mandir=/usr/share/man --infodir=/usr/share/info > --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib > --disable-dependency-tracking --disable-silent-rules --libdir=/usr/lib64 > --docdir=/usr/share/doc/rsyslog-8. --enable-gene > rate-man-pages --enable-imfile --enable-impstats --enable-imptcp > --enable-imttcp --enable-mmanon --enable-mmaudit --enable-mmfields > --enable-mmjsonparse --enable-mmpstrucdata --enable-mmsequ > ence --enable-mmutf8fix --enable-mail --enable-omprog --enable-omruleset > --enable-omstdout --enable-omuxsock --enable-pmaixforwardedfrom > --enable-pmciscoios --enable-pmcisconames --enable-pm > lastmsg --enable-pmrfc3164sd --enable-pmsnare --disable-libdbi > --disable-ommongodb --disable-mysql --disable-oracle --disable-pgsql > --disable-omhiredis --enable-debug --enable-diagtools --en > able-imdiag --enable-memcheck --enable-rtinst --enable-valgrind > --disable-elasticsearch --enable-libgcrypt --enable-jemalloc > --disable-gssapi-krb5 --disable-mmnormalize --disable-omudpspoof > --disable-omrabbitmq --disable-relp --disable-rfc3195 > --disable-mmrfc5424addhmac --disable-snmp --disable-mmsnmptrapd > --disable-gnutls --disable-imjournal --disable-omjournal --disable-usert > ools --disable-imzmq3 --disable-omzmq3 > --with-systemdsystemunitdir=/usr/lib/systemd/system --enable-testbench Not sure about > libgcc_s.so.1 must be installed for pthread_cancel to work => > # locate libgcc_s.so.1 > /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 > /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/32/libgcc_s.so.1 and # pwd /var/tmp/portage/app-admin/rsyslog-8./work/rsyslog-8./tests # srcdir=. ./imptcp_conndrop.sh TEST: [imptcp_conndrop.sh]: test imptcp with random connection drops rsyslogd started with pid 3462 00020 open connections starting run 1 Sending 5 messages. 0005 messages sent runtime: 2.442 00020 close connections -D option initiated 2477 connection closures End of tcpflood Run imdiag[13500]: mainqueue empty libgcc_s.so.1 must be installed for pthread_cancel to work ./diag.sh: line 16: 3462 Aborted (core dumped) $valgrind ../tools/rsyslogd -u2 -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
On 2014-11-11 22:02, Thomas D. wrote: > Not sure about > >> libgcc_s.so.1 must be installed for pthread_cancel to work > > => > >> # locate libgcc_s.so.1 >> /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 >> /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/32/libgcc_s.so.1 > > and > > # pwd > /var/tmp/portage/app-admin/rsyslog-8./work/rsyslog-8./tests > > # srcdir=. ./imptcp_conndrop.sh > > TEST: [imptcp_conndrop.sh]: test imptcp with random connection drops > rsyslogd started with pid 3462 > 00020 open connections > starting run 1 > Sending 5 messages. > 0005 messages sent > runtime: 2.442 > 00020 close connections > -D option initiated 2477 connection closures > End of tcpflood Run > imdiag[13500]: mainqueue empty > libgcc_s.so.1 must be installed for pthread_cancel to work > ./diag.sh: line 16: 3462 Aborted (core dumped) > $valgrind ../tools/rsyslogd -u2 -n -irsyslog$3.pid > -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 Back trace: > #0 0x7f623d0fcec7 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:55 > #1 0x7f623d0fe2ca in __GI_abort () at abort.c:89 > #2 0x0042fd8f in sigsegvHdlr (signum=6) at debug.c:852 > #3 > #4 0x7f623d0fcec7 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:55 > #5 0x7f623d0fe2ca in __GI_abort () at abort.c:89 > #6 0x7f623d13af34 in __libc_message (do_abort=do_abort@entry=1, > fmt=fmt@entry=0x7f623d22b3cf "%s") at ../sysdeps/posix/libc_fatal.c:175 > #7 0x7f623d13af5e in __GI___libc_fatal (message=0x7f623e2cb3d8 > "libgcc_s.so.1 must be installed for pthread_cancel to work\n") at > ../sysdeps/posix/libc_fatal.c:186 > #8 0x7f623e2c9864 in pthread_cancel_init () at > ../sysdeps/nptl/unwind-forcedunwind.c:64 > #9 0x7f623e2c994c in _Unwind_ForcedUnwind (exc=0x7f623bdd0d70, > stop=stop@entry=0x7f623e2c7c60 , stop_argument=0x7f623bdcfeb0) > at ../sysdeps/nptl/unwind-forcedunwind.c:129 > #10 0x7f623e2c7dd0 in __GI___pthread_unwind (buf=) at > unwind.c:129 > #11 0x7f623e2c2465 in __do_cancel () at pthreadP.h:277 > #12 __pthread_exit (value=) at pthread_exit.c:29 > #13 0x0045df82 in thrdStarter (arg=0x7f623c82d3e0) at ../threads.c:227 > #14 0x7f623e2c13c4 in start_thread (arg=0x7f623bdd0700) at > pthread_create.c:310 > #15 0x7f623d1ada2d in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 Using glibc-2.20. Related to: https://sourceware.org/bugzilla/show_bug.cgi?id=13119 ? At least when running the example from the bug report I am getting a similar back trace: > #0 0x7fc39c017ec7 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:55 > #1 0x7fc39c0192ca in __GI_abort () at abort.c:89 > #2 0x7fc39c055f34 in __libc_message (do_abort=do_abort@entry=1, > fmt=fmt@entry=0x7fc39c1463cf "%s") at ../sysdeps/posix/libc_fatal.c:175 > #3 0x7fc39c055f5e in __GI___libc_fatal (message=0x7fc39c3903d8 > "libgcc_s.so.1 must be installed for pthread_cancel to work\n") at > ../sysdeps/posix/libc_fatal.c:186 > #4 0x7fc39c38e864 in pthread_cancel_init () at > ../sysdeps/nptl/unwind-forcedunwind.c:64 > #5 0x7fc39c38e94c in _Unwind_ForcedUnwind (exc=0x7fc39bfe2d70, > stop=stop@entry=0x7fc39c38cc60 , stop_argument=0x7fc39bfe1f70) > at ../sysdeps/nptl/unwind-forcedunwind.c:129 > #6 0x7fc39c38cdd0 in __GI___pthread_unwind (buf=) at > unwind.c:129 > #7 0x7fc39c387465 in __do_cancel () at pthreadP.h:277 > #8 __pthread_exit (value=) at pthread_exit.c:29 > #9 0x00400721 in foo (ptr=0x0) at foo.c:15 > #10 0x7fc39c3863c4 in start_thread (arg=0x7fc39bfe2700) at > pthread_create.c:310 > #11 0x7fc39c0c8a2d in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
If thread _ cancel does not work all sorts of things can go wrong. Do you know why the platform is defective? Any way to detect this? So that we during configure can tell it does not work? Rainer Sent from phone, thus brief. Am 11.11.2014 22:21 schrieb "Thomas D." : > On 2014-11-11 22:02, Thomas D. wrote: > > Not sure about > > > >> libgcc_s.so.1 must be installed for pthread_cancel to work > > > > => > > > >> # locate libgcc_s.so.1 > >> /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 > >> /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/32/libgcc_s.so.1 > > > > and > > > > # pwd > > /var/tmp/portage/app-admin/rsyslog-8./work/rsyslog-8./tests > > > > # srcdir=. ./imptcp_conndrop.sh > > > > > TEST: [imptcp_conndrop.sh]: test imptcp with random connection drops > > rsyslogd started with pid 3462 > > 00020 open connections > > starting run 1 > > Sending 5 messages. > > 0005 messages sent > > runtime: 2.442 > > 00020 close connections > > -D option initiated 2477 connection closures > > End of tcpflood Run > > imdiag[13500]: mainqueue empty > > libgcc_s.so.1 must be installed for pthread_cancel to work > > ./diag.sh: line 16: 3462 Aborted (core dumped) > > $valgrind ../tools/rsyslogd -u2 -n -irsyslog$3.pid > > -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 > > Back trace: > > > #0 0x7f623d0fcec7 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:55 > > #1 0x7f623d0fe2ca in __GI_abort () at abort.c:89 > > #2 0x0042fd8f in sigsegvHdlr (signum=6) at debug.c:852 > > #3 > > #4 0x7f623d0fcec7 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:55 > > #5 0x7f623d0fe2ca in __GI_abort () at abort.c:89 > > #6 0x7f623d13af34 in __libc_message (do_abort=do_abort@entry=1, > fmt=fmt@entry=0x7f623d22b3cf "%s") at ../sysdeps/posix/libc_fatal.c:175 > > #7 0x7f623d13af5e in __GI___libc_fatal (message=0x7f623e2cb3d8 > "libgcc_s.so.1 must be installed for pthread_cancel to work\n") at > ../sysdeps/posix/libc_fatal.c:186 > > #8 0x7f623e2c9864 in pthread_cancel_init () at > ../sysdeps/nptl/unwind-forcedunwind.c:64 > > #9 0x7f623e2c994c in _Unwind_ForcedUnwind (exc=0x7f623bdd0d70, > stop=stop@entry=0x7f623e2c7c60 , > stop_argument=0x7f623bdcfeb0) at ../sysdeps/nptl/unwind-forcedunwind.c:129 > > #10 0x7f623e2c7dd0 in __GI___pthread_unwind (buf=) at > unwind.c:129 > > #11 0x7f623e2c2465 in __do_cancel () at pthreadP.h:277 > > #12 __pthread_exit (value=) at pthread_exit.c:29 > > #13 0x0045df82 in thrdStarter (arg=0x7f623c82d3e0) at > ../threads.c:227 > > #14 0x7f623e2c13c4 in start_thread (arg=0x7f623bdd0700) at > pthread_create.c:310 > > #15 0x7f623d1ada2d in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 > > Using glibc-2.20. > > Related to: https://sourceware.org/bugzilla/show_bug.cgi?id=13119 ? > > At least when running the example from the bug report I am getting a > similar back trace: > > > #0 0x7fc39c017ec7 in __GI_raise (sig=sig@entry=6) at > ../sysdeps/unix/sysv/linux/raise.c:55 > > #1 0x7fc39c0192ca in __GI_abort () at abort.c:89 > > #2 0x7fc39c055f34 in __libc_message (do_abort=do_abort@entry=1, > fmt=fmt@entry=0x7fc39c1463cf "%s") at ../sysdeps/posix/libc_fatal.c:175 > > #3 0x7fc39c055f5e in __GI___libc_fatal (message=0x7fc39c3903d8 > "libgcc_s.so.1 must be installed for pthread_cancel to work\n") at > ../sysdeps/posix/libc_fatal.c:186 > > #4 0x7fc39c38e864 in pthread_cancel_init () at > ../sysdeps/nptl/unwind-forcedunwind.c:64 > > #5 0x7fc39c38e94c in _Unwind_ForcedUnwind (exc=0x7fc39bfe2d70, > stop=stop@entry=0x7fc39c38cc60 , > stop_argument=0x7fc39bfe1f70) at ../sysdeps/nptl/unwind-forcedunwind.c:129 > > #6 0x7fc39c38cdd0 in __GI___pthread_unwind (buf=) at > unwind.c:129 > > #7 0x7fc39c387465 in __do_cancel () at pthreadP.h:277 > > #8 __pthread_exit (value=) at pthread_exit.c:29 > > #9 0x00400721 in foo (ptr=0x0) at foo.c:15 > > #10 0x7fc39c3863c4 in start_thread (arg=0x7fc39bfe2700) at > pthread_create.c:310 > > #11 0x7fc39c0c8a2d in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:109 > > > -Thomas > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyon
Re: [rsyslog] ruleset with only stop
On Tue, 11 Nov 2014, Rainer Gerhards wrote: 2014-11-11 17:22 GMT+01:00 David Lang : On Tue, 11 Nov 2014, Brian Knox wrote: Rainer, I agree that an empty ruleset is an oddity. In our case, the short answer is that we are generating configurations from templates using chef, and the ability to have a ruleset that simply discards would make part of that process much simpler for us. It is admittedly an edge case. It's an edge case, but I think it's one that should be supported if possible. automated config generation is very useful, and being able to group rules into rulesets and call them with the calling function not having any idea of what is going to happen with the logs at that point is very useful, being able to have a high level config split the logs up and call different rulesets on different logs without having to worry if the ruleset does something with the logs or just throws them away is _very_ useful. So it is a corner case, but I think it's a valuable one to support. ack I would suggest that at config load time, that this is an odd enough corner case that it's worth logging a "ruleset X can't do anything with the logs" message, not just for the case where the only action is to throw it away, but also for the cases where the conditions in a ruleset cannot possibly match any log message (if priority = info then *.crit also cannot match anything for example) Let's start with what we have on the table. I think it is best to add a ruleset parameter "permitEmpty=on". That way, we don't generate error/warning messages when the user is aware of what he is doing. In any manual case (without config gen), I'd still say that's an error indication. I think that this is a sufficently corner case that I'm not sure it's worth the extra complexity to silence the warning. I think that people who do this intentionally can just ignore the log message. On the topic of no filter matches. That's quite complex, as you would need to evaluate all the filters and possible conditions. Not sure if it can even reliably done. Am I overlooking something? I am not saying that it should try to catch every possible case, but I was thinking that the configuration optimization step would "optomize away" some impossible combinations, and that could result in an empty ruleset. David Lang ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
if you can find the diag.sh that it's using, look at what binaries it's calling and do an ldd on them, it's likely that that library isn't in the ldconfig path. David Lang On Tue, 11 Nov 2014, Thomas D. wrote: Date: Tue, 11 Nov 2014 22:02:41 +0100 From: Thomas D. Reply-To: rsyslog-users To: rsyslog@lists.adiscon.com Subject: Re: [rsyslog] Feedback Request: do we still need -devel versions? Hi, On 2014-11-11 15:24, Rainer Gerhards wrote: side-note: It looks that with my last commit, the testbench in master branch should now work. I couldn't test all possible option/platform combinations. So it would be good if someone (Thomas? Anyone else?) could try to break it and report back. "imptcp_conndrop.sh" is failing, but this time the test suite won't wait forever, nice! ;) [...] PASS: failover-no-rptd-vg.sh PASS: udp-msgreduc-vg.sh PASS: udp-msgreduc-orgmsg-vg.sh PASS: tcp-msgreduc-vg.sh PASS: manyptcp.sh PASS: imptcp_large.sh PASS: imptcp_addtlframedelim.sh libgcc_s.so.1 must be installed for pthread_cancel to work FAIL: imptcp_conndrop.sh PASS: rscript_replace.sh PASS: rscript_replace_complex.sh PASS: rscript_wrap2.sh PASS: rscript_wrap3.sh [...] Testsuite summary for rsyslog 8.5.0 # TOTAL: 124 # PASS: 118 # SKIP: 5 # XFAIL: 0 # FAIL: 1 # XPASS: 0 # ERROR: 0 See tests/test-suite.log Please report to rsyslog@lists.adiscon.com tests/test-suite.log: https://bpaste.net/show/a488b224cfd9 rsyslog was build with: /var/tmp/portage/app-admin/rsyslog-8./work/rsyslog-8./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --disable-dependency-tracking --disable-silent-rules --libdir=/usr/lib64 --docdir=/usr/share/doc/rsyslog-8. --enable-gene rate-man-pages --enable-imfile --enable-impstats --enable-imptcp --enable-imttcp --enable-mmanon --enable-mmaudit --enable-mmfields --enable-mmjsonparse --enable-mmpstrucdata --enable-mmsequ ence --enable-mmutf8fix --enable-mail --enable-omprog --enable-omruleset --enable-omstdout --enable-omuxsock --enable-pmaixforwardedfrom --enable-pmciscoios --enable-pmcisconames --enable-pm lastmsg --enable-pmrfc3164sd --enable-pmsnare --disable-libdbi --disable-ommongodb --disable-mysql --disable-oracle --disable-pgsql --disable-omhiredis --enable-debug --enable-diagtools --en able-imdiag --enable-memcheck --enable-rtinst --enable-valgrind --disable-elasticsearch --enable-libgcrypt --enable-jemalloc --disable-gssapi-krb5 --disable-mmnormalize --disable-omudpspoof --disable-omrabbitmq --disable-relp --disable-rfc3195 --disable-mmrfc5424addhmac --disable-snmp --disable-mmsnmptrapd --disable-gnutls --disable-imjournal --disable-omjournal --disable-usert ools --disable-imzmq3 --disable-omzmq3 --with-systemdsystemunitdir=/usr/lib/systemd/system --enable-testbench Not sure about libgcc_s.so.1 must be installed for pthread_cancel to work => # locate libgcc_s.so.1 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.3/32/libgcc_s.so.1 and # pwd /var/tmp/portage/app-admin/rsyslog-8./work/rsyslog-8./tests # srcdir=. ./imptcp_conndrop.sh TEST: [imptcp_conndrop.sh]: test imptcp with random connection drops rsyslogd started with pid 3462 00020 open connections starting run 1 Sending 5 messages. 0005 messages sent runtime: 2.442 00020 close connections -D option initiated 2477 connection closures End of tcpflood Run imdiag[13500]: mainqueue empty libgcc_s.so.1 must be installed for pthread_cancel to work ./diag.sh: line 16: 3462 Aborted (core dumped) $valgrind ../tools/rsyslogd -u2 -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] ruleset with only stop
If was able to use an empty ruleset, a warning resulting from that wouldn't bother me at all. Brian On Tue, Nov 11, 2014 at 4:25 PM, David Lang wrote: > On Tue, 11 Nov 2014, Rainer Gerhards wrote: > > 2014-11-11 17:22 GMT+01:00 David Lang : >> >> On Tue, 11 Nov 2014, Brian Knox wrote: >>> >>> Rainer, >>> I agree that an empty ruleset is an oddity. In our case, the short answer is that we are generating configurations from templates using chef, and the ability to have a ruleset that simply discards would make part of that process much simpler for us. It is admittedly an edge case. >>> It's an edge case, but I think it's one that should be supported if >>> possible. >>> >>> automated config generation is very useful, and being able to group rules >>> into rulesets and call them with the calling function not having any idea >>> of what is going to happen with the logs at that point is very useful, >>> being able to have a high level config split the logs up and call >>> different >>> rulesets on different logs without having to worry if the ruleset does >>> something with the logs or just throws them away is _very_ useful. >>> >>> So it is a corner case, but I think it's a valuable one to support. >>> >>> >>> ack >> >> >> I would suggest that at config load time, that this is an odd enough >>> corner case that it's worth logging a "ruleset X can't do anything with >>> the >>> logs" message, not just for the case where the only action is to throw it >>> away, but also for the cases where the conditions in a ruleset cannot >>> possibly match any log message (if priority = info then *.crit also >>> cannot >>> match anything for example) >>> >>> >>> Let's start with what we have on the table. I think it is best to add a >> ruleset parameter "permitEmpty=on". That way, we don't generate >> error/warning messages when the user is aware of what he is doing. In any >> manual case (without config gen), I'd still say that's an error >> indication. >> > > I think that this is a sufficently corner case that I'm not sure it's > worth the extra complexity to silence the warning. I think that people who > do this intentionally can just ignore the log message. > > On the topic of no filter matches. That's quite complex, as you would need >> to evaluate all the filters and possible conditions. Not sure if it can >> even reliably done. Am I overlooking something? >> > > I am not saying that it should try to catch every possible case, but I was > thinking that the configuration optimization step would "optomize away" > some impossible combinations, and that could result in an empty ruleset. > > David Lang > > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] tls + tcp input
Is it possible to specify TLS on a per input basis for the TCP input, or is it all or nothing? Brian ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
On 2014-11-11 22:27, David Lang wrote: > if you can find the diag.sh that it's using, look at what binaries it's > calling > and do an ldd on them, it's likely that that library isn't in the ldconfig > path. To make sure the build environment doesn't have the problem (because when I manually run the test, the test seems to pass) I added some debugging code to "imptcp_conndrop.sh", see https://bpaste.net/show/c52e4da0a446 Now the output looks like https://bpaste.net/show/b35599ee85c0 Looks good, not? ...and the manual run passed again: http://pastebin.com/raw.php?i=niXvTeUt Not sure what's going on :/ -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] Feedback Request: do we still need -devel versions?
On 2014-11-11 22:24, Rainer Gerhards wrote: > If thread _ cancel does not work all sorts of things can go wrong. Do you > know why the platform is defective? Any way to detect this? So that we > during configure can tell it does not work? At the moment I don't understand what's going on. The core dump created in rsyslog's test seems to be identical with the one you'll get when running the code from glibc's bug tracker like I quoted in my previous mail. But because when I manually run the failed rsyslog test again after "make check" the test will pass... no pthread_cancel error. So *I* cannot tell you at the moment if there's a problem with pthread_cancel at all you need to get around. -Thomas ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [rsyslog] [Lognorm] Tokenized-multivalue field-type for liblognorm
Sorry, this should be discussed on rsyslog mailing list, adding rsyslog as
well.
Here are the pull requests(in order):
https://github.com/rsyslog/libestr/pull/1
https://github.com/rsyslog/liblognorm/pull/8
https://github.com/rsyslog/rsyslog/pull/149
On Wed, Nov 12, 2014 at 9:55 AM, singh.janmejay
wrote:
> The array-subscripting patches are ready, I'll send a pull request on
> github.
>
> It is of the form $!foo[1]!bar[2]!baz.
>
> On Wed, Nov 12, 2014 at 9:52 AM, singh.janmejay
> wrote:
>
>> How do we want foreach support to be?
>>
>> Possible places are:
>> 1. a foreach loop-construct along-side flow-control structures in rscript
>> 2. a foreach value-producing templateElement in template
>>
>> The tradeoff is, someone looking to do some useful re-structuring of
>> output, may want to call exec-template several times in approach-1, but
>> they can just emit out what they want by placing property-extractors in
>> foreach block in approch-2.
>>
>> The counter argument is, templates can't assign variables, execute
>> functions, do arithmetic-operations etc, so usefulness of foreach in a
>> template will be limited to most basic of message transformation cases.
>> Additionally, string-template as of now has no clean way of implementing
>> foreach. It'll require some involved syntax.
>>
>> To me approach-1 seems better suited for such a thing (purely because
>> if-else, the other flow control structure is implemented at this level, and
>> variable assignment, functions etc make arbitrary transformation easier).
>>
>> Once we have decided where to place foreach, let us zero in on its
>> syntax. To seed the thought, how about this:
>>
>> set $.grault = "";
>> foreach($!foo as $.bar) {
>> set $.baz = $.bar!quux;
>> foreach($.baz as $.corge) {
>> if ( $.corge contains "#" ) then {
>> set $.grault = $.grault & $.corge;
>> }
>> }
>> }
>>
>> Alternatively:
>> foreach($.bar <- $!foo) {...}
>> foreach($.bar in $!foo) {...}
>> foreach($.bar : $!foo) {...}
>>
>>
>>
>> On Fri, Oct 31, 2014 at 6:46 PM, singh.janmejay > > wrote:
>>
>>> Sure, I'll fork on github.
>>>
>>> On Fri, Oct 31, 2014 at 6:42 PM, Rainer Gerhards <
>>> rgerha...@hq.adiscon.com> wrote:
>>>
2014-10-31 14:08 GMT+01:00 singh.janmejay :
> Cool, I'll implement $!foo!bar[0].
>
> +1
> Let us process this patch-set, because is kinda hard to keep track of
> old patches and re-send in one shot.
>
>
would you mind cloning on github and maintain a feature branch there?
That would make it much easier for me, as I could merge the branch when you
are done. If not, it's no problem and I'll maintain that branch.
i'll send the new patch once done(i'll now only get to work on it on
> monday).
>
>
I haven't had a chance to look as I am now busy building test
environments and looking at the testbench [yes, one guy!] ;) But I see
Pavel has asked some questions. He recently did a lot of work on the lib,so
it is best to coordinate that part with him.
Rainer
Do existing patches look ok except for the indexed-addressing feature?
>
> On Fri, Oct 31, 2014 at 4:15 PM, David Lang wrote:
>
>> On Fri, 31 Oct 2014, singh.janmejay wrote:
>>
>> Yes, I didn't have a need to address tokens individually, but you
>>> have a
>>> point.
>>>
>>> Any suggestions on what we want to do for addressing array elements?
>>>
>>> I wonder if its possible to do in $!... notation without breaking
>>> backward
>>> compatibility. How about a function?
>>>
>>> I'll be happy to implement support for addressing it in $!...
>>> notation if
>>> don't mind breaking a corner case in backward compatibility. Eg.
>>> $!foo!bar![0] ? Its kinda ugly though, or so I think.
>>>
>>
>> I was thinking just $!foo!bar[0] it's a bit ugly, but not too bad. It
>> does mean that you can't have '[' in a variable name, but I don't think
>> that's likely to be a real problem. I don't think there's ever a really
>> clean way to do something like $!foo[2]!bar[2]!baz no matter what your
>> syntax, it gets messy.
>>
>> for templates, we probably need some sort of foreach(array, pattern)
>> function that takes the pattern and repeats it for each item in the
>> array.
>>
>> David Lang
>>
>>
>> On Fri, Oct 31, 2014 at 3:44 PM, David Lang wrote:
>>>
>>> On Fri, 31 Oct 2014, singh.janmejay wrote:
It writes it as a json array, here is a fragment from my manual
tests:
>
> [ "15", "26", "15" ]
>
>
right, but how do you access it in rsyslog?
if you have { 'foo': { 'bar': '10' } } you access this as $!foo!bar
and
get the result '10'
what would you use to access the value '26' in your example?
