Re: [rsyslog] Compiling rsyslog-8.15.0 : undefined reference to `rpl_malloc'

2016-01-15 Thread Rainer Gerhards
2016-01-15 19:07 GMT+01:00 Rainer Gerhards :
> 2016-01-15 18:59 GMT+01:00 Girish Kumar :
>> Thanks Rainer for the suggestion.
>>
>> Still I am getting same error after commenting AC_FUNC_MALLOC and 
>> AC_FUNC_REALLOC. I commented  this both in libestr and rsylog
>>
>> /home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/7.X.X.R01/armv7/lib/libestr.so:
>>  undefined reference to `rpl_malloc'
>>
>> Could you please let me know what is the problem.
>
> https://www.google.de/search?q=%60rpl_malloc&oq=%60rpl_malloc&aqs=chrome..69i57j0l5.543j0j7&sourceid=chrome&es_sm=93&ie=UTF-8#q=rpl_malloc
>
> first two restults look promising. That's the way how I found out ;)

I should add that it just worked for me following the advise to
comment out these macros. So I do not know any internals of this
autoconf macro problem.

Rainer
>
> HTH
> Rainer
>
>> Am I missing any library ?
>>
>> Regards,
>> Girish
>>
>> -Original Message-
>> From: rsyslog-boun...@lists.adiscon.com 
>> [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Wednesday, January 13, 2016 7:26 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Compiling rsyslog-8.15.0 : undefined reference to 
>> `rpl_malloc'
>>
>> I recently stumbled over this myself when compiling with clang. This seems 
>> to be caused by problems in those autoconf macros in
>> configure.ac:
>>
>> AC_FUNC_MALLOC
>> AC_FUNC_REALLOC
>>
>> On the web, you find suggestions that everything is fine if these are 
>> commented out. As it looks, this seems to be the case, but I have not yet 
>> fully evaluated the impact.
>>
>> HTH
>> Rainer
>>
>> 2016-01-13 14:47 GMT+01:00 Girish Kumar :
>>> Hi  All,
>>>
>>> I am getting following error while compiling rsyslog-8.15.0 for armv7
>>> platform  .  Could you please help on this
>>>
>>>
>>>
>>> configure:4039: arm-unknown-linux-gnueabi-gcc -mcpu=cortex-a9
>>> -mabi=aapcs-linux -msoft-float -O -fPIC -fno-omit-frame-pointer
>>> -mapcs-frame -mno-sched-prolog
>>> -I/soft/gnu/cross/linux_64/13/arm/sysroot/usr/include
>>> -I/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
>>> 7.X.X.R01/armv7/include -mcpu=cortex-a9 -mabi=aapcs-linux -msoft-float
>>> -O -fno-omit-frame-pointer -mapcs-frame -mno-sched-prolog
>>> -I/soft/gnu/cross/linux_64/13/arm/sysroot/usr/include
>>> -I/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
>>> 7.X.X.R01/armv7/include -mcpu=cortex-a9 -mabi=aapcs-linux -msoft-float
>>> -L/soft/gnu/cross/linux_64/13/arm/sysroot/lib
>>> -L/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
>>> 7.X.X.R01/armv7/lib -lcrypto -lestr conftest.c  >&5
>>> /home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/7.X.X.R01/armv7/lib/libestr.so:
>>>  undefined reference to `rpl_malloc'
>>>
>>>
>>> Regards,
>>> Girish
>>> ___
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
>>> LIKE THAT.
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This 
>> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our 
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
>> LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] Compiling rsyslog-8.15.0 : undefined reference to `rpl_malloc'

2016-01-15 Thread Rainer Gerhards
2016-01-15 18:59 GMT+01:00 Girish Kumar :
> Thanks Rainer for the suggestion.
>
> Still I am getting same error after commenting AC_FUNC_MALLOC and 
> AC_FUNC_REALLOC. I commented  this both in libestr and rsylog
>
> /home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/7.X.X.R01/armv7/lib/libestr.so:
>  undefined reference to `rpl_malloc'
>
> Could you please let me know what is the problem.

https://www.google.de/search?q=%60rpl_malloc&oq=%60rpl_malloc&aqs=chrome..69i57j0l5.543j0j7&sourceid=chrome&es_sm=93&ie=UTF-8#q=rpl_malloc

first two restults look promising. That's the way how I found out ;)

HTH
Rainer

> Am I missing any library ?
>
> Regards,
> Girish
>
> -Original Message-
> From: rsyslog-boun...@lists.adiscon.com 
> [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, January 13, 2016 7:26 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Compiling rsyslog-8.15.0 : undefined reference to 
> `rpl_malloc'
>
> I recently stumbled over this myself when compiling with clang. This seems to 
> be caused by problems in those autoconf macros in
> configure.ac:
>
> AC_FUNC_MALLOC
> AC_FUNC_REALLOC
>
> On the web, you find suggestions that everything is fine if these are 
> commented out. As it looks, this seems to be the case, but I have not yet 
> fully evaluated the impact.
>
> HTH
> Rainer
>
> 2016-01-13 14:47 GMT+01:00 Girish Kumar :
>> Hi  All,
>>
>> I am getting following error while compiling rsyslog-8.15.0 for armv7
>> platform  .  Could you please help on this
>>
>>
>>
>> configure:4039: arm-unknown-linux-gnueabi-gcc -mcpu=cortex-a9
>> -mabi=aapcs-linux -msoft-float -O -fPIC -fno-omit-frame-pointer
>> -mapcs-frame -mno-sched-prolog
>> -I/soft/gnu/cross/linux_64/13/arm/sysroot/usr/include
>> -I/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
>> 7.X.X.R01/armv7/include -mcpu=cortex-a9 -mabi=aapcs-linux -msoft-float
>> -O -fno-omit-frame-pointer -mapcs-frame -mno-sched-prolog
>> -I/soft/gnu/cross/linux_64/13/arm/sysroot/usr/include
>> -I/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
>> 7.X.X.R01/armv7/include -mcpu=cortex-a9 -mabi=aapcs-linux -msoft-float
>> -L/soft/gnu/cross/linux_64/13/arm/sysroot/lib
>> -L/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
>> 7.X.X.R01/armv7/lib -lcrypto -lestr conftest.c  >&5
>> /home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/7.X.X.R01/armv7/lib/libestr.so:
>>  undefined reference to `rpl_malloc'
>>
>>
>> Regards,
>> Girish
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites 
>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
>> THAT.
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This 
> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our 
> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
> LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] Compiling rsyslog-8.15.0 : undefined reference to `rpl_malloc'

2016-01-15 Thread Girish Kumar
Thanks Rainer for the suggestion.

Still I am getting same error after commenting AC_FUNC_MALLOC and 
AC_FUNC_REALLOC. I commented  this both in libestr and rsylog

/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/7.X.X.R01/armv7/lib/libestr.so:
 undefined reference to `rpl_malloc'

Could you please let me know what is the problem. Am I missing any library ?

Regards,
Girish

-Original Message-
From: rsyslog-boun...@lists.adiscon.com 
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards
Sent: Wednesday, January 13, 2016 7:26 PM
To: rsyslog-users
Subject: Re: [rsyslog] Compiling rsyslog-8.15.0 : undefined reference to 
`rpl_malloc'

I recently stumbled over this myself when compiling with clang. This seems to 
be caused by problems in those autoconf macros in
configure.ac:

AC_FUNC_MALLOC
AC_FUNC_REALLOC

On the web, you find suggestions that everything is fine if these are commented 
out. As it looks, this seems to be the case, but I have not yet fully evaluated 
the impact.

HTH
Rainer

2016-01-13 14:47 GMT+01:00 Girish Kumar :
> Hi  All,
>
> I am getting following error while compiling rsyslog-8.15.0 for armv7 
> platform  .  Could you please help on this
>
>
>
> configure:4039: arm-unknown-linux-gnueabi-gcc -mcpu=cortex-a9 
> -mabi=aapcs-linux -msoft-float -O -fPIC -fno-omit-frame-pointer 
> -mapcs-frame -mno-sched-prolog  
> -I/soft/gnu/cross/linux_64/13/arm/sysroot/usr/include 
> -I/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
> 7.X.X.R01/armv7/include -mcpu=cortex-a9 -mabi=aapcs-linux -msoft-float 
> -O -fno-omit-frame-pointer -mapcs-frame -mno-sched-prolog  
> -I/soft/gnu/cross/linux_64/13/arm/sysroot/usr/include 
> -I/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
> 7.X.X.R01/armv7/include -mcpu=cortex-a9 -mabi=aapcs-linux -msoft-float  
> -L/soft/gnu/cross/linux_64/13/arm/sysroot/lib 
> -L/home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/
> 7.X.X.R01/armv7/lib -lcrypto -lestr conftest.c  >&5
> /home/gkuma020/PERFORCE/7.X.X.R01-gkuma020-new/engr/sw/tps/built/46/7.X.X.R01/armv7/lib/libestr.so:
>  undefined reference to `rpl_malloc'
>
>
> Regards,
> Girish
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE 
> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites 
> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is 
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our 
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] Transaction support for omkafka

2016-01-15 Thread Brian Knox
Hello Ralph! I'm the initial author of the omhiredis output. I haven't
looked at the omkafka source code yet so I can't speak to it's complexity -
but I can say that writing the transaction support for omhiredis was fairly
simple.  It was just implementing the beginTransaction / endTransaction
macro blocks in addition to the doAction block.

If knowing at the rsyslog level what is going on with your queues is
important, I'd lean towards taking a crack at transaction support in
omkafka.

I'm currently using omelasticsearch heavily - and being able to look in my
imstats logs to see what's happening with my queues and actions definitely
makes it simple to tell what is going on at a glance.

Just my thoughts!
Cheers,
Brian

On Fri, Jan 15, 2016 at 12:31 AM, Ralph Caraveo  wrote:

> Hello,
>
> (I apologize if this got sent twice but I may have sent it too soon before
> the mailing list registration process was completed.)
>
> I'm hoping the group can provide some guidance around a requirement we
> have to have transactional support around having an Rsyslog OM module that
> writes to Kafka.
>
> What we'd like to do, is leverage consuming data from Rsyslog and posting
> to Kafka however it looks like the OMKafka module doesn't currently support
> transactions when posting to Kafka in the event of an error.  So, if we
> write to Kafka, and an error occurs, it looks like we lose the log-line for
> that particular Action item unless we write it to a fallback log file.
>
> Additionally, we've looked at writing our own Kafka Producer using the
> OMProg style where we consume off of STDIN and then connect to kafka and
> produce data.  Unfortunately this approach also doesn't allow us to
> communicate back to Rsyslog that a failure has happened in the event of an
> error.
>
> We tried to deal with this by writing to a fallback text file, and this
> works great when there are errors with Kafka, but if the process dies
> between receiving from STDIN and before writing to Kafka, we can still
> potentially lose messages.
>
> Additionally, I noticed that the OMHIREDIS client does support
> transactions, so it sounds like we want the design of OMHIREDIS (where it
> utilizes transactions) but with that support in OMKafka.
>
> I'm just looking for a recommendation on a way forward from the group.  If
> it makes more sense to enhance OMKafka to have transaction support or if we
> try to go down the path of adding some type of transaction support to the
> Omprog module which would allow us to continue using our custom Kafka
> producer of which is written in Go.
>
> Thanks for anyone's time around this!
>
> Ralph
> @deckarep
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] Feedback request

2016-01-15 Thread David Lang

On Fri, 15 Jan 2016, David Lang wrote:


On Fri, 15 Jan 2016, Rainer Gerhards wrote:


Hi folks,

I would like to get some feedback on this potentially
compatibility-breaking but long-standing bug:

https://github.com/rsyslog/rsyslog/pull/604


My feeling is that fixing the off-by-one error is good, but eliminating the 
fields that you use to populate a standard property should not be done by 
default, especially if it hasn't been done up to now.


Adding an option to do so is reasonable, but someone may want to check if the 
property was populated from JSON or from some other fallback (plus it would 
break existing configs)


as an opt-in option, it is a good idea. As a change in behavior it's 
horrible.


If we didn't have any existing behavior, there would be room for a lot more 
discussion. I'm not sure which way that discussion would end up (I see valid 
reasons on both sides) but breaking existing configs would be a major issue, 
and this is nowhere near a big enough reason.




put a tickler somewhere so that we can have this discussion when we look at 
moving to 9.x, that would be a time to consider changeing the defaults. We have 
a few other things that fall in this category as well, but nothing big.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] Feedback request

2016-01-15 Thread David Lang

On Fri, 15 Jan 2016, Rainer Gerhards wrote:


Hi folks,

I would like to get some feedback on this potentially
compatibility-breaking but long-standing bug:

https://github.com/rsyslog/rsyslog/pull/604


My feeling is that fixing the off-by-one error is good, but eliminating the 
fields that you use to populate a standard property should not be done by 
default, especially if it hasn't been done up to now.


Adding an option to do so is reasonable, but someone may want to check if the 
property was populated from JSON or from some other fallback (plus it would 
break existing configs)


as an opt-in option, it is a good idea. As a change in behavior it's horrible.

If we didn't have any existing behavior, there would be room for a lot more 
discussion. I'm not sure which way that discussion would end up (I see valid 
reasons on both sides) but breaking existing configs would be a major issue, and 
this is nowhere near a big enough reason.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


[rsyslog] Feedback request

2016-01-15 Thread Rainer Gerhards
Hi folks,

I would like to get some feedback on this potentially
compatibility-breaking but long-standing bug:

https://github.com/rsyslog/rsyslog/pull/604

Thanks,
Rainer
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.