I'm running 8.21 (self compiled) with all the latest libraries and am not having problems with queues.

the error you are getting (fatal error on disk queue 'action 5 queue[DA]', emergency switch to direct mode) is one I've run across when there are permission problems with the directory that the queue is in, or if there are old queue files and no .qi file

David Lang

On Wed, 27 Jul 2016, Alec Swan wrote:

Date: Wed, 27 Jul 2016 12:10:32 -0600
From: Alec Swan <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] libfastjson 0.99.3 released

Rainer, per your request I re-ran the test and was able to start rsyslog
with the newly downloaded binaries. The following shows test output. I also
included the logs from /var/log/messages which indicate that there might be
a problem with some queues. Could you please confirm that this is not a
regression of the queue related issues fixed in 8.18.0 or 8.19.0?

# yum erase rsyslog-mmutf8fix rsyslog-elasticsearch rsyslog-mmnormalize
liblognorm1 rsyslog json-c libfastjson4
...
Removed:
 json-c.x86_64 0:0.11-11.el6                     libfastjson4.x86_64
0:0.99.3-2.el6            liblognorm1.x86_64 0:1.1.3-1.el6
rsyslog.x86_64 0:8.20.0-1.el6
 rsyslog-elasticsearch.x86_64 0:8.20.0-1.el6
rsyslog-mmnormalize.x86_64 0:8.20.0-1.el6     rsyslog-mmutf8fix.x86_64
0:8.20.0-1.el6

Complete!

# rpm -qa | grep rsys

# yumdownloader --resolve rsyslog-mmnormalize-8.20.0
rsyslog-elasticsearch-8.20.0 rsyslog-mmutf8fix-8.20.0
...
rsyslog_v8

 | 2.5 kB     00:00
rsyslog_v8/primary_db

| 264 kB     00:00
--> Running transaction check
---> Package rsyslog-elasticsearch.x86_64 0:8.20.0-1.el6 will be installed
--> Processing Dependency: rsyslog = 8.20.0-1.el6 for package:
rsyslog-elasticsearch-8.20.0-1.el6.x86_64
---> Package rsyslog-mmnormalize.x86_64 0:8.20.0-1.el6 will be installed
--> Processing Dependency: liblognorm.so.5()(64bit) for package:
rsyslog-mmnormalize-8.20.0-1.el6.x86_64
---> Package rsyslog-mmutf8fix.x86_64 0:8.20.0-1.el6 will be installed
--> Running transaction check
---> Package liblognorm2.x86_64 0:2.0.0-1.el6 will be installed
--> Processing Dependency: libfastjson.so.4()(64bit) for package:
liblognorm2-2.0.0-1.el6.x86_64
---> Package rsyslog.x86_64 0:8.20.0-1.el6 will be installed
--> Running transaction check
---> Package libfastjson4.x86_64 0:0.99.3-2.el6 will be installed
--> Finished Dependency Resolution
rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm

|  13 kB     00:00
rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm

|  24 kB     00:00
rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm

|  12 kB     00:00
rsyslog-8.20.0-1.el6.x86_64.rpm

| 691 kB     00:00
liblognorm2-2.0.0-1.el6.x86_64.rpm

 |  70 kB     00:00
libfastjson4-0.99.3-2.el6.x86_64.rpm

 |  46 kB     00:00

# ls
libfastjson4-0.99.3-2.el6.x86_64.rpm  rsyslog-8.20.0-1.el6.x86_64.rpm
        rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm
liblognorm2-2.0.0-1.el6.x86_64.rpm
rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm
rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm

# rpm -ivh *.rpm
warning: libfastjson4-0.99.3-2.el6.x86_64.rpm: Header V3 RSA/SHA1
Signature, key ID e00b8985: NOKEY
Preparing...                ###########################################
[100%]
  1:libfastjson4           ########################################### [
17%]
  2:rsyslog                ########################################### [
33%]
  3:liblognorm2            ########################################### [
50%]
  4:rsyslog-mmnormalize    ########################################### [
67%]
  5:rsyslog-elasticsearch  ########################################### [
83%]
  6:rsyslog-mmutf8fix      ###########################################
[100%]

# ps -ax | grep rsyslog
Warning: bad syntax, perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ
3494 pts/0    S+     0:00 grep rsyslog

# service rsyslog start
Starting system logger:                                    [  OK  ]

# ps -ax | grep rsyslog
Warning: bad syntax, perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ
3578 ?        Ssl    0:00 /sbin/rsyslogd -i /var/run/syslogd.pid
3602 pts/0    S+     0:00 grep rsyslog

# tail /var/log/messages
Jul 27 18:06:10 rsyslogd: [origin software="rsyslogd" swVersion="8.20.0"
x-pid="3526" x-info="http://www.rsyslog.com";] exiting on signal 15.
Jul 27 18:06:11 rsyslogd: [origin software="rsyslogd" swVersion="8.20.0"
x-pid="3578" x-info="http://www.rsyslog.com";] start
Jul 27 18:06:11 rsyslogd-2040: fatal error on disk queue 'action 5
queue[DA]', emergency switch to direct mode [v8.20.0 try
http://www.rsyslog.com/e/2040 ]
Jul 27 18:06:11 rsyslogd-2040: fatal error on disk queue 'action 3
queue[DA]', emergency switch to direct mode [v8.20.0 try
http://www.rsyslog.com/e/2040 ]
Jul 27 18:06:11 rsyslogd-2040: fatal error on disk queue 'action 0
queue[DA]', emergency switch to direct mode [v8.20.0 try
http://www.rsyslog.com/e/2040 ]
Jul 27 18:06:11 rsyslogd-2221: module 'imuxsock' already in this config,
cannot be added  [v8.20.0 try http://www.rsyslog.com/e/2221 ]
Jul 27 18:06:11 rsyslogd-2221: module 'imklog' already in this config,
cannot be added  [v8.20.0 try http://www.rsyslog.com/e/2221 ]

Thanks,

Alec

On Wed, Jul 27, 2016 at 11:03 AM, Rainer Gerhards <[email protected]>
wrote:

As I said, I tried for almost two weeks with different settings, for sure
more than 100 tries. This is just the latest state. In the end I focused on
trusty, to save some time. If we can get that pic error go away, we can
look at the other ones.

Rainer

Sent from phone, thus brief.

Am 27.07.2016 18:44 schrieb "David Lang" <[email protected]>:

On Wed, 27 Jul 2016, Rainer Gerhards wrote:

2016-07-27 16:41 GMT+02:00 David Lang <[email protected]>:

On Wed, 27 Jul 2016, Rainer Gerhards wrote:

That's unfortunately not the case for the Ubuntu repo. I always
consistently recompiled everything. Actually the problem occurs during
that
process, when e.g. liblognorm is compiled.



I am only building for trusty, but I am using the rsyslog-pkg-ubuntu
scripts
to do the compiles, everything up to the point of uploading the results
to
the PPA

are you getting the failure on trusty, or on other versions?


I get them on trusty and also on other versions. See here:

https://launchpad.net/~adiscon/+archive/ubuntu/v8-devel/+packages

I have the impression that the exact versions which fail change, but
trusty seems always affected.


what I am seeing at that link is an error that it wasn't able to install
libfastjson-dev (exact message: Depends: libfastjson-dev but it is not
installable)

hmm, digging further, it looks like the trusty amd64 build errors out
with
the -fPIC message, but all the other errors I see from the 22nd are
uninstallable libfastjson-dev

and indeed, there is not a libfastjson package for xenial or wily

have you tried rebuilding libfastjson just in case something went wrong
there?

David Lang

Rainer


David Lang

Rainer

Sent from phone, thus brief.

Am 27.07.2016 12:06 schrieb "David Lang" <[email protected]>:

On Wed, 27 Jul 2016, Rainer Gerhards wrote:


Thanks for the elaborate output. I think we may be getting closer.
This triggered my attention:

--> Processing Dependency: libjson-c.so.2()(64bit) for package:
liblognorm1-1.1.3-1.el6.x86_64

I think we need to rebuild liblognorm with the new libfastjson
package. Otherwise we can see subtle problems with the json
handlers,
as there core structures are different (much more in 0.99.3 than
0.99.2 of libfastjson). This doesn't explain yet the line number
problems we saw, but in anycase that's something to look at. I will
let you know when we have updates in the repo.


Yes, as I have been building with the git master for libfastjson and
liblognorm for several months, I found that if I didn't recompile
everything at once to be on the new version of libraries that both
used,
bad things happened.

I've been suspecting that this sort of skew has been a large part of
the
repo problems of the last few weeks as new versions of libraries have
been
released at different times than rsyslog packages. I've been
suspecting
that if everything was recompiled, things would work (so 8.19.0
packages
break with the new libraries, but I expected 8.20 packages compiled
against
the new libraries to then work)

David Lang

In parallel, I also install a CentOS6 (don't have RHEL) machine and


see what exactly it installes when we pull from the testing repo.

Thanks again,
Rainer

2016-07-27 1:27 GMT+02:00 Alec Swan <[email protected]>:

I configured /etc/yum.repos.d/rsyslog.repo to point to the "testing"
rsyslog repo and removed all other yum repos. After that I
downloaded
all
RPMS from the "testing" repo using yumdownloader and installed them
with
"rpm -ivh". The end result is the same. The following shows the
output
of
the test.

# more /etc/yum.repos.d/rsyslog.repo
[rsyslog_v8]
name=Adiscon CentOS-$releasever - local packages for $basearch
baseurl=http://rpms.adiscon.com/testing/epel-$releasever/$basearch
enabled=1
gpgcheck=0
gpgkey=http://rpms.adiscon.com/RPM-GPG-KEY-Adiscon
protect=1

# yumdownloader --resolve rsyslog-mmnormalize-8.20.0
rsyslog-elasticsearch-8.20.0 rsyslog-mmutf8fix-8.20.0
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
...
rsyslog_v8

  | 2.5 kB     00:00
rsyslog_v8/primary_db

 | 264 kB     00:00
--> Running transaction check
---> Package rsyslog-elasticsearch.x86_64 0:8.20.0-1.el6 will be
installed
--> Processing Dependency: rsyslog = 8.20.0-1.el6 for package:
rsyslog-elasticsearch-8.20.0-1.el6.x86_64
---> Package rsyslog-mmnormalize.x86_64 0:8.20.0-1.el6 will be
installed
--> Processing Dependency: liblognorm.so.2()(64bit) for package:
rsyslog-mmnormalize-8.20.0-1.el6.x86_64
---> Package rsyslog-mmutf8fix.x86_64 0:8.20.0-1.el6 will be
installed
--> Running transaction check
---> Package liblognorm1.x86_64 0:1.1.3-1.el6 will be installed
--> Processing Dependency: libjson-c.so.2()(64bit) for package:
liblognorm1-1.1.3-1.el6.x86_64
---> Package rsyslog.x86_64 0:8.20.0-1.el6 will be installed
--> Processing Dependency: libfastjson.so.4()(64bit) for package:
rsyslog-8.20.0-1.el6.x86_64
--> Running transaction check
---> Package json-c.x86_64 0:0.11-11.el6 will be installed
---> Package libfastjson4.x86_64 0:0.99.3-2.el6 will be installed
--> Finished Dependency Resolution
rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm

 |  13 kB     00:00
rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm

 |  24 kB     00:00
rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm

 |  12 kB     00:00
rsyslog-8.20.0-1.el6.x86_64.rpm

 | 690 kB     00:00
json-c-0.11-11.el6.x86_64.rpm

 |  51 kB     00:00
libfastjson4-0.99.3-2.el6.x86_64.rpm

  |  46 kB     00:00
liblognorm1-1.1.3-1.el6.x86_64.rpm

  |  46 kB     00:00

# ls
json-c-0.11-11.el6.x86_64.rpm
liblognorm1-1.1.3-1.el6.x86_64.rpm
 rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm
 rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm
libfastjson4-0.99.3-2.el6.x86_64.rpm
rsyslog-8.20.0-1.el6.x86_64.rpm
rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm

# rpm -ivh *.rpm
warning: libfastjson4-0.99.3-2.el6.x86_64.rpm: Header V3 RSA/SHA1
Signature, key ID e00b8985: NOKEY
Preparing...
###########################################
[100%]
   1:libfastjson4
 ###########################################
[
14%]
   2:rsyslog
###########################################
[
29%]
   3:json-c
 ###########################################
[
43%]
   4:liblognorm1
###########################################
[
57%]
   5:rsyslog-mmnormalize
###########################################
[
71%]
   6:rsyslog-elasticsearch
###########################################
[
86%]
   7:rsyslog-mmutf8fix
###########################################
[100%]

# service rsyslog start
Starting system logger:                                    [  OK  ]

# ps -ax | grep rsys
Warning: bad syntax, perhaps a bogus '-'? See
/usr/share/doc/procps-3.2.8/FAQ
 1020 pts/0    S+     0:00 grep rsys

# dmesg
...
rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp
00007f799e66f7e8
error 4 in libc-2.12.so[7f79a5d25000+18a000]

# service rsyslog start
Starting system logger:                                    [  OK  ]

# dmesg
...
rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp
00007f799e66f7e8
error 4 in libc-2.12.so[7f79a5d25000+18a000]
rs:main Q:Reg[1063]: segfault at 1b ip 00007f9a44249778 sp
00007f9a3c1de878
error 6 in libfastjson.so.4.0.0[7f9a44246000+9000]

# service rsyslog start
Starting system logger:                                    [  OK  ]

# dmesg
...
rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp
00007f799e66f7e8
error 4 in libc-2.12.so[7f79a5d25000+18a000]
rs:main Q:Reg[1063]: segfault at 1b ip 00007f9a44249778 sp
00007f9a3c1de878
error 6 in libfastjson.so.4.0.0[7f9a44246000+9000]
rs:main Q:Reg[1093]: segfault at 0 ip 00007f32e00c22b6 sp
00007f32d88e47e8
error 4 in libc-2.12.so[7f32dff9a000+18a000]


Thanks,

Alec

On Tue, Jul 26, 2016 at 10:18 AM, Rainer Gerhards <
[email protected]>
wrote:

Am 26.07.2016 17:58 schrieb "Alec Swan" <[email protected]>:




I downloaded the packages from adiscon "testing" repo, then
uploaded
them
to our local Nexus repo and then installed them with yum. So, the
source

is

our local Nexus repo.

Unfortunately, the "testing" repo does not seem to have libee.so

dependency

required by libfastjson, so I can't just use "testing" repo to
install

all

rsyslog dependencies.


Something is fishy here. I just checked the spec file, and
libfastjson
does
not have a dependency on libee. Is there a problem with the
staging
process? Or can you try from the original repo, not the mirror?

Rainer



Thanks,

Alec

On Tue, Jul 26, 2016 at 9:36 AM, Rainer Gerhards <

[email protected]


wrote:

2016-07-26 17:35 GMT+02:00 Alec Swan <[email protected]>:



Rainer, per Florian's request in "experimental rpms for

rsyslog-8.20.0" I



downloaded RPMs from the "testing" repo baseurl=


http://rpms.adiscon.com/testing/epel-$releasever/$basearch



Can you check where the system says it actually installed them
from.
The line numbers do not match the RPMs Florian created, thus I
ask.

Rainer



Thanks,

Alec

On Tue, Jul 26, 2016 at 7:30 AM, Rainer Gerhards <

[email protected]>

wrote:

I today tried quite a while to reproduce the issue with your
config,


unfortunately without any success. For now, I conclude that
this
is
not a general problem, but one induced by your environment and
data.

I have then looked again at the traces and the few line
numbers
it
contains. Now comes the interesting fact. I notice that there
is a
mismatch between the line numbers shown and the code that
actually

was



used to build the packages (judging from git and the source RPMs).



So



I wonder what might have gone wrong here. Can you please let me
know


which exact repro you installed from and potentially which
other

repos



you have configured on that machine. It might also be useful to



query



the system from where exactly the packages have ben installed. I



found



[1], which hopefully helps to get that information.



Once I have this information, I can check if there are any
version
quirks (note that commit [2] fixed a situation that could
cause
exactly what you see). If there is no issue with the package

sources,



I unfortunately need your help in further troubleshooting this in



your



environment. But let's first see what the repo check will bring
up.



Rainer

[1]







http://serverfault.com/questions/62026/how-to-know-from-which-yum-repository-a-package-has-been-installed

[2]









https://github.com/rsyslog/libfastjson/commit/0499ec8df09c913156ead426c171031b566dbe7d


2016-07-23 19:28 GMT+02:00 Alec Swan <[email protected]>:

Yes, that should be OK. Is there a link to the build server/CI
env

I



can



use?



Thanks,

Alec

On Sat, Jul 23, 2016 at 7:08 AM, Rainer Gerhards <

[email protected]>

wrote:

Thanks. Would it be OK for your if a somewhat mangled version
of



these



configs show up inside the testbench? I think about adding a



specific



test for it, that would be great for the future as well. I'd



probably



try to strip it a bit down (e.g. no ES but writing to file, which


would enable runs on all CI environments and when no ES is

available



-- I assume this would also trigger the bug).



Please let me know,
Rainer

2016-07-22 17:46 GMT+02:00 Alec Swan <[email protected]>:

Rainer, I attached /etc/rsyslog.conf and

/etc/rsyslog.d/rsyslog-custom.conf


Thanks,

Alec

On Fri, Jul 22, 2016 at 12:42 AM, Rainer Gerhards <

[email protected]>

wrote:

unfortunately, we do not seem to have debug symbols, so I



don't



see



much more. Can you share your full config (including all



includes, if



any)? Maybe I can get it to abort in my lab as well.



Rainer

2016-07-21 22:02 GMT+02:00 Alec Swan <[email protected]
:

Rainer, here is the output from running rsyslog under

valgrind:




[root valgrind-3.11.0]# valgrind rsyslogd
==2371== Memcheck, a memory error detector
==2371== Copyright (C) 2002-2015, and GNU GPL'd, by
Julian

Seward



et



al.



==2371== Using Valgrind-3.11.0 and LibVEX; rerun with -h for



copyright



info



==2371== Command: rsyslogd
==2371==
==2371== Conditional jump or move depends on
uninitialised

value(s)



==2371==    at 0x16A414: cnfexprDestruct (in /sbin/rsyslogd)


==2371==    by 0x16B36D: cnfstmtOptimize (in
/sbin/rsyslogd)
==2371==    by 0x146317: ??? (in /sbin/rsyslogd)
==2371==    by 0x12E5BE: llExecFunc (in /sbin/rsyslogd)
==2371==    by 0x146176: rulesetOptimizeAll (in

/sbin/rsyslogd)



==2371==    by 0x123EF6: ??? (in /sbin/rsyslogd)


==2371==    by 0x158A68: ??? (in /sbin/rsyslogd)
==2371==    by 0x159008: main (in /sbin/rsyslogd)
==2371==
==2375== Warning: invalid file descriptor 99990 in
syscall

close()



==2371==


==2371== HEAP SUMMARY:
==2371==     in use at exit: 209,122 bytes in 2,121
blocks
==2371==   total heap usage: 11,004 allocs, 8,883 frees,

4,319,525



bytes



allocated


==2371==
==2371== LEAK SUMMARY:
==2371==    definitely lost: 26 bytes in 1 blocks
==2371==    indirectly lost: 0 bytes in 0 blocks
==2371==      possibly lost: 88 bytes in 2 blocks
==2371==    still reachable: 209,008 bytes in 2,118
blocks
==2371==         suppressed: 0 bytes in 0 blocks
==2371== Rerun with --leak-check=full to see details of

leaked



memory



==2371==


==2371== For counts of detected and suppressed errors,
rerun

with:



-v



==2371== Use --track-origins=yes to see where uninitialised



values



come



from



==2371== ERROR SUMMARY: 1 errors from 1 contexts

(suppressed:



114



from 7)



[root valgrind-3.11.0]# ==2375== Thread 6 rs:main Q:Reg:


==2375== Invalid read of size 8
==2375==    at 0x56607BE: fjson_object_iter_next
(json_object_iterator.c:119)
==2375==    by 0x5660863: fjson_object_iter_begin
(json_object_iterator.c:77)
==2375==    by 0x126F09: ??? (in /sbin/rsyslogd)
==2375==    by 0x1270A0: msgAddJSON (in /sbin/rsyslogd)
==2375==    by 0x7242F2D: ??? (in

/lib64/rsyslog/mmnormalize.so)



==2375==    by 0x14EA64: ??? (in /sbin/rsyslogd)


==2375==    by 0x14F561: ??? (in /sbin/rsyslogd)
==2375==    by 0x14F923: ??? (in /sbin/rsyslogd)
==2375==    by 0x146AB6: ??? (in /sbin/rsyslogd)
==2375==    by 0x1473DA: ??? (in /sbin/rsyslogd)
==2375==    by 0x156FE3: ??? (in /sbin/rsyslogd)
==2375==    by 0x1457D8: ??? (in /sbin/rsyslogd)
==2375==  Address 0xf57d338 is 0 bytes after a block of
size

72



alloc'd



==2375==    at 0x4A05FEF: calloc (vg_replace_malloc.c:711)


==2375==    by 0x7659871: json_object_new

(json_object.c:178)



==2375==    by 0x765A61A: json_object_new_object



(json_object.c:354)



==2375==    by 0x744A355: ln_normalize (in



/usr/lib64/liblognorm.so.2.0.0)

==2375==    by 0x7242EEB: ??? (in

/lib64/rsyslog/mmnormalize.so)



==2375==    by 0x14EA64: ??? (in /sbin/rsyslogd)


==2375==    by 0x14F561: ??? (in /sbin/rsyslogd)
==2375==    by 0x14F923: ??? (in /sbin/rsyslogd)
==2375==    by 0x146AB6: ??? (in /sbin/rsyslogd)
==2375==    by 0x1473DA: ??? (in /sbin/rsyslogd)
==2375==    by 0x156FE3: ??? (in /sbin/rsyslogd)
==2375==    by 0x1457D8: ??? (in /sbin/rsyslogd)
==2375==
==2375== Invalid read of size 4
==2375==    at 0x5660778: fjson_object_get

(json_object.c:176)



==2375==    by 0x126F57: ??? (in /sbin/rsyslogd)


==2375==    by 0x1270A0: msgAddJSON (in /sbin/rsyslogd)
==2375==    by 0x7242F2D: ??? (in

/lib64/rsyslog/mmnormalize.so)



==2375==    by 0x14EA64: ??? (in /sbin/rsyslogd)


==2375==    by 0x14F561: ??? (in /sbin/rsyslogd)
==2375==    by 0x14F923: ??? (in /sbin/rsyslogd)
==2375==    by 0x146AB6: ??? (in /sbin/rsyslogd)
==2375==    by 0x1473DA: ??? (in /sbin/rsyslogd)
==2375==    by 0x156FE3: ??? (in /sbin/rsyslogd)
==2375==    by 0x1457D8: ??? (in /sbin/rsyslogd)
==2375==    by 0x13FB5D: wtiWorker (in /sbin/rsyslogd)
==2375==  Address 0x60000001e is not stack'd, malloc'd or

(recently)



free'd



==2375==
==2375==
==2375== Process terminating with default action of
signal

11



(SIGSEGV)



==2375==  Access not within mapped region at address



0x60000001E



==2375==    at 0x5660778: fjson_object_get



(json_object.c:176)



==2375==    by 0x126F57: ??? (in /sbin/rsyslogd)


==2375==    by 0x1270A0: msgAddJSON (in /sbin/rsyslogd)
==2375==    by 0x7242F2D: ??? (in

/lib64/rsyslog/mmnormalize.so)



==2375==    by 0x14EA64: ??? (in /sbin/rsyslogd)


==2375==    by 0x14F561: ??? (in /sbin/rsyslogd)
==2375==    by 0x14F923: ??? (in /sbin/rsyslogd)
==2375==    by 0x146AB6: ??? (in /sbin/rsyslogd)
==2375==    by 0x1473DA: ??? (in /sbin/rsyslogd)
==2375==    by 0x156FE3: ??? (in /sbin/rsyslogd)
==2375==    by 0x1457D8: ??? (in /sbin/rsyslogd)
==2375==    by 0x13FB5D: wtiWorker (in /sbin/rsyslogd)
==2375==  If you believe this happened as a result of a

stack



==2375==  overflow in your program's main thread (unlikely



but



==2375==  possible), you can try to increase the size of the


==2375==  main thread stack using the --main-stacksize=

flag.



==2375==  The main thread stack size used in this run was



10485760.



==2375==


==2375== HEAP SUMMARY:
==2375==     in use at exit: 2,611,014 bytes in 11,732

blocks



==2375==   total heap usage: 27,013 allocs, 15,281 frees,



8,524,299



bytes



allocated


==2375==
==2375== LEAK SUMMARY:
==2375==    definitely lost: 4,117 bytes in 11 blocks
==2375==    indirectly lost: 24,922 bytes in 20 blocks
==2375==      possibly lost: 4,464 bytes in 12 blocks
==2375==    still reachable: 2,577,511 bytes in 11,689

blocks



==2375==         suppressed: 0 bytes in 0 blocks


==2375== Rerun with --leak-check=full to see details of

leaked



memory



==2375==


==2375== For counts of detected and suppressed errors,
rerun

with:



-v



==2375== Use --track-origins=yes to see where uninitialised



values



come



from



==2375== ERROR SUMMARY: 5 errors from 3 contexts

(suppressed:



120



from 7)




[root valgrind-3.11.0]# rsyslogd -v
rsyslogd 8.20.0, compiled with:
PLATFORM: x86_64-redhat-linux-gnu
PLATFORM (lsb_release -d):
FEATURE_REGEXP: Yes
GSSAPI Kerberos 5 support: No
FEATURE_DEBUG (debug build, slow code): No
32bit Atomic operations supported: Yes
64bit Atomic operations supported: Yes
memory allocator: system default
Runtime Instrumentation (slow code): No
uuid support: Yes
Number of Bits in RainerScript integers: 64

See http://www.rsyslog.com for more information.

[root valgrind-3.11.0]# dmesg
..
rs:main Q:Reg[1776]: segfault at 65 ip 00007f0769bcc2b6
sp

00007f0762de87e8

error 4 in libc-2.12.so[7f0769aa4000+18a000]

Thanks,

Alec

On Wed, Jul 20, 2016 at 11:32 PM, Rainer Gerhards <

[email protected]>

wrote:

2016-07-20 23:02 GMT+02:00 Alec Swan <[email protected]
:



I used test RPMs to upgrade from 8.19.0 to 8.20.0.

However,



now



when I



start rsyslog it logs the following error message to



dmesg



and



shuts



rsyslog down:


"rs:main Q:Reg[23801]: segfault at 1a ip
00007fa95ac9f778

sp



00007fa94f5fd878 error 6 in



libfastjson.so.4.0.0[7fa95ac9c000+9000]"




These are the packages that were installed/updated:
Jul 20 20:53:11 yum[22693]: Installed:

libfastjson4-0.99.3-2.el6.x86_64



Jul 20 20:53:11 yum[22693]: Updated:



rsyslog-8.20.0-1.el6.x86_64



Jul 20 20:53:11 yum[22693]: Updated:



rsyslog-mmutf8fix-8.20.0-1.el6.x86_64

Jul 20 20:53:11 yum[22693]: Updated:

rsyslog-mmnormalize-8.20.0-1.el6.x86_64

Jul 20 20:53:11 yum[22693]: Updated:
rsyslog-elasticsearch-8.20.0-1.el6.x86_64


would it be possible that you do a run under valgrind

control?




Rainer



Thanks,

Alec

On Wed, Jul 20, 2016 at 12:11 PM, Michael Biebl <

[email protected]>



wrote:




2016-07-12 23:12 GMT+02:00 David Lang <[email protected]>:



On Tue, 12 Jul 2016, Michael Biebl wrote:

Fwiw, this is the first release I felt comfortable



enough



uploading



to



Debian unstable.


It just hit the archive a few hours ago:
https://tracker.debian.org/pkg/libfastjson

I hope we don't break the API that much anymore

before



1.0.0.




I'm now waiting for a liblognorm 1.1.4 release which

compiles



against



libfastjson 0.99.3.





My understanding is that we are going to get

liblognorm 2



this



cycle



(after



being delayed for the last couple of cycles)



Ok, so what's the plan here? Which liblognorm version

should I



pick?



At least they currently released version doesn't compile



against



the



latest libfastjson release (which is not a surprise,



after



the



API



rework, which I'm thankful for).



Regards,
Michael

--
Why is it that all of the instruments seeking

intelligent



life



in



the



universe are pointed away from Earth?


_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow

https://twitter.com/rgerhards



NOTE WELL: This is a PUBLIC mailing list, posts are



ARCHIVED



by a



myriad



of sites beyond our control. PLEASE UNSUBSCRIBE and DO



NOT



POST if



you



DON'T LIKE THAT.



_______________________________________________


rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow

https://twitter.com/rgerhards



NOTE WELL: This is a PUBLIC mailing list, posts are



ARCHIVED



by



a



myriad



of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT



POST if



you



DON'T LIKE THAT.

...
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to