2016-07-27 16:41 GMT+02:00 David Lang <[email protected]>: > On Wed, 27 Jul 2016, Rainer Gerhards wrote: > >> That's unfortunately not the case for the Ubuntu repo. I always >> consistently recompiled everything. Actually the problem occurs during >> that >> process, when e.g. liblognorm is compiled. > > > I am only building for trusty, but I am using the rsyslog-pkg-ubuntu scripts > to do the compiles, everything up to the point of uploading the results to > the PPA > > are you getting the failure on trusty, or on other versions?
I get them on trusty and also on other versions. See here: https://launchpad.net/~adiscon/+archive/ubuntu/v8-devel/+packages I have the impression that the exact versions which fail change, but trusty seems always affected. Rainer > > David Lang > >> Rainer >> >> Sent from phone, thus brief. >> >> Am 27.07.2016 12:06 schrieb "David Lang" <[email protected]>: >> >>> On Wed, 27 Jul 2016, Rainer Gerhards wrote: >>> >>> >>>> Thanks for the elaborate output. I think we may be getting closer. >>>> This triggered my attention: >>>> >>>> --> Processing Dependency: libjson-c.so.2()(64bit) for package: >>>> liblognorm1-1.1.3-1.el6.x86_64 >>>> >>>> I think we need to rebuild liblognorm with the new libfastjson >>>> package. Otherwise we can see subtle problems with the json handlers, >>>> as there core structures are different (much more in 0.99.3 than >>>> 0.99.2 of libfastjson). This doesn't explain yet the line number >>>> problems we saw, but in anycase that's something to look at. I will >>>> let you know when we have updates in the repo. >>>> >>> >>> Yes, as I have been building with the git master for libfastjson and >>> liblognorm for several months, I found that if I didn't recompile >>> everything at once to be on the new version of libraries that both used, >>> bad things happened. >>> >>> I've been suspecting that this sort of skew has been a large part of the >>> repo problems of the last few weeks as new versions of libraries have >>> been >>> released at different times than rsyslog packages. I've been suspecting >>> that if everything was recompiled, things would work (so 8.19.0 packages >>> break with the new libraries, but I expected 8.20 packages compiled >>> against >>> the new libraries to then work) >>> >>> David Lang >>> >>> In parallel, I also install a CentOS6 (don't have RHEL) machine and >>>> >>>> see what exactly it installes when we pull from the testing repo. >>>> >>>> Thanks again, >>>> Rainer >>>> >>>> 2016-07-27 1:27 GMT+02:00 Alec Swan <[email protected]>: >>>> >>>>> I configured /etc/yum.repos.d/rsyslog.repo to point to the "testing" >>>>> rsyslog repo and removed all other yum repos. After that I downloaded >>>>> all >>>>> RPMS from the "testing" repo using yumdownloader and installed them >>>>> with >>>>> "rpm -ivh". The end result is the same. The following shows the output >>>>> of >>>>> the test. >>>>> >>>>> # more /etc/yum.repos.d/rsyslog.repo >>>>> [rsyslog_v8] >>>>> name=Adiscon CentOS-$releasever - local packages for $basearch >>>>> baseurl=http://rpms.adiscon.com/testing/epel-$releasever/$basearch >>>>> enabled=1 >>>>> gpgcheck=0 >>>>> gpgkey=http://rpms.adiscon.com/RPM-GPG-KEY-Adiscon >>>>> protect=1 >>>>> >>>>> # yumdownloader --resolve rsyslog-mmnormalize-8.20.0 >>>>> rsyslog-elasticsearch-8.20.0 rsyslog-mmutf8fix-8.20.0 >>>>> Loaded plugins: fastestmirror >>>>> Loading mirror speeds from cached hostfile >>>>> ... >>>>> rsyslog_v8 >>>>> >>>>> | 2.5 kB 00:00 >>>>> rsyslog_v8/primary_db >>>>> >>>>> | 264 kB 00:00 >>>>> --> Running transaction check >>>>> ---> Package rsyslog-elasticsearch.x86_64 0:8.20.0-1.el6 will be >>>>> installed >>>>> --> Processing Dependency: rsyslog = 8.20.0-1.el6 for package: >>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64 >>>>> ---> Package rsyslog-mmnormalize.x86_64 0:8.20.0-1.el6 will be >>>>> installed >>>>> --> Processing Dependency: liblognorm.so.2()(64bit) for package: >>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64 >>>>> ---> Package rsyslog-mmutf8fix.x86_64 0:8.20.0-1.el6 will be installed >>>>> --> Running transaction check >>>>> ---> Package liblognorm1.x86_64 0:1.1.3-1.el6 will be installed >>>>> --> Processing Dependency: libjson-c.so.2()(64bit) for package: >>>>> liblognorm1-1.1.3-1.el6.x86_64 >>>>> ---> Package rsyslog.x86_64 0:8.20.0-1.el6 will be installed >>>>> --> Processing Dependency: libfastjson.so.4()(64bit) for package: >>>>> rsyslog-8.20.0-1.el6.x86_64 >>>>> --> Running transaction check >>>>> ---> Package json-c.x86_64 0:0.11-11.el6 will be installed >>>>> ---> Package libfastjson4.x86_64 0:0.99.3-2.el6 will be installed >>>>> --> Finished Dependency Resolution >>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm >>>>> >>>>> | 13 kB 00:00 >>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm >>>>> >>>>> | 24 kB 00:00 >>>>> rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm >>>>> >>>>> | 12 kB 00:00 >>>>> rsyslog-8.20.0-1.el6.x86_64.rpm >>>>> >>>>> | 690 kB 00:00 >>>>> json-c-0.11-11.el6.x86_64.rpm >>>>> >>>>> | 51 kB 00:00 >>>>> libfastjson4-0.99.3-2.el6.x86_64.rpm >>>>> >>>>> | 46 kB 00:00 >>>>> liblognorm1-1.1.3-1.el6.x86_64.rpm >>>>> >>>>> | 46 kB 00:00 >>>>> >>>>> # ls >>>>> json-c-0.11-11.el6.x86_64.rpm >>>>> liblognorm1-1.1.3-1.el6.x86_64.rpm >>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm >>>>> rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm >>>>> libfastjson4-0.99.3-2.el6.x86_64.rpm rsyslog-8.20.0-1.el6.x86_64.rpm >>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm >>>>> >>>>> # rpm -ivh *.rpm >>>>> warning: libfastjson4-0.99.3-2.el6.x86_64.rpm: Header V3 RSA/SHA1 >>>>> Signature, key ID e00b8985: NOKEY >>>>> Preparing... ########################################### >>>>> [100%] >>>>> 1:libfastjson4 ########################################### >>>>> [ >>>>> 14%] >>>>> 2:rsyslog ########################################### >>>>> [ >>>>> 29%] >>>>> 3:json-c ########################################### >>>>> [ >>>>> 43%] >>>>> 4:liblognorm1 ########################################### >>>>> [ >>>>> 57%] >>>>> 5:rsyslog-mmnormalize ########################################### >>>>> [ >>>>> 71%] >>>>> 6:rsyslog-elasticsearch ########################################### >>>>> [ >>>>> 86%] >>>>> 7:rsyslog-mmutf8fix ########################################### >>>>> [100%] >>>>> >>>>> # service rsyslog start >>>>> Starting system logger: [ OK ] >>>>> >>>>> # ps -ax | grep rsys >>>>> Warning: bad syntax, perhaps a bogus '-'? See >>>>> /usr/share/doc/procps-3.2.8/FAQ >>>>> 1020 pts/0 S+ 0:00 grep rsys >>>>> >>>>> # dmesg >>>>> ... >>>>> rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp >>>>> 00007f799e66f7e8 >>>>> error 4 in libc-2.12.so[7f79a5d25000+18a000] >>>>> >>>>> # service rsyslog start >>>>> Starting system logger: [ OK ] >>>>> >>>>> # dmesg >>>>> ... >>>>> rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp >>>>> 00007f799e66f7e8 >>>>> error 4 in libc-2.12.so[7f79a5d25000+18a000] >>>>> rs:main Q:Reg[1063]: segfault at 1b ip 00007f9a44249778 sp >>>>> 00007f9a3c1de878 >>>>> error 6 in libfastjson.so.4.0.0[7f9a44246000+9000] >>>>> >>>>> # service rsyslog start >>>>> Starting system logger: [ OK ] >>>>> >>>>> # dmesg >>>>> ... >>>>> rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp >>>>> 00007f799e66f7e8 >>>>> error 4 in libc-2.12.so[7f79a5d25000+18a000] >>>>> rs:main Q:Reg[1063]: segfault at 1b ip 00007f9a44249778 sp >>>>> 00007f9a3c1de878 >>>>> error 6 in libfastjson.so.4.0.0[7f9a44246000+9000] >>>>> rs:main Q:Reg[1093]: segfault at 0 ip 00007f32e00c22b6 sp >>>>> 00007f32d88e47e8 >>>>> error 4 in libc-2.12.so[7f32dff9a000+18a000] >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Alec >>>>> >>>>> On Tue, Jul 26, 2016 at 10:18 AM, Rainer Gerhards < >>>>> [email protected]> >>>>> wrote: >>>>> >>>>> Am 26.07.2016 17:58 schrieb "Alec Swan" <[email protected]>: >>>>>> >>>>>> >>>>>>> >>>>>>> I downloaded the packages from adiscon "testing" repo, then uploaded >>>>>>> them >>>>>>> to our local Nexus repo and then installed them with yum. So, the >>>>>>> source >>>>>>> >>>>>> is >>>>>> >>>>>>> our local Nexus repo. >>>>>>> >>>>>>> Unfortunately, the "testing" repo does not seem to have libee.so >>>>>>> >>>>>> dependency >>>>>> >>>>>>> required by libfastjson, so I can't just use "testing" repo to >>>>>>> install >>>>>>> >>>>>> all >>>>>> >>>>>>> rsyslog dependencies. >>>>>>> >>>>>> >>>>>> Something is fishy here. I just checked the spec file, and libfastjson >>>>>> does >>>>>> not have a dependency on libee. Is there a problem with the staging >>>>>> process? Or can you try from the original repo, not the mirror? >>>>>> >>>>>> Rainer >>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Alec >>>>>>> >>>>>>> On Tue, Jul 26, 2016 at 9:36 AM, Rainer Gerhards < >>>>>>> >>>>>> [email protected] >>>>>> >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> 2016-07-26 17:35 GMT+02:00 Alec Swan <[email protected]>: >>>>>>>> >>>>>>>> >>>>>>>>> Rainer, per Florian's request in "experimental rpms for >>>>>>>>> >>>>>>>> rsyslog-8.20.0" I >>>>>> >>>>>> >>>>>>> downloaded RPMs from the "testing" repo baseurl= >>>>>>>>> >>>>>>>>> http://rpms.adiscon.com/testing/epel-$releasever/$basearch >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Can you check where the system says it actually installed them from. >>>>>>>> The line numbers do not match the RPMs Florian created, thus I ask. >>>>>>>> >>>>>>>> Rainer >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Alec >>>>>>>>> >>>>>>>>> On Tue, Jul 26, 2016 at 7:30 AM, Rainer Gerhards < >>>>>>>>> >>>>>>>> [email protected]> >>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I today tried quite a while to reproduce the issue with your >>>>>>>>> config, >>>>>>>>>> >>>>>>>>>> unfortunately without any success. For now, I conclude that this >>>>>>>>>> is >>>>>>>>>> not a general problem, but one induced by your environment and >>>>>>>>>> data. >>>>>>>>>> >>>>>>>>>> I have then looked again at the traces and the few line numbers it >>>>>>>>>> contains. Now comes the interesting fact. I notice that there is a >>>>>>>>>> mismatch between the line numbers shown and the code that actually >>>>>>>>>> >>>>>>>>> was >>>>>> >>>>>> >>>>>>> used to build the packages (judging from git and the source RPMs). >>>>>>>>>> >>>>>>>>>> >>>>>>>>> So >>>>>> >>>>>> >>>>>>> I wonder what might have gone wrong here. Can you please let me know >>>>>>>>>> >>>>>>>>>> which exact repro you installed from and potentially which other >>>>>>>>>> >>>>>>>>> repos >>>>>> >>>>>> >>>>>>> you have configured on that machine. It might also be useful to >>>>>>>>>> >>>>>>>>>> >>>>>>>>> query >>>>>> >>>>>> >>>>>>> the system from where exactly the packages have ben installed. I >>>>>>>>>> >>>>>>>>>> >>>>>>>>> found >>>>>> >>>>>> >>>>>>> [1], which hopefully helps to get that information. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Once I have this information, I can check if there are any version >>>>>>>>>> quirks (note that commit [2] fixed a situation that could cause >>>>>>>>>> exactly what you see). If there is no issue with the package >>>>>>>>>> >>>>>>>>> sources, >>>>>> >>>>>> >>>>>>> I unfortunately need your help in further troubleshooting this in >>>>>>>>>> >>>>>>>>>> >>>>>>>>> your >>>>>> >>>>>> >>>>>>> environment. But let's first see what the repo check will bring up. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Rainer >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> http://serverfault.com/questions/62026/how-to-know-from-which-yum-repository-a-package-has-been-installed >>>>>> >>>>>>> [2] >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> https://github.com/rsyslog/libfastjson/commit/0499ec8df09c913156ead426c171031b566dbe7d >>>>>> >>>>>>> >>>>>>>>>> 2016-07-23 19:28 GMT+02:00 Alec Swan <[email protected]>: >>>>>>>>>> >>>>>>>>>>> Yes, that should be OK. Is there a link to the build server/CI >>>>>>>>>>> env >>>>>>>>>>> >>>>>>>>>> I >>>>>> >>>>>> >>>>>>> can >>>>>>>> >>>>>>>> >>>>>>>>> use? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Alec >>>>>>>>>>> >>>>>>>>>>> On Sat, Jul 23, 2016 at 7:08 AM, Rainer Gerhards < >>>>>>>>>>> >>>>>>>>>> [email protected]> >>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Thanks. Would it be OK for your if a somewhat mangled version of >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> these >>>>>>>> >>>>>>>> >>>>>>>>> configs show up inside the testbench? I think about adding a >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> specific >>>>>> >>>>>> >>>>>>> test for it, that would be great for the future as well. I'd >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> probably >>>>>> >>>>>> >>>>>>> try to strip it a bit down (e.g. no ES but writing to file, which >>>>>>>>>>>> >>>>>>>>>>>> would enable runs on all CI environments and when no ES is >>>>>>>>>>>> >>>>>>>>>>> available >>>>>> >>>>>> >>>>>>> -- I assume this would also trigger the bug). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Please let me know, >>>>>>>>>>>> Rainer >>>>>>>>>>>> >>>>>>>>>>>> 2016-07-22 17:46 GMT+02:00 Alec Swan <[email protected]>: >>>>>>>>>>>> >>>>>>>>>>>>> Rainer, I attached /etc/rsyslog.conf and >>>>>>>>>>>>> >>>>>>>>>>>> /etc/rsyslog.d/rsyslog-custom.conf >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Alec >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Jul 22, 2016 at 12:42 AM, Rainer Gerhards < >>>>>>>>>>>>> >>>>>>>>>>>> [email protected]> >>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> unfortunately, we do not seem to have debug symbols, so I >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> don't >>>>>> >>>>>> >>>>>>> see >>>>>>>> >>>>>>>> >>>>>>>>> much more. Can you share your full config (including all >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> includes, if >>>>>>>> >>>>>>>> >>>>>>>>> any)? Maybe I can get it to abort in my lab as well. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Rainer >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2016-07-21 22:02 GMT+02:00 Alec Swan <[email protected]>: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Rainer, here is the output from running rsyslog under >>>>>>>>>>>>>>> >>>>>>>>>>>>>> valgrind: >>>>>> >>>>>> >>>>>>> >>>>>>>>>>>>>>> [root valgrind-3.11.0]# valgrind rsyslogd >>>>>>>>>>>>>>> ==2371== Memcheck, a memory error detector >>>>>>>>>>>>>>> ==2371== Copyright (C) 2002-2015, and GNU GPL'd, by Julian >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Seward >>>>>>>> >>>>>>>> >>>>>>>>> et >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> al. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> ==2371== Using Valgrind-3.11.0 and LibVEX; rerun with -h for >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> copyright >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> info >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== Command: rsyslogd >>>>>>>>>>>>>>> ==2371== >>>>>>>>>>>>>>> ==2371== Conditional jump or move depends on uninitialised >>>>>>>>>>>>>>> >>>>>>>>>>>>>> value(s) >>>>>>>> >>>>>>>> >>>>>>>>> ==2371== at 0x16A414: cnfexprDestruct (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== by 0x16B36D: cnfstmtOptimize (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2371== by 0x146317: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2371== by 0x12E5BE: llExecFunc (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2371== by 0x146176: rulesetOptimizeAll (in >>>>>>>>>>>>>>> >>>>>>>>>>>>>> /sbin/rsyslogd) >>>>>> >>>>>> >>>>>>> ==2371== by 0x123EF6: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== by 0x158A68: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2371== by 0x159008: main (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2371== >>>>>>>>>>>>>>> ==2375== Warning: invalid file descriptor 99990 in syscall >>>>>>>>>>>>>>> >>>>>>>>>>>>>> close() >>>>>>>> >>>>>>>> >>>>>>>>> ==2371== >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== HEAP SUMMARY: >>>>>>>>>>>>>>> ==2371== in use at exit: 209,122 bytes in 2,121 blocks >>>>>>>>>>>>>>> ==2371== total heap usage: 11,004 allocs, 8,883 frees, >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 4,319,525 >>>>>>>> >>>>>>>> >>>>>>>>> bytes >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> allocated >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== >>>>>>>>>>>>>>> ==2371== LEAK SUMMARY: >>>>>>>>>>>>>>> ==2371== definitely lost: 26 bytes in 1 blocks >>>>>>>>>>>>>>> ==2371== indirectly lost: 0 bytes in 0 blocks >>>>>>>>>>>>>>> ==2371== possibly lost: 88 bytes in 2 blocks >>>>>>>>>>>>>>> ==2371== still reachable: 209,008 bytes in 2,118 blocks >>>>>>>>>>>>>>> ==2371== suppressed: 0 bytes in 0 blocks >>>>>>>>>>>>>>> ==2371== Rerun with --leak-check=full to see details of >>>>>>>>>>>>>>> >>>>>>>>>>>>>> leaked >>>>>> >>>>>> >>>>>>> memory >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ==2371== >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== For counts of detected and suppressed errors, rerun >>>>>>>>>>>>>>> >>>>>>>>>>>>>> with: >>>>>>>> >>>>>>>> >>>>>>>>> -v >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ==2371== Use --track-origins=yes to see where uninitialised >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> values >>>>>>>> >>>>>>>> >>>>>>>>> come >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> from >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2371== ERROR SUMMARY: 1 errors from 1 contexts >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (suppressed: >>>>>> >>>>>> >>>>>>> 114 >>>>>>>> >>>>>>>> >>>>>>>>> from 7) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> [root valgrind-3.11.0]# ==2375== Thread 6 rs:main Q:Reg: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== Invalid read of size 8 >>>>>>>>>>>>>>> ==2375== at 0x56607BE: fjson_object_iter_next >>>>>>>>>>>>>>> (json_object_iterator.c:119) >>>>>>>>>>>>>>> ==2375== by 0x5660863: fjson_object_iter_begin >>>>>>>>>>>>>>> (json_object_iterator.c:77) >>>>>>>>>>>>>>> ==2375== by 0x126F09: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1270A0: msgAddJSON (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x7242F2D: ??? (in >>>>>>>>>>>>>>> >>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) >>>>>> >>>>>> >>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== Address 0xf57d338 is 0 bytes after a block of size >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 72 >>>>>> >>>>>> >>>>>>> alloc'd >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> ==2375== at 0x4A05FEF: calloc (vg_replace_malloc.c:711) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x7659871: json_object_new >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (json_object.c:178) >>>>>> >>>>>> >>>>>>> ==2375== by 0x765A61A: json_object_new_object >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (json_object.c:354) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ==2375== by 0x744A355: ln_normalize (in >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> /usr/lib64/liblognorm.so.2.0.0) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x7242EEB: ??? (in >>>>>>>>>>>>>>> >>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) >>>>>> >>>>>> >>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== >>>>>>>>>>>>>>> ==2375== Invalid read of size 4 >>>>>>>>>>>>>>> ==2375== at 0x5660778: fjson_object_get >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (json_object.c:176) >>>>>> >>>>>> >>>>>>> ==2375== by 0x126F57: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x1270A0: msgAddJSON (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x7242F2D: ??? (in >>>>>>>>>>>>>>> >>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) >>>>>> >>>>>> >>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x13FB5D: wtiWorker (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== Address 0x60000001e is not stack'd, malloc'd or >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (recently) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> free'd >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== >>>>>>>>>>>>>>> ==2375== >>>>>>>>>>>>>>> ==2375== Process terminating with default action of signal >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 11 >>>>>> >>>>>> >>>>>>> (SIGSEGV) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> ==2375== Access not within mapped region at address >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 0x60000001E >>>>>> >>>>>> >>>>>>> ==2375== at 0x5660778: fjson_object_get >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (json_object.c:176) >>>>>> >>>>>> >>>>>>> ==2375== by 0x126F57: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x1270A0: msgAddJSON (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x7242F2D: ??? (in >>>>>>>>>>>>>>> >>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) >>>>>> >>>>>> >>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== by 0x13FB5D: wtiWorker (in /sbin/rsyslogd) >>>>>>>>>>>>>>> ==2375== If you believe this happened as a result of a >>>>>>>>>>>>>>> >>>>>>>>>>>>>> stack >>>>>> >>>>>> >>>>>>> ==2375== overflow in your program's main thread (unlikely >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> but >>>>>> >>>>>> >>>>>>> ==2375== possible), you can try to increase the size of the >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== main thread stack using the --main-stacksize= >>>>>>>>>>>>>>> >>>>>>>>>>>>>> flag. >>>>>> >>>>>> >>>>>>> ==2375== The main thread stack size used in this run was >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 10485760. >>>>>>>> >>>>>>>> >>>>>>>>> ==2375== >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== HEAP SUMMARY: >>>>>>>>>>>>>>> ==2375== in use at exit: 2,611,014 bytes in 11,732 >>>>>>>>>>>>>>> >>>>>>>>>>>>>> blocks >>>>>> >>>>>> >>>>>>> ==2375== total heap usage: 27,013 allocs, 15,281 frees, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 8,524,299 >>>>>>>> >>>>>>>> >>>>>>>>> bytes >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> allocated >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== >>>>>>>>>>>>>>> ==2375== LEAK SUMMARY: >>>>>>>>>>>>>>> ==2375== definitely lost: 4,117 bytes in 11 blocks >>>>>>>>>>>>>>> ==2375== indirectly lost: 24,922 bytes in 20 blocks >>>>>>>>>>>>>>> ==2375== possibly lost: 4,464 bytes in 12 blocks >>>>>>>>>>>>>>> ==2375== still reachable: 2,577,511 bytes in 11,689 >>>>>>>>>>>>>>> >>>>>>>>>>>>>> blocks >>>>>> >>>>>> >>>>>>> ==2375== suppressed: 0 bytes in 0 blocks >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== Rerun with --leak-check=full to see details of >>>>>>>>>>>>>>> >>>>>>>>>>>>>> leaked >>>>>> >>>>>> >>>>>>> memory >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ==2375== >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== For counts of detected and suppressed errors, rerun >>>>>>>>>>>>>>> >>>>>>>>>>>>>> with: >>>>>>>> >>>>>>>> >>>>>>>>> -v >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> ==2375== Use --track-origins=yes to see where uninitialised >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> values >>>>>>>> >>>>>>>> >>>>>>>>> come >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> from >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> ==2375== ERROR SUMMARY: 5 errors from 3 contexts >>>>>>>>>>>>>>> >>>>>>>>>>>>>> (suppressed: >>>>>> >>>>>> >>>>>>> 120 >>>>>>>> >>>>>>>> >>>>>>>>> from 7) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>> [root valgrind-3.11.0]# rsyslogd -v >>>>>>>>>>>>>>> rsyslogd 8.20.0, compiled with: >>>>>>>>>>>>>>> PLATFORM: x86_64-redhat-linux-gnu >>>>>>>>>>>>>>> PLATFORM (lsb_release -d): >>>>>>>>>>>>>>> FEATURE_REGEXP: Yes >>>>>>>>>>>>>>> GSSAPI Kerberos 5 support: No >>>>>>>>>>>>>>> FEATURE_DEBUG (debug build, slow code): No >>>>>>>>>>>>>>> 32bit Atomic operations supported: Yes >>>>>>>>>>>>>>> 64bit Atomic operations supported: Yes >>>>>>>>>>>>>>> memory allocator: system default >>>>>>>>>>>>>>> Runtime Instrumentation (slow code): No >>>>>>>>>>>>>>> uuid support: Yes >>>>>>>>>>>>>>> Number of Bits in RainerScript integers: 64 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> See http://www.rsyslog.com for more information. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [root valgrind-3.11.0]# dmesg >>>>>>>>>>>>>>> .. >>>>>>>>>>>>>>> rs:main Q:Reg[1776]: segfault at 65 ip 00007f0769bcc2b6 sp >>>>>>>>>>>>>>> >>>>>>>>>>>>>> 00007f0762de87e8 >>>>>>>>>>>>>> >>>>>>>>>>>>>>> error 4 in libc-2.12.so[7f0769aa4000+18a000] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Alec >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 11:32 PM, Rainer Gerhards < >>>>>>>>>>>>>>> >>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2016-07-20 23:02 GMT+02:00 Alec Swan <[email protected]>: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I used test RPMs to upgrade from 8.19.0 to 8.20.0. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> However, >>>>>> >>>>>> >>>>>>> now >>>>>>>> >>>>>>>> >>>>>>>>> when I >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> start rsyslog it logs the following error message to >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> dmesg >>>>>> >>>>>> >>>>>>> and >>>>>>>> >>>>>>>> >>>>>>>>> shuts >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> rsyslog down: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> "rs:main Q:Reg[23801]: segfault at 1a ip 00007fa95ac9f778 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> sp >>>>>> >>>>>> >>>>>>> 00007fa94f5fd878 error 6 in >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> libfastjson.so.4.0.0[7fa95ac9c000+9000]" >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> These are the packages that were installed/updated: >>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Installed: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> libfastjson4-0.99.3-2.el6.x86_64 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> rsyslog-8.20.0-1.el6.x86_64 >>>>>>>> >>>>>>>> >>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> rsyslog-mmutf8fix-8.20.0-1.el6.x86_64 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: >>>>>>>>>>>>>>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> would it be possible that you do a run under valgrind >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> control? >>>>>> >>>>>> >>>>>>> >>>>>>>>>>>>>>>> Rainer >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Alec >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 12:11 PM, Michael Biebl < >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2016-07-12 23:12 GMT+02:00 David Lang <[email protected]>: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Tue, 12 Jul 2016, Michael Biebl wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Fwiw, this is the first release I felt comfortable >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> enough >>>>>> >>>>>> >>>>>>> uploading >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Debian unstable. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> It just hit the archive a few hours ago: >>>>>>>>>>>>>>>>>>>> https://tracker.debian.org/pkg/libfastjson >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I hope we don't break the API that much anymore >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> before >>>>>> >>>>>> >>>>>>> 1.0.0. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'm now waiting for a liblognorm 1.1.4 release which >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> compiles >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> against >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> libfastjson 0.99.3. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> My understanding is that we are going to get >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> liblognorm 2 >>>>>> >>>>>> >>>>>>> this >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> cycle >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> (after >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> being delayed for the last couple of cycles) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Ok, so what's the plan here? Which liblognorm version >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> should I >>>>>>>> >>>>>>>> >>>>>>>>> pick? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> At least they currently released version doesn't compile >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> against >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> the >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> latest libfastjson release (which is not a surprise, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> after >>>>>> >>>>>> >>>>>>> the >>>>>>>> >>>>>>>> >>>>>>>>> API >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> rework, which I'm thankful for). >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>> Michael >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>> Why is it that all of the instruments seeking >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> intelligent >>>>>> >>>>>> >>>>>>> life >>>>>>>> >>>>>>>> >>>>>>>>> in >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> the >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> universe are pointed away from Earth? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>>>>>>>> What's up with rsyslog? Follow >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> https://twitter.com/rgerhards >>>>>>>> >>>>>>>> >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> ARCHIVED >>>>>> >>>>>> >>>>>>> by a >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> myriad >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> NOT >>>>>> >>>>>> >>>>>>> POST if >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> you >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>>>>>>> What's up with rsyslog? Follow >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://twitter.com/rgerhards >>>>>> >>>>>> >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ARCHIVED >>>>>> >>>>>> >>>>>>> by >>>>>>>> >>>>>>>> >>>>>>>>> a >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> myriad >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> POST if >>>>>>>> >>>>>>>> >>>>>>>>> you >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>> rsyslog mailing list >>>>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>>>>>>> What's up with rsyslog? Follow >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://twitter.com/rgerhards >>>>>> >>>>>> >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ARCHIVED >>>>>> >>>>>> >>>>>>> by a >>>>>>>> >>>>>>>> >>>>>>>>> myriad >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> POST if >>>>>>>> >>>>>>>> >>>>>>>>> you >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
