Hi All, still my rsyslog not stable, it is restarting with below information. Could you please help us.
Jun 29 11:37:47 127.0.0.1 kernel: imudp(w0)[1327844]: segfault at 27f6584009f40 ip 00007f65a85f2cf7 sp 00007f65a2fc91a0 error 4 in libc-2.28.so[7f65a8558000+1bc000] Jun 29 11:37:47 127.0.0.1 kernel: Code: 8b 40 08 48 83 f8 0f 0f 86 b6 03 00 00 48 39 c2 0f 82 ad 03 00 00 49 8b 10 48 83 e2 f8 48 39 ca 0f 85 c5 06 00 00 49 8b 51 18 <4c> 39 4a 10 0f 85 df 04 00 00 49 39 69 10 0f 85 d5 04 00 00 a8 01 Jun 29 11:37:47 127.0.0.1 systemd[1]: Started Process Core Dump (PID 1395302/UID 0). Jun 29 11:37:47 127.0.0.1 systemd-coredump[1395303]: Process 1327836 (rsyslogd) of user 0 dumped core.#012#012Stack trace of thread 1327844:#012#0 0x00007f65a85f2cf7 _int_malloc (libc.so.6)#012#1 0x00007f65a85f4842 malloc (libc.so.6)#012#2 0x000055d82dcb035a msgSetFromSockinfo (rsyslogd)#012#3 0x00007f65a5c0dc01 rcvMainLoop (imudp.so)#012#4 0x00007f65a5c0e200 wrkr (imudp.so)#012#5 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#6 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327837:#012#0 0x00007f65a98b4aa4 read (libpthread.so.0)#012#1 0x00007f65a66370ce readklog (imklog.so)#012#2 0x00007f65a663741d klogLogKMsg (imklog.so)#012#3 0x00007f65a663658c runInput (imklog.so)#012#4 0x000055d82dcda281 thrdStarter (rsyslogd)#012#5 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#6 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327836:#012#0 0x00007f65a867f331 pselect (libc.so.6)#012#1 0x000055d82dc860a9 mainloop (rsyslogd)#012#2 0x000055d82dc82e64 main (rsyslogd)#012#3 0x00007f65a8592ca3 __libc_start_main (libc.so.6)#012#4 0x000055d82dc8322e _start (rsyslogd)#012#012Stack trace of thread 1327842:#012#0 0x00007f65a8687a27 epoll_wait (libc.so.6)#012#1 0x00007f65a5c0d8b8 rcvMainLoop (imudp.so)#012#2 0x00007f65a5c0e200 wrkr (imudp.so)#012#3 0x00007f65a5c0e34f runInput (imudp.so)#012#4 0x000055d82dcda281 thrdStarter (rsyslogd)#012#5 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#6 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327839:#012#0 0x00007f65a867cbd6 ppoll (libc.so.6)#012#1 0x00007f65a8dfc23f fd_wait_for_event (libsystemd.so.0)#012#2 0x00007f65a8dd849b sd_journal_wait (libsystemd.so.0)#012#3 0x00007f65a6226c40 runInput (imjournal.so)#012#4 0x000055d82dcda281 thrdStarter (rsyslogd)#012#5 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#6 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327840:#012#0 0x00007f65a867f21f __select (libc.so.6)#012#1 0x000055d82dcb9af7 srSleep (rsyslogd)#012#2 0x00007f65a6020960 runInput (impstats.so)#012#3 0x000055d82dcda281 thrdStarter (rsyslogd)#012#4 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#5 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327843:#012#0 0x00007f65a98b144c pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x00007f65a5e19344 wrkr (imptcp.so)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327841:#012#0 0x00007f65a8687a27 epoll_wait (libc.so.6)#012#1 0x00007f65a5e1965c runInput (imptcp.so)#012#2 0x000055d82dcda281 thrdStarter (rsyslogd)#012#3 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#4 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395226:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395150:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395071:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395128:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395236:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327846:#012#0 0x00007f65a98b144c pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc8a2c wtiWorker (rsyslogd)#012#2 0x000055d82dcc71bf wtpWorker (rsyslogd)#012#3 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#4 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395183:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395246:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1328952:#012#0 0x00007f65a98b144c pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc8a2c wtiWorker (rsyslogd)#012#2 0x000055d82dcc71bf wtpWorker (rsyslogd)#012#3 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#4 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395234:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395160:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395189:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395282:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1327854:#012#0 0x00007f65a98b144c pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc8a2c wtiWorker (rsyslogd)#012#2 0x000055d82dcc71bf wtpWorker (rsyslogd)#012#3 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#4 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1394253:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395283:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395286:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1394917:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012#012Stack trace of thread 1395290:#012#0 0x00007f65a98b179a pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)#012#1 0x000055d82dcc5edc asyncWriterThread (rsyslogd)#012#2 0x00007f65a98ab1cf start_thread (libpthread.so.0)#012#3 0x00007f65a8591d83 __clone (libc.so.6)#012 Thanks & Regards Vijay Kumar Kanukula On Mon, 20 Jun 2022 at 22:54, vijay kumar <[email protected]> wrote: > Hi Rainer, > > Do you have any luck with asan logs??? > > Thanks & Regards > Vijay Kumar Kanukula > > On Fri, 17 Jun 2022 at 14:11, vijay kumar <[email protected]> > wrote: > >> Hello Rainer, >> >> When we installed ASAN related RPMS to capture the logs rsyslog restart >> for every 2 to 3 mins, it was purely unstable so we downgraded immediately. >> Attaching ASAN logs captured day before yesterday. >> >> Thanks & Regards >> Vijay Kumar Kanukula >> >> On Fri, 17 Jun 2022 at 13:44, Rainer Gerhards <[email protected]> >> wrote: >> >>> can you please post rsyslog -v output as well as the current ASAN report? >>> >>> Thanks, >>> Rainer >>> >>> El vie, 17 jun 2022 a las 10:12, vijay kumar >>> (<[email protected]>) escribió: >>> > >>> > HI Rainer/David/Marisuz, >>> > >>> > Could you please help me with creating one input rule with a queue as >>> Marisuz suggested. I was failing to create a rule. >>> > >>> > If I change anything in my 90-inputs.conf file do I need to make any >>> changes in 30-rules.conf??? >>> > >>> > @Rainer : I upgraded to the latest version of rsyslog and tested. But >>> still my rsyslog restarts continuously. It was not stable. >>> > >>> > Thanks & Regards >>> > Vijay Kumar Kanukula >>> > >>> > >>> > >>> > >>> > >>> > On Fri, 17 Jun 2022 at 12:55, Rainer Gerhards < >>> [email protected]> wrote: >>> >> >>> >> it would be good to find the root cause. The best would be to use the >>> >> current 8.2206.0 version and see if it works. If so, all is fine. If >>> >> not, we should try to debug the issue. >>> >> >>> >> Rainer >>> >> >>> >> El jue, 16 jun 2022 a las 16:22, vijay kumar via rsyslog >>> >> (<[email protected]>) escribió: >>> >> > >>> >> > Hi Team, >>> >> > >>> >> > My rsyslog service is getting restarted very frequently and we >>> understand >>> >> > it is due to race between the various threads, which causes one >>> thread to >>> >> > free a message field while another tries to read/write it. >>> >> > >>> >> > log: >>> >> > === >>> >> > ==3035157==ERROR: AddressSanitizer: heap-buffer-overflow on address >>> >> > 0x61e000000ac2 at pc 0x7f6085ac62fd bp 0x7f6079d755e0 sp >>> 0x7f6079d74d88 >>> >> > READ of size 2627 at 0x61e000000ac2 thread T3 (in:imjournal) >>> >> > #0 0x7f6085ac62fc (/lib64/libasan.so.5+0xb92fc) >>> >> > #1 0x7f607d5066a5 in readJSONfromJournalMsg imjournal.c:288 >>> >> > #2 0x7f607d5066a5 in readjournal imjournal.c:497 >>> >> > #3 0x55ae0523366e in thrdStarter ../threads.c:243 >>> >> > #4 0x7f60855dd1ce in start_thread >>> (/lib64/libpthread.so.0+0x81ce) >>> >> > #5 0x7f60835a1d82 in clone (/lib64/libc.so.6+0x39d82) >>> >> > >>> >> > 0x61e000000ac2 is located 0 bytes to the right of 2626-byte region >>> >> > [0x61e000000080,0x61e000000ac2) >>> >> > allocated by thread T3 (in:imjournal) here: >>> >> > #0 0x7f6085afcfe8 in __interceptor_realloc >>> (/lib64/libasan.so.5+0xeffe8) >>> >> > #1 0x7f6084af1a8b (/lib64/libsystemd.so.0+0x82a8b) >>> >> > >>> >> > Thread T3 (in:imjournal) created by T0 here: >>> >> > #0 0x7f6085a5fea3 in __interceptor_pthread_create >>> >> > (/lib64/libasan.so.5+0x52ea3) >>> >> > #1 0x55ae05234470 in thrdCreate ../threads.c:289 >>> >> > >>> >> > SUMMARY: AddressSanitizer: heap-buffer-overflow >>> >> > (/lib64/libasan.so.5+0xb92fc) >>> >> > Shadow bytes around the buggy address: >>> >> > 0x0c3c7fff8100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >>> >> > 0x0c3c7fff8110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >>> >> > 0x0c3c7fff8120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >>> >> > 0x0c3c7fff8130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >>> >> > 0x0c3c7fff8140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >>> >> > =>0x0c3c7fff8150: 00 00 00 00 00 00 00 00[02]fa fa fa fa fa fa fa >>> >> > 0x0c3c7fff8160: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c3c7fff8170: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c3c7fff8180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c3c7fff8190: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c3c7fff81a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > Shadow byte legend (one shadow byte represents 8 application bytes): >>> >> > Addressable: 00 >>> >> > Partially addressable: 01 02 03 04 05 06 07 >>> >> > Heap left redzone: fa >>> >> > Freed heap region: fd >>> >> > Stack left redzone: f1 >>> >> > Stack mid redzone: f2 >>> >> > Stack right redzone: f3 >>> >> > Stack after return: f5 >>> >> > Stack use after scope: f8 >>> >> > Global redzone: f9 >>> >> > Global init order: f6 >>> >> > Poisoned by user: f7 >>> >> > Container overflow: fc >>> >> > Array cookie: ac >>> >> > Intra object redzone: bb >>> >> > ASan internal: fe >>> >> > Left alloca redzone: ca >>> >> > Right alloca redzone: cb >>> >> > ================================================================= >>> >> > ==3035157==ERROR: AddressSanitizer: heap-use-after-free on address >>> >> > 0x60c0005d85c0 at pc 0x7f6085a4db27 bp 0x7f6070621e50 sp >>> 0x7f60706215f8 >>> >> > READ of size 128 at 0x60c0005d85c0 thread T9 (rs:main Q:Reg) >>> >> > #0 0x7f6085a4db26 (/lib64/libasan.so.5+0x40b26) >>> >> > #1 0x55ae0516d061 in msgSetFromSockinfo >>> (/usr/sbin/rsyslogd+0x390061) >>> >> > #2 0x55ae0516e522 in MsgDup msg.c:1129 >>> >> > #3 0x55ae05205d58 in execCall ruleset.c:290 >>> >> > #4 0x55ae05205d58 in scriptExec ruleset.c:608 >>> >> > #5 0x55ae05206532 in execIf ruleset.c:313 >>> >> > #6 0x55ae05206532 in scriptExec ruleset.c:614 >>> >> > #7 0x55ae05208868 in processBatch ruleset.c:660 >>> >> > #8 0x55ae050ae50c in msgConsumer rsyslogd.c:694 >>> >> > #9 0x55ae051f100d in ConsumerReg queue.c:2145 >>> >> > #10 0x55ae051e0804 in wtiWorker wti.c:428 >>> >> > #11 0x55ae051d9dd5 in wtpWorker wtp.c:435 >>> >> > #12 0x7f60855dd1ce in start_thread >>> (/lib64/libpthread.so.0+0x81ce) >>> >> > #13 0x7f60835a1d82 in clone (/lib64/libc.so.6+0x39d82) >>> >> > >>> >> > 0x60c0005d85c0 is located 0 bytes inside of 128-byte region >>> >> > [0x60c0005d85c0,0x60c0005d8640) >>> >> > freed by thread T6 (imudp(w1)) here: >>> >> > #0 0x7f6085afc7e0 in __interceptor_free >>> (/lib64/libasan.so.5+0xef7e0) >>> >> > #1 0x55ae0515a3a5 in MsgSetRcvFromWithoutAddRef msg.c:471 >>> >> > >>> >> > previously allocated by thread T6 (imudp(w1)) here: >>> >> > #0 0x7f6085afcba8 in __interceptor_malloc >>> (/lib64/libasan.so.5+0xefba8) >>> >> > #1 0x55ae0516cff4 in msgSetFromSockinfo >>> (/usr/sbin/rsyslogd+0x38fff4) >>> >> > >>> >> > Thread T9 (rs:main Q:Reg) created by T3 (in:imjournal) here: >>> >> > #0 0x7f6085a5fea3 in __interceptor_pthread_create >>> >> > (/lib64/libasan.so.5+0x52ea3) >>> >> > #1 0x55ae051dc9d8 in wtpStartWrkr wtp.c:497 >>> >> > #2 0x55ae051dc9d8 in wtpAdviseMaxWorkers wtp.c:570 >>> >> > >>> >> > Thread T6 (imudp(w1)) created by T0 here: >>> >> > #0 0x7f6085a5fea3 in __interceptor_pthread_create >>> >> > (/lib64/libasan.so.5+0x52ea3) >>> >> > #1 0x55ae05234470 in thrdCreate ../threads.c:289 >>> >> > >>> >> > SUMMARY: AddressSanitizer: heap-use-after-free >>> (/lib64/libasan.so.5+0x40b26) >>> >> > Shadow bytes around the buggy address: >>> >> > 0x0c18800b3060: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa >>> >> > 0x0c18800b3070: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd >>> >> > 0x0c18800b3080: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd >>> >> > 0x0c18800b3090: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa >>> >> > 0x0c18800b30a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd >>> >> > =>0x0c18800b30b0: fa fa fa fa fa fa fa fa[fd]fd fd fd fd fd fd fd >>> >> > 0x0c18800b30c0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa >>> >> > 0x0c18800b30d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd >>> >> > 0x0c18800b30e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c18800b30f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c18800b3100: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd >>> >> > Shadow byte legend (one shadow byte represents 8 application bytes): >>> >> > Addressable: 00 >>> >> > Partially addressable: 01 02 03 04 05 06 07 >>> >> > Heap left redzone: fa >>> >> > Freed heap region: fd >>> >> > Stack left redzone: f1 >>> >> > Stack mid redzone: f2 >>> >> > Stack right redzone: f3 >>> >> > Stack after return: f5 >>> >> > Stack use after scope: f8 >>> >> > Global redzone: f9 >>> >> > Global init order: f6 >>> >> > Poisoned by user: f7 >>> >> > Container overflow: fc >>> >> > Array cookie: ac >>> >> > Intra object redzone: bb >>> >> > ASan internal: fe >>> >> > Left alloca redzone: ca >>> >> > Right alloca redzone: cb >>> >> > ================================================================= >>> >> > ==3035157==ERROR: AddressSanitizer: heap-use-after-free on address >>> >> > 0x6040000285a0 at pc 0x55ae0520a723 bp 0x7f60669659a0 sp >>> 0x7f6066965990 >>> >> > WRITE of size 4 at 0x6040000285a0 thread T16 (rs:qradar.local) >>> >> > #0 0x55ae0520a722 in propDestruct prop.c:63 >>> >> > #1 0x55ae05170299 in MsgSetRcvFromIPWithoutAddRef msg.c:457 >>> >> > #2 0x55ae05170299 in resolveDNS msg.c:522 >>> >> > #3 0x55ae0517064a in getRcvFromIP msg.c:558 >>> >> > #4 0x55ae05179317 in MsgGetProp (/usr/sbin/rsyslogd+0x39c317) >>> >> > #5 0x55ae0524aece in tplToString ../template.c:207 >>> >> > #6 0x55ae0522a4ab in prepareDoActionParams ../action.c:1114 >>> >> > #7 0x55ae0522a4ab in processMsgMain ../action.c:1648 >>> >> > #8 0x55ae0522c279 in doSubmitToActionQ ../action.c:1825 >>> >> > #9 0x55ae05205826 in execAct ruleset.c:209 >>> >> > #10 0x55ae05205826 in scriptExec ruleset.c:599 >>> >> > #11 0x55ae05208868 in processBatch ruleset.c:660 >>> >> > #12 0x55ae050ae50c in msgConsumer rsyslogd.c:694 >>> >> > #13 0x55ae051f100d in ConsumerReg queue.c:2145 >>> >> > #14 0x55ae051e0804 in wtiWorker wti.c:428 >>> >> > #15 0x55ae051d9dd5 in wtpWorker wtp.c:435 >>> >> > #16 0x7f60855dd1ce in start_thread >>> (/lib64/libpthread.so.0+0x81ce) >>> >> > #17 0x7f60835a1d82 in clone (/lib64/libc.so.6+0x39d82) >>> >> > >>> >> > 0x6040000285a0 is located 16 bytes inside of 48-byte region >>> >> > [0x604000028590,0x6040000285c0) >>> >> > freed by thread T16 (rs:qradar.local) here: >>> >> > #0 0x7f6085afc7e0 in __interceptor_free >>> (/lib64/libasan.so.5+0xef7e0) >>> >> > #1 0x55ae0515a3a5 in MsgSetRcvFromWithoutAddRef msg.c:471 >>> >> > >>> >> > previously allocated by thread T16 (rs:qradar.local) here: >>> >> > #0 0x7f6085afcdb0 in calloc (/lib64/libasan.so.5+0xefdb0) >>> >> > #1 0x55ae0520a439 in propConstruct prop.c:56 >>> >> > >>> >> > Thread T16 (rs:qradar.local) created by T9 (rs:main Q:Reg) here: >>> >> > #0 0x7f6085a5fea3 in __interceptor_pthread_create >>> >> > (/lib64/libasan.so.5+0x52ea3) >>> >> > #1 0x55ae051dc9d8 in wtpStartWrkr wtp.c:497 >>> >> > #2 0x55ae051dc9d8 in wtpAdviseMaxWorkers wtp.c:570 >>> >> > >>> >> > SUMMARY: AddressSanitizer: heap-use-after-free prop.c:63 in >>> propDestruct >>> >> > Shadow bytes around the buggy address: >>> >> > 0x0c087fffd060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa >>> >> > 0x0c087fffd070: fa fa 00 00 00 00 00 fa fa fa fa fa fa fa fa fa >>> >> > 0x0c087fffd080: fa fa 00 00 00 00 00 fa fa fa fd fd fd fd fd fd >>> >> > 0x0c087fffd090: fa fa 00 00 00 00 00 fa fa fa fd fd fd fd fd fa >>> >> > 0x0c087fffd0a0: fa fa 00 00 00 00 00 fa fa fa fa fa fa fa fa fa >>> >> > =>0x0c087fffd0b0: fa fa fd fd[fd]fd fd fd fa fa fa fa fa fa fa fa >>> >> > 0x0c087fffd0c0: fa fa fd fd fd fd fd fd fa fa fa fa fa fa fa fa >>> >> > 0x0c087fffd0d0: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd >>> >> > 0x0c087fffd0e0: fa fa fa fa fa fa fa fa fa fa fd fd fd fd fd fa >>> >> > 0x0c087fffd0f0: fa fa fa fa fa fa fa fa fa fa fd fd fd fd fd fa >>> >> > 0x0c087fffd100: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd >>> >> > Shadow byte legend (one shadow byte represents 8 application bytes): >>> >> > Addressable: 00 >>> >> > Partially addressable: 01 02 03 04 05 06 07 >>> >> > Heap left redzone: fa >>> >> > Freed heap region: fd >>> >> > Stack left redzone: f1 >>> >> > Stack mid redzone: f2 >>> >> > Stack right redzone: f3 >>> >> > Stack after return: f5 >>> >> > Stack use after scope: f8 >>> >> > Global redzone: f9 >>> >> > Global init order: f6 >>> >> > Poisoned by user: f7 >>> >> > Container overflow: fc >>> >> > Array cookie: ac >>> >> > Intra object redzone: bb >>> >> > ASan internal: fe >>> >> > Left alloca redzone: ca >>> >> > Right alloca redzone: cb >>> >> > ==3035157==ABORTING >>> >> > >>> >> > ================================= >>> >> > what is the possible solution ??? >>> >> > >>> >> > would be to have multiple rsyslog instances, which is possible if >>> the >>> >> > traffic is split between ports. If yes could you please suggest how >>> to >>> >> > configure?? >>> >> > >>> >> > Thanks & Regards >>> >> > Vijay Kumar Kanukula >>> >> > _______________________________________________ >>> >> > rsyslog mailing list >>> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog >>> >> > http://www.rsyslog.com/professional-services/ >>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>> you DON'T LIKE THAT. >>> >> _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
