It was looking very similar to the issue that Thomas reported on http://www.gossamer-threads.com/lists/rsyslog/users/16204?do=post_view_threaded
To double-check, I cherry-picked the two patches in that PR, which fixed the failing json_array_looping test. So I'll now focus on Chenlin's problem. To rule-out the same problem (the one that PR fixes), Chenlin, can you confirm which version of rsyslog are you on? On Thu, Apr 2, 2015 at 6:58 PM, singh.janmejay <singh.janme...@gmail.com> wrote: > I couldn't spend too much time on it, but just saw it reproduce on my > local env (also running Ubuntu 12.04). > > Will again get a chance to look at in a a few days (likely Monday). > > Again, sorry for the delay. > > On Thu, Apr 2, 2015 at 6:43 PM, singh.janmejay <singh.janme...@gmail.com> > wrote: >> Foreach support is a recent addition. >> >> On Thu, Apr 2, 2015 at 3:34 PM, singh.janmejay <singh.janme...@gmail.com> >> wrote: >>> I saw that mail, but by that time I had already setup the environment >>> to reproduce it with Chenlin's config. >>> >>> I should pursue the CI failure first. Didn't know it was happening in every >>> run. >>> >>> On Thu, Apr 2, 2015 at 2:04 PM, Rainer Gerhards >>> <rgerha...@hq.adiscon.com> wrote: >>>> I >>>> >>>> 2015-04-02 7:35 GMT+02:00 singh.janmejay <singh.janme...@gmail.com>: >>>>> Hadn't noticed the use of msg key early-on, so "]}" shouldn't be a >>>>> problem, but I can't reproduce it with input i crafted to work with >>>>> it. >>>>> Here is the crafted input: >>>>> @cee:{"msg" : ["G\"foo\":\"10\"}", "G\"bar\":\"20\""]} >>>>> >>>>> This is what does fail it, im using a bad type (it doesn't do type-check >>>>> yet): >>>>> @cee:{"msg" : "G\"foo\":\"10\"}", "bar" : ["foo"]} >>>>> >>>>> Here it ending in value of 'bar' satisfies the ']}' conditional, but >>>>> msg isn't an array. I'll add array-typecheck, but other than this I >>>>> have been unable to reproduce it, so this may not fix your problem. >>>>> >>>>> I think your input line (or entire input file) will help. >>>> >>>> No sure if you overlooked that posting. As it looks, the problem seems >>>> to be reproducible with the testbench as well. At least this commit >>>> continously fails: >>>> >>>> https://github.com/rsyslog/rsyslog/pull/286 >>>> >>>> I've also sent a build log via te ML a couple of days ago. >>>> >>>> HTH >>>> Rainer >>>>> >>>>> On Thu, Apr 2, 2015 at 9:39 AM, singh.janmejay <singh.janme...@gmail.com> >>>>> wrote: >>>>>> Hi Chenlin, >>>>>> >>>>>> I finally have some time to look at it. (Sorry for the delay.) >>>>>> >>>>>> Can you please share a sample log-line? Feel free to anonymize it, but >>>>>> please keep the structure and types as is. >>>>>> >>>>>> I see that you are checking $msg ends with "]}", that means it is an >>>>>> object, not an array. >>>>>> >>>>>> If my interpretation is correct, trying to iterate over object instead >>>>>> of array may be causing this problem. (I haven't looked at code yet to >>>>>> verify this manifestation though). >>>>>> >>>>>> On Tue, Mar 31, 2015 at 2:32 PM, Rainer Gerhards >>>>>> <rgerha...@hq.adiscon.com> wrote: >>>>>>> 2015-03-30 19:55 GMT+02:00 singh.janmejay <singh.janme...@gmail.com>: >>>>>>>> Sorry, I haven't had a chance to look at it yet. >>>>>>> >>>>>>> Take your time, I know how challenging it is at times ;) I just wanted >>>>>>> to spread the good news that we have a good repro. >>>>>>> >>>>>>> Rainer >>>>>>>> >>>>>>>> Will get to this asap. >>>>>>>> >>>>>>>> On Mon, Mar 30, 2015 at 2:36 PM, Rainer Gerhards >>>>>>>> <rgerha...@hq.adiscon.com> wrote: >>>>>>>>> Co-incidentally, an unrelated testbench run also experienced problems >>>>>>>>> in this area. I have attached the testbench log. >>>>>>>>> >>>>>>>>> Here is the valgrind report: >>>>>>>>> >>>>>>>>> 5812.499032607:main Q:Reg/w0 : wti 0x614e9e0: we need to create a new >>>>>>>>> action worker instance for action 2 >>>>>>>>> >>>>>>>>> 5812.499181273:main Q:Reg/w0 : Action 2 transitioned to state: itx >>>>>>>>> >>>>>>>>> 5812.499314576:main Q:Reg/w0 : FOREACH .corge IN >>>>>>>>> >>>>>>>>> 5812.499625664:main Q:Reg/w0 : var '.quux!bar' >>>>>>>>> >>>>>>>>> 5812.500078770:main Q:Reg/w0 : eval expr 0x60849e0, type 'V[86]' >>>>>>>>> >>>>>>>>> ==2284== Thread 6: >>>>>>>>> >>>>>>>>> ==2284== Invalid read of size 8 >>>>>>>>> >>>>>>>>> ==2284== at 0x587B134: lh_table_lookup_entry (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x587B1A8: lh_table_lookup (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x416F98: jsonVarExtract (msg.c:4218) >>>>>>>>> >>>>>>>>> ==2284== by 0x41C82F: msgGetJSONPropJSON (msg.c:2848) >>>>>>>>> >>>>>>>>> ==2284== by 0x4562D0: cnfexprEval (rainerscript.c:1785) >>>>>>>>> >>>>>>>>> ==2284== by 0x45842E: cnfexprEvalCollection (rainerscript.c:2483) >>>>>>>>> >>>>>>>>> ==2284== by 0x435CFC: scriptExec (ruleset.c:271) >>>>>>>>> >>>>>>>>> ==2284== by 0x435D6B: scriptExec (ruleset.c:284) >>>>>>>>> >>>>>>>>> ==2284== by 0x436233: processBatch (ruleset.c:503) >>>>>>>>> >>>>>>>>> ==2284== by 0x44464B: msgConsumer (rsyslogd.c:576) >>>>>>>>> >>>>>>>>> ==2284== by 0x432AC2: ConsumerReg (queue.c:1897) >>>>>>>>> >>>>>>>>> ==2284== by 0x42F55E: wtiWorker (wti.c:334) >>>>>>>>> >>>>>>>>> ==2284== Address 0x61645c8 is 8 bytes before a block of size 5 >>>>>>>>> alloc'd >>>>>>>>> >>>>>>>>> ==2284== at 0x4C2B6CD: malloc (in >>>>>>>>> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) >>>>>>>>> >>>>>>>>> ==2284== by 0x5D0B081: strdup (strdup.c:43) >>>>>>>>> >>>>>>>>> ==2284== by 0x5878A5C: json_object_object_add (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x41E419: msgAddJSON (msg.c:4379) >>>>>>>>> >>>>>>>>> ==2284== by 0x41F2E0: msgSetJSONFromVar (msg.c:4554) >>>>>>>>> >>>>>>>>> ==2284== by 0x435D4E: scriptExec (ruleset.c:283) >>>>>>>>> >>>>>>>>> ==2284== by 0x436233: processBatch (ruleset.c:503) >>>>>>>>> >>>>>>>>> ==2284== by 0x44464B: msgConsumer (rsyslogd.c:576) >>>>>>>>> >>>>>>>>> ==2284== by 0x432AC2: ConsumerReg (queue.c:1897) >>>>>>>>> >>>>>>>>> ==2284== by 0x42F55E: wtiWorker (wti.c:334) >>>>>>>>> >>>>>>>>> ==2284== by 0x42E6C6: wtpWorker (wtp.c:389) >>>>>>>>> >>>>>>>>> ==2284== by 0x5050E99: start_thread (pthread_create.c:308) >>>>>>>>> >>>>>>>>> ==2284== >>>>>>>>> >>>>>>>>> ==2284== Jump to the invalid address stated on the next line >>>>>>>>> >>>>>>>>> ==2284== at 0x0: ??? >>>>>>>>> >>>>>>>>> ==2284== by 0x587B136: lh_table_lookup_entry (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x587B1A8: lh_table_lookup (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x416F98: jsonVarExtract (msg.c:4218) >>>>>>>>> >>>>>>>>> ==2284== by 0x41C82F: msgGetJSONPropJSON (msg.c:2848) >>>>>>>>> >>>>>>>>> ==2284== by 0x4562D0: cnfexprEval (rainerscript.c:1785) >>>>>>>>> >>>>>>>>> ==2284== by 0x45842E: cnfexprEvalCollection (rainerscript.c:2483) >>>>>>>>> >>>>>>>>> ==2284== by 0x435CFC: scriptExec (ruleset.c:271) >>>>>>>>> >>>>>>>>> ==2284== by 0x435D6B: scriptExec (ruleset.c:284) >>>>>>>>> >>>>>>>>> ==2284== by 0x436233: processBatch (ruleset.c:503) >>>>>>>>> >>>>>>>>> ==2284== by 0x44464B: msgConsumer (rsyslogd.c:576) >>>>>>>>> >>>>>>>>> ==2284== by 0x432AC2: ConsumerReg (queue.c:1897) >>>>>>>>> >>>>>>>>> ==2284== Address 0x0 is not stack'd, malloc'd or (recently) free'd >>>>>>>>> >>>>>>>>> ==2284== >>>>>>>>> >>>>>>>>> ==2284== >>>>>>>>> >>>>>>>>> ==2284== Process terminating with default action of signal 11 >>>>>>>>> (SIGSEGV) >>>>>>>>> >>>>>>>>> ==2284== Bad permissions for mapped region at address 0x0 >>>>>>>>> >>>>>>>>> ==2284== at 0x0: ??? >>>>>>>>> >>>>>>>>> ==2284== by 0x587B136: lh_table_lookup_entry (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x587B1A8: lh_table_lookup (in >>>>>>>>> /usr/lib/x86_64-linux-gnu/libjson.so.0.0.1) >>>>>>>>> >>>>>>>>> ==2284== by 0x416F98: jsonVarExtract (msg.c:4218) >>>>>>>>> >>>>>>>>> ==2284== by 0x41C82F: msgGetJSONPropJSON (msg.c:2848) >>>>>>>>> >>>>>>>>> ==2284== by 0x4562D0: cnfexprEval (rainerscript.c:1785) >>>>>>>>> >>>>>>>>> ==2284== by 0x45842E: cnfexprEvalCollection (rainerscript.c:2483) >>>>>>>>> >>>>>>>>> ==2284== by 0x435CFC: scriptExec (ruleset.c:271) >>>>>>>>> >>>>>>>>> ==2284== by 0x435D6B: scriptExec (ruleset.c:284) >>>>>>>>> >>>>>>>>> ==2284== by 0x436233: processBatch (ruleset.c:503) >>>>>>>>> >>>>>>>>> ==2284== by 0x44464B: msgConsumer (rsyslogd.c:576) >>>>>>>>> >>>>>>>>> ==2284== by 0x432AC2: ConsumerReg (queue.c:1897) >>>>>>>>> >>>>>>>>> ==2284== >>>>>>>>> >>>>>>>>> ==2284== HEAP SUMMARY: >>>>>>>>> >>>>>>>>> ==2284== in use at exit: 961,978 bytes in 1,632 blocks >>>>>>>>> >>>>>>>>> ==2284== total heap usage: 2,448 allocs, 816 frees, 1,023,184 bytes >>>>>>>>> allocated >>>>>>>>> >>>>>>>>> ==2284== >>>>>>>>> >>>>>>>>> ==2284== LEAK SUMMARY: >>>>>>>>> >>>>>>>>> ==2284== definitely lost: 6 bytes in 1 blocks >>>>>>>>> >>>>>>>>> ==2284== indirectly lost: 0 bytes in 0 blocks >>>>>>>>> >>>>>>>>> ==2284== possibly lost: 2,592 bytes in 9 blocks >>>>>>>>> >>>>>>>>> ==2284== still reachable: 959,380 bytes in 1,622 blocks >>>>>>>>> >>>>>>>>> ==2284== suppressed: 0 bytes in 0 blocks >>>>>>>>> >>>>>>>>> ==2284== Rerun with --leak-check=full to see details of leaked memory >>>>>>>>> >>>>>>>>> ==2284== >>>>>>>>> >>>>>>>>> ==2284== For counts of detected and suppressed errors, rerun with: -v >>>>>>>>> >>>>>>>>> ==2284== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from >>>>>>>>> 2) >>>>>>>>> >>>>>>>>> receiving response: Connection reset by peer >>>>>>>>> >>>>>>>>> ./diag.sh: line 17: 2284 Killed $valgrind >>>>>>>>> ../tools/rsyslogd -C -n -irsyslog$3.pid -M../runtime/.libs:../.libs >>>>>>>>> -f$srcdir/testsuites/$2 >>>>>>>>> >>>>>>>>> FAIL: json_array_looping.sh >>>>>>>>> >>>>>>>>> Rainer >>>>>>>>> >>>>>>>>> 2015-03-21 17:50 GMT+01:00 chenlin rao <rao.chen...@gmail.com>: >>>>>>>>>> thanks for the information. But rsyslogd also fault after I change >>>>>>>>>> mmjsonparse action config as `action ( type="mmjsonparse" >>>>>>>>>> name="action_jsonarray-parse"`. output as follow: >>>>>>>>>> >>>>>>>>>> 6008.997308131:main Q:Reg/w1 : eval expr 0x52f18f0, return datatype >>>>>>>>>> 'J' >>>>>>>>>> >>>>>>>>>> 6009.002050327:main Q:Reg/w1 : ACTION 16 >>>>>>>>>> [builtin:omfwd:action(type="builtin:omfwd" ...)] >>>>>>>>>> >>>>>>>>>> 6009.002357554:main Q:Reg/w1 : executing action 16 >>>>>>>>>> >>>>>>>>>> 6009.002454575:main Q:Reg/w1 : Called action, logging to >>>>>>>>>> builtin:omfwd >>>>>>>>>> >>>>>>>>>> ==9947== Thread 22: >>>>>>>>>> >>>>>>>>>> ==9947== Invalid read of size 4 >>>>>>>>>> >>>>>>>>>> ==9947== at 0x3A044093A0: pthread_mutex_lock (in /lib64/ >>>>>>>>>> libpthread-2.12.so) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x438053: qqueueEnqMsg (queue.c:2856) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x4421AB: doSubmitToActionQ (action.c:1403) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439D6C: scriptExec (ruleset.c:197) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439ECC: scriptExec (ruleset.c:284) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x43A40B: processBatch (ruleset.c:503) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x44A403: msgConsumer (rsyslogd.c:575) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x438D12: ConsumerReg (queue.c:1897) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x43314B: wtiWorker (wti.c:334) >>>>>>>>>> >>>>>>>>>> ==9947== Address 0x10 is not stack'd, malloc'd or (recently) free'd >>>>>>>>>> >>>>>>>>>> ==9947== >>>>>>>>>> >>>>>>>>>> ==9947== >>>>>>>>>> >>>>>>>>>> ==9947== Process terminating with default action of signal 11 >>>>>>>>>> (SIGSEGV) >>>>>>>>>> >>>>>>>>>> ==9947== Access not within mapped region at address 0x10 >>>>>>>>>> >>>>>>>>>> ==9947== at 0x3A044093A0: pthread_mutex_lock (in /lib64/ >>>>>>>>>> libpthread-2.12.so) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x438053: qqueueEnqMsg (queue.c:2856) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x4421AB: doSubmitToActionQ (action.c:1403) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439D6C: scriptExec (ruleset.c:197) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439ECC: scriptExec (ruleset.c:284) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x43A40B: processBatch (ruleset.c:503) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x44A403: msgConsumer (rsyslogd.c:575) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x438D12: ConsumerReg (queue.c:1897) >>>>>>>>>> >>>>>>>>>> ==9947== by 0x43314B: wtiWorker (wti.c:334) >>>>>>>>>> >>>>>>>>>> ==9947== If you believe this happened as a result of a stack >>>>>>>>>> >>>>>>>>>> ==9947== overflow in your program's main thread (unlikely but >>>>>>>>>> >>>>>>>>>> ==9947== possible), you can try to increase the size of the >>>>>>>>>> >>>>>>>>>> ==9947== main thread stack using the --main-stacksize= flag. >>>>>>>>>> >>>>>>>>>> ==9947== The main thread stack size used in this run was 10485760. >>>>>>>>>> >>>>>>>>>> ==9947== >>>>>>>>>> >>>>>>>>>> ==9947== HEAP SUMMARY: >>>>>>>>>> >>>>>>>>>> ==9947== in use at exit: 1,181,215,468 bytes in 282,120 blocks >>>>>>>>>> >>>>>>>>>> ==9947== total heap usage: 379,556 allocs, 97,436 frees, >>>>>>>>>> 2,096,403,341 >>>>>>>>>> bytes allocated >>>>>>>>>> >>>>>>>>>> ==9947== >>>>>>>>>> >>>>>>>>>> ==9947== LEAK SUMMARY: >>>>>>>>>> >>>>>>>>>> ==9947== definitely lost: 516 bytes in 12 blocks >>>>>>>>>> >>>>>>>>>> ==9947== indirectly lost: 0 bytes in 0 blocks >>>>>>>>>> >>>>>>>>>> ==9947== possibly lost: 33,561,793 bytes in 24 blocks >>>>>>>>>> >>>>>>>>>> ==9947== still reachable: 1,147,653,159 bytes in 282,084 blocks >>>>>>>>>> >>>>>>>>>> ==9947== suppressed: 0 bytes in 0 blocks >>>>>>>>>> >>>>>>>>>> ==9947== Rerun with --leak-check=full to see details of leaked memory >>>>>>>>>> >>>>>>>>>> ==9947== >>>>>>>>>> >>>>>>>>>> ==9947== For counts of detected and suppressed errors, rerun with: -v >>>>>>>>>> >>>>>>>>>> ==9947== Use --track-origins=yes to see where uninitialised values >>>>>>>>>> come from >>>>>>>>>> >>>>>>>>>> ==9947== ERROR SUMMARY: 25 errors from 5 contexts (suppressed: 100 >>>>>>>>>> from 9) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Seems very similar with before. >>>>>>>>>> >>>>>>>>>> BTW: is it the "workerthreads" options make sense to >>>>>>>>>> mmjsonparse/mmfields >>>>>>>>>> etc? >>>>>>>>>> >>>>>>>>>> 2015-03-22 0:07 GMT+08:00 Rainer Gerhards <rgerha...@hq.adiscon.com>: >>>>>>>>>> >>>>>>>>>>> 2015-03-21 14:50 GMT+01:00 chenlin rao <rao.chen...@gmail.com>: >>>>>>>>>>> > $MaxMessageSize 32m >>>>>>>>>>> > module( load="imtcp" ) >>>>>>>>>>> > module( load="imuxsock" ) >>>>>>>>>>> > module( load="imklog" ) >>>>>>>>>>> > module( load="mmfields" ) >>>>>>>>>>> > module( load="mmjsonparse" ) >>>>>>>>>>> > module( load="omelasticsearch" ) >>>>>>>>>>> > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >>>>>>>>>>> > $RepeatedMsgReduction off >>>>>>>>>>> > $WorkDirectory /data0/rsyslog >>>>>>>>>>> > $MainMsgQueueFilename mainQ >>>>>>>>>>> > $MainMsgQueueType LinkedList >>>>>>>>>>> > $MainMsgQueueMaxFileSize 512M >>>>>>>>>>> > $MainMsgQueueMaxDiskSpace 20G >>>>>>>>>>> > $PreserveFQDN on >>>>>>>>>>> > input(type="imtcp" port="514") >>>>>>>>>>> > template(name="local6IndexName" type="list") { >>>>>>>>>>> > constant(value="logstash-mweibo-") >>>>>>>>>>> > property(name="timereported" dateFormat="rfc3339" >>>>>>>>>>> > position.from="1" >>>>>>>>>>> > position.to="4") >>>>>>>>>>> > constant(value=".") >>>>>>>>>>> > property(name="timereported" dateFormat="rfc3339" >>>>>>>>>>> > position.from="6" >>>>>>>>>>> > position.to="7") >>>>>>>>>>> > constant(value=".") >>>>>>>>>>> > property(name="timereported" dateFormat="rfc3339" >>>>>>>>>>> > position.from="9" >>>>>>>>>>> > position.to="10") >>>>>>>>>>> > } >>>>>>>>>>> > template( name="local6TypeName" type="string" >>>>>>>>>>> > string="%programname%" ) >>>>>>>>>>> > template( name="local6JsonArray" type="list" ) { >>>>>>>>>>> > constant(value="{\"@timestamp\":\"") >>>>>>>>>>> > property(name="timereported" >>>>>>>>>>> > dateFormat="rfc3339") >>>>>>>>>>> > constant(value="\",\"host\":\"") property(name="hostname") >>>>>>>>>>> > constant(value="\",") property(name="$.line" >>>>>>>>>>> > position.from="2") >>>>>>>>>>> > } >>>>>>>>>>> > Ruleset( name="forwardRuleSetJsonArray" ) >>>>>>>>>>> > { >>>>>>>>>>> > action ( type="mmjsonparse" >>>>>>>>>>> > name="action_jsonarray-parse" >>>>>>>>>>> > queue.size="3000" >>>>>>>>>>> > queue.dequeuebatchsize="300" >>>>>>>>>>> > queue.maxdiskspace="15G" >>>>>>>>>>> > queue.checkpointinterval="10" >>>>>>>>>>> > queue.type="linkedlist" >>>>>>>>>>> > queue.workerthreads="30" >>>>>>>>>>> > queue.workerthreadminimummessages="100" >>>>>>>>>>> > queue.maxfilesize="500M" >>>>>>>>>>> > queue.saveonshutdown="on" >>>>>>>>>>> > ) >>>>>>>>>>> >>>>>>>>>>> running mmjsonparse on a queue (asynchronously!) does not make any >>>>>>>>>>> sense. When running async, the json properties will never be seen by >>>>>>>>>>> the rest of the ruleset. >>>>>>>>>>> >>>>>>>>>>> Maybe this even triggers the fault... >>>>>>>>>>> >>>>>>>>>>> Rainer >>>>>>>>>>> > # maybe recieved a too large data being truncated >>>>>>>>>>> > if ( re_match($msg, ']}$') ) then { >>>>>>>>>>> > foreach ($.line in $!msg) do { >>>>>>>>>>> > action ( >>>>>>>>>>> > # test for other output modules >>>>>>>>>>> > # type="omfile" file="/data/rsyslog-debug.log" >>>>>>>>>>> > # type="omfwd" Target="127.0.0.1" Port="5140" >>>>>>>>>>> Protocol="tcp" >>>>>>>>>>> > type="omelasticsearch" server="es.domain.com" >>>>>>>>>>> > dynSearchIndex="on" searchIndex="local6IndexName" >>>>>>>>>>> > dynSearchType="on" >>>>>>>>>>> > searchType="local6TypeName" asyncrepl="on" bulkmode="on" >>>>>>>>>>> > template="local6JsonArray" >>>>>>>>>>> > name="action_json2log-es1003" >>>>>>>>>>> > queue.filename="action_json2log-es1003" >>>>>>>>>>> > queue.size="10000" >>>>>>>>>>> > queue.dequeuebatchsize="2000" >>>>>>>>>>> > queue.maxdiskspace="15G" >>>>>>>>>>> > queue.discardseverity="3" >>>>>>>>>>> > queue.checkpointinterval="10" >>>>>>>>>>> > queue.type="linkedlist" >>>>>>>>>>> > queue.workerthreads="5" >>>>>>>>>>> > queue.workerthreadminimummessages="2000" >>>>>>>>>>> > queue.maxfilesize="500M" >>>>>>>>>>> > queue.saveonshutdown="on" >>>>>>>>>>> > ) >>>>>>>>>>> > } >>>>>>>>>>> > stop >>>>>>>>>>> > } >>>>>>>>>>> > } >>>>>>>>>>> > >>>>>>>>>>> > if ( $programname == 'mobile_client_net_fatal_error' ) then >>>>>>>>>>> > { >>>>>>>>>>> > call forwardRuleSetJsonArray >>>>>>>>>>> > stop >>>>>>>>>>> > } >>>>>>>>>>> > >>>>>>>>>>> *.info;mail.none;authpriv.none;cron.none;local6.none;local7.none;user.none >>>>>>>>>>> > /var/log/messages >>>>>>>>>>> > authpriv.* >>>>>>>>>>> > /var/log/secure >>>>>>>>>>> > mail.* >>>>>>>>>>> > /var/log/maillog >>>>>>>>>>> > cron.* >>>>>>>>>>> > /var/log/cron >>>>>>>>>>> > uucp,news.crit >>>>>>>>>>> > /var/log/spooler >>>>>>>>>>> > >>>>>>>>>>> > 2015-03-21 13:14 GMT+08:00 singh.janmejay >>>>>>>>>>> > <singh.janme...@gmail.com>: >>>>>>>>>>> > >>>>>>>>>>> >> Can you please share your config as well? >>>>>>>>>>> >> >>>>>>>>>>> >> Also, I'll likely be able to look at it only after 27th Mar. As >>>>>>>>>>> >> of now I >>>>>>>>>>> >> don't have access to a computer and github doesn't seem to show >>>>>>>>>>> >> line >>>>>>>>>>> >> numbers while browsing code using mobile phone. >>>>>>>>>>> >> >>>>>>>>>>> >> -- >>>>>>>>>>> >> Regards, >>>>>>>>>>> >> Janmejay >>>>>>>>>>> >> >>>>>>>>>>> >> PS: Please blame the typos in this mail on my phone's >>>>>>>>>>> >> uncivilized soft >>>>>>>>>>> >> keyboard sporting it's not-so-smart-assist technology. >>>>>>>>>>> >> >>>>>>>>>>> >> On Mar 21, 2015 8:49 AM, "chenlin rao" <rao.chen...@gmail.com> >>>>>>>>>>> >> wrote: >>>>>>>>>>> >> >>>>>>>>>>> >> > I try to build rsyslogd from github master with "./configure >>>>>>>>>>> >> --enable-debug >>>>>>>>>>> >> > --enable-valgrind --enable-memcheck --enable-elasticsearch >>>>>>>>>>> >> > --enable-mmjsonparse --enable-mmsequence --enable-mmfields >>>>>>>>>>> >> > --disable-liblogging-stdlog --enable-omruleset" >>>>>>>>>>> >> > >>>>>>>>>>> >> > then process exit with: >>>>>>>>>>> >> > >>>>>>>>>>> >> > 7778.301430428:main Q[DA]:Reg/w0: ACTION 16 >>>>>>>>>>> >> > [builtin:omfwd:action(type="builtin:omfwd" ...)] >>>>>>>>>>> >> > 7778.301676909:main Q[DA]:Reg/w0: executing action 16 >>>>>>>>>>> >> > 7778.301791807:main Q[DA]:Reg/w0: Called action, logging to >>>>>>>>>>> builtin:omfwd >>>>>>>>>>> >> > ==20833== Thread 2: >>>>>>>>>>> >> > ==20833== Invalid read of size 4 >>>>>>>>>>> >> > ==20833== at 0x3A044093A0: pthread_mutex_lock (in /lib64/ >>>>>>>>>>> >> > libpthread-2.12.so) >>>>>>>>>>> >> > ==20833== by 0x438053: qqueueEnqMsg (queue.c:2856) >>>>>>>>>>> >> > ==20833== by 0x4421AB: doSubmitToActionQ (action.c:1403) >>>>>>>>>>> >> > ==20833== by 0x439D6C: scriptExec (ruleset.c:197) >>>>>>>>>>> >> > ==20833== by 0x439ECC: scriptExec (ruleset.c:284) >>>>>>>>>>> >> > ==20833== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>>> >> > ==20833== by 0x43A40B: processBatch (ruleset.c:503) >>>>>>>>>>> >> > ==20833== by 0x44A403: msgConsumer (rsyslogd.c:575) >>>>>>>>>>> >> > ==20833== by 0x438D12: ConsumerReg (queue.c:1897) >>>>>>>>>>> >> > ==20833== by 0x43314B: wtiWorker (wti.c:334) >>>>>>>>>>> >> > ==20833== by 0x431BA6: wtpWorker (wtp.c:389) >>>>>>>>>>> >> > ==20833== by 0x3A044079D0: start_thread (in /lib64/ >>>>>>>>>>> libpthread-2.12.so >>>>>>>>>>> >> ) >>>>>>>>>>> >> > ==20833== Address 0x10 is not stack'd, malloc'd or (recently) >>>>>>>>>>> >> > free'd >>>>>>>>>>> >> > ==20833== >>>>>>>>>>> >> > ==20833== >>>>>>>>>>> >> > ==20833== Process terminating with default action of signal 11 >>>>>>>>>>> (SIGSEGV) >>>>>>>>>>> >> > ==20833== Access not within mapped region at address 0x10 >>>>>>>>>>> >> > ==20833== at 0x3A044093A0: pthread_mutex_lock (in /lib64/ >>>>>>>>>>> >> > libpthread-2.12.so) >>>>>>>>>>> >> > ==20833== by 0x438053: qqueueEnqMsg (queue.c:2856) >>>>>>>>>>> >> > ==20833== by 0x4421AB: doSubmitToActionQ (action.c:1403) >>>>>>>>>>> >> > ==20833== by 0x439D6C: scriptExec (ruleset.c:197) >>>>>>>>>>> >> > ==20833== by 0x439ECC: scriptExec (ruleset.c:284) >>>>>>>>>>> >> > ==20833== by 0x439CD9: scriptExec (ruleset.c:416) >>>>>>>>>>> >> > ==20833== by 0x43A40B: processBatch (ruleset.c:503) >>>>>>>>>>> >> > ==20833== by 0x44A403: msgConsumer (rsyslogd.c:575) >>>>>>>>>>> >> > ==20833== by 0x438D12: ConsumerReg (queue.c:1897) >>>>>>>>>>> >> > ==20833== by 0x43314B: wtiWorker (wti.c:334) >>>>>>>>>>> >> > ==20833== by 0x431BA6: wtpWorker (wtp.c:389) >>>>>>>>>>> >> > ==20833== by 0x3A044079D0: start_thread (in /lib64/ >>>>>>>>>>> libpthread-2.12.so >>>>>>>>>>> >> ) >>>>>>>>>>> >> > ==20833== If you believe this happened as a result of a stack >>>>>>>>>>> >> > ==20833== overflow in your program's main thread (unlikely but >>>>>>>>>>> >> > ==20833== possible), you can try to increase the size of the >>>>>>>>>>> >> > ==20833== main thread stack using the --main-stacksize= flag. >>>>>>>>>>> >> > ==20833== The main thread stack size used in this run was >>>>>>>>>>> >> > 10485760. >>>>>>>>>>> >> > ==20833== >>>>>>>>>>> >> > ==20833== HEAP SUMMARY: >>>>>>>>>>> >> > ==20833== in use at exit: 44,605,326 bytes in 5,019 blocks >>>>>>>>>>> >> > ==20833== total heap usage: 10,310 allocs, 5,291 frees, >>>>>>>>>>> >> > 78,621,963 >>>>>>>>>>> >> bytes >>>>>>>>>>> >> > allocated >>>>>>>>>>> >> > ==20833== >>>>>>>>>>> >> > ==20833== LEAK SUMMARY: >>>>>>>>>>> >> > ==20833== definitely lost: 516 bytes in 12 blocks >>>>>>>>>>> >> > ==20833== indirectly lost: 0 bytes in 0 blocks >>>>>>>>>>> >> > ==20833== possibly lost: 33,559,553 bytes in 17 blocks >>>>>>>>>>> >> > ==20833== still reachable: 11,045,257 bytes in 4,990 blocks >>>>>>>>>>> >> > ==20833== suppressed: 0 bytes in 0 blocks >>>>>>>>>>> >> > ==20833== Rerun with --leak-check=full to see details of >>>>>>>>>>> >> > leaked memory >>>>>>>>>>> >> > ==20833== >>>>>>>>>>> >> > ==20833== For counts of detected and suppressed errors, rerun >>>>>>>>>>> >> > with: -v >>>>>>>>>>> >> > ==20833== Use --track-origins=yes to see where uninitialised >>>>>>>>>>> >> > values >>>>>>>>>>> come >>>>>>>>>>> >> > from >>>>>>>>>>> >> > ==20833== ERROR SUMMARY: 11 errors from 3 contexts >>>>>>>>>>> >> > (suppressed: 100 >>>>>>>>>>> from >>>>>>>>>>> >> 9) >>>>>>>>>>> >> > 已杀死 >>>>>>>>>>> >> > >>>>>>>>>>> >> > 2015-03-20 23:29 GMT+08:00 singh.janmejay >>>>>>>>>>> >> > <singh.janme...@gmail.com>: >>>>>>>>>>> >> > >>>>>>>>>>> >> > > Can you please build with debug symbols and repeat the >>>>>>>>>>> >> > > valgrind run? >>>>>>>>>>> >> > > >>>>>>>>>>> >> > > -- >>>>>>>>>>> >> > > Regards, >>>>>>>>>>> >> > > Janmejay >>>>>>>>>>> >> > > >>>>>>>>>>> >> > > PS: Please blame the typos in this mail on my phone's >>>>>>>>>>> >> > > uncivilized >>>>>>>>>>> soft >>>>>>>>>>> >> > > keyboard sporting it's not-so-smart-assist technology. >>>>>>>>>>> >> > > >>>>>>>>>>> >> > > On Mar 20, 2015 6:03 PM, "chenlin rao" >>>>>>>>>>> >> > > <rao.chen...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >> > > >>>>>>>>>>> >> > > > btw: if I change omelasticsearch/omfwd to omfile, rsyslogd >>>>>>>>>>> >> > > > would >>>>>>>>>>> be >>>>>>>>>>> >> > > fine... >>>>>>>>>>> >> > > > >>>>>>>>>>> >> > > > 2015-03-20 20:13 GMT+08:00 chenlin rao >>>>>>>>>>> >> > > > <rao.chen...@gmail.com>: >>>>>>>>>>> >> > > > >>>>>>>>>>> >> > > > > 3498.767218405:main Q[DA]:Reg/w0: rainerscript: var >>>>>>>>>>> >> > > > > 200:!msg: >>>>>>>>>>> '[ { >>>>>>>>>>> >> > > "uid": >>>>>>>>>>> >> > > > > "1941604034", "request_header": >>>>>>>>>>> >> > > "{\"Accept-Encoding\":\"gzip,deflate\"}", >>>>>>>>>>> >> > > > > "network_type": "wifi", "end_time": "1426836307406", >>>>>>>>>>> >> > > > > "dns_ip": >>>>>>>>>>> >> > > > > "218.15.203.34,192.168.1.1", "response_code": "200", >>>>>>>>>>> >> "response_data": >>>>>>>>>>> >> > > > "{}", >>>>>>>>>>> >> > > > > "start_time": "1426836307328", "act": "net_fatal_error", >>>>>>>>>>> >> > > > > "type": >>>>>>>>>>> >> > > > > "net_fatal_error", "request_url": "https:\/\/api.weibo.cn >>>>>>>>>>> >> > > > >>>>>>>>>>> >> > > >>>>>>>>>>> >> > >>>>>>>>>>> >> >>>>>>>>>>> \/2\/client\/url_list?accuracy_level=0&c=android&i=9eef7ba&s=0088f6a4&ua=Xiaomi-MI%202S__weibo__5.0.0__android__android4.4.4&wm=5311_4002&v_f=2&from=1050095010&gsid=4uzH00573m6YZU5oVhYra896c0y&lang=zh_CN&skin=default&oldwm=5311_4002", >>>>>>>>>>> >> > > > > "response_status_line": "HTTP\/1.1 200 OK" }, { "uid": >>>>>>>>>>> >> "1941604034", >>>>>>>>>>> >> > > > > "request_header": >>>>>>>>>>> >> > > > > "{\"Accept-Encoding\":\"gzip,deflate\"}", >>>>>>>>>>> >> > > > "network_type": >>>>>>>>>>> >> > > > > "wifi", "end_time": "1426836320563", "dns_ip": >>>>>>>>>>> >> > > > "218.15.203.34,192.168.1.1", >>>>>>>>>>> >> > > > > "response_code": "200", "response_data": "{}", >>>>>>>>>>> >> > > > > "start_time": >>>>>>>>>>> >> > > > > "1426836320505", "act": "net_fatal_error", "type": >>>>>>>>>>> >> "net_fatal_error", >>>>>>>>>>> >> > > > > "request_url": "https:\/\/api.weibo.cn >>>>>>>>>>> >> > > > >>>>>>>>>>> >> > > >>>>>>>>>>> >> > >>>>>>>>>>> >> >>>>>>>>>>> \/2\/client\/url_list?accuracy_level=0&c=android&i=9eef7ba&s=0088f6a4&ua=Xiaomi-MI%202S__weibo__5.0.0__android__android4.4.4&wm=5311_4002&v_f=2&from=1050095010&gsid=4uzH00573m6YZU5oVhYra896c0y&lang=zh_CN&skin=default&oldwm=5311_4002", >>>>>>>>>>> >> > > > > "response_status_line": "HTTP\/1.1 200 OK" } ]' >>>>>>>>>>> >> > > > > 3498.767338611:main Q[DA]:Reg/w0: eval expr 0x62e63a0, >>>>>>>>>>> >> > > > > return >>>>>>>>>>> >> > datatype >>>>>>>>>>> >> > > > 'J' >>>>>>>>>>> >> > > > > 3498.770732236:main Q[DA]:Reg/w0: ACTION 17 >>>>>>>>>>> >> > > > > [builtin:omfwd:action(type="builtin:omfwd" ...)] >>>>>>>>>>> >> > > > > 3498.770963455:main Q[DA]:Reg/w0: executing action 17 >>>>>>>>>>> >> > > > > 3498.771052511:main Q[DA]:Reg/w0: Called action, logging >>>>>>>>>>> >> > > > > to >>>>>>>>>>> >> > > builtin:omfwd >>>>>>>>>>> >> > > > > ==30138== Thread 3: >>>>>>>>>>> >> > > > > ==30138== Invalid read of size 4 >>>>>>>>>>> >> > > > > ==30138== at 0x4E3E3A0: pthread_mutex_lock (in /lib64/ >>>>>>>>>>> >> > > > > libpthread-2.12.so) >>>>>>>>>>> >> > > > > ==30138== by 0x14228D: qqueueEnqMsg (in >>>>>>>>>>> >> > > > > /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x14BB5B: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143C4C: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143DAC: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143BB9: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x1442CB: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x15336B: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143112: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x13D671: wtiWorker (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x13D1C1: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x4E3C9D0: start_thread (in /lib64/ >>>>>>>>>>> >> > libpthread-2.12.so) >>>>>>>>>>> >> > > > > ==30138== Address 0x10 is not stack'd, malloc'd or >>>>>>>>>>> >> > > > > (recently) >>>>>>>>>>> >> free'd >>>>>>>>>>> >> > > > > ==30138== >>>>>>>>>>> >> > > > > ==30138== >>>>>>>>>>> >> > > > > ==30138== Process terminating with default action of >>>>>>>>>>> >> > > > > signal 11 >>>>>>>>>>> >> > > (SIGSEGV) >>>>>>>>>>> >> > > > > ==30138== Access not within mapped region at address >>>>>>>>>>> >> > > > > 0x10 >>>>>>>>>>> >> > > > > ==30138== at 0x4E3E3A0: pthread_mutex_lock (in /lib64/ >>>>>>>>>>> >> > > > > libpthread-2.12.so) >>>>>>>>>>> >> > > > > ==30138== by 0x14228D: qqueueEnqMsg (in >>>>>>>>>>> >> > > > > /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x14BB5B: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143C4C: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143DAC: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143BB9: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x1442CB: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x15336B: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x143112: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x13D671: wtiWorker (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x13D1C1: ??? (in /sbin/rsyslogd) >>>>>>>>>>> >> > > > > ==30138== by 0x4E3C9D0: start_thread (in /lib64/ >>>>>>>>>>> >> > libpthread-2.12.so) >>>>>>>>>>> >> > > > > ==30138== If you believe this happened as a result of a >>>>>>>>>>> >> > > > > stack >>>>>>>>>>> >> > > > > ==30138== overflow in your program's main thread >>>>>>>>>>> >> > > > > (unlikely but >>>>>>>>>>> >> > > > > ==30138== possible), you can try to increase the size >>>>>>>>>>> >> > > > > of the >>>>>>>>>>> >> > > > > ==30138== main thread stack using the --main-stacksize= >>>>>>>>>>> >> > > > > flag. >>>>>>>>>>> >> > > > > ==30138== The main thread stack size used in this run >>>>>>>>>>> >> > > > > was >>>>>>>>>>> >> 10485760. >>>>>>>>>>> >> > > > > ==30138== >>>>>>>>>>> >> > > > > ==30138== HEAP SUMMARY: >>>>>>>>>>> >> > > > > ==30138== in use at exit: 1,078,127,378 bytes in >>>>>>>>>>> >> > > > > 291,997 >>>>>>>>>>> blocks >>>>>>>>>>> >> > > > > ==30138== total heap usage: 614,872 allocs, 322,875 >>>>>>>>>>> >> > > > > frees, >>>>>>>>>>> >> > > > 1,586,744,205 >>>>>>>>>>> >> > > > > bytes allocated >>>>>>>>>>> >> > > > > ==30138== >>>>>>>>>>> >> > > > > ==30138== LEAK SUMMARY: >>>>>>>>>>> >> > > > > ==30138== definitely lost: 33,555,027 bytes in 13 >>>>>>>>>>> >> > > > > blocks >>>>>>>>>>> >> > > > > ==30138== indirectly lost: 0 bytes in 0 blocks >>>>>>>>>>> >> > > > > ==30138== possibly lost: 5,760 bytes in 18 blocks >>>>>>>>>>> >> > > > > ==30138== still reachable: 1,044,566,591 bytes in >>>>>>>>>>> >> > > > > 291,966 >>>>>>>>>>> blocks >>>>>>>>>>> >> > > > > ==30138== suppressed: 0 bytes in 0 blocks >>>>>>>>>>> >> > > > > ==30138== Rerun with --leak-check=full to see details of >>>>>>>>>>> >> > > > > leaked >>>>>>>>>>> >> > memory >>>>>>>>>>> >> > > > > ==30138== >>>>>>>>>>> >> > > > > ==30138== For counts of detected and suppressed errors, >>>>>>>>>>> >> > > > > rerun >>>>>>>>>>> with: >>>>>>>>>>> >> > -v >>>>>>>>>>> >> > > > > ==30138== Use --track-origins=yes to see where >>>>>>>>>>> >> > > > > uninitialised >>>>>>>>>>> values >>>>>>>>>>> >> > > come >>>>>>>>>>> >> > > > > from >>>>>>>>>>> >> > > > > ==30138== ERROR SUMMARY: 1809 errors from 5 contexts >>>>>>>>>>> (suppressed: >>>>>>>>>>> >> 131 >>>>>>>>>>> >> > > > from >>>>>>>>>>> >> > > > > 9) >>>>>>>>>>> >> > > > > Killed >>>>>>>>>>> >> > > > > >>>>>>>>>>> >> > > > > >>>>>>>>>>> >> > > > > 2015-03-20 8:50 GMT+08:00 singh.janmejay < >>>>>>>>>>> singh.janme...@gmail.com >>>>>>>>>>> >> >: >>>>>>>>>>> >> > > > > >>>>>>>>>>> >> > > > >> Can you please run this with valgrind and share its >>>>>>>>>>> >> > > > >> output on >>>>>>>>>>> >> crash? >>>>>>>>>>> >> > > > >> >>>>>>>>>>> >> > > > >> -- >>>>>>>>>>> >> > > > >> Regards, >>>>>>>>>>> >> > > > >> Janmejay >>>>>>>>>>> >> > > > >> >>>>>>>>>>> >> > > > >> PS: Please blame the typos in this mail on my phone's >>>>>>>>>>> uncivilized >>>>>>>>>>> >> > soft >>>>>>>>>>> >> > > > >> keyboard sporting it's not-so-smart-assist technology. >>>>>>>>>>> >> > > > >> >>>>>>>>>>> >> > > > >> On Mar 19, 2015 11:10 PM, "chenlin rao" >>>>>>>>>>> >> > > > >> <rao.chen...@gmail.com >>>>>>>>>>> > >>>>>>>>>>> >> > > wrote: >>>>>>>>>>> >> > > > >> >>>>>>>>>>> >> > > > >> > Hello everyone. >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > I just learnt a foreach syntax from >>>>>>>>>>> >> > > `src/tests/json_array_looping.sh`, >>>>>>>>>>> >> > > > >> so I >>>>>>>>>>> >> > > > >> > try to parse my logdata(yes, long json array in msg) >>>>>>>>>>> >> > > > >> > as >>>>>>>>>>> follow: >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > ``` >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > $MaxMessageSize 256k >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > template( name="local6JsonArray" type="string" >>>>>>>>>>> >> > string="%$.line%\n" ) >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > Ruleset( name="forwardRuleSetJsonArray" ) >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > { >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > action( type="mmjsonparse" ) >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > foreach ($.line in $!msg) do { >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > action ( type="omfile" file="/data0/logfile" >>>>>>>>>>> >> > > > >> > template="local6JsonArray") >>>>>>>>>>> >> > > > >> > } >>>>>>>>>>> >> > > > >> > } >>>>>>>>>>> >> > > > >> > ``` >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > But I always got segment fault after restarted few >>>>>>>>>>> >> > > > >> > minutes. >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > Before the rsyslogd died , I can watch some correct >>>>>>>>>>> >> > > > >> > lines in >>>>>>>>>>> the >>>>>>>>>>> >> > > > >> > "/data0/logfile". >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > While running `rsyslogd -dn`, I watch the last five >>>>>>>>>>> >> > > > >> > lines >>>>>>>>>>> were: >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > ``` >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > 3295.820218609:main Q:Reg/w0 : FOREACH .line IN >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > 3295.820227632:main Q:Reg/w0 : var '!msg' >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > 3295.820240109:main Q:Reg/w0 : eval expr >>>>>>>>>>> >> > > > >> > 0x7fe2044fefa0, >>>>>>>>>>> type >>>>>>>>>>> >> > > 'V[86]' >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > 3295.821747011:main Q:Reg/w0 : rainerscript: var >>>>>>>>>>> >> > > > >> > 200:!msg: >>>>>>>>>>> >> > > '{"msg":[{ >>>>>>>>>>> >> > > > >> > "end_time":"142...longmsghere....\"verieval expr >>>>>>>>>>> 0x7fe2044fefa0, >>>>>>>>>>> >> > > > return >>>>>>>>>>> >> > > > >> > datatype 'J' >>>>>>>>>>> >> > > > >> > ``` >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > The length of such line is about 20k~30k, far away >>>>>>>>>>> >> > > > >> > below my >>>>>>>>>>> >> > > > >> > $MaxMessageSize. >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> > So, why rsyslogd segment fault? >>>>>>>>>>> >> > > > >> > _______________________________________________ >>>>>>>>>>> >> > > > >> > rsyslog mailing list >>>>>>>>>>> >> > > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> >> > > > >> > http://www.rsyslog.com/professional-services/ >>>>>>>>>>> >> > > > >> > What's up with rsyslog? Follow >>>>>>>>>>> >> > > > >> > https://twitter.com/rgerhards >>>>>>>>>>> >> > > > >> > NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>>>>> >> > > > >> > ARCHIVED >>>>>>>>>>> by >>>>>>>>>>> >> a >>>>>>>>>>> >> > > > myriad >>>>>>>>>>> >> > > > >> > of sites beyond our control. PLEASE UNSUBSCRIBE and >>>>>>>>>>> >> > > > >> > DO NOT >>>>>>>>>>> POST >>>>>>>>>>> >> if >>>>>>>>>>> >> > > you >>>>>>>>>>> >> > > > >> > DON'T LIKE THAT. >>>>>>>>>>> >> > > > >> > >>>>>>>>>>> >> > > > >> _______________________________________________ >>>>>>>>>>> >> > > > >> rsyslog mailing list >>>>>>>>>>> >> > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> >> > > > >> http://www.rsyslog.com/professional-services/ >>>>>>>>>>> >> > > > >> What's up with rsyslog? Follow >>>>>>>>>>> >> > > > >> https://twitter.com/rgerhards >>>>>>>>>>> >> > > > >> NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>>>>> >> > > > >> ARCHIVED >>>>>>>>>>> by a >>>>>>>>>>> >> > > myriad >>>>>>>>>>> >> > > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO >>>>>>>>>>> >> > > > >> NOT >>>>>>>>>>> POST if >>>>>>>>>>> >> > you >>>>>>>>>>> >> > > > >> DON'T LIKE THAT. >>>>>>>>>>> >> > > > >> >>>>>>>>>>> >> > > > > >>>>>>>>>>> >> > > > > >>>>>>>>>>> >> > > > _______________________________________________ >>>>>>>>>>> >> > > > rsyslog mailing list >>>>>>>>>>> >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> >> > > > http://www.rsyslog.com/professional-services/ >>>>>>>>>>> >> > > > What's up with rsyslog? Follow >>>>>>>>>>> >> > > > https://twitter.com/rgerhards >>>>>>>>>>> >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are >>>>>>>>>>> >> > > > ARCHIVED by a >>>>>>>>>>> >> > myriad >>>>>>>>>>> >> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>> >> > > > POST if >>>>>>>>>>> >> you >>>>>>>>>>> >> > > > DON'T LIKE THAT. >>>>>>>>>>> >> > > > >>>>>>>>>>> >> > > _______________________________________________ >>>>>>>>>>> >> > > rsyslog mailing list >>>>>>>>>>> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> >> > > http://www.rsyslog.com/professional-services/ >>>>>>>>>>> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>>>>>>>>>> >> > > by a >>>>>>>>>>> >> myriad >>>>>>>>>>> >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>> >> > > POST if >>>>>>>>>>> you >>>>>>>>>>> >> > > DON'T LIKE THAT. >>>>>>>>>>> >> > > >>>>>>>>>>> >> > _______________________________________________ >>>>>>>>>>> >> > rsyslog mailing list >>>>>>>>>>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> >> > http://www.rsyslog.com/professional-services/ >>>>>>>>>>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>>>>>>>>>> >> > by a >>>>>>>>>>> myriad >>>>>>>>>>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>>> >> > POST if you >>>>>>>>>>> >> > DON'T LIKE THAT. >>>>>>>>>>> >> _______________________________________________ >>>>>>>>>>> >> rsyslog mailing list >>>>>>>>>>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> >> http://www.rsyslog.com/professional-services/ >>>>>>>>>>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>>>>>>>> >> a myriad >>>>>>>>>>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>>>>> >> if you >>>>>>>>>>> >> DON'T LIKE THAT. >>>>>>>>>>> >> >>>>>>>>>>> > _______________________________________________ >>>>>>>>>>> > rsyslog mailing list >>>>>>>>>>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> > http://www.rsyslog.com/professional-services/ >>>>>>>>>>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>>> > myriad >>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>>>> you >>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> rsyslog mailing list >>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>>> myriad >>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>>>> you >>>>>>>>>>> DON'T LIKE THAT. >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> rsyslog mailing list >>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>>> POST if you DON'T LIKE THAT. >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> rsyslog mailing list >>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>>>>>> POST if you DON'T LIKE THAT. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Janmejay >>>>>>>> http://codehunk.wordpress.com >>>>>>>> _______________________________________________ >>>>>>>> rsyslog mailing list >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>>> if you DON'T LIKE THAT. >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>>>> if you DON'T LIKE THAT. >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Janmejay >>>>>> http://codehunk.wordpress.com >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Janmejay >>>>> http://codehunk.wordpress.com >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>> >>> >>> >>> -- >>> Regards, >>> Janmejay >>> http://codehunk.wordpress.com >> >> >> >> -- >> Regards, >> Janmejay >> http://codehunk.wordpress.com > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.