Tests for replace, wrap(2) and wrap(3) + a bug-fix in replace. Sorry for the passing more patches, Im using patches/mail to complete things that are half-done, will start using github fork to manage fresh work. Last patch, I guess.
On Fri, Oct 31, 2014 at 7:31 AM, singh.janmejay <[email protected]> wrote: > > > On Thu, Oct 30, 2014 at 5:00 PM, Rainer Gerhards <[email protected] > > wrote: > >> 014-10-30 12:28 GMT+01:00 singh.janmejay <[email protected]>: >> >> > Patch for rsyslog-doc. >> > >> > >> Thanks! >> >> >> > I coudn't find the repo on git.adiscon.com, so this patch is done over >> the >> > github mirror. >> > >> > >> This repo only exists on github. In any case, github is more or less the >> primary these days, because it's easier to work with their pull requests. >> >> Will merge within the next couple of minutes (so no extra confirmation for >> that ;)). >> >> Rainer >> >> > On Wed, Oct 29, 2014 at 8:33 PM, singh.janmejay < >> [email protected]> >> > wrote: >> > >> > > Sure. >> > > >> > > -- >> > > Regards, >> > > Janmejay >> > > >> > > PS: Please blame the typos in this mail on my phone's uncivilized soft >> > > keyboard sporting it's not-so-smart-assist technology. >> > > >> > > On Oct 29, 2014 8:31 PM, "Rainer Gerhards" <[email protected]> >> > > wrote: >> > > >> > >> excellent, thx! If you need help, let me know. >> > >> >> > >> Rainer >> > >> >> > >> 2014-10-29 16:00 GMT+01:00 singh.janmejay <[email protected] >> >: >> > >> >> > >> > No, I'll add docs, I want to learn how to do it. Let me handle it. >> > >> > >> > >> > -- >> > >> > Regards, >> > >> > Janmejay >> > >> > >> > >> > PS: Please blame the typos in this mail on my phone's uncivilized >> soft >> > >> > keyboard sporting it's not-so-smart-assist technology. >> > >> > >> > >> > On Oct 29, 2014 7:30 PM, "Rainer Gerhards" < >> [email protected]> >> > >> > wrote: >> > >> > >> > >> > > Looks fine to me, except for a minor code formatting nit (we use >> tab >> > >> and >> > >> > > not fixed spaces - bad choice at that time ;)). >> > >> > > >> > >> > > I have now also merged these two. >> > >> > > >> > >> > > It would be great if you could add some doc to the rsyslog-doc >> > >> project. >> > >> > If >> > >> > > there is no time, let me know and I'll at least mention the new >> > >> functions >> > >> > > in it. >> > >> > > >> > >> > > Thanks again! >> > >> > > Rainer >> > >> > > >> > >> > > 2014-10-21 9:38 GMT+02:00 singh.janmejay < >> [email protected] >> > >: >> > >> > > >> > >> > > > Hi Rainer, David, >> > >> > > > >> > >> > > > Do the patches look ok? >> > >> > > > >> > >> > > > On Fri, Oct 17, 2014 at 3:56 PM, singh.janmejay < >> > >> > > [email protected]> >> > >> > > > wrote: >> > >> > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > On Tue, Oct 14, 2014 at 12:18 PM, singh.janmejay < >> > >> > > > [email protected] >> > >> > > > > > wrote: >> > >> > > > > >> > >> > > > >> Yes, I'll make necessary changes and send the patch again. >> > >> > > > >> >> > >> > > > >> -- >> > >> > > > >> Regards, >> > >> > > > >> Janmejay >> > >> > > > >> >> > >> > > > >> PS: Please blame the typos in this mail on my phone's >> > uncivilized >> > >> > soft >> > >> > > > >> keyboard sporting it's not-so-smart-assist technology. >> > >> > > > >> On Oct 14, 2014 12:02 PM, "Rainer Gerhards" < >> > >> > > [email protected]> >> > >> > > > >> wrote: >> > >> > > > >> >> > >> > > > >>> 2014-10-14 8:30 GMT+02:00 singh.janmejay < >> > >> [email protected] >> > >> > >: >> > >> > > > >>> >> > >> > > > >>> > > > - replace(operand, substring_to_be_replaced, >> > >> its_replacement) >> > >> > > > >>> > > > - wrap(operand, string_to_wrap_around_it) >> > >> > > > >>> > > > >> > >> > > > >>> > > >> > >> > > > >>> > > - concat(operand_1, operand_2) >> > >> > > > >>> > > > >> > >> > > > >>> > > >> > >> > > > >>> > > >> > >> > > > >>> > > I think you overlooked the concatenation operation. you >> > can >> > >> > > simply >> > >> > > > do >> > >> > > > >>> > > >> > >> > > > >>> > > operand1 & operand2 >> > >> > > > >>> > > >> > >> > > > >>> > > eg: "foo" & "bar" ==> "foobar" >> > >> > > > >>> > > >> > >> > > > >>> > > I think concat() is equivalent to this. Am I right? >> > >> > > > >>> > > >> > >> > > > >>> > >> > >> > > > >>> > It is the same. I didn't know it was available as an >> > >> operator, my >> > >> > > > bad. >> > >> > > > >>> Will >> > >> > > > >>> > remove the concat implementation in the reworked patch. >> > >> > > > >>> > >> > >> > > > >>> >> > >> > > > >>> yeah the doc is pretty sparse. >> > >> > > > >>> >> > >> > > > >>> If I understand you correctly, I will wait for a new patch >> set >> > >> > before >> > >> > > > >>> merging the existing one. Right? >> > >> > > > >>> >> > >> > > > >>> Rainer >> > >> > > > >>> _______________________________________________ >> > >> > > > >>> rsyslog mailing list >> > >> > > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> > > > >>> http://www.rsyslog.com/professional-services/ >> > >> > > > >>> What's up with rsyslog? Follow >> https://twitter.com/rgerhards >> > >> > > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are >> ARCHIVED >> > by >> > >> a >> > >> > > > myriad >> > >> > > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> > POST >> > >> if >> > >> > > you >> > >> > > > >>> DON'T LIKE THAT. >> > >> > > > >>> >> > >> > > > >> >> > >> > > > > >> > >> > > > > Hi, >> > >> > > > > >> > >> > > > > Sorry for breaking the continuation here, I finally managed >> to >> > get >> > >> > some >> > >> > > > > time to work on this today. >> > >> > > > > >> > >> > > > > Patches are attached. Just for clarity, I am attaching >> > >> all(actually >> > >> > 2) >> > >> > > > > relevant patches(even though one of them is not changed). >> > >> > > > > >> > >> > > > > Changes made: >> > >> > > > > - removed the patch that implemented concat >> > >> > > > > - enhanced the wrap method to have 2 variants wrap(str, >> wrapper) >> > >> and >> > >> > > > > wrap(str, wrapper, wrapper_escape) >> > >> > > > > - kept the replace (and wrap first cut) patch untouched. >> > >> > > > > >> > >> > > > > >> > >> > > > > -- >> > >> > > > > Regards, >> > >> > > > > Janmejay >> > >> > > > > http://codehunk.wordpress.com >> > >> > > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > -- >> > >> > > > Regards, >> > >> > > > Janmejay >> > >> > > > http://codehunk.wordpress.com >> > >> > > > _______________________________________________ >> > >> > > > rsyslog mailing list >> > >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> > > > http://www.rsyslog.com/professional-services/ >> > >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >> by a >> > >> > myriad >> > >> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST if >> > >> you >> > >> > > > DON'T LIKE THAT. >> > >> > > > >> > >> > > _______________________________________________ >> > >> > > rsyslog mailing list >> > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> > > http://www.rsyslog.com/professional-services/ >> > >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > >> myriad >> > >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> if >> > you >> > >> > > DON'T LIKE THAT. >> > >> > > >> > >> > _______________________________________________ >> > >> > rsyslog mailing list >> > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> > http://www.rsyslog.com/professional-services/ >> > >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > myriad >> > >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > >> > DON'T LIKE THAT. >> > >> > >> > >> _______________________________________________ >> > >> rsyslog mailing list >> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> > >> http://www.rsyslog.com/professional-services/ >> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > >> DON'T LIKE THAT. >> > >> >> > > >> > >> > >> > -- >> > Regards, >> > Janmejay >> > http://codehunk.wordpress.com >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > The last doc-patch had a typo, this fixes it, and applies after it. > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > -- Regards, Janmejay http://codehunk.wordpress.com
From bf64196ad5790fdce8f1f35939a7699d1aad6cdb Mon Sep 17 00:00:00 2001 From: Janmejay Singh <[email protected]> Date: Fri, 31 Oct 2014 18:43:19 +0530 Subject: [PATCH] - fixed a bug in replace - added tests for replace - added tests for wrap(2) and wrap(3) --- .gitignore | 1 + grammar/rainerscript.c | 41 ++++++++++++++------------- tests/Makefile.am | 4 +++ tests/rscript_replace.sh | 13 +++++++++ tests/rscript_replace_complex.sh | 16 +++++++++++ tests/rscript_wrap2.sh | 14 +++++++++ tests/rscript_wrap3.sh | 14 +++++++++ tests/testsuites/complex_replace_input | 3 ++ tests/testsuites/rscript_replace.conf | 13 +++++++++ tests/testsuites/rscript_replace_complex.conf | 11 +++++++ tests/testsuites/rscript_wrap2.conf | 9 ++++++ tests/testsuites/rscript_wrap3.conf | 9 ++++++ tests/testsuites/wrap3_input | 1 + 13 files changed, 130 insertions(+), 19 deletions(-) create mode 100755 tests/rscript_replace.sh create mode 100755 tests/rscript_replace_complex.sh create mode 100755 tests/rscript_wrap2.sh create mode 100755 tests/rscript_wrap3.sh create mode 100644 tests/testsuites/complex_replace_input create mode 100644 tests/testsuites/rscript_replace.conf create mode 100644 tests/testsuites/rscript_replace_complex.conf create mode 100644 tests/testsuites/rscript_wrap2.conf create mode 100644 tests/testsuites/rscript_wrap3.conf create mode 100644 tests/testsuites/wrap3_input diff --git a/.gitignore b/.gitignore index 92b8e6e..abb0a55 100644 --- a/.gitignore +++ b/.gitignore @@ -54,5 +54,6 @@ checklog *.log logfile* *.conf +!tests/testsuites/*.conf log /target diff --git a/grammar/rainerscript.c b/grammar/rainerscript.c index e02df91..b9b4b36 100644 --- a/grammar/rainerscript.c +++ b/grammar/rainerscript.c @@ -1481,7 +1481,6 @@ doFunc_exec_template(struct cnffunc *__restrict__ const func, return; } - static inline es_str_t* doFuncReplace(struct var *__restrict__ const operandVal, struct var *__restrict__ const findVal, struct var *__restrict__ const replaceWithVal) { int freeOperand, freeFind, freeReplacement; @@ -1493,40 +1492,44 @@ doFuncReplace(struct var *__restrict__ const operandVal, struct var *__restrict_ uint lfind = es_strlen(findStr); uint lReplaceWith = es_strlen(replaceWithStr); uint size = 0; - uchar* src = es_getBufAddr(str); + uchar* src_buff = es_getBufAddr(str); uint i, j; - for(i = j = 0; i < es_strlen(str); i++, size++) { + for(i = j = 0; i <= es_strlen(str); i++, size++) { if (j == lfind) { size = size - lfind + lReplaceWith; j = 0; - } else if (src[i] == find[j]) { - j++; - } else if (j > 0) { - j = 0; } + if (i == es_strlen(str)) break; + if (src_buff[i] == find[j]) { + j++; + } else if (j > 0) { + i -= (j - 1); + size -= (j - 1); + j = 0; + } } es_str_t *res = es_newStr(size); unsigned char* dest = es_getBufAddr(res); uint k, s; - for(i = j = k = s = 0; i < es_strlen(str); i++, s++) { + for(i = j = k = s = 0; i <= es_strlen(str); i++, s++) { if (j == lfind) { for (k = 0; k < lReplaceWith; k++) { dest[s - j + k] = replaceWith[k]; } s = s - j + lReplaceWith; j = 0; - dest[s] = src[i]; - } else { - if (src[i] == find[j]) { - j++; - } else if (j > 0) { - j = 0; - for (k = 0; k < j; k++) { - dest[s - j + k] = find[k]; - } - } - dest[s] = src[i]; } + if (i == es_strlen(str)) break; + if (src_buff[i] == find[j]) { + j++; + } else { + if (j > 0) { + i -= j; + s -= j; + j = 0; + } + dest[s] = src_buff[i]; + } } res->lenStr = size; if(freeOperand) es_deleteStr(str); diff --git a/tests/Makefile.am b/tests/Makefile.am index 55ef2e0..09d84d8 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -78,6 +78,10 @@ TESTS += \ rscript_prifilt.sh \ rscript_optimizer1.sh \ rscript_ruleset_call.sh \ + rscript_replace.sh \ + rscript_replace_complex.sh \ + rscript_wrap2.sh \ + rscript_wrap3.sh \ rs_optimizer_pri.sh \ cee_simple.sh \ cee_diskqueue.sh \ diff --git a/tests/rscript_replace.sh b/tests/rscript_replace.sh new file mode 100755 index 0000000..0de13e5 --- /dev/null +++ b/tests/rscript_replace.sh @@ -0,0 +1,13 @@ +# added 2014-10-31 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_replace.sh\]: test for replace script-function +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_replace.conf +./tcpflood -m 1 -I testsuites/date_time_msg +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh content-check "date time: Thu 0ct0ber 30 13:20:18 IST 2014" +source $srcdir/diag.sh exit diff --git a/tests/rscript_replace_complex.sh b/tests/rscript_replace_complex.sh new file mode 100755 index 0000000..6c01369 --- /dev/null +++ b/tests/rscript_replace_complex.sh @@ -0,0 +1,16 @@ +# added 2014-10-31 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_replace_complex.sh\]: a more complex test for replace script-function +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_replace_complex.conf +./tcpflood -m 1 -I testsuites/complex_replace_input +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +cp rsyslog.out.log /tmp/ +source $srcdir/diag.sh content-check "try to replace rsyslog and syrsyslog with rrsyslog" +source $srcdir/diag.sh content-check "try to replace hello_world in hello_worldlo and helhello_world with hello_world_world" +source $srcdir/diag.sh content-check "try to FBB in FBB_quux and quux_FBB with FBB" +source $srcdir/diag.sh exit diff --git a/tests/rscript_wrap2.sh b/tests/rscript_wrap2.sh new file mode 100755 index 0000000..8995fd4 --- /dev/null +++ b/tests/rscript_wrap2.sh @@ -0,0 +1,14 @@ +# added 2014-11-03 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_wrap2.sh\]: a test for wrap\(2\) script-function +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_wrap2.conf +./tcpflood -m 1 -I testsuites/date_time_msg +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +cp rsyslog.out.log /tmp/ +source $srcdir/diag.sh content-check "**foo says at Thu Oct 30 13:20:18 IST 2014 random number is 19597**" +source $srcdir/diag.sh exit diff --git a/tests/rscript_wrap3.sh b/tests/rscript_wrap3.sh new file mode 100755 index 0000000..9e9c282 --- /dev/null +++ b/tests/rscript_wrap3.sh @@ -0,0 +1,14 @@ +# added 2014-11-03 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_wrap3.sh\]: a test for wrap\(3\) script-function +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_wrap3.conf +./tcpflood -m 1 -I testsuites/wrap3_input +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +cp rsyslog.out.log /tmp/ +source $srcdir/diag.sh content-check "bcdefbcfoo says a abcESCdefb has ESCbcdefbc" +source $srcdir/diag.sh exit diff --git a/tests/testsuites/complex_replace_input b/tests/testsuites/complex_replace_input new file mode 100644 index 0000000..81ecbfc --- /dev/null +++ b/tests/testsuites/complex_replace_input @@ -0,0 +1,3 @@ +<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: try to replace syslog and sysyslog with rsyslog +<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: try to replace hello in hellolo and helhello with hello_world +<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: try to foo_bar_baz in foo_bar_baz_quux and quux_foo_bar_baz with FBB diff --git a/tests/testsuites/rscript_replace.conf b/tests/testsuites/rscript_replace.conf new file mode 100644 index 0000000..d721db3 --- /dev/null +++ b/tests/testsuites/rscript_replace.conf @@ -0,0 +1,13 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$.replaced_msg%\n") + +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") + +template(name="date_time" type="list") { + property(name="msg" regex.Expression="Thu .+ 2014" regex.Type="ERE" regex.Match="0") +} + +set $.replaced_msg = replace("date time: " & exec_template("date_time"), "O" & "ct", replace("october", "o", "0")); + +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/rscript_replace_complex.conf b/tests/testsuites/rscript_replace_complex.conf new file mode 100644 index 0000000..f9f7865 --- /dev/null +++ b/tests/testsuites/rscript_replace_complex.conf @@ -0,0 +1,11 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$.replaced_msg%\n") + +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") + +set $.replaced_msg = replace($msg, "syslog", "rsyslog") +set $.replaced_msg = replace($.replaced_msg, "hello", "hello_world") +set $.replaced_msg = replace($.replaced_msg, "foo_bar_baz", "FBB"); + +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/rscript_wrap2.conf b/tests/testsuites/rscript_wrap2.conf new file mode 100644 index 0000000..9a26d96 --- /dev/null +++ b/tests/testsuites/rscript_wrap2.conf @@ -0,0 +1,9 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$.replaced_msg%\n") + +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") + +set $.replaced_msg = wrap("foo says" & $msg, "*" & "*"); + +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/rscript_wrap3.conf b/tests/testsuites/rscript_wrap3.conf new file mode 100644 index 0000000..8caca96 --- /dev/null +++ b/tests/testsuites/rscript_wrap3.conf @@ -0,0 +1,9 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$.replaced_msg%\n") + +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") + +set $.replaced_msg = wrap("foo says" & $msg, "bc" & "def" & "bc", "ES" & "C"); + +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/wrap3_input b/tests/testsuites/wrap3_input new file mode 100644 index 0000000..18aa3cb --- /dev/null +++ b/tests/testsuites/wrap3_input @@ -0,0 +1 @@ +<167>Mar 6 16:57:54 172.20.245.8 %PIX-7-710005: a abcbcdefbcdefb has bcdefbc -- 2.0.4
_______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
