Got it. Yes, I misunderstood. Thanks
-----Original Message----- From: rsyslog [mailto:[email protected]] On Behalf Of Rich Megginson via rsyslog Sent: Wednesday, April 03, 2019 12:36 PM To: [email protected] Cc: Rich Megginson Subject: Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and forwarding I think you are misunderstanding. There is no parameter named "metadata". Perhaps you are referring to this section? "Metadata The imfile module supports message metadata. It supports the following data items: filename Name of the file where the message originated from. This is most useful when using wildcards inside file monitors, because it then is the only way to know which file the message originated from. The value can be accessed using the %$!metadata!filename% property. Note: For symlink-ed files this does not contain name of the actual file (source of the data) but name of the symlink (file which matched configured input). fileoffset Offset of the file in bytes at the time the message was read. The offset reported is from the start of the line. This information can be useful when recreating multi-line files that may have been accessed or transmitted non-sequentially. The value can be accessed using the %$!metadata!fileoffset% property. Metadata is only present if enabled. By default it is enabled for input() statements that contain wildcards. For all others, it is disabled by default. It can explicitly be turned on or off via the addMetadata input() parameter, which always overrides the default. " This means that if you set the parameter `addMetadata="on"` in the imfile module or input, imfile will add two message properties to your message - $!metadata!filename and $!metadata!fileoffset On 4/3/19 11:29 AM, Gorman, Kevin via rsyslog wrote: > According to the imfile doc, metadata is in imfile with parameters filename > or fileoffset. Or am I misunderstanding again? > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.rsyslog.com_d > oc_master_configuration_modules_imfile.html-3Fhighlight-3Dimfile-23met > adata&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_ > yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRKcQQkrM19uW1nEexoCAF > WtUhaDSG3nr8&s=PXYQOLyTg2UJaFUshiYOD0spqnkONzsaHsRxb1brAW8&e= > > -----Original Message----- > From: rsyslog [mailto:[email protected]] On Behalf Of > Rich Megginson via rsyslog > Sent: Wednesday, April 03, 2019 12:25 PM > To: [email protected] > Cc: Rich Megginson > Subject: Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and > forwarding > > On 4/3/19 11:19 AM, Gorman, Kevin via rsyslog wrote: >> The *.* call linux_forward was the key. Much cleaner. Thanks! >> I also see the audit logs in the debug log and at the other end in tcpdump. >> I don't see the messages in the remote log system data, but that's another >> issue and product receiving the logs. >> >> I added addMetadata to the module load and a metadata entry to the input but >> get debug errors. I guess metadata isn't supported in this old version or >> am I missing something else? >> >> ... >> rsyslogd: error during parsing file /etc/rsyslog.d/rsyslog.all.conf, >> on or before line 1: parameter 'addMetadata' not known -- typo in config >> file? [v8.24.0-34.el7 try >> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_e_2207&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprdhxODG4D7SWxzm4IWHX6tUA4NIWXUg8A&s=D1oad95iuL_gck0_iSR9iVXJMOT5Y7fJfCIfAaB_HpY&e= >> ] ... >> 1789.633979501:main thread : Called LogMsg, msg: error during parsing >> file /etc/rsyslog.d/rsyslog.linux.conf, on or before line 17: parameter >> 'metadata' not known -- typo in config file? >> rsyslogd: error during parsing file >> /etc/rsyslog.d/rsyslog.linux.conf, >> on or before line 17: parameter 'metadata' not known -- typo in config file? >> [v8.24.0-34.el7 try >> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_e_2207&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprdhxODG4D7SWxzm4IWHX6tUA4NIWXUg8A&s=D1oad95iuL_gck0_iSR9iVXJMOT5Y7fJfCIfAaB_HpY&e= >> ] ... >> >> module(load="imfile" mode="inotify" addMetadata="on") >> >> global ( >> parser.dropTrailingLFOnReception="on" >> parser.escapeControlCharactersOnReceive="on" >> workDirectory="/var/lib/rsyslog" >> ) >> >> ruleset( >> name="linux_forward" >> queue.type="LinkedList" >> queue.filename="FwdRule0" >> queue.maxDiskSpace="1g" >> queue.saveOnShutdown="on" >> ) { >> action( >> type="omfwd" >> target="2001:4888:a00:3154:f0:ff2:0:b01" # logserver VIP >> protocol="tcp" >> port="5544" >> action.resumeRetryCount="-1" >> ) >> } >> >> *.* call linux_forward >> >> input( >> type="imfile" >> ruleset="linux_forward" >> tag="audit" >> file="/var/log/audit/audit.log" >> metadata="filename" > > 1) there is no parameter "metadata" - you mean "addmetadata", except . . . > > 2) . . . you already added it to the module load > > >> ) >> >> -----Original Message----- >> From: Rainer Gerhards [mailto:[email protected]] >> Sent: Wednesday, April 03, 2019 10:26 AM >> To: Gorman, Kevin >> Cc: rsyslog-users >> Subject: Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and >> forwarding >> >> El mié., 3 abr. 2019 a las 17:13, Gorman, Kevin >> (<[email protected]>) escribió: >>> I'm using any and all docs on >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.rsyslog.com_doc_v8-2Dstable&d=DwIFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=l98nK1LGc7XvzQocBxz0C-beuBDTTA_0A-BxOzUwFG0&s=M4v2Mr09OpuPe6E_YDxsM_PPIOsvcPbnav9gu73kKaE&e=, >>> posts on the internet, etc. The thing is there are few actual >>> configuration examples and info on how to actually use the various syntax >>> for rsyslog, but lots of references on what individual parts do, which >>> makes it difficult for someone not familiar with it to get anything done. >> I asked because the parameters you gave simply did not exist, thus the >> syntax error. The online doc is for the currently selected version. >> With outdated versions like 8.24.0 you should probably stick to the OS >> provided doc package. That just as info. >> >> It is impossible to provide full config examples for everything - there are >> more potential configs than atoms in the universe (really!). >> It would be good if users would contribute their configs to the doc, >> but this does not happen so far :-( >>> The version is 8.24 on RHEL 7.6 using the default rsyslog.conf. Here is the >>> complete config. >>> >>> There is no problem collecting or forwarding log data defined in >>> rsyslog.conf. I simply want to define the address, port and protocol once >>> and only once. >>> >>> I am having trouble forwarding the other 2 sample log files, or rather >>> audit.log since ansible.log is always empty. Audit.log is not defined in >>> rsyslog.conf and is pretty busy, but I don't see data in tcpdump on the >>> local server or in the remote log server. >>> >>> From the debug output below, everything looks ok to me. >>> >> Let me have a look. Warning: I don't remember what was at the time we >> had 8.24 (probably 2.5 yrs ago), >> >>> # cat rsyslog.all.conf >>> module(load="imfile" mode="inotify") >>> >>> global ( >>> parser.dropTrailingLFOnReception="on" >>> parser.escapeControlCharactersOnReceive="on" >>> workDirectory="/var/lib/rsyslog" >>> ) >>> >>> ruleset(name="linux_forward") { >>> action( >>> type="omfwd" >>> target="2001:4888:a00:3154:f0:ff2:0:b01" # logserver VIP >>> protocol="tcp" >>> port="5544" >>> ) >>> stop >> stop is unnecessary here, but causes no issue. >>> } >>> *.* action( >>> type="omfwd" >>> target="2001:4888:a00:3154:f0:ff2:0:b01" >>> port="5544" >>> protocol="tcp" >>> ) >> replace by >> >> call linux_forward >> >>> # cat rsyslog.linux.conf >>> >>> input( >>> type="imfile" >>> ruleset="linux_forward" >>> tag="ansible" >>> file="/var/log/ansible.log" >>> ) >>> >>> input( >>> type="imfile" >>> ruleset="linux_forward" >>> tag="audit" >>> file="/var/log/audit/audit.log" >>> ) >> looks good >> >>> # grep -v ^# /etc/rsyslog.conf >>> >>> $ModLoad imuxsock # provides support for local system logging (e.g. >>> via logger command) $ModLoad imjournal # provides access to the >>> systemd journal $WorkDirectory /var/lib/rsyslog >>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat >>> $IncludeConfig /etc/rsyslog.d/*.conf $OmitLocalLogging on >>> $IMJournalStateFile imjournal.state >>> *.info;mail.none;authpriv.none;cron.none /var/log/messages >>> authpriv.* /var/log/secure >>> mail.* -/var/log/maillog >>> cron.* /var/log/cron >>> *.emerg :omusrmsg:* >>> uucp,news.crit /var/log/spooler >>> local7.* /var/log/boot.log >> looks good >> >>> # rsyslogd -N2 2>&1 >>> rsyslogd: version 8.24.0-34.el7, config validation run (level 2), >>> master config /etc/rsyslog.conf >>> >>> 2612.391124552:main thread : debug level 2 set via config file >>> 2612.391137687:main thread : This is rsyslog version 8.24.0-34.el7 >>> 2612.391142024:main thread : config parser: reached end of file >>> /etc/rsyslog.d/rsyslog.debug.conf >> at least one of the includede files is missing - if others are missing they >> may have problems... >> >> [snip] >> >>> rsyslogd: End of config validation run. Bye. >>> >> except for the call, I don't see an issue. >> >> As a test, do an upgrade to the currently supported version. If it works, >> you have hit an already solved bug. >> >> HTH >> Rainer >>> >>> -----Original Message----- >>> From: Rainer Gerhards [mailto:[email protected]] >>> Sent: Wednesday, April 03, 2019 2:04 AM >>> To: rsyslog-users >>> Cc: Gorman, Kevin >>> Subject: Re: [rsyslog] [E] Re: Help with newer syntax a ruleset and >>> forwarding >>> >>>> *.* action( >>>> type="omfwd" >>>> ruleset="linux_forward" >>>> tag="rsyslog" >>>> name="rsyslog" >>>> ) >>> Which doc do you use as reference for this? Or, more general, which doc >>> link do you use to craft the configs? >>> >>> Rainer >> _______________________________________________ >> rsyslog mailing list >> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net >> _ >> mailman_listinfo_rsyslog&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb >> 6 >> __0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDApr >> d >> hxODG4D7SWxzm4IWHX6tUA4NIWXUg8A&s=EP2RkF3kjg8N0t0ZFWe0OIxZoWj_0yRwwMl >> H >> rP5Ujig&e= >> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_p >> r >> ofessional-2Dservices_&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6_ >> _ >> 0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprdh >> x >> ODG4D7SWxzm4IWHX6tUA4NIWXUg8A&s=vfDgmm3sswgPwiM9cKhMxcb5ZtvGLaCMIu9Ib >> q zYD7o&e= What's up with rsyslog? Follow >> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rger >> h >> ards&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_ >> y >> qJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprdhxODG4D7SWxzm4IWHX6 >> t UA4NIWXUg8A&s=LX43BRfSGCnC58D6rd3aGieZshxb3xMT1HCAcaZGqKk&e= >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. > > _______________________________________________ > rsyslog mailing list > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_ > mailman_listinfo_rsyslog&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6 > __0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprd > hxODG4D7SWxzm4IWHX6tUA4NIWXUg8A&s=EP2RkF3kjg8N0t0ZFWe0OIxZoWj_0yRwwMlH > rP5Ujig&e= > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_pr > ofessional-2Dservices_&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__ > 0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprdhx > ODG4D7SWxzm4IWHX6tUA4NIWXUg8A&s=vfDgmm3sswgPwiM9cKhMxcb5ZtvGLaCMIu9Ibq > zYD7o&e= What's up with rsyslog? Follow > https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerh > ards&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_y > qJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=Li_EIXuDAprdhxODG4D7SWxzm4IWHX6t > UA4NIWXUg8A&s=LX43BRfSGCnC58D6rd3aGieZshxb3xMT1HCAcaZGqKk&e= > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ > rsyslog mailing list > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_ > mailman_listinfo_rsyslog&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6 > __0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRK > cQQkrM19uW1nEexoCAFWtUhaDSG3nr8&s=FYBbHG0yD94SasPLJntoLUWymPAp_3UntywB > 1RpH8QQ&e= > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_pr > ofessional-2Dservices_&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__ > 0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRKcQ > QkrM19uW1nEexoCAFWtUhaDSG3nr8&s=jJ_WPtWojOAw7DEGHY2--f0siSw1rXuRclqwxK > qvK0c&e= What's up with rsyslog? Follow > https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerh > ards&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_y > qJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRKcQQkrM19uW1nEexoCAFW > tUhaDSG3nr8&s=tv4M5Ofu3rFSgBahQrVDXw6u1xPAeOi3sNd4b1QReqM&e= > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRKcQQkrM19uW1nEexoCAFWtUhaDSG3nr8&s=FYBbHG0yD94SasPLJntoLUWymPAp_3UntywB1RpH8QQ&e= https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRKcQQkrM19uW1nEexoCAFWtUhaDSG3nr8&s=jJ_WPtWojOAw7DEGHY2--f0siSw1rXuRclqwxKqvK0c&e= What's up with rsyslog? Follow https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=stJ62L_yqJWGrxRl6wWPMpHwvVUmOyXDBgn3Xxfk-6U&m=8UjAK-mpyqRKcQQkrM19uW1nEexoCAFWtUhaDSG3nr8&s=tv4M5Ofu3rFSgBahQrVDXw6u1xPAeOi3sNd4b1QReqM&e= NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
