Re: [rsyslog] mmnormalize with mutiple input: conditionals?

[David Lang]

On Wed, 30 Nov 2016, mosto...@gmail.com wrote:

I'm still not able to get it, and *perhaps reproduced an issue related to 
properties not being accessible*...
Consider the following relay.conf:
  global(
  MaxMessageSize="32k"
  workDirectory="/var/spool/rsyslog"
  parser.escapeControlCharactersOnReceive="off"
  )

  template(
       name="json"
  string="<%pri%>%timestamp:::date-rfc3339% %hostname%
  logs/%$!data!group%/%$!data!aapp%: %$!data%"
       type="string"
  )

first off, this template is still not forming a valid syslog message.

syslog messages can be one of two formats

rfc-3164

<44>Jan 12 11:22:33 host program[pid]: message
(with [pid] being optional)

or rfc-5424

<44>1 2003-08-24T05:14:15.000003-07:00 host.f.d.n program pid - message

what you are producing is neither because you are using the wrong time format, 
change date-rfc3339 to date-rfc3164.


  module(load="omrelp")
  ruleset(name="relp") {
       set $!data!aapp=field($programname,"/",2);
       set $!data!file="$!metadata!filename";
       set $!data!group=field($programname,"/",1);
       set $!data!msg=$msg;
       action(
  action.reportSuspension="on"
  action.resumeRetryCount="-1"
  port="20514"
  queue.maxdiskspace="5M"
  queue.SaveOnShutdown="on"
  queue.type="LinkedList"
  target="server"
  template="json"
  type="omrelp"
       )
  }
  ruleset(name="apps") {
       call relp
       stop
  }

  module(load="imfile")

  input(type="imfile" file="/logs/apache/app1/app.log"
  tag="group/app1" addMetadata="on" ruleset="apps"
  PersistStateInterval="1")

  input(type="imfile" file="/logs/apache/app2/app.log"
  tag="group/app2" addMetadata="on" ruleset="apps"
  PersistStateInterval="1")
  ruleset(name="app_server1") {
       set $!data!containerApps="app1,app2";
       call relp
       stop
  }

  input(type="imfile" file="/logs/server1/app1.log"
  tag="group1/server1" addMetadata="on" ruleset="app_server1"
  startmsg.regex="^####" readTimeout="5" PersistStateInterval="1")
  input(type="imfile" file="/logs/server1/app2.log"
  tag="group1/server1" addMetadata="on" ruleset="app_server1"
  startmsg.regex="^####" readTimeout="5" PersistStateInterval="1")

  ruleset(name="app_server2") {
       set $!data!containerApps="app2,app3";
       call relp
       stop
  }
  input(type="imfile" file="/logs/server2/app2.log"
  tag="group2/server2" addMetadata="on" ruleset="app_server2"
  startmsg.regex="^####" readTimeout="5" PersistStateInterval="1")
  input(type="imfile" file="/logs/server2/app3.log"
  tag="group2/server2" addMetadata="on" ruleset="app_server2"
  startmsg.regex="^####" readTimeout="5" PersistStateInterval="1")


With this configuration I'm getting messages like:

  200 syslog 911 <133>2016-11-30T20:02:30.210405+01:00 my-rsyslog
  logs/group/***FIELD NOT FOUND***: { "aapp": "***FIELD NOT FOUND***",
  "file": "\/logs\/apache\/app1\/app.log", "group": "group", "msg":
  "17 mar 2016 13:27:28,934  INFO REDACTED..." }


Why /aapp/ field isn't found?

any time you don't get fields like you expect, write a log in the format 
RSYSLOG_DebugFormat and look at what is actually set

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.