No problem at all. -- Dave Caplinger, Director of Architecture | 402.361.3063 Solutionary | Relevant . Intelligent . Security
On Nov 14, 2013, at 3:50 AM, Rainer Gerhards <[email protected]> wrote: > Hi Dave, > > thanks for the excellent description. Do you mind if I add it to the sites > FAQ? > > Rainer > > > On Wed, Nov 13, 2013 at 10:06 PM, Dave Caplinger < > [email protected]> wrote: > >> On Nov 13, 2013, at 1:40 PM, Leggett, Torrance I. <[email protected]> >> wrote: >>> >>> The ${MainMsg/Action}ResumeRetryCount seems to look like a 'legacy' >> style option, but I don't see the equivalent in the ranier-script style: >>> >>> action( type="omfwd" >>> target="127.0.0.1" >>> port="5544" >>> protocol="udp" >>> template="RSYSLOG_TraditionalForwardFormat" >>> queue.type="LinkedList" >>> queue.filename="logstash" >>> queue.size="1000000" >>> queue.highwatermark="60000" >>> queue.lowwatermark="50000" >>> queue.maxdiskspace="1g" >>> queue.saveonshutdown="on" >>> ) >> >> It is: >> >> action.resumeretrycount="-1" >> >> Also, I recommend giving your action a 'name="whatever"' parameter entry >> as well (as you'll see below). >> >> >>> Second, I've seen the pstats module, but do you have some examples of >> how I could be using it to tell what's going on? >> >> First, enable the module with something like: >> >> module(load="impstats" interval="660" severity="7") >> >> This will start generating logs tagged with "rsyslogd-pstats" every 600 >> seconds. If you like, you can use that tag to filter them into their own >> file: >> >> if $syslogtag contains 'rsyslogd-pstats' then { >> action(type="omfile" queue.type="linkedlist" >> queue.discardmark="980" name="pstats" file="/var/log/pstats") >> stop >> } >> >> You'll wind up with several log lines at each interval, all showing >> current counters (since rsyslog restart). So to determine inter-interval >> deltas, you'd have to import these into a spreadsheet. (Newer rsyslog can >> emit just the deltas in the log lines, but that's in v7.5.x I believe.) >> >> For example, if you want to filter based on some property (such as source >> IP address) and send the matching logs to both a local file and on to a >> remote destination, you might use something like: >> >> if $fromhost-ip == >> [ "1.1.1.1", \ >> "2.2.2.2" ] \ >> then { >> action (type="omfwd" queue.type="linkedlist" queue.discardmark="980" >> action.resumeretrycount="-1" name="NET.forward" >> target="10.10.10.10" >> port="514" protocol="tcp") >> action (type="omfile" queue.type="linkedlist" queue.discardmark="980" >> name="NET.local" file="/var/log/messages") >> stop >> } >> >> Which is a log flow like: >> >> source -> imudp -> main Q -> NET.local (to local files) & NET.forward >> (to remote) >> >> >> Here's an example of a batch of pstats output (re-ordered slightly) from >> the above config: >> >> Nov 13 14:31:35 loghost rsyslogd-pstats: imudp(*:514): submitted=23035 >> Nov 13 14:31:35 loghost rsyslogd-pstats: main Q: size=15 enqueued=89624087 >> full=0 discarded.full=0 discarded.nf=0 maxqsize=444 >> Nov 13 14:31:35 loghost rsyslogd-pstats: NET.local: size=0 enqueued=11541 >> full=0 discarded.full=0 discarded.nf=0 maxqsize=7 >> Nov 13 14:31:35 loghost rsyslogd-pstats: NET.local: processed=11541 >> failed=0 >> Nov 13 14:31:35 loghost rsyslogd-pstats: NET.forward: size=0 >> enqueued=11541 full=0 discarded.full=0 discarded.nf=0 maxqsize=7 >> Nov 13 14:31:35 loghost rsyslogd-pstats: NET.forward: processed=11541 >> failed=0 >> Nov 13 14:31:35 loghost rsyslogd-pstats: pstats: size=0 enqueued=65508 >> full=0 discarded.full=0 discarded.nf=0 maxqsize=25 >> Nov 13 14:31:35 loghost rsyslogd-pstats: pstats: processed=65500 failed=0 >> >> >> In this case we have: >> >> 1) A UDP input (imudp) >> >> This logs message counts "submitted" to rsyslog via UDP port 514. >> >> 2) A main queue (main Q) >> >> This shows messages entering the queue (enqueued), as well as any dropped >> messages (discarded.full=0, discarded.nf=0). It also shows how many >> times the queue has become completely full (full=0) and it keeps a running >> total of the maximum size the queue has ever hit (maxqsize=444). (All >> these counters are since rsyslog startup.) >> >> 3) Two output/action queues (NET.local, NET.forward) >> >> These logs queue stats like above, as well as successfully "processed" >> (via omfile and omfwd in this case), indicating successful delivery to >> their final destination (local file or remote TCP receiver, in this case). >> >> 4) Another queue to handle pstats output itself (as I described above) >> >> This example doesn't happen to include DA-mode, which adds another pstats >> log line for the DA portion of the associated action queue. >> >> If you don't give your action queues names, you'll wind up with pstats >> logs referring to things like "action 2", and have a hard time figuring out >> what is going on. >> >> A well-behaved queue will have zero discarded.full and discarded.nf, and >> a low maxqsize, meaning that everything entering the queue is leaving >> promptly. In a backlog situation, you'll see size and maxqsize for an >> action/output queue increase over time, until maxqsize hits your configured >> queue.size parameter. Then the main Q will start increasing in size (and >> maxqsize) until it approaches and exceeds full. Then the discarded.nfand >> discarded.full counters will start climbing. >> >> Hope this helps, >> >> - Dave >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
