I thought there was an option to have the input module keep per-sender stats,
but I'm not finding it now.
with dyn_stats() see
https://www.rsyslog.com/doc/master/configuration/dyn_stats.html
you create a stats type with the dyn_stats() call and you count with the
dyn_inc() call
stats inervals are tied to the interval you set for the impstats, so if you want
per hour or per day stats, it will take post-processing of the stats log
remember, you can log the stats via syslog, at which point you can do all the
filtering that you can do with any other log message
This will not tell you about any errors, but the other rsyslog log messages
should do that.
Simple Event Correlator is a program that you can have watch logs to alert on
specific log messages, or summarize logs over time. Splunk and Elasticsearch are
commonly used for this sort of thing (and having them summarize stats is FAR
more efficient than having them count the number of messages directly)
does this point you in the right direction? if it's still unclear, ask more
questions
David Lang
On Thu, 18 Feb 2021, odrzen wrote:
Date: Thu, 18 Feb 2021 23:13:36 +0000
From: odrzen <odr...@protonmail.com>
To: David Lang <da...@lang.hm>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] Information about incoming logs.
So far, looking for what you are telling us, I have been able to add the
following configuration `/etc/rsyslog.conf` :
```
module(load="impstats"
interval="600"
severity="7"
log.syslog="off"
log.file="/var/log/rsyslog-stats/stats.log")
global(senders.keepTrack="on")
dyn_stats(name="msg_per_host")
```
and here are the results I get every 10 minutes:
```
Fri Feb 19 00:52:08 2021: global: origin=dynstats msg_per_host.ops_overflow=0
msg_per_host.new_metric_add=0 msg_per_host.no_metric=0
msg_per_host.metrics_purged=0 msg_per_host.ops_ignored=0
msg_per_host.purge_triggered=0
Fri Feb 19 00:52:08 2021: imuxsock: origin=imuxsock submitted=0
ratelimit.discarded=0 ratelimit.numratelimiters=0
Fri Feb 19 00:52:08 2021: dynafile cache Auditlog: origin=omfile requests=1638
level0=1020 missed=76 evicted=66 maxused=10 closetimeouts=0
Fri Feb 19 00:52:08 2021: action 0: origin=core.action processed=1638 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: dynafile cache RemoteLogs: origin=omfile
requests=75872 level0=8240 missed=941 evicted=931 maxused=10 closetimeouts=0
Fri Feb 19 00:52:08 2021: action 1: origin=core.action processed=75872 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 2: origin=core.action processed=74218 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 3: origin=core.action processed=1638 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 4: origin=core.action processed=0 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 5: origin=core.action processed=16 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 6: origin=core.action processed=0 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 7: origin=core.action processed=0 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: action 8: origin=core.action processed=0 failed=0
suspended=0 suspended.duration=0 resumed=0
Fri Feb 19 00:52:08 2021: msg_per_host: origin=dynstats.bucket
Fri Feb 19 00:52:08 2021: imudp(*:514): origin=imudp submitted=0
Fri Feb 19 00:52:08 2021: imudp(*:514): origin=imudp submitted=0
Fri Feb 19 00:52:08 2021: imtcp(6514): origin=imtcp submitted=75441
Fri Feb 19 00:52:08 2021: resource-usage: origin=impstats utime=6334580
stime=7795002 maxrss=9256 minflt=2781 majflt=6 inblock=1320 oublock=98680
nvcsw=146787 nivcsw=40
Fri Feb 19 00:52:08 2021: main Q: origin=core.queue size=0 enqueued=75872
full=0 discarded.full=0 discarded.nf=0 maxqsize=85
Fri Feb 19 00:52:08 2021: imudp(w0): origin=imudp called.recvmmsg=0
called.recvmsg=0 msgs.received=0
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_1.com messages=21
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_2.com messages=21814
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_3.com messages=5096
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_4.com messages=25
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_5.com messages=37
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_6.com messages=151
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_7.com messages=13
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_8.com messages=7460
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_9.com messages=21
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_10.com messages=5118
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_12.com messages=5099
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_13.com messages=14
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_14.com messages=7820
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_15.com messages=123
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_16.com messages=15170
Fri Feb 19 00:52:08 2021: _sender_stat: sender=example_17.com messages=7459
```
About the "the imtcp module to record per-sender stats", I'm not sure I found what you
mean, because I only found this parameter in this `dyn_stats(name="msg_per_host")` order.
About dyn_stats(), how can I create my "own" statistics ? For example, can I
create reports per/hour or per/days ? In addition, can I include more information in
these reports ?
In addition, can I see if there are problems with some machines with connection
( for example wrong certificates ) or bottleneck problems ?
Thank you very much for your time. I appreciate your help.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, February 17, 2021 1:29 AM, David Lang <da...@lang.hm> wrote:
take a look at the impstats module, and the ability for the imtcp module to
record per-sender stats
if that isn't enough, you can create your own stats via the dyn_stats()
functions
David Lang
On Tue, 16 Feb 2021, odrzen via rsyslog wrote:
Date: Tue, 16 Feb 2021 23:04:40 +0000
From: odrzen via rsyslog rsyslog@lists.adiscon.com
Reply-To: odrzen odr...@protonmail.com,
rsyslog-users rsyslog@lists.adiscon.com
To: "rsyslog@lists.adiscon.com" rsyslog@lists.adiscon.com
Cc: odrzen odr...@protonmail.com
Subject: [rsyslog] Information about incoming logs.
Hello rsyslog community,
I'm new to rsyslog, but have already successfully configured some machines to
send their logs using mutual TLS authentication.
So far so good, but I have some questions :
How can I see from the rsyslog server side how many and which machines send
logs ?
With the command "systemctl status rsyslog" I have a view on what happens and
which machines have problems, but it is not very clear.
Can I somehow see which machines are successfully communicating and sending
their logs ?
Another very interesting information for me, would be to be able to see the
time of the last update for each machine.
For example:
Domain Status Last update
domain1.com [Connection OK] 04:04:33
domain2.com [Connection FAILED] 02:32:03
domain3.com [Connection OK] 04:02:12
....
Can I get this kind of information from the rsyslog server ?
Thanks in advance.
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
