splitting this off as I'm going on a bit of a tangent here
On Fri, 25 Oct 2013, Rainer Gerhards wrote:
Also, as a slight tangent, if it's possible to have mmpstats variables
available as global variables (or similar) then much of the manual counting
may not be needed as the pstats counters could be used instead.
Could be done, but (probably) requires some time. Also note that these
counts are NOT accurate - so they are only good if a rough number is used.
Keeping them accurate would very considerably slow down the engine. But it
definitely makes sense to have them available, for other uses as well
(think about switching from one output to another based on the number of
messages inside the main queue). I'd still consider this as a new feature
request.
I agree it's a different feature request, I think access to rsyslog internal
data (pstats, batch size, and future similar things) would be a useful
capability to get at some point. I was bringing it up now so that we could try
to make sure the design doesn't make it impossible to do this.
You mention that pstats counts are not completely accurate for performance
reasons.
First off, I want to make sure I understand the problem.
Is it that there could be races between different threads, or is it that since
the report takes time to generate, not all the numbers are from the same point
in time, and the process of reporting and then zeroing them may race with them
being modified?
I'm wondering if the right approach here is to emulate the Linux Kernel per-cpu
variables, have each thread maintain it's own copy of any counters that it's
updating, and then have a way for the master thread to signal the other threads
that it needs them to report their values (and optionally reset them)
This would let each thread maintain their counters without expensive atomic
operations (which should improve performance if care is paid to alignment and
cacheline separation) and make everything as accurate as it can be without
having a global 'stop everything' flag while the counters are read/reset, and
even that may not be that bad if it's infrequent enough (especially since the
hotpath of updates would now be faster and rsyslog is dealing with a handful of
threads, not hundreds or thousands)
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
