On Wed, 7 Oct 2015, singh.janmejay wrote:
--
Regards,
Janmejay
PS: Please blame the typos in this mail on my phone's uncivilized soft
keyboard sporting it's not-so-smart-assist technology.
On Oct 7, 2015 3:26 AM, "David Lang" <da...@lang.hm> wrote:
On Wed, 7 Oct 2015, singh.janmejay wrote:
--
Regards,
Janmejay
PS: Please blame the typos in this mail on my phone's uncivilized soft
keyboard sporting it's not-so-smart-assist technology.
On Oct 6, 2015 10:32 PM, "David Lang" <da...@lang.hm> wrote:
On Tue, 6 Oct 2015, singh.janmejay wrote:
It is possible to use global-variables (it'll require some
enhancements, table-support etc), but it'll be very inefficient
compared to this approach. For instance, choice of data-structure etc
allows making the solution a lot more efficient.
As for the data structures, Rainer has been identifying inefficencies in
how json-c works and working to improve them
That optimizes variable system. But it still is a general propose
variable
system. It can't and shouldn't understand relationship between variables.
what relationships are there between the different metrics?
The fact that they are read in one shot, reported and reset.
I don't understand why you couldn't do $\stats!metric1, $\stats!metric2, etc.
There is no reason why this couldn't have the same locking as your new
structure.
Here its possible to locklessly increment counters in most cases, so
its overhead is a lot lesser than global-variables.
how can you manage counters in multiple threads without locks?
Especially
when dealing with batches.
Consider a trie based implementation. With bounded fanout-factor, it's
O(1)
wrt metric-names cardinality. It also has very little lock contention
involved. Usually operations work with read-locks, only when new metric
is
initialized it requires a write lock on patent node. If recycles are few
and far apart, lock contention would be negligible.
if you have multiple threads that may need to update the same metric at
the same time, a tree doesn't eliminate the locking.
The only situation involving a lock that is contended for, is when a metric
is to be initialized. Consider this trie:
A -> B -> C
-> D
Now for incrementing key ABC, no contention exists, because it involves
read-locks only. It just uses atomic-increment to bump the counter at node
C. Same for ABD.
But ABE will require a write-lock at node B, because node E doesn't exist
yet. However key with not shared parent can again be initialized
concurrently. Init operation can be amortized over a large set of increment
operations making its cost negligible(this is the knob that reset interval
exposes).
you are misunderstanding me
If you have two threads that both want to change the counter at node C, you have
to have locking to keep them from having problems.
Similarly, when one thread goes to output the stats, you need to lock them so
that there isn't a lost increment between the time that you read the stat and
the time you zero it.
That's the locking that's expensive.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
