On Sat, 14 Dec 2013, RB wrote:
On Sat, Dec 14, 2013 at 5:24 AM, Rainer Gerhards
<rgerha...@hq.adiscon.com> wrote:
well, technically it's for sure possible, it's just another of these 24h
things. Technically, it's a question of interface, and insofar of which
types of modules. Obviously, these will be slower, and how slow is another
interface/effort question.
Thinking about this, one could probably also claim the answer is "yes, you
can write OUTPUT modules in any language", it's just a doc issue. In fact,
omprog can be used as an interface here. It's actually not even a bad
interface...
Again, something learned ;)
Probably the cheapest (implementation) "binding" for rsyslog would be
a system() like call. Execute the subprogram with /bin/sh -c and
communicate with structured messages on STDIO.
a real module binding would be far more complex. It would allow the module (in
whatever language) access to the rsyslog queues and other data structures. This
is possible, but not easy by any means.
One big problem is that currently rsyslog does all this work in a threaded
environment. It may make sense in v9 or v10 to shift from a default
shared-everything threading model to a explicit shared memory multiprocess
model. At that point having one of the processes use a different language would
not be that hard.
But in the current threaded model, having one thread run a different language
would be very, very hard.
The other issue here is performance. Rsyslog goes to a LOT of effort to be fast.
Some of the things that have made very noticable diffences in performance in
rsyslog are things that seem like they should be very minor. Think about these
things and then think about what would be involved to define interfaces in a
multi-language safe way.
things that have resulted in noticable speedups have been:
removing gettimeofday() calls.
it used to be that rsyslog recorded when a message arrived, when it was put on
the main queue, when it was moved to an action queue, when it was pulled from
the action queue, and when it was delivered
now, high performance users configure rsyslog so that it only does one
gettimeofday() call per hundred (ot thousand) messages that arrive and use that
one time for every message
string modules
it used to be that the default template (<%pri%>%timestamp% %hostname%
%syslogtag%%msg%) was interpreted by the rsyslog engine for every message that
was output
now string modules written in C create these strings rather than interpreting
the template. This resulted in a double-digit % performance improvement
With optimizations like these in use, changing things to allow for a module
written in a different language to have access to the rsyslog internals as would
be needed for a high-performance interface seems like it will probably end up
hurting the rsyslog performance overall.
That being said, I am very much in favor of multi-process with explicit sharing
rather than multi-threaded with implicit sharing, but getting all the interfaces
correct and fast would be a VERY hard task.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
