> I have looked further at the code. I think I see part of the > inefficiency that happens in the regular case (with default > parameters). The queue subsystem was originally written with a very > generic scope, intended to handle things other than messages. However, > we only process message objects via it (I need to check in detail if > there are tiny exceptions, but I don't think so). This generic code has > a high price, it uses various levels of indirection and does NOT use > the optimized message creation paths e.g. it does a time system call > for each message being read, even thought that time is immediately > discarded. As past experience told us, that time call alone is very > problematic. > > I also see other inefficiencies in regard to the encoding respective > the way the in-memory object is restored. There is even a lot of room > with the current encoding, not to mention the binary header David > mentioned. > > I think a technically sound plan is to first make the queue-subsystem > non-generic so that it knows what it deals with (messages!). With that > extra information, I can than make the reader (hopefully much) more > efficient. I think all other performance enhancements need to go after > that. Moving the code to be msg-specific (non-generic) is required in > any case as all other optimizations discussed so far rely on the fact > that msg objects are being processed.
The current git master head now contains a refactored version that implements many of the things I mentioned above. On my development machine, I get around a 10 to 15% speedup (but this number is always questionable, as it is virtualized, no testing done on real hardware). I'll try today to check a little bit more on the refactoring approach, I think there is at least one more spot that I can change with relatively little effort (and hopefully big effect). Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
