Limiting the number of batches within a given interval is conceptually similar to the index refresh rate setting in elasticsearch itself. At first blush I like this idea, as it is simple to understand the impact on when a given log line will be available within elasticsearch if I know this interval and know my index refresh interval.
On Sat, Aug 29, 2015 at 12:21 PM, David Lang <da...@lang.hm> wrote: > On Mon, 24 Aug 2015, Rainer Gerhards wrote: > > I am mostly with Radu on this topic. I think there are some use cases >> where it really would be advantageous to submit a larger batch, even >> if this means waiting. True, these use cases were very seldom in the >> early days of rsyslog and may still be, but I think it's something one >> might validly want. >> > > The thought hit me that we are loking at this wrong. > > The problem is overloading the receiver with too many small batches. > > rather than trying to define batch size, isn't what we really want to have > is a limit on how many batches we send in a given timeframe? possibly with > a 'escape clause' that says tht if we are sending maxbatch size messages > for the entire timeframe we do something (spawn a new sending thread, > temporarily allow higher sending rates, or just let the backlog accumulate > are all valid choices under different conditions) > > thoughts? > > David Lang > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
