On 12/28/2012 02:47 AM, John P. Rouillard wrote: > >> In my own environment, I've seen accidental differences of 8-10KB >> which disappear when a new dump file is created in the next second. >> >> However, it is important to know that the read offset is not always >> reflecting the amount of already processed input data. Since reading >>from file is done by up to 1KB chunks, these chunks could easily >> contain several lines. As a consequence, only the first line is >> matched against rules immediately, while the rest will reside in a >> buffer for a very short amount of time. > > I wonder if the "read offset" could be quickly turned into a > "processed offset" (which is what I attempt to capture) using: > > read offset - (size of buffer contents - end of current line in buffer)
In principle, it would be possible to output the number of read but yet unprocessed bytes. However, SEC will invoke read(2) system call only if there is no complete line found among buffered bytes. As long as complete lines can be fetched from the IO buffer, read(2) is not called. Therefore, the read offset can not jump far ahead of actual processing offset (unless, of course, there are huge lines in input files). > >> Nevertheless, any real lag can be certainly detected by comparing >> recorded read offsets with file sizes. > > Yup. I think that would work to throw into an rrd for monitoring. > > One thing I was considering was writing a perl function that I could > call from a calender rule that would allow me to dump the stats on a > regular basis into and rrd or other database. How easily could I reuse > your changes in such a senario? One simple scenario would be to deliver SIGUSR1 signal to SEC process itself, and then parse out necessary data from the dump file. I suppose the dump_data() function could also be invoked directly with the main:: prefix, without generating the signal. kind regards, risto > > -- > -- rouilj > John Rouillard > =========================================================================== > My employers don't acknowledge my existence much less my opinions. > > ------------------------------------------------------------------------------ > Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and > much more. Get web development skills now with LearnDevNow - > 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. > SALE $99.99 this month only -- learn more at: > http://p.sf.net/sfu/learnmore_122812 > _______________________________________________ > Simple-evcorr-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > ------------------------------------------------------------------------------ Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
