2013/9/30 John P. Rouillard <[email protected]>
>
> In message
> <cagfjscmfak2z+n7j8bbzjp8cc3zpms1pa_exp+mo5mvtuf_...@mail.gmail.com> ,
> Risto Vaarandi writes:
> >2013/9/30 John P. Rouillard <[email protected]>
> >> I had to define an interval today for a SEC ruleset. Since calendar
> >> rules only fire at an exact time, the calendar rule to define my
> >> interval had to fire every minute during the interval to make sure
> >> that the interval context was defined in case sec restarted during the
> >> interval.
> >>
> >> This took three calendar rules and seems to be a waste of cpu
> >> cycles even with a context of [ ! interval_active ].
> >>
> >> Would it be useful to define an optional time2 parameter (and possibly
> >> action2) for Calendar with the following semantics:
> >>
> >> if the calendar rule has not yet been triggered, and the current
> >> time is between time and time2, execute action.
> >>
> >> At time time2, execute action2.
> >>
> >> This means at SEC startup, the calendar rule will trigger if the time
> >> is between the two times. This means that the rule should be triggered
> >> without the delay between SEC startup and the next minute
> >> interval. Also this will allow a more compact way to specify two
> >> actions that should run at an absolute time relative to each other.
> >
> >that's an interesting proposal. For addressing this problem, I have
> usually
> >renewed my critical time interval contexts every minute between interval
> >endpoints, and having them had a lifetime of 1 minute. This workaround
> >works nicely with time specifications like '* * * * *' or '* 1-5 * * 0',
> >since at startup sec will run actions of calendar rules which match the
> >given moment, doing this all *before* entering the main event processing
> >loop.
>
> Ah ok, I didn't realise that SEC scanned the calendar rules first. Has
> that changed in the past couple pof years?
Exactly, that change was introduced into the 2.6.1 version which was
released in September 2011. Since this particular version, the first scan
of Calendar rules appears immediately after SEC_STARTUP event has been
generated (if the generation of this event has been configured with
--intevents, of course).
> I'll have to go back and
> look at my test cases then because I managed to have a context not
> created with a time spec of:
>
> time = * * * * *
> action = create ALIVE
>
> type = single
> ptype = regexp
> pattern = .
> desc = is ALIVE set
> context = ! ALIVE
> action = write - not alive
>
> when the first event was processed.
>
> I needed a time interval of 17:15 to 06:30 which ended up with needing
> four rules:
>
> time = 15-59 17 * * *
> time = 0-59 18-23 * * *
> time = 0-59 0-5 * * *
> time = 0-30 6 * * *
>
> (I thought 18-23,0-5 would work but it didn't.)
>
Hmm, that's strange because it should work. In your example below, I
modified 'time=* * * * *' into
time = * 20-23,3-8,12-19,0-2,9-11 * * *
(plus several other modifications) and had the context set up properly.
Why wasn't your time specification working, did you get a syntax error into
the log or was one of the ranges ignored?
However, if by accident a space is left after the comma between 18-23 and
0-5, yielding
time=* 18-23, 0-5 * * *
this is still regarded as a valid time specification, since optionally the
time field takes the 6th element for matching specific years within given
century. Therefore, the specification with a typo would actually mean
"every minute between 18.00 and 23.59, in days 1-5 of any month plus the
last day of any month".
>
> Also can you test this:
>
> type = calendar
> desc = test context
> time = * * * * *
> context = ALIVE
> action = create ALIVE 60
>
> type = single
> desc = prime
> ptype = substr
> pattern = prime
> action = create ALIVE 60
>
> type = single
> desc = test
> ptype = substr
> pattern = test
> context = ALIVE
> action = write - I am alive
>
> type = single
> desc = test
> ptype = tvalue
> pattern = true
> action = write - I am dead
>
> If I send on stdin:
>
> ( echo prime; yes test ) | sec ...
>
> I end up dead at some point (which is quite disturbing 8-) ). Usually
> in under 24 hours. This smells like a timing issue somehwere, possibly
> in perl. I did at one point use a lifetime of 62 seconds which
> IIRC worked fine, but I would be interested if the 60 second period
> works for you.
>
It is somewhat hard to guess what might be wrong, since on a normal system
with reasonable amount of computing resources the first Calendar rule from
your example is processed within the 1st second of each minute.
On the other hand, the ALIVE context is expired when
(current_time - context_creation_time > 60)
Since the context is created from the Calendar rule, it should normally
expire in the 2nd second of each minute, and would thus normally survive
until the next run of the Calendar rule. Nevertheless, several
circumstances might interfere with this balance. First, if the system is
heavily overloaded and sec is not receiving enough cpu time, processing of
calendar rules could take place later than expected (this rare, but
possible).
More commonly, however, this scenario could happen because of slow IO. In
order to make sec usable for unix pipelines, sec actions which write to
standard output do it in a blocking manner, and this includes 'write -
<string>'. For example, if you invoke
( echo prime; yes test ) | sec | less
it is very easy to reproduce the "dead" event by just leaving 'less'
untouched in a terminal window for a few moments. The pipe buffer between
sec and less will fill up very quickly, and as a result, the sec process
will block on a pipe. The same kind of blocking could happen for any slow
reader, and it could accidentally offset the processing of the Calendar
rule.
I am not sure if my guesswork is of any help, but that's what I'm
suspecting...
kind regards,
risto
>
> If not then you may want to bump your lifetimes to make sure you don't
> process an event with a missing context.
>
> Because of this I create an infinite lifetime context with rules that
> fire every minute during the interval. Since a restart will delete the
> context I only use a single specific min/hour time rule to delete the
> context.
>
> >Fortunately, the calendar rules are treated separately from other
> >rules which match regular input events, and the list of these rules is
> >scanned at most once a second. Thus the number of calendar rules has much
> >less impact on performance than regular rules which could be evaluated
> >thousands of times per second.
>
> Well that is good at least.
>
> --
> -- rouilj
> John Rouillard
> ===========================================================================
> My employers don't acknowledge my existence much less my opinions.
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> Simple-evcorr-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
