Hi,
Sorry, i couldn´t answer before but i thought something like that:
Here i can detect and create a context for each user.
type = Single
ptype = Cached
desc = $0
continue = TakeNext
context = !BYTES_$+{user}
pattern = pattern
action = create BYTES_$+{user} 300;
We check that the context exists and we add bytes in each context:
type = Single
ptype = Cached
desc = $0
continue = TakeNext
context = BYTES_$+{user}
pattern = pattern_que_necesites
action = *add* BYTES_$+{user} $bytes_uploaded
Finally i pop the result:
type = Single
ptype = Cached
desc = $+{user}
continue = TakeNext
context = BYTES_$+{user}
pattern = pattern
action = *pop* BYTES_$+{user} %bytes_uploaded_$+{user} ; \
I don´t know how read the bytes added to the context. is it possible?
Regards.
2015-09-30 14:34 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
> hi Jaren,
> for using a separate window for each individual user, you can't use
> SingleWithThreshold operations, since their internal counters reflect event
> count, and the occurrence of a matching events always adds 1 to the
> counter. Since you want to implement event counters where values greater
> than 1 can be added, the rules need some additional Perl code. For
> instance, the following ruleset addresses the counting scheme for users
> Korsakof, Bob and Alice, and triggers an alarm if one of those users
> uploads more than 100MB within 900 seconds (15 minutes):
>
> # Clean entries from the %bytes hash table which are older than 15 minutes
>
> type=Calendar
> time=* * * * *
> desc=clean old entries from the bytes hash table
> action=lcall %o -> ( sub { my($user); my($time) = time(); \
> foreach $user (keys %bytes) { \
> if ($bytes{$user}->[-1]->[0] < $time - 900) \
> { delete $bytes{$user}; } } } )
>
> # Suppress the user if he/she is not Korsakof, Bob or Alice
>
> type=Suppress
> ptype=RegExp
> pattern=^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ (?!(?:Korsakof|Bob|Alice)
> )\S+ \d+
>
> # Use the %bytes hash table for memorizing each upload event for the user,
> # and record the time and bytecount for each upload. When the upload event
> # occurs, drop all previous upload events which are older than 15 minutes,
> # and sum the bytecounts of remaining uploads. If the sum is greater than
> # 100MB, produce an alarm, and suppress further alarms for 2 hours.
>
> type=Single
> ptype=RegExp
> pattern=^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ ([\w.@-]+) (\d+)
> context=$1 $2 -> ( sub { my($time) = time(); my($sum) = 0; \
> push @{$bytes{$_[0]}}, [ $time, $_[1] ]; \
> while ($bytes{$_[0]}->[0]->[0] < $time - 900) \
> { shift @{$bytes{$_[0]}}; } \
> map { $sum += $_->[1] } @{$bytes{$_[0]}}; \
> return ($sum > 104857600); } ) \
> && !SUPPRESS_ALARMS_FOR_$1
> desc=User $1 has transmitted too many bytes
> action=write - %s; create SUPPRESS_ALARMS_FOR_$1 7200
>
>
> The most complex part of the above ruleset is the context expression of
> the last rule which employs the %bytes hash table for memorizing all
> uploads for the last 15 minutes. As in my previous ruleset example, the
> hash table keys are user names, but this time each user name points to a
> list of upload events. Each upload event is in turn memorized as a tuple
> (time_of_upload, bytecount). In other words, $bytes{"Bob"}->[0]->[0] refers
> to the time of the earliest upload event for user Bob, while
> $bytes{"Alice"}->[0]->[1] refers to the bytecount of the earliest upload
> event for user Alice. The Perl function in the context expression uses the
> time and bytecount fields for dropping events older than 15 minutes first,
> and then summing up the bytes of remaining events and comparing the sum to
> 104857600 (100MB). This ensures that you are always doing thresholding for
> the last 15 minutes.
>
> If you wish to filter relevant user names in a different way (e.g., you
> want to exclude few user names and apply thresholding for the rest), you
> need to modify the regular expression of the second rule accordingly.
>
> Hope this helps,
> risto
>
> 2015-09-30 14:20 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
>
>> Hi,
>>
>> Thanks for your help. I was thinking and i can not block user during 2H
>> without alerts or in any time.
>> I was thinking and how can i add a temporaly window?.Can i change the
>> alert to SingleWithThreshold and just add a window variable without
>> thresh?Could i limit the number of users?
>>
>> The idea its like that: User Jaren has uploaded 100MB in 15 min.
>>
>>
>>
>> Regards.
>>
>> 2015-09-30 13:19 GMT+02:00 Jaren Peich <burkol...@gmail.com>:
>>
>>> Hi,
>>>
>>> Thanks for your help. I was thinking and i can not block user during 2H
>>> without alerts or in any time.
>>> I was thinking and how can i add a temporaly window?.Can i change the
>>> alert to SingleWithThreshold and just add a window variable without
>>> thresh?Could i limit the number of users?
>>>
>>> The idea its like that: User Jaren has uploaded 100MB in 15 min.
>>>
>>>
>>>
>>> Regards.
>>>
>>> 2015-09-29 16:19 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
>>>
>>>> hi Jaren,
>>>> you could try the following simple ruleset and see if this fits your
>>>> needs:
>>>>
>>>> type=Calendar
>>>> time=0 * * * *
>>>> desc=drop byte counters
>>>> action=lcall %o -> ( sub { %bytes = () } )
>>>>
>>>> type=Single
>>>> ptype=RegExp
>>>> pattern=^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ ([\w.@-]+) (\d+)
>>>> context=$1 $2 -> ( sub { ($bytes{$_[0]} += $_[1]) > 10000 } ) && \
>>>> !SUPPRESS_ALARMS_FOR_$1
>>>> desc=User $1 has transmitted too many bytes
>>>> action=write - %s; create SUPPRESS_ALARMS_FOR_$1 7200
>>>>
>>>> This ruleset employs the Perl hash table %bytes for keeping track of
>>>> user data transfers. The keys to %bytes hash table are user names, and the
>>>> value which corresponds to each user name is the number of bytes
>>>> transmitted.
>>>>
>>>> Since you didn't mention in which particular time window you would like
>>>> to accomplish upload tracking, I have chosen a fixed window of 1 hour which
>>>> is applied to all users. The purpose of the first rule is to drop byte
>>>> counters for all users once every hour. If you wish to accomplish the
>>>> monitoring in a larger window, it is easy to adjust the 'time' field of the
>>>> Calendar rule accordingly (the field takes a value in crontab syntax).
>>>>
>>>> The second rule matches individual upload events with the regular
>>>> expression
>>>>
>>>> ^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ ([\w.@-]+) (\d+)
>>>>
>>>> which sets $1 to user name and $2 to transmitted bytes. I have assumed
>>>> that legal characters for user names are letters, digits, underscores,
>>>> dots, dashes and @-signs, and therefore I've used ([\w.@-]+) for
>>>> matching the username. If you want your username to be just any sequence of
>>>> non-whitespace characters, you can use (\S+) instead. In order to track the
>>>> number of bytes user has uploaded, the rule uses the following context
>>>> expression
>>>>
>>>> $1 $2 -> ( sub { ($bytes{$_[0]} += $_[1]) > 10000 } ) && \
>>>> !SUPPRESS_ALARMS_FOR_$1
>>>>
>>>> which consists of two operands joined by logical AND (&&).
>>>>
>>>> The first operand '$1 $2 -> ( sub { ($bytes{$_[0]} += $_[1]) > 10000 }
>>>> )' is a call to a precompiled perl function which simply adds bytes for the
>>>> current upload to the total of the given user. Note that the variables $1
>>>> (user name) and $2 (bytes) serve as input parameters for the function.
>>>> Also, the function checks if the resulting user total exceeds the threshold
>>>> (I have used 10000 bytes for threshold which probably needs replacement by
>>>> a more reasonable value). The second operand of the context expression
>>>> !SUPPRESS_ALARMS_FOR_$1 is true if the context SUPPRESS_ALARMS_FOR_$1 does
>>>> not exist (which is the case if we haven't generated alarm for the given
>>>> user within the last 2 hours -- see the following explanation). If both
>>>> operands are true (in other words, the upload threshold is violated and we
>>>> haven't produced an alert about the user in last 2h), the rule will write a
>>>> warning string to standard output with the 'write' action. Also, after
>>>> generating the alarm, the rule creates the SUPPRESS_ALARMS_FOR_$1 context
>>>> with a lifetime of 7200 seconds. Since the context will stay around for 2
>>>> hours, it suppresses further alarms for the same user for this time frame.
>>>>
>>>> Hope this helps,
>>>> risto
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2015-09-29 11:21 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
>>>>
>>>>> Hi,
>>>>>
>>>>> Sorry for my English and thanks for the perl library. Thanks for
>>>>> accepting me in mailing list. :)
>>>>>
>>>>> I have to program a code with SEC to detect a quantity of bytes
>>>>> uploaded to the web from one user with a limit. I dont know how to sum the
>>>>> quantity of bytes that comes from each proxy log line till reach the limit
>>>>> and generate an alert.
>>>>>
>>>>> Log Line
>>>>>
>>>>> Time: Method "Url" User Bytes
>>>>> 29/09/2015 10:14:POST "www.google.com" Korsakof 1250
>>>>>
>>>>> Thank you. Regards.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Simple-evcorr-users mailing list
>>>>> Simple-evcorr-users@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>>>>
>>>>>
>>>>
>>>
>>
>
------------------------------------------------------------------------------
Full-scale, agent-less Infrastructure Monitoring from a single dashboard
Integrate with 40+ ManageEngine ITSM Solutions for complete visibility
Physical-Virtual-Cloud Infrastructure monitoring from one console
Real user monitoring with APM Insights and performance trend reports
Learn More http://pubads.g.doubleclick.net/gampad/clk?id=247754911&iu=/4140
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
