hi Jaren,
for using a separate window for each individual user, you can't use
SingleWithThreshold operations, since their internal counters reflect event
count, and the occurrence of a matching events always adds 1 to the
counter. Since you want to implement event counters where values greater
than 1 can be added, the rules need some additional Perl code. For
instance, the following ruleset addresses the counting scheme for users
Korsakof, Bob and Alice, and triggers an alarm if one of those users
uploads more than 100MB within 900 seconds (15 minutes):
# Clean entries from the %bytes hash table which are older than 15 minutes
type=Calendar
time=* * * * *
desc=clean old entries from the bytes hash table
action=lcall %o -> ( sub { my($user); my($time) = time(); \
foreach $user (keys %bytes) { \
if ($bytes{$user}->[-1]->[0] < $time - 900) \
{ delete $bytes{$user}; } } } )
# Suppress the user if he/she is not Korsakof, Bob or Alice
type=Suppress
ptype=RegExp
pattern=^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ (?!(?:Korsakof|Bob|Alice)
)\S+ \d+
# Use the %bytes hash table for memorizing each upload event for the user,
# and record the time and bytecount for each upload. When the upload event
# occurs, drop all previous upload events which are older than 15 minutes,
# and sum the bytecounts of remaining uploads. If the sum is greater than
# 100MB, produce an alarm, and suppress further alarms for 2 hours.
type=Single
ptype=RegExp
pattern=^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ ([\w.@-]+) (\d+)
context=$1 $2 -> ( sub { my($time) = time(); my($sum) = 0; \
push @{$bytes{$_[0]}}, [ $time, $_[1] ]; \
while ($bytes{$_[0]}->[0]->[0] < $time - 900) \
{ shift @{$bytes{$_[0]}}; } \
map { $sum += $_->[1] } @{$bytes{$_[0]}}; \
return ($sum > 104857600); } ) \
&& !SUPPRESS_ALARMS_FOR_$1
desc=User $1 has transmitted too many bytes
action=write - %s; create SUPPRESS_ALARMS_FOR_$1 7200
The most complex part of the above ruleset is the context expression of the
last rule which employs the %bytes hash table for memorizing all uploads
for the last 15 minutes. As in my previous ruleset example, the hash table
keys are user names, but this time each user name points to a list of
upload events. Each upload event is in turn memorized as a tuple
(time_of_upload, bytecount). In other words, $bytes{"Bob"}->[0]->[0] refers
to the time of the earliest upload event for user Bob, while
$bytes{"Alice"}->[0]->[1] refers to the bytecount of the earliest upload
event for user Alice. The Perl function in the context expression uses the
time and bytecount fields for dropping events older than 15 minutes first,
and then summing up the bytes of remaining events and comparing the sum to
104857600 (100MB). This ensures that you are always doing thresholding for
the last 15 minutes.
If you wish to filter relevant user names in a different way (e.g., you
want to exclude few user names and apply thresholding for the rest), you
need to modify the regular expression of the second rule accordingly.
Hope this helps,
risto
2015-09-30 14:20 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> Hi,
>
> Thanks for your help. I was thinking and i can not block user during 2H
> without alerts or in any time.
> I was thinking and how can i add a temporaly window?.Can i change the
> alert to SingleWithThreshold and just add a window variable without
> thresh?Could i limit the number of users?
>
> The idea its like that: User Jaren has uploaded 100MB in 15 min.
>
>
>
> Regards.
>
> 2015-09-30 13:19 GMT+02:00 Jaren Peich <burkol...@gmail.com>:
>
>> Hi,
>>
>> Thanks for your help. I was thinking and i can not block user during 2H
>> without alerts or in any time.
>> I was thinking and how can i add a temporaly window?.Can i change the
>> alert to SingleWithThreshold and just add a window variable without
>> thresh?Could i limit the number of users?
>>
>> The idea its like that: User Jaren has uploaded 100MB in 15 min.
>>
>>
>>
>> Regards.
>>
>> 2015-09-29 16:19 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
>>
>>> hi Jaren,
>>> you could try the following simple ruleset and see if this fits your
>>> needs:
>>>
>>> type=Calendar
>>> time=0 * * * *
>>> desc=drop byte counters
>>> action=lcall %o -> ( sub { %bytes = () } )
>>>
>>> type=Single
>>> ptype=RegExp
>>> pattern=^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ ([\w.@-]+) (\d+)
>>> context=$1 $2 -> ( sub { ($bytes{$_[0]} += $_[1]) > 10000 } ) && \
>>> !SUPPRESS_ALARMS_FOR_$1
>>> desc=User $1 has transmitted too many bytes
>>> action=write - %s; create SUPPRESS_ALARMS_FOR_$1 7200
>>>
>>> This ruleset employs the Perl hash table %bytes for keeping track of
>>> user data transfers. The keys to %bytes hash table are user names, and the
>>> value which corresponds to each user name is the number of bytes
>>> transmitted.
>>>
>>> Since you didn't mention in which particular time window you would like
>>> to accomplish upload tracking, I have chosen a fixed window of 1 hour which
>>> is applied to all users. The purpose of the first rule is to drop byte
>>> counters for all users once every hour. If you wish to accomplish the
>>> monitoring in a larger window, it is easy to adjust the 'time' field of the
>>> Calendar rule accordingly (the field takes a value in crontab syntax).
>>>
>>> The second rule matches individual upload events with the regular
>>> expression
>>>
>>> ^\d+\/\d+\/\d+ \d{2}:\d{2}:\s*POST \S+ ([\w.@-]+) (\d+)
>>>
>>> which sets $1 to user name and $2 to transmitted bytes. I have assumed
>>> that legal characters for user names are letters, digits, underscores,
>>> dots, dashes and @-signs, and therefore I've used ([\w.@-]+) for
>>> matching the username. If you want your username to be just any sequence of
>>> non-whitespace characters, you can use (\S+) instead. In order to track the
>>> number of bytes user has uploaded, the rule uses the following context
>>> expression
>>>
>>> $1 $2 -> ( sub { ($bytes{$_[0]} += $_[1]) > 10000 } ) && \
>>> !SUPPRESS_ALARMS_FOR_$1
>>>
>>> which consists of two operands joined by logical AND (&&).
>>>
>>> The first operand '$1 $2 -> ( sub { ($bytes{$_[0]} += $_[1]) > 10000 }
>>> )' is a call to a precompiled perl function which simply adds bytes for the
>>> current upload to the total of the given user. Note that the variables $1
>>> (user name) and $2 (bytes) serve as input parameters for the function.
>>> Also, the function checks if the resulting user total exceeds the threshold
>>> (I have used 10000 bytes for threshold which probably needs replacement by
>>> a more reasonable value). The second operand of the context expression
>>> !SUPPRESS_ALARMS_FOR_$1 is true if the context SUPPRESS_ALARMS_FOR_$1 does
>>> not exist (which is the case if we haven't generated alarm for the given
>>> user within the last 2 hours -- see the following explanation). If both
>>> operands are true (in other words, the upload threshold is violated and we
>>> haven't produced an alert about the user in last 2h), the rule will write a
>>> warning string to standard output with the 'write' action. Also, after
>>> generating the alarm, the rule creates the SUPPRESS_ALARMS_FOR_$1 context
>>> with a lifetime of 7200 seconds. Since the context will stay around for 2
>>> hours, it suppresses further alarms for the same user for this time frame.
>>>
>>> Hope this helps,
>>> risto
>>>
>>>
>>>
>>>
>>>
>>> 2015-09-29 11:21 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
>>>
>>>> Hi,
>>>>
>>>> Sorry for my English and thanks for the perl library. Thanks for
>>>> accepting me in mailing list. :)
>>>>
>>>> I have to program a code with SEC to detect a quantity of bytes
>>>> uploaded to the web from one user with a limit. I dont know how to sum the
>>>> quantity of bytes that comes from each proxy log line till reach the limit
>>>> and generate an alert.
>>>>
>>>> Log Line
>>>>
>>>> Time: Method "Url" User Bytes
>>>> 29/09/2015 10:14:POST "www.google.com" Korsakof 1250
>>>>
>>>> Thank you. Regards.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Simple-evcorr-users mailing list
>>>> Simple-evcorr-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>>>
>>>>
>>>
>>
>
------------------------------------------------------------------------------
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
