Hello Risto
Thank you for the suggestion, yes i thought about direct lookup of hash
table on the regular expression but i agree that its not an efficient way
with lot of side effects.
Got the basic idea of my initial usecase. will work on the loops for
testing different log sources.
Regards,
Santhosh
On Wed, Sep 4, 2019 at 7:42 PM Risto Vaarandi <risto.vaara...@gmail.com>
wrote:
> hi Santhosh,
>
> is my understanding correct that you would like to match an IP address
> with a regular expression, and perform a lookup into %ioc hash table within
> the same regular expression? I need to study the documentation before
> suggesting how this could be done, but I would advise against this
> approach. Firstly, if you embed code in the regular expression, it would
> become lot more complex and less readable. And secondly, matching such an
> expression against input events might become more expensive. Therefore, it
> is better to perform hash table lookups in a separate code block after a
> successful regular expression match. Since IP addresses can only appear in
> two locations in your input events, you could rewrite the PerlFunc pattern
> to extract both IP addresses and have two statements for checking their
> presence in %ioc hash table. For example:
>
> type=Single
> ptype=PerlFunc
> pattern=sub { if ($_[0] !~ /ASA-\S+: Teardown \S+ connection \d+ for
> outside: ([\d.]+).* identity: ([\d.]+)/) { return 0; } \
> if (exists($ioc{$1})) { return ($1, $ioc{$1}); } \
> if (exists($ioc{$2})) { return ($2, $ioc{$2}); } \
> return 0; }
> desc=Connection to IP address $1 with IoC information $2
> action=pipe 'Log matched IoC:%{.nl}IoC: $2%{.nl}Log: $0' /bin/mail
> mail...@example.com
>
> If both IP addresses are present in %ioc hash table, the above rule will
> return the IP address after "outside:" field with the ioc information.
>
> Also, if you have other input events with different formats which might
> contain an arbitrary number of IP addresses, you can include a loop in
> PerlFunc pattern for extracting IP addresses iteratively with a regular
> expression. If you are interested in a relevant example, please let me know
> and I will post it to mailing list.
>
> kind regards,
> risto
>
> Kontakt Santhosh Kumar (<santhoshkmrre...@gmail.com>) kirjutas kuupäeval
> K, 4. september 2019 kell 05:50:
>
>> Hello Risto
>>
>> Though this query is not related. Just got curious about using the
>> variable in a pattern directly instead of matching later.
>>
>> Log: "ASA-6-302016: Teardown UDP connection 806353 for outside:
>> 187.189.195.208 <http://187.189.195.208:8443/>/24057 to
>>
>> identity: 172.18.124.136/161 duration 0:02:01 bytes 313"
>>
>> As per the rule, IPv4 is extracted to a variable $1 and match against %ioc
>> hash table. Instead, is it possible to match the IOC IP's with any part of
>> the log(either in outside:([\d.]+) or in identity:([\d.]+)).
>>
>> Regards,
>>
>> san
>>
>>
>>
>> On Tue, Sep 3, 2019 at 12:47 PM Santhosh Kumar <
>> santhoshkmrre...@gmail.com> wrote:
>>
>>> Hello risto
>>>
>>> I ran the tests with real logs. Suggested method works exactly as
>>> expected.
>>>
>>> This resolves many of my other queries. Thank you for prompt response.
>>>
>>> Regards,
>>> Santhosh
>>>
>>>
>>> On Fri, Aug 30, 2019, 20:59 Risto Vaarandi <risto.vaara...@gmail.com>
>>> wrote:
>>>
>>>> hi Santhosh,
>>>>
>>>> since your task involves not only matching IP addresses against a
>>>> blacklist but also includes reporting IoC information for a bad IP address,
>>>> I would recommend loading IoC data from file into a Perl hash which allows
>>>> for quick lookups. The example ruleset below uses a global hash %ioc which
>>>> is a global variable and can thus be accessed from all rules:
>>>>
>>>> type=Single
>>>> ptype=RegExp
>>>> pattern=^(SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART)$
>>>> context=SEC_INTERNAL_EVENT
>>>> desc=load IoC information on SEC startup and HUP or ABRT signals
>>>> action=lcall %count -> ( sub { %ioc = (); \
>>>> if (!open(FILE, "/home/risto/IOC_data_proposal.txt")) { return
>>>> -1; } \
>>>> while (<FILE>) { if (/^\s*(([\d.]+):\d+\s*-.*)/) { \
>>>> $ioc{$2} = $1; } \
>>>> } close(FILE); return scalar keys %ioc; } ); \
>>>> logonly Loaded %count entries from IoC data file
>>>>
>>>> type=Single
>>>> ptype=PerlFunc
>>>> pattern=sub { if ($_[0] !~ /ASA-\S+: Teardown \S+ connection \d+ for
>>>> outside: ([\d.]+)/) { return 0; } \
>>>> if (!exists($ioc{$1})) { return 0; } return ($1, $ioc{$1}); }
>>>> desc=Connection to IP address $1 with IoC information $2
>>>> action=pipe 'Log matched IoC:%{.nl}IoC: $2%{.nl}Log: $0' /bin/mail
>>>> some...@example.com
>>>>
>>>> The first rule loads IoC information from file into %ioc hash table
>>>> whenever SEC is started or HUP or ABRT signal is received by SEC. IP
>>>> addresses serve as keys of the hash table, while each value is an entire
>>>> line from the IoC file. For example, if the file contains the following two
>>>> lines
>>>>
>>>> 187.163.222.244:465 - emotet
>>>> 187.189.195.208:8443 - emotet
>>>>
>>>> the %ioc hash table will contain the following mappings (keys and
>>>> values are separated by ->):
>>>>
>>>> 187.163.222.244 -> 187.163.222.244:465 - emotet
>>>> 187.189.195.208 -> 187.189.195.208:8443 - emotet
>>>>
>>>> Currently, the first rule assumes that IoC file is in the same format
>>>> as you described in your e-mail, and the rule uses regular expression
>>>> ^\s*(([\d.]+):\d+\s*-.*) for parsing the file and extracting relevant
>>>> information. Should the format of the file change, this regular expression
>>>> needs to be adjusted accordingly. Also, the rule finds the number of
>>>> entries loaded from IoC file, stores it in %count action list variable and
>>>> logs a debug message with this value into SEC log file. If the rule was
>>>> unable to open the file, the value -1 is logged which is useful for
>>>> troubleshooting purposes.
>>>>
>>>> The second rule uses PerlFunc pattern for matching incoming ASA
>>>> firewall events and first verifies that incoming event matches the regular
>>>> expression
>>>> ASA-\S+: Teardown \S+ connection \d+ for outside: ([\d.]+)
>>>> If there is a match, IP address of remote host is extracted and
>>>> assigned to $1 variable, and %ioc hash table is looked up for IoC
>>>> information for that IP address. If lookup is successful, PerlFunc pattern
>>>> returns a list with two elements (IP address, IoC info) which are mapped to
>>>> match variables $1 and $2 by SEC ($0 variable will hold the entire matching
>>>> event log line). The match variables are then used by 'pipe' action for
>>>> sending an e-mail to relevant mailbox.
>>>>
>>>> Hope this helps,
>>>> risto
>>>>
>>>>
--
Regards,
SanthoshKumar S
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
