Hello Thanks for the brief..!!
In my case, udgram creates a event loop however tcpsock functionality achived the goal. Regarding second scenario, the goal is to match the IP address and notify. "ASA-6-302016: Teardown UDP connection 806353 for outside: 187.189.195.208 <http://187.189.195.208:8443/>/24057 to identity: 172.18.124.136/161 duration 0:02:01 bytes 313" List of IOC's stored in a file(IOC_data_proposal.txt) as below. the rule suppose to match the IP address from the file(IOC_data_proposal.txt) though the file contains other informations like port number and name of the malware, etc. which is not part of the log. The reason for not removing extra data(port, ioc name) from the file( IOC_data_proposal.txt) is that its required for a notification. ************************ IOC_data_proposal.txt 187.163.222.244:465 - emotet 187.189.195.208:8443 - emotet 188.166.253.46:8080 - emotet 189.209.217.49:80 <http://189.209.217.49/> - heartbleed ************************ Expected mail Output: Log Matched IOC, IOC: 187.189.195.208:8443 - emotet Log: "ASA-6-302016: Teardown UDP connection 806353 for outside: 187.189.195.208 <http://187.189.195.208:8443/>/24057 to identity: 172.18.124.136/161 duration 0:02:01 bytes 313" regards, Santhosh On Wed, Aug 28, 2019 at 4:18 AM Risto Vaarandi <risto.vaara...@gmail.com> wrote: > hi Santhosh, > > Kontakt Santhosh Kumar (<santhoshkmrre...@gmail.com>) kirjutas kuupäeval > T, 27. august 2019 kell 04:55: > >> Hello Risto >> >> >> I’ve been running tests on SEC for a while and stuck with below points. >> I’m not familiar with Perl though I tried to find a solution from sec mail >> bucket but no luck, please suggest if this can be achieved with high >> performance, >> >> >> >> 1. I could see a log drops when I tested with the event rate of 15000 >> logs/sec. A simple SEC rule to receive and forward all the logs to a >> destination. The output shows relatively less number of logs. This also >> increases the cpu usage from 0.3% to 45% >> >> ************************ >> >> Type=single >> >> Ptype=regexp >> >> Pattern=([.\d]+) >> >> Desc=$1 >> >> Action=pipe $0 nc syslog101 514 >> >> ************************ >> > > The above rule is very inefficient, since 'pipe' action pipes the value of > $0 to "nc syslog101 514" command and then closes the pipe, forcing the nc > command to terminate. In other words, each time the above rule matches an > event, new command is forked, and if you have 15000 events per second, the > rule attempts to fork 15000 processes each second. This imposes > considerable load on the system and if you send in events at a high rate, > the rule might easily exhaust your system resources. > > Instead of forking nc on each matching event, I would recommend to utilize > 'tcpsock' action which transmits events over a single TCP socket (since you > haven't used -u flag with nc tool, I assume that your syslog server listens > on port 514/tcp). For example, consider the following rule (the rule > terminates each transmitted line with the newline): > > type=single > ptype=regexp > pattern=([.\d]+) > desc=test event $1 > action=tcpsock syslog101:514 $0%.nl > > If your syslog server speaks BSD syslog protocol ( > https://tools.ietf.org/html/rfc3164), and incoming events are not in that > format, you could use sec builtin action list variables for formatting the > event and providing fields that syslog server expects (such as timestamp). > For example, the following rule transmits each event line over TCP in BSD > syslog format with priority 14 (facility of "user" and severity of "info"), > with hostname "myhost", with program name "myprog", and using newline as a > separator between messages: > > type=single > ptype=regexp > pattern=([.\d]+) > desc=test event $1 > action=tcpsock syslog101:514 <14>%.monstr %.mdaystr %.hmsstr myhost > myprog: $0%.nl > > Finally, as David suggested, you can also pass messages to local syslog > server via /dev/log socket, and let the local syslog server handle the > messages (note that unlike for 'tcpsock' in previous example, there is no > need for hostname and terminating newline for 'udgram' action): > > type=single > ptype=regexp > pattern=([.\d]+) > desc=test event $1 > action=udgram /dev/log <14>%.monstr %.mdaystr %.hmsstr myprog: $0 > > If your local syslog server is rsyslog, you could have the following > rsyslog rule for forwarding messages: > > if $programname == "myprog" then @@syslog101:514 > > As you can see, there are several ways for achieving your goal, and > hopefully above examples are helpful for selecting the most convenient > solution. > > >> >> 1. On a different scenario, I was interested to match the logs with >> list of IOC’s. Here i was trying to mail the detected log along with IOC >> name. I could achieve it to certain level as mentioned in example but no >> luck with this cases, "Split IP's from the IOC file and use it on the >> “pattern” to match IP from logs" >> >> ************************ >> >> IOC_data_proposal.txt >> >> 187.163.222.244:465 - emotet >> >> 187.189.195.208:8443 - emotet >> >> 188.166.253.46:8080 - emotet >> >> 189.209.217.49:80 - heartbleed >> >> ************************ >> >> Please check and share some insights. >> > > I am not sure I fully understood what exactly you want to achieve here. > Can you provide some examples of input events and what output you would > like to generate on each match? > > kind regards, > risto > > > >> >> >> >> Eg: I currently tested below case and its working fine as this is a >> straight forward IOC matches. >> >> ************************ >> >> #Current Rule for matching IOC: >> >> type=Single >> >> ptype=RegExp >> >> pattern=(?:SEC_STARTUP|SEC_RESTART|SEC_SOFTRESTART) >> >> desc=load IOC data >> >> action=logonly; delete IP; create IP; \ >> >> lcall %iocevents -> (sub{scalar `cat >> /usr/local/bin/sec-rules/ioc_data.txt`}); >> \ >> >> cevent IOC_IP 0 %iocevents; >> >> >> >> type=Single >> >> ptype=RegExp >> >> pattern=. >> >> context=IOC_IP >> >> desc=create an entry >> >> action=logonly; alias IOC IOC_$0 >> >> >> >> type=Single >> >> ptype=regexp >> >> context=IOC_$2 >> >> pattern= syslog.*hostname=([\w\-\d]+).*IP=([\d\.]+) >> >> desc=Matched host & ip: $2 && $3 >> >> action=pipe '$0' mail -s ‘%s’ ‘test123.gmail.com’ >> >> >> >> IOC_data.txt >> >> 187.163.222.244 >> >> 187.189.195.208 >> >> 188.166.253.46 >> >> 189.209.217.49 >> >> 187.163.222.244 >> >> 187.189.195.208 >> >> 188.166.253.46 >> >> 189.209.217.49 >> >> ************************ >> >> >> >> Regards, >> >> san >> > -- Regards, SanthoshKumar S
_______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
