[ https://issues.apache.org/jira/browse/METRON-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Justin Leet updated METRON-157: ------------------------------- Fix Version/s: (was: Next + 1) 0.7.1 > Create CEF Parser > ----------------- > > Key: METRON-157 > URL: https://issues.apache.org/jira/browse/METRON-157 > Project: Metron > Issue Type: New Feature > Reporter: Domenic Puzio > Assignee: Kyle Richardson > Priority: Minor > Labels: platform > Fix For: 0.7.1 > > > Create a parser for CEF (Common Event Format). CEF is a very common > formatting for security data sources; it is used by FireEye, Adallom, Imperva > WAF, CyberArk, and others. The parser should be flexible enough to work for > any of these data sources. CEF uses shorthand field names, so field names > should be changed to human-readable and Metron-friendly equivalents. CEF > custom labels (cs1Label, flexString1Label, etc.) should be converted > appropriately. > Below are sample messages and their expected parsed output. > Adallom CEF > 2016-04-01T09:29:11.356-0400 > CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity > policy 'User download/view file' was triggered by 'scolb...@gmail.com' > suser=wander...@rock.com start=1459517280810 end=1459517280810 > audits=["AVPR-4oIPeFmuZ3CKKrg","AVPR-wx80cd9PUpAu2aj","AVPR-6XGPeFmuZ3CKKvx","AVPSALn_qE4Kgs_8_yK9","AVPSASW3gw_f3aEvgEmi"] > services=["APPID_SXC"] users=["lva...@hotmail.com"] > cs6=https://abcd-remote.console.arc.com/#/alerts/56fe779ee4b0459f4e9a484a > cs6Label=consoleUrl > ... > {"source.type":"adallom","device_version":"1.0","severity":"0","device_product":"Adallom","services":"[\"APPID_SXC\"]","src_username":"wander...@rock.com","message":"Activity > policy 'User download\/view file' was triggered by > 'scolb...@gmail.com'","users":"[\"lva...@hotmail.com\"]","consoleUrl":"https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a","event_class_id":"56fe779ee4b0459f4e9a484a","original_string":"2016-04-01T09:29:11.356-0400 > > CEF:0|Adallom|Adallom|1.0|56fe779ee4b0459f4e9a484a|ALERT_CABINET_EVENT_MATCH_AUDIT|0|msg=Activity > policy 'User download\/view file' was triggered by 'scolb...@gmail.com' > suser=wander...@rock.com start=1459517280810 end=1459517280810 > audits=[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"] > services=[\"APPID_SXC\"] users=[\"lva...@hotmail.com\"] > cs6=https:\/\/abcd-remote.console.arc.com\/#\/alerts\/56fe779ee4b0459f4e9a484a > cs6Label=consoleUrl","header":"2016-04-01T09:29:11.356-0400 > CEF:0","event_name":"ALERT_CABINET_EVENT_MATCH_AUDIT","startTime":"1459517280810","device_vendor":"Adallom","endTime":"1459517280810","audits":"[\"AVPR-4oIPeFmuZ3CKKrg\",\"AVPR-wx80cd9PUpAu2aj\",\"AVPR-6XGPeFmuZ3CKKvx\",\"AVPSALn_qE4Kgs_8_yK9\",\"AVPSASW3gw_f3aEvgEmi\"]","timestamp":1459502951000} > CyberArk CEF > Mar 21 14:05:02 HHHPVATN1 CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve > password|5|act=Retrieve password suser=spilgrim fname=Root\ABC phobos3 - COMP > dvc=120.99.70.3 shost=10.44.134.78 dhost= duser= externalId= app= reason= > cs1Label="Affected User Name" cs1= cs2Label="Safe Name" cs2=Security > Vulnerability Mgmt cs3Label="Device Type" cs3= cs4Label="Database" cs4= > cs5Label="Other info" cs5=101.198.70.93 cn1Label="Request Id" cn1= > cn2Label="Ticket Id" cn2=Needed to verify config files being pulled > msg=Needed to verify config files being pulled > ... > {"timestamp":1458569102000,"source.type":"cyberark","device_version":"7.20.0091","device_product":"Vault","fileName":"Root\\ABC > phobos3 - COMP","src_username":"spilgrim","\"Other > info\"":"101.198.70.93","\"Ticket Id\"":"Needed to verify config files being > pulled > ","deviceAddress":"120.99.70.3","severity":"5","deviceAction":"Retrieve > password","message":"Needed to verify config files being > pulled","event_class_id":"295","original_string":"Mar 21 14:05:02 HHHPVATN1 > CEF:0|Cyber-Ark|Vault|7.20.0091|295|Retrieve password|5|act=Retrieve password > suser=spilgrim fname=Root\\ABC phobos3 - COMP dvc=120.99.70.3 > shost=10.44.134.78 dhost= duser= externalId= app= reason= cs1Label=\"Affected > User Name\" cs1= cs2Label=\"Safe Name\" cs2=Security Vulnerability Mgmt > cs3Label=\"Device Type\" cs3= cs4Label=\"Database\" cs4= cs5Label=\"Other > info\" cs5=101.198.70.93 cn1Label=\"Request Id\" cn1= cn2Label=\"Ticket Id\" > cn2=Needed to verify config files being pulled msg=Needed to verify config > files being pulled","\"Safe Name\"":"Security Vulnerability > Mgmt","header":"Mar 21 14:05:02 HHHPVATN1 CEF:0","event_name":"Retrieve > password","device_vendor":"Cyber-Ark","src_hostname":"10.44.134.78"} > WAF CEF > <14>CEF:0|Imperva Inc.|SecureSphere|10.0.0.4_16|ABC - Secure Login.vm Page > Rate Limit UK - Source IP||High|act=alert dst=17.43.200.42 dpt=88 > duser=${Alert.username} src=10.31.45.69 spt=34435 proto=TCP rt=31 March 2016 > 13:04:55 cat=Alert cs1= cs1Label=Policy cs2=ABC-Secure cs2Label=ServerGroup > cs3=servers_svc cs3Label=ServiceName cs4=server_app cs4Label=ApplicationName > cs5=QA cs5Label=Description > ... > {"source.type":"waf","device_version":"10.0.0.4_16","severity":"High","device_product":"SecureSphere","ServerGroup":"ABC-Secure","ApplicationName":"server_app","Description":"QA","deviceAction":"alert","ip_dst_port":"88","dst_username":"${Alert.username}","priority":"14","deviceEventCategory":"Alert","protocol":"TCP","ip_dst_addr":"17.43.200.42","ip_src_port":"34435","event_class_id":"ABC > - Secure Login.vm Page Rate Limit UK - Source > IP","ServiceName":"servers_svc","original_string":"<14>CEF:0|Imperva > Inc.|SecureSphere|10.0.0.4_16|ABC - Secure Login.vm Page Rate Limit UK - > Source IP||High|act=alert dst=17.43.200.42 dpt=88 duser=${Alert.username} > src=10.31.45.69 spt=34435 proto=TCP rt=31 March 2016 13:04:55 cat=Alert cs1= > cs1Label=Policy cs2=ABC-Secure cs2Label=ServerGroup cs3=servers_svc > cs3Label=ServiceName cs4=server_app cs4Label=ApplicationName cs5=QA > cs5Label=Description","header":"<14>CEF:0","device_vendor":"Imperva > Inc.","ip_src_addr":"10.31.45.69","timestamp":1459429495000} -- This message was sent by Atlassian JIRA (v7.6.3#76005)