Christian Tramnitz created METRON-940:
-----------------------------------------
Summary: problems with current Palo Alto schema for CEF parser
Key: METRON-940
URL: https://issues.apache.org/jira/browse/METRON-940
Project: Metron
Issue Type: Bug
Affects Versions: 0.4
Environment: full-dev 0.4.0 master
Reporter: Christian Tramnitz
The current Palo Alto parser (schema on top of CEF parser) seems to use a
custom field definition.
As far as I can tell there is no "standard" definition for a CEF message in
PaloAlto as the scheme can be freely defined. However, there is a documented
example and I would suggest to base the Metron upon this documented definition
rather than a custom definition.
Alternatively we could come up with our message format definition for Palo Alto
CEF, but then we need to document what needs to be done on the Firewall to get
these.
This is a sanitized sample message for threat and traffic:
{noformat}
<14>1 2017-05-08T23:22:00+00:00 10.1.1.1 - - - - CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|url|THREAT|1|rt=May 08 2017 23:22:00 GMT
deviceExternalId=00000000000 src=192.168.1.2 dst=10.28.1.1
sourceTranslatedAddress=10.200.0.1 destinationTranslatedAddress=10.28.1.1
cs1Label=Rule cs1=rulename suser= duser= app=ssl cs3Label=Virtual System
cs3=vsys1 cs4Label=Source Zone cs4=private cs5Label=Destination Zone
cs5=untrust deviceInboundInterface=vlan.1001 deviceOutboundInterface=vlan.1
cs6Label=LogProfile cs6=Syslog cn1Label=SessionID cn1=53493 cnt=1 spt=59950
dpt=443 sourceTranslatedPort=30630 destinationTranslatedPort=443
flexString1Label=Flags flexString1=0x40b000 proto=tcp act=alert
request=\"www.example.com/\" cs2Label=URL Category cs2=unknown
flexString2Label=Direction flexString2=client-to-server externalId=9868673
requestContext= cat=(9999) filePath= fileId=0 fileHash=
requestClientApplication= fileType= panosxforwarderfor= panosreferer= suid=
msg= duid= oldFileId=0 PanOSDGl1=16 PanOSDGl2=11 PanOSDGl3=0 PanOSDGl4=0
PanOSVsysName= dvchost=firewall
{noformat}
{noformat}
<14>1 2017-05-08T23:22:00+00:00 10.12.1.1 - - - - CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|drop|TRAFFIC|1|rt=May 08 2017 23:21:59 GMT
deviceExternalId=00000000000 src=100.1.2.3 dst=120.1.2.3
sourceTranslatedAddress=0.0.0.0 destinationTranslatedAddress=0.0.0.0
cs1Label=Rule cs1=DropLog suser= duser= app=not-applicable cs3Label=Virtual
System cs3=vsys1 cs4Label=Source Zone cs4=untrust cs5Label=Destination Zone
cs5=untrust deviceInboundInterface=vlan.1 deviceOutboundInterface=
cs6Label=LogProfile cs6=Syslog cn1Label=SessionID cn1=0 cnt=1 spt=7297 dpt=123
sourceTranslatedPort=0 destinationTranslatedPort=0 flexString1Label=Flags
flexString1=0x0 proto=udp act=deny flexNumber1Label=Total bytes flexNumber1=67
in=67 out=0 cn2Label=Packets cn2=1 PanOSPacketsReceived=0 PanOSPacketsSent=1
start=May 08 2017 23:21:59 GMT cn3Label=Elapsed time in seconds cn3=0
cs2Label=URL Category cs2=any externalId=3342330262 reason=policy-deny
PanOSDGl1=16 PanOSDGl2=11 PanOSDGl3=0 PanOSDGl4=0 PanOSVsysName=
dvchost=firewall cat=from-policy
{noformat}
Using the following definitions:
{noformat}
Traffic:
CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|$subtype|$type|1|rt=$cefformatted-receive_time
deviceExternalId=$serial src=$src dst=$dst sourceTranslatedAddress=$natsrc
destinationTranslatedAddress=$natdst cs1Label=Rule cs1=$rule suser=$srcuser
duser=$dstuser app=$app cs3Label=Virtual System cs3=$vsys cs4Label=Source Zone
cs4=$from cs5Label=Destination Zone cs5=$to deviceInboundInterface=$inbound_if
deviceOutboundInterface=$outbound_if cs6Label=LogProfile cs6=$logset
cn1Label=SessionID cn1=$sessionid cnt=$repeatcnt spt=$sport dpt=$dport
sourceTranslatedPort=$natsport destinationTranslatedPort=$natdport
flexString1Label=Flags flexString1=$flags proto=$proto act=$action
flexNumber1Label=Total bytes flexNumber1=$bytes in=$bytes_sent
out=$bytes_received cn2Label=Packets cn2=$packets
PanOSPacketsReceived=$pkts_received PanOSPacketsSent=$pkts_sent
start=$cef-formatted-time_generated cn3Label=Elapsed time in seconds
cn3=$elapsed cs2Label=URL Category cs2=$category externalId=$seqno
reason=$session_end_reason PanOSDGl1=$dg_hier_level_1
PanOSDGl2=$dg_hier_level_2 PanOSDGl3=$dg_hier_level_3
PanOSDGl4=$dg_hier_level_4 PanOSVsysName=$vsys_name dvchost=$device_name
cat=$action_source
Threat:
CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|$subtype|$type|$number-ofseverity|rt=$cef-formatted-receive_time
deviceExternalId=$serial src=$src dst=$dst sourceTranslatedAddress=$natsrc
destinationTranslatedAddress=$natdst cs1Label=Rule cs1=$rule suser=$srcuser
duser=$dstuser app=$app cs3Label=Virtual System cs3=$vsys cs4Label=Source Zone
cs4=$from cs5Label=Destination Zone cs5=$to deviceInboundInterface=$inbound_if
deviceOutboundInterface=$outbound_if cs6Label=LogProfile cs6=$logset
cn1Label=SessionID cn1=$sessionid cnt=$repeatcnt spt=$sport dpt=$dport
sourceTranslatedPort=$natsport destinationTranslatedPort=$natdport
flexString1Label=Flags flexString1=$flags proto=$proto act=$action
request=$misc cs2Label=URL Category cs2=$category flexString2Label=Direction
flexString2=$direction externalId=$seqno requestContext=$contenttype
cat=$threatid filePath=$cloud fileId=$pcap_id fileHash=$filedigest
requestClientApplication=$user_agent fileType=$filetype panosxforwarderfor=$xff
panosreferer=$referer suid=$sender msg=$subject duid=$recipient
oldFileId=$reportid PanOSDGl1=$dg_hier_level_1 PanOSDGl2=$dg_hier_level_2
PanOSDGl3=$dg_hier_level_3 PanOSDGl4=$dg_hier_level_4 PanOSVsysName=$vsys_name
dvchost=$device_name
Config:
CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|$result|$type|1|rt=$cefformatted-receive_time
deviceExternalId=$serial shost=$host cs3Label=Virtual System cs3=$vsys act=$cmd
duser=$admin destinationServiceName=$client msg=$path externalId=$seqno
PanOSDGl1=$dg_hier_level_1 PanOSDGl2=$dg_hier_level_2
PanOSDGl3=$dg_hier_level_3 PanOSDGl4=$dg_hier_level_4 PanOSVsysName=$vsys_name
dvchost=$device_name Optional: cs1Label=Before Change Detail
cs1=$before-change-detail cs2Label=After Change Detail cs2=$after-change-detail
System:
CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|$subtype|$type|$number-ofseverity|rt=$cef-formatted-receive_time
deviceExternalId=$serial cs3Label=Virtual System cs3=$vsys fname=$object
flexString2Label=Module flexString2=$module msg=$opaque externalId=$seqno
cat=$eventid PanOSDGl1=$dg_hier_level_1 PanOSDGl2=$dg_hier_level_2
PanOSDGl3=$dg_hier_level_3 PanOSDGl4=$dg_hier_level_4 PanOSVsysName=$vsys_name
dvchost=$device_name
HIP Match:
CEF:0|Palo Alto
Networks|PAN-OS|7.0.0|$matchtype|$type|1|rt=$cefformatted-receive_time
deviceExternalId=$serial suser=$srcuser cs3Label=Virtual System cs3=$vsys
shost=$machinename src=$src cnt=$repeatcnt externalId=$seqno cat=$matchname
cs2Label=Operating System cs2=$os PanOSDGl1=$dg_hier_level_1
PanOSDGl2=$dg_hier_level_2 PanOSDGl3=$dg_hier_level_3
PanOSDGl4=$dg_hier_level_4 PanOSVsysName=$vsys_name dvchost=$device_name
{noformat}
as per attached CEF example documentation from Palo Alto (I'm attaching
documentation for version 6.1 and 7.0, 7.0 still works with 7.1 while PAN-OS is
untested for now)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
