[
https://issues.apache.org/jira/browse/METRON-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Austin updated METRON-181:
-------------------------------
Description:
Create a Parser for Infoblox log lines. This data source has several different
formats that should be parsed as specified below:
- DNS query message:
<30>Mar 31 13:48:57 10.216.9.216 named[19446]: client 10.216.65.24#59335
(stmt-filenet-ich-server.data.org): query:
stmt-stmt-filenet-ich-server.data.org IN A + (10.216.9.215)
...
{"process_id":"19446","process":"named","dns_record_type":"IN","dns_query":"stmt-filenet-ich-server.data.org","source.type":"Infoblox","ip_address":"10.216.9.216","priority":"30","dns_result":"success","original_string":"<30>Mar
31 13:48:57 10.216.9.216 named[19446]: client 10.216.65.24#59335
(stmt-filenet-ich-server.data.org): query: stmt-filenet-ich-server.data.org IN
A +
(10.216.9.215)","ip_src_port":"59335","dns_server_interface":"10.216.9.215","dns_action_type":"query","dns_bind_parameters":"+","ip_src_addr":"10.216.65.24","timestamp":1459432137000}
- DNS Error message:
<30>Mar 31 09:48:59 19.244.24.107 named[22289]: error (unexpected RCODE
REFUSED) resolving '171.43.244.14.IN-ADDR.ARPA/PTR/IN': 218.178.70.134#53
...
{"process_id":"22289","process":"named","dns_record_type":"PTR","dns_query":"171.43.244.14.IN-ADDR.ARPA","dns_forward_server":"218.178.70.134","source.type":"Infoblox","ip_address":"19.244.24.107","priority":"30","dns_result":"error","original_string":"<30>Mar
31 09:48:59 19.244.24.107 named[22289]: error (unexpected RCODE REFUSED)
resolving '171.43.244.14.IN-ADDR.ARPA\/PTR\/IN':
218.178.70.134#53","dns_forward_port":"53","dns_forward_return_code":"REFUSED","dns_action_type":"query","timestamp":1459417739000}
- DNS Zone Update message:
<30>Mar 31 09:48:59 101.37.216.113 named[22628]: zone
...
{"process_id":"22628","process":"named","original_string":"<30>Mar 31 09:48:59
101.37.216.113 named[22628]:
zone","source.type":"Infoblox","ip_address":"101.37.216.113","priority":"30","dns_action_type":"zone_update","timestamp":1459417739000}
- DNS Update failure message:
<27>Mar 31 09:48:54 101.37.216.136 named[4018]: client 101.155.18.101#61440:
update 'sharedbuilder.com/IN' denied
...
{"process_id":"4018","process":"named","src_ip_addr":"101.155.18.101","dns_record_type":"IN","dns_update_target":"sharedbuilder.com","source.type":"Infoblox","ip_address":"101.37.216.136","priority":"27","src_ip_port":"61440","dns_result":"denied","original_string":"<27>Mar
31 09:48:54 101.37.216.136 named[4018]: client 101.155.18.101#61440: update
'sharedbuilder.com\/IN'
denied","dns_action_type":"update","timestamp":1459417734000}
- DNS Update success message:
<30>Mar 31 09:48:58 101.101.11.143 named[12172]: client 101.14.113.105#6714/key
dhcp_updater_default: updating zone '218.110.in-addr.arpa/IN': adding an RR at
'151.147.218.210.in-addr.arpa' PTR
...
{"process_id":"12172","process":"named","src_ip_addr":"101.14.113.105","dns_auth_keyname":"dhcp_updater_default","source.type":"Infoblox","ip_address":"101.101.11.143","priority":"30","dns_update_message":"updating
zone '218.110.in-addr.arpa\/IN': adding an RR at
'151.147.218.210.in-addr.arpa'
PTR","src_ip_port":"6714","original_string":"<30>Mar 31 09:48:58 101.101.11.143
named[12172]: client 101.14.113.105#6714\/key dhcp_updater_default: updating
zone '218.110.in-addr.arpa\/IN': adding an RR at '15.147.218.210.in-addr.arpa'
PTR","timestamp":1459417738000}
- DHCP Request message:
<30>Mar 31 09:48:59 101.24.112.103 dhcpd[6947]: DHCPREQUEST for 101.116.73.120
from 01:1b:53:51:6d:12 (SEP001B535C6DE2) via 101.116.72.222 uid
01:10:11:53:5c:61:e2 (RENEW)
...
{"process_id":"6947","src_mac":"01:1b:53:51:6d:12","process":"dhcpd","dhcp_relay_ip":"101.116.72.222","dhcp_type":"DHCPREQUEST","dhcp_uid":"01:10:11:53:5c:61:e2","dhcp_options":"RENEW","source.type":"Infoblox","ip_address":"101.24.112.103","message":"DHCPREQUEST
for 101.116.73.120 from 01:1b:53:51:6d:12 (SEP001B535C6DE2) via 101.116.72.222
uid 01:10:11:53:5c:61:e2
(RENEW)","priority":"30","dhcp_hostname":"SEP001B535C6DE2","original_string":"<30>Mar
31 09:48:59 101.24.112.103 dhcpd[6947]: DHCPREQUEST for 101.116.73.120 from
01:1b:53:51:6d:12 (SEP001B535C6DE2) via 101.116.72.222 uid 01:10:11:53:5c:61:e2
(RENEW)","dhcp_requested_ip":"101.116.73.120","timestamp":1459417739000}
- DHCP Acknowledgement message:
<30>Mar 31 09:48:59 101.14.123.101 dhcpd[17697]: DHCPACK on 101.115.176.151 to
01:17:95:e2:05:c4 (SEP0017955205C4) via eth2 relay 101.115.76.213
lease-duration 691084 (RENEW) uid 01:0d:17:95:52:15:c4
...
{"process_id":"17697","process":"dhcpd","dhcp_relay_ip":"101.115.76.213","dhcp_type":"DHCPACK","dhcp_uid":"01:0d:17:95:52:15:c4","dhcp_options":"RENEW","source.type":"Infoblox","ip_address":"101.14.123.101","message":"DHCPACK
on 101.115.176.151 to 01:17:95:e2:05:c4 (SEP0017955205C4) via eth2 relay
101.115.76.213 lease-duration 691084 (RENEW) uid
01:0d:17:95:52:15:c4","priority":"30","dst_mac":"01:0d:17:95:52:15:c4","dhcp_hostname":"SEP0017955205C4","original_string":"<30>Mar
31 09:48:59 101.14.123.101 dhcpd[17697]: DHCPACK on 101.115.176.151 to
01:17:95:e2:05:c4 (SEP0017955205C4) via eth2 relay 101.115.76.213
lease-duration 691084 (RENEW) uid
01:0d:17:95:52:15:c","dhcp_lease_duration":"691084","dhcp_interface":"eth2","timestamp":1459417739000}
was:
Create a Parser for Infoblox log lines. This data source has several different
formats that should be parsed as specified below:
- DNS query message:
<30>Mar 31 13:48:57 10.26.9.26 named[19446]: client 10.26.65.240#59335
(stmt-filenet-nch-server.uscards.cof): query:
stmt-filenet-nch-server.uscards.cof IN A + (10.26.9.25)
...
{"process_id":"19446","process":"named","dns_record_type":"IN","dns_query":"stmt-filenet-nch-server.uscards.cof","source.type":"Infoblox","ip_address":"10.26.9.26","priority":"30","dns_result":"success","original_string":"<30>Mar
31 13:48:57 10.26.9.26 named[19446]: client 10.26.65.240#59335
(stmt-filenet-nch-server.uscards.cof): query:
stmt-filenet-nch-server.uscards.cof IN A +
(10.26.9.25)","ip_src_port":"59335","dns_server_interface":"10.26.9.25","dns_action_type":"query","dns_bind_parameters":"+","ip_src_addr":"10.26.65.240","timestamp":1459432137000}
- DNS Error message:
<30>Mar 31 09:48:59 199.244.214.107 named[22289]: error (unexpected RCODE
REFUSED) resolving '71.43.244.104.IN-ADDR.ARPA/PTR/IN': 208.78.70.34#53
...
{"process_id":"22289","process":"named","dns_record_type":"PTR","dns_query":"71.43.244.104.IN-ADDR.ARPA","dns_forward_server":"208.78.70.34","source.type":"Infoblox","ip_address":"199.244.214.107","priority":"30","dns_result":"error","original_string":"<30>Mar
31 09:48:59 199.244.214.107 named[22289]: error (unexpected RCODE REFUSED)
resolving '71.43.244.104.IN-ADDR.ARPA\/PTR\/IN':
208.78.70.34#53","dns_forward_port":"53","dns_forward_return_code":"REFUSED","dns_action_type":"query","timestamp":1459417739000}
- DNS Zone Update message:
<30>Mar 31 09:48:59 10.37.216.13 named[22628]: zone
...
{"process_id":"22628","process":"named","original_string":"<30>Mar 31 09:48:59
10.37.216.13 named[22628]:
zone","source.type":"Infoblox","ip_address":"10.37.216.13","priority":"30","dns_action_type":"zone_update","timestamp":1459417739000}
- DNS Update failure message:
<27>Mar 31 09:48:54 10.37.216.36 named[4018]: client 10.155.8.101#61440: update
'sharebuilder.com/IN' denied
...
{"process_id":"4018","process":"named","src_ip_addr":"10.155.8.101","dns_record_type":"IN","dns_update_target":"sharebuilder.com","source.type":"Infoblox","ip_address":"10.37.216.36","priority":"27","src_ip_port":"61440","dns_result":"denied","original_string":"<27>Mar
31 09:48:54 10.37.216.36 named[4018]: client 10.155.8.101#61440: update
'sharebuilder.com\/IN'
denied","dns_action_type":"update","timestamp":1459417734000}
- DNS Update success message:
<30>Mar 31 09:48:58 10.10.1.43 named[12172]: client 10.14.3.105#6714/key
dhcp_updater_default: updating zone '218.10.in-addr.arpa/IN': adding an RR at
'15.147.218.10.in-addr.arpa' PTR
...
{"process_id":"12172","process":"named","src_ip_addr":"10.14.3.105","dns_auth_keyname":"dhcp_updater_default","source.type":"Infoblox","ip_address":"10.10.1.43","priority":"30","dns_update_message":"updating
zone '218.10.in-addr.arpa\/IN': adding an RR at '15.147.218.10.in-addr.arpa'
PTR","src_ip_port":"6714","original_string":"<30>Mar 31 09:48:58 10.10.1.43
named[12172]: client 10.14.3.105#6714\/key dhcp_updater_default: updating zone
'218.10.in-addr.arpa\/IN': adding an RR at '15.147.218.10.in-addr.arpa'
PTR","timestamp":1459417738000}
- DHCP Request message:
<30>Mar 31 09:48:59 10.24.2.103 dhcpd[6947]: DHCPREQUEST for 10.116.73.20 from
00:1b:53:5c:6d:e2 (SEP001B535C6DE2) via 10.116.72.2 uid 01:00:1b:53:5c:6d:e2
(RENEW)
...
{"process_id":"6947","src_mac":"00:1b:53:5c:6d:e2","process":"dhcpd","dhcp_relay_ip":"10.116.72.2","dhcp_type":"DHCPREQUEST","dhcp_uid":"01:00:1b:53:5c:6d:e2","dhcp_options":"RENEW","source.type":"Infoblox","ip_address":"10.24.2.103","message":"DHCPREQUEST
for 10.116.73.20 from 00:1b:53:5c:6d:e2 (SEP001B535C6DE2) via 10.116.72.2 uid
01:00:1b:53:5c:6d:e2
(RENEW)","priority":"30","dhcp_hostname":"SEP001B535C6DE2","original_string":"<30>Mar
31 09:48:59 10.24.2.103 dhcpd[6947]: DHCPREQUEST for 10.116.73.20 from
00:1b:53:5c:6d:e2 (SEP001B535C6DE2) via 10.116.72.2 uid 01:00:1b:53:5c:6d:e2
(RENEW)","dhcp_requested_ip":"10.116.73.20","timestamp":1459417739000}
- DHCP Acknowledgement message:
<30>Mar 31 09:48:59 10.14.3.101 dhcpd[17697]: DHCPACK on 10.115.76.151 to
00:17:95:52:05:c4 (SEP0017955205C4) via eth2 relay 10.115.76.3 lease-duration
691084 (RENEW) uid 01:00:17:95:52:05:c4
...
<30>Mar 31 09:48:59 10.14.3.101 dhcpd[17697]: DHCPACK on 10.115.76.151 to
00:17:95:52:05:c4 (SEP0017955205C4) via eth2 relay 10.115.76.3 lease-duration
691084 (RENEW) uid 01:00:17:95:52:05:c4
> Create Infoblox Parser
> ----------------------
>
> Key: METRON-181
> URL: https://issues.apache.org/jira/browse/METRON-181
> Project: Metron
> Issue Type: New Feature
> Reporter: Phil Austin
> Priority: Minor
>
> Create a Parser for Infoblox log lines. This data source has several
> different formats that should be parsed as specified below:
> - DNS query message:
> <30>Mar 31 13:48:57 10.216.9.216 named[19446]: client 10.216.65.24#59335
> (stmt-filenet-ich-server.data.org): query:
> stmt-stmt-filenet-ich-server.data.org IN A + (10.216.9.215)
> ...
> {"process_id":"19446","process":"named","dns_record_type":"IN","dns_query":"stmt-filenet-ich-server.data.org","source.type":"Infoblox","ip_address":"10.216.9.216","priority":"30","dns_result":"success","original_string":"<30>Mar
> 31 13:48:57 10.216.9.216 named[19446]: client 10.216.65.24#59335
> (stmt-filenet-ich-server.data.org): query: stmt-filenet-ich-server.data.org
> IN A +
> (10.216.9.215)","ip_src_port":"59335","dns_server_interface":"10.216.9.215","dns_action_type":"query","dns_bind_parameters":"+","ip_src_addr":"10.216.65.24","timestamp":1459432137000}
> - DNS Error message:
> <30>Mar 31 09:48:59 19.244.24.107 named[22289]: error (unexpected RCODE
> REFUSED) resolving '171.43.244.14.IN-ADDR.ARPA/PTR/IN': 218.178.70.134#53
> ...
> {"process_id":"22289","process":"named","dns_record_type":"PTR","dns_query":"171.43.244.14.IN-ADDR.ARPA","dns_forward_server":"218.178.70.134","source.type":"Infoblox","ip_address":"19.244.24.107","priority":"30","dns_result":"error","original_string":"<30>Mar
> 31 09:48:59 19.244.24.107 named[22289]: error (unexpected RCODE REFUSED)
> resolving '171.43.244.14.IN-ADDR.ARPA\/PTR\/IN':
> 218.178.70.134#53","dns_forward_port":"53","dns_forward_return_code":"REFUSED","dns_action_type":"query","timestamp":1459417739000}
> - DNS Zone Update message:
> <30>Mar 31 09:48:59 101.37.216.113 named[22628]: zone
> ...
> {"process_id":"22628","process":"named","original_string":"<30>Mar 31
> 09:48:59 101.37.216.113 named[22628]:
> zone","source.type":"Infoblox","ip_address":"101.37.216.113","priority":"30","dns_action_type":"zone_update","timestamp":1459417739000}
> - DNS Update failure message:
> <27>Mar 31 09:48:54 101.37.216.136 named[4018]: client 101.155.18.101#61440:
> update 'sharedbuilder.com/IN' denied
> ...
> {"process_id":"4018","process":"named","src_ip_addr":"101.155.18.101","dns_record_type":"IN","dns_update_target":"sharedbuilder.com","source.type":"Infoblox","ip_address":"101.37.216.136","priority":"27","src_ip_port":"61440","dns_result":"denied","original_string":"<27>Mar
> 31 09:48:54 101.37.216.136 named[4018]: client 101.155.18.101#61440: update
> 'sharedbuilder.com\/IN'
> denied","dns_action_type":"update","timestamp":1459417734000}
> - DNS Update success message:
> <30>Mar 31 09:48:58 101.101.11.143 named[12172]: client
> 101.14.113.105#6714/key dhcp_updater_default: updating zone
> '218.110.in-addr.arpa/IN': adding an RR at '151.147.218.210.in-addr.arpa' PTR
> ...
> {"process_id":"12172","process":"named","src_ip_addr":"101.14.113.105","dns_auth_keyname":"dhcp_updater_default","source.type":"Infoblox","ip_address":"101.101.11.143","priority":"30","dns_update_message":"updating
> zone '218.110.in-addr.arpa\/IN': adding an RR at
> '151.147.218.210.in-addr.arpa'
> PTR","src_ip_port":"6714","original_string":"<30>Mar 31 09:48:58
> 101.101.11.143 named[12172]: client 101.14.113.105#6714\/key
> dhcp_updater_default: updating zone '218.110.in-addr.arpa\/IN': adding an RR
> at '15.147.218.210.in-addr.arpa' PTR","timestamp":1459417738000}
> - DHCP Request message:
> <30>Mar 31 09:48:59 101.24.112.103 dhcpd[6947]: DHCPREQUEST for
> 101.116.73.120 from 01:1b:53:51:6d:12 (SEP001B535C6DE2) via 101.116.72.222
> uid 01:10:11:53:5c:61:e2 (RENEW)
> ...
> {"process_id":"6947","src_mac":"01:1b:53:51:6d:12","process":"dhcpd","dhcp_relay_ip":"101.116.72.222","dhcp_type":"DHCPREQUEST","dhcp_uid":"01:10:11:53:5c:61:e2","dhcp_options":"RENEW","source.type":"Infoblox","ip_address":"101.24.112.103","message":"DHCPREQUEST
> for 101.116.73.120 from 01:1b:53:51:6d:12 (SEP001B535C6DE2) via
> 101.116.72.222 uid 01:10:11:53:5c:61:e2
> (RENEW)","priority":"30","dhcp_hostname":"SEP001B535C6DE2","original_string":"<30>Mar
> 31 09:48:59 101.24.112.103 dhcpd[6947]: DHCPREQUEST for 101.116.73.120 from
> 01:1b:53:51:6d:12 (SEP001B535C6DE2) via 101.116.72.222 uid
> 01:10:11:53:5c:61:e2
> (RENEW)","dhcp_requested_ip":"101.116.73.120","timestamp":1459417739000}
> - DHCP Acknowledgement message:
> <30>Mar 31 09:48:59 101.14.123.101 dhcpd[17697]: DHCPACK on 101.115.176.151
> to 01:17:95:e2:05:c4 (SEP0017955205C4) via eth2 relay 101.115.76.213
> lease-duration 691084 (RENEW) uid 01:0d:17:95:52:15:c4
> ...
> {"process_id":"17697","process":"dhcpd","dhcp_relay_ip":"101.115.76.213","dhcp_type":"DHCPACK","dhcp_uid":"01:0d:17:95:52:15:c4","dhcp_options":"RENEW","source.type":"Infoblox","ip_address":"101.14.123.101","message":"DHCPACK
> on 101.115.176.151 to 01:17:95:e2:05:c4 (SEP0017955205C4) via eth2 relay
> 101.115.76.213 lease-duration 691084 (RENEW) uid
> 01:0d:17:95:52:15:c4","priority":"30","dst_mac":"01:0d:17:95:52:15:c4","dhcp_hostname":"SEP0017955205C4","original_string":"<30>Mar
> 31 09:48:59 101.14.123.101 dhcpd[17697]: DHCPACK on 101.115.176.151 to
> 01:17:95:e2:05:c4 (SEP0017955205C4) via eth2 relay 101.115.76.213
> lease-duration 691084 (RENEW) uid
> 01:0d:17:95:52:15:c","dhcp_lease_duration":"691084","dhcp_interface":"eth2","timestamp":1459417739000}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
