Re: [pmacct-discussion] pmaactd bgp as-path missing on sql inserts
Hey Paolo, Thanks for all your help. Those suggestions did the trick! Best, --Derrick ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] pmaactd bgp as-path missing on sql inserts
Hi Derrick, Perfect. About your questions: 1. src_host, dst_host and src_net, dst_net are mutual exclusive in the sense they are multiplexed on the same field. If you remove src_net and dst_net from the 'aggregate' you will get individual hosts. If you also need IP prefixes readily available (ie. you don't want to bother applying masks to IP addresses) you can fire a second plugin that accounts on IP prefixes (so remove src_host, dst_host from the 'aggregate' there). Let me know if this would work for you. 2. Sure: you are looking for bgp_agent_map directive. Look CONFIG-KEYS and 'examples/agent_to_peer.map.example' for further info. Cheers, Paolo On Tue, Oct 29, 2013 at 07:37:27PM -0700, Derrick Sawyer wrote: Hey Paolo, My apologies, that did the trick. I thought I had all the necessary configurations but overlooked that one. Your right as-path is working but the src as-path is what I really needed. I also have another couple of questions: 1. How do I obtain the actual src/dst host IP? What is being captured is the IP network not the IP host. 2. Is there a way to map bgp lookups? I have several routers and would like particular flows from a router to have pmacctd do the the src as-path lookup on that router. I can post these questions in a new thread if that is preferable. Also thanks for the quick response! Best, --Derrick ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] pmaactd bgp as-path missing on sql inserts
Hi Derrick, Excellent capturing of information of yours. From the SQL inserts you posted it's only evident that the src_as_path is not working properly - ie. does not say anything about as_path: you sure you did verify the same problem with that primitive? If yes, can you post something about it? About src_as_path, i see you miss the following statement in the config: bgp_src_as_path_type: bgp Can you please add it and let me know if then it appears to work OK? Cheers, Paolo On Mon, Oct 28, 2013 at 04:19:07PM -0700, Derrick Sawyer wrote: Hi, I am using pmacct 1.5.0rc1 and running into a issue in which the as-path lookup via BGP is not being inserted into postgres. *Configuration:* ! Defaults debug: true daemonize: false plugins: pgsql[5mins], pgsql[hourly] nfacctd_port: 7000 nfacctd_time_new: true interface: eth0 nfacctd_as_new: bgp nfacctd_net: bgp nfacctd_peer_as: true nfacctd_renormalize: true plugin_buffer_size: 10240 plugin_pipe_size: 1024000 geoip_ipv4_file: /usr/share/GeoIP/GeoIP.dat geoip_ipv6_file: /usr/share/GeoIP/GeoIPv6.dat pkt_len_distrib_bins: 0-199,200-399,400-599,600-799,800-999,1000-1499,1500-9000 !BGP bgp_daemon: true bgp_daemon_ip: x.x.x.x bgp_daemon_max_peers: 100 bgp_aspath_radius: 15 bgp_peer_src_as_type: bgp bgp_agent_map: /opt/src/pmacctd/etc/pmacct-agent_bgp.map pre_tag_map: /opt/src/pmacctd/etc/pretag.map bgp_daemon_msglog: true !SQL sql_user: pmacctd sql_passwd: sql_optimize_clauses: true sql_dont_try_update: true sql_table_type: bgp aggregate: src_mac, dst_mac, src_host, dst_host, src_net, dst_net, src_mask, dst_mask, src_as, dst_as, src_port, dst_port, tos, proto, flows, tag, tcpflags, in_iface, out_iface, as_path, sampling_rate, src_host_country, dst_host_country, pkt_len_distrib, timestamp_start, timestamp_end, src_as_path !5 min sql_refresh_time[5mins]: 60 sql_history[5mins]: 1m sql_history_roundoff[5mins]: m sql_table[5mins]: acct_5mins !1 hour sql_refresh_time[hourly]: 3600 sql_history[hourly]: 1h sql_history_roundoff[hourly]: h sql_table[hourly]: acct_hourly *BGP msglog:* INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '198.206.8.0/21' Path: '65501 1299 3257 4436 29761 36352' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] w Prefix: '8.20.2.0/24' Path: '65501 2914 174 35873' Comms: '' EComms: '' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '192.58.232.0/24' Path: '65501 2914 209 6629' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '198.206.8.0/21' Path: '65501 2914 4436 29761 36352' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '192.58.232.0/24' Path: '65501 2914 3356 6629 6629 6629 6629 6629' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '80.67.188.0/24' Path: '65501 1299 3257 42456 60197' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '89.234.141.0/24' Path: '65501 1299 42456 60630' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '80.67.188.0/24' Path: '65501 1299 42456 60197' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] w Prefix: '186.235.48.0/21' Path: '65501 1299 701 7738 263096' Comms: '' EComms: '' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '186.235.48.0/21' Path: '65501 1299 701 7738 263096' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] w Prefix: '186.235.48.0/21' Path: '65501 1299 701 7738 263096' Comms: '' EComms: '' DEBUG ( default/core/BGP ): [Id: x.x.x.x] BGP_KEEPALIVE received DEBUG ( default/core/BGP ): [Id: x.x.x.x] BGP_KEEPALIVE sent *SQL insert:* INFO ( 5mins/pgsql ): *** Purging cache - START *** DEBUG ( 5mins/pgsql ): INSERT INTO acct_5mins (stamp_updated, stamp_inserted, mac_src, mac_dst, ip_src, ip_dst, as_src, iface_in, iface_out, mask_src, mask_dst, as_dst, as_path, as_path_src, port_src, port_dst, tcp_flags, tos, ip_proto, country_ip_src, country_ip_dst, sampling_rate, pkt_len_distrib, timestamp_start, timestamp_start_residual, timestamp_end, timestamp_end_residual, agent_id, packets, bytes, flows) VALUES (ABSTIME(138301)::Timestamp, ABSTIME(1382999700)::Timestamp, '00:00:00:00:00:00', '00:00:00:00:00:00', '199.7.69.0', 'x.x.x.x', 12008, 564, 698, 24, 27, 0, '', '', 53, 50346, 0, 0, 17, 'US', '--', 1, '0-199', ABSTIME(1383000342)::Timestamp, 17900, ABSTIME(1383000342)::Timestamp, 17900, 1, 1, 62, 1) DEBUG ( 5mins/pgsql ): INSERT INTO acct_5mins (stamp_updated, stamp_inserted, mac_src, mac_dst, ip_src, ip_dst, as_src, iface_in, iface_out, mask_src, mask_dst, as_dst, as_path, as_path_src, port_src, port_dst,
Re: [pmacct-discussion] pmaactd bgp as-path missing on sql inserts
Hey Paolo, My apologies, that did the trick. I thought I had all the necessary configurations but overlooked that one. Your right as-path is working but the src as-path is what I really needed. I also have another couple of questions: 1. How do I obtain the actual src/dst host IP? What is being captured is the IP network not the IP host. 2. Is there a way to map bgp lookups? I have several routers and would like particular flows from a router to have pmacctd do the the src as-path lookup on that router. I can post these questions in a new thread if that is preferable. Also thanks for the quick response! Best, --Derrick ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] pmaactd bgp as-path missing on sql inserts
Hi, I am using pmacct 1.5.0rc1 and running into a issue in which the as-path lookup via BGP is not being inserted into postgres. *Configuration:* ! Defaults debug: true daemonize: false plugins: pgsql[5mins], pgsql[hourly] nfacctd_port: 7000 nfacctd_time_new: true interface: eth0 nfacctd_as_new: bgp nfacctd_net: bgp nfacctd_peer_as: true nfacctd_renormalize: true plugin_buffer_size: 10240 plugin_pipe_size: 1024000 geoip_ipv4_file: /usr/share/GeoIP/GeoIP.dat geoip_ipv6_file: /usr/share/GeoIP/GeoIPv6.dat pkt_len_distrib_bins: 0-199,200-399,400-599,600-799,800-999,1000-1499,1500-9000 !BGP bgp_daemon: true bgp_daemon_ip: x.x.x.x bgp_daemon_max_peers: 100 bgp_aspath_radius: 15 bgp_peer_src_as_type: bgp bgp_agent_map: /opt/src/pmacctd/etc/pmacct-agent_bgp.map pre_tag_map: /opt/src/pmacctd/etc/pretag.map bgp_daemon_msglog: true !SQL sql_user: pmacctd sql_passwd: sql_optimize_clauses: true sql_dont_try_update: true sql_table_type: bgp aggregate: src_mac, dst_mac, src_host, dst_host, src_net, dst_net, src_mask, dst_mask, src_as, dst_as, src_port, dst_port, tos, proto, flows, tag, tcpflags, in_iface, out_iface, as_path, sampling_rate, src_host_country, dst_host_country, pkt_len_distrib, timestamp_start, timestamp_end, src_as_path !5 min sql_refresh_time[5mins]: 60 sql_history[5mins]: 1m sql_history_roundoff[5mins]: m sql_table[5mins]: acct_5mins !1 hour sql_refresh_time[hourly]: 3600 sql_history[hourly]: 1h sql_history_roundoff[hourly]: h sql_table[hourly]: acct_hourly *BGP msglog:* INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '198.206.8.0/21' Path: '65501 1299 3257 4436 29761 36352' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] w Prefix: '8.20.2.0/24' Path: '65501 2914 174 35873' Comms: '' EComms: '' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '192.58.232.0/24' Path: '65501 2914 209 6629' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '198.206.8.0/21' Path: '65501 2914 4436 29761 36352' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '192.58.232.0/24' Path: '65501 2914 3356 6629 6629 6629 6629 6629' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '80.67.188.0/24' Path: '65501 1299 3257 42456 60197' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '89.234.141.0/24' Path: '65501 1299 42456 60630' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '80.67.188.0/24' Path: '65501 1299 42456 60197' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] w Prefix: '186.235.48.0/21' Path: '65501 1299 701 7738 263096' Comms: '' EComms: '' INFO ( default/core/BGP ): [Id: x.x.x.x] u Prefix: '186.235.48.0/21' Path: '65501 1299 701 7738 263096' Comms: '' EComms: '' LP: '100' MED: '0' Nexthop: 'x.x.x.x' INFO ( default/core/BGP ): [Id: x.x.x.x] w Prefix: '186.235.48.0/21' Path: '65501 1299 701 7738 263096' Comms: '' EComms: '' DEBUG ( default/core/BGP ): [Id: x.x.x.x] BGP_KEEPALIVE received DEBUG ( default/core/BGP ): [Id: x.x.x.x] BGP_KEEPALIVE sent *SQL insert:* INFO ( 5mins/pgsql ): *** Purging cache - START *** DEBUG ( 5mins/pgsql ): INSERT INTO acct_5mins (stamp_updated, stamp_inserted, mac_src, mac_dst, ip_src, ip_dst, as_src, iface_in, iface_out, mask_src, mask_dst, as_dst, as_path, as_path_src, port_src, port_dst, tcp_flags, tos, ip_proto, country_ip_src, country_ip_dst, sampling_rate, pkt_len_distrib, timestamp_start, timestamp_start_residual, timestamp_end, timestamp_end_residual, agent_id, packets, bytes, flows) VALUES (ABSTIME(138301)::Timestamp, ABSTIME(1382999700)::Timestamp, '00:00:00:00:00:00', '00:00:00:00:00:00', '199.7.69.0', 'x.x.x.x', 12008, 564, 698, 24, 27, 0, '', '', 53, 50346, 0, 0, 17, 'US', '--', 1, '0-199', ABSTIME(1383000342)::Timestamp, 17900, ABSTIME(1383000342)::Timestamp, 17900, 1, 1, 62, 1) DEBUG ( 5mins/pgsql ): INSERT INTO acct_5mins (stamp_updated, stamp_inserted, mac_src, mac_dst, ip_src, ip_dst, as_src, iface_in, iface_out, mask_src, mask_dst, as_dst, as_path, as_path_src, port_src, port_dst, tcp_flags, tos, ip_proto, country_ip_src, country_ip_dst, sampling_rate, pkt_len_distrib, timestamp_start, timestamp_start_residual, timestamp_end, timestamp_end_residual, agent_id, packets, bytes, flows) VALUES (ABSTIME(138301)::Timestamp, ABSTIME(1382999700)::Timestamp, '00:00:00:00:00:00', '00:00:00:00:00:00', '64.34.160.0', 'x.x.x.x', 13768, 564, 698, 20, 27, 0, '', '', 53, 44730, 0, 0, 17, 'US', '--', 1, '0-199', ABSTIME(1383000347)::Timestamp, 53200, ABSTIME(1383000347)::Timestamp, 53200, 1, 1, 182, 1) DEBUG ( 5mins/pgsql ): INSERT INTO acct_5mins (stamp_updated, stamp_inserted, mac_src, mac_dst, ip_src, ip_dst, as_src, iface_in, iface_out,
