Hi Noriyuki-san, Good to know all worked with the peer_src_as. Wrt the equivalent feature for AS-PATH, it seems you never aggregate over the src_as_path primitive in your config:
aggregate[inbound]: src_host, dst_host, src_as, peer_src_as, peer_src_ip, as_path Here as_path should be replaced by src_as_path. Give it a try and let me know if it works. You are right with your observation about BGP data & restart. Although the bgp_table_dump_refresh_time setting should have no influence as that is the time interval for pmacct to write BGP tables to some output file/ AMQP exchange. It should be more connected to the arrival of data from the BGP speaker you peer to. Regards, Paolo On Thu, Apr 09, 2015 at 01:04:22PM +0900, Maoke wrote: > dear Paolo, > > thanks a lot for the quick reply! it really works. and i also checked out > the information in the CONFIG_KEYS regarding the reverse lookup issue. on > the other hand, however, i also tried with the "bgp_src_as_path_type: bgp" > primitive but it looks not working for the source AS path. i attach the > updated part of my configuration file and the corresponding result on the > tail of this mail. > > another observation: i noticed that when the process restarted, sometimes > for a quite long while we cannot get the BGP-related pieces in the memory > dump at all and then they appear. is that because that the BGP thread has > not dumped updated entries of the route, for the time being? i tried to set > the bgp_table_dump_refresh_time shorter (300 -> 60) and it sounds that the > zero-results last not so long right now. > > thanks and regards, > - noriyuki > > >> current pmacctd.conf > … > ! > pmacctd_net: bgp > bgp_peer_src_as_type: bgp > bgp_src_as_path_type: bgp > aggregate[inbound]: src_host, dst_host, src_as, peer_src_as, > peer_src_ip, as_path > aggregate[outbound]: src_host, dst_host, dst_as, peer_dst_as, > peer_dst_ip, as_path > aggregate_filter[inbound]: dst net 2001:db8:3::/48 > aggregate_filter[outbound]: src net 2001:db8:3::/48 > ! > … > > >> current result > ~/pmacct_work$ pmacct -s -p /tmp/collect.pipe-eth0-in > SRC_AS AS_PATH PEER_SRC_AS PEER_SRC_IP > SRC_IP DST_IP > PACKETS BYTES > 0 ^$ 0 0 > 2001:db8:3:3::3 > 2001:db8:3:3::1 927 728068 > 0 ^$ 0 0 > 2001:db8:2:2::1 > 2001:db8:3:6::2 74 9720 > 0 ^$ 0 0 > fe80::42:acff:fe11:2 > 2001:db8:3:3::1 46 7632 > 0 ^$ 0 0 > 2001:db8:3:6::2 > 2001:db8:3:3::1 94 12628 > 0 ^$ 0 0 > 2001:db8:2:ff3::2 > 2001:db8:3:6::2 7 560 > 65530 ^$ 65530 0 > 2001:db8:2:2::1 > 2001:db8:3:6::2 51 6672 > 0 ^$ 0 0 > fe80::1 > 2001:db8:3:3::3 11 792 > 0 ^$ 0 0 > 2001:db8:1:1::2 > 2001:db8:3:6::2 12 864 > 0 ^$ 0 0 > 2001:db8:3:3::1 > 2001:db8:3:6::2 110 9232 > 65533 ^$ 65530 0 > 2001:db8:2:ff3::2 > 2001:db8:3:6::2 7 560 > 0 ^$ 0 0 > 2001:db8:3:3::1 > 2001:db8:3:3::3 981 79538 > > For a total of: 11 entries > > ~/pmacct_work$ pmacct -s -p /tmp/collect.pipe-eth0-out > DST_AS AS_PATH PEER_DST_AS PEER_DST_IP > SRC_IP DST_IP > PACKETS BYTES > 0 ^$ 0 2001:db8:3:3::1 > 2001:db8:3:3::1 > 2001:db8:3:3::3 532 44294 > 0 ^$ 0 0 > 2001:db8:3:6::2 > 2001:db8:1:1::2 6 720 > 0 ^$ 0 0 > 2001:db8:3:3::3 > 2001:db8:3:3::1 548 615490 > 0 ^$ 0 0 > 2001:db8:3:3::3 fe80::1 > 12 768 > 0 ^$ 0 0 > 2001:db8:3:6::2 > 2001:db8:3:3::1 60 7792 > 0 ^$ 0 0 > 2001:db8:3:6::2 > 2001:db8:2:2::1 95 7676 > 0 ^$ 0 0 > 2001:db8:3:3::1 > 2001:df0:232:eea0::fff4 1024 98304 > 0 ^$ 0 2001:db8:3:3::1 > 2001:db8:3:3::1 > 2001:db8:3:6::2 44 3644 > 0 ^$ 0 0 > 2001:db8:3:3::3 > 2001:db8:2:ff3::2 3 384 > 0 ^$ 0 2001:db8:3:3::1 > 2001:db8:3:6::2 > 2001:db8:3:3::1 34 4836 > 65530 65530 65530 2001:db8:1:1::2 > 2001:db8:3:6::2 > 2001:db8:2:2::1 59 4724 > 0 ^$ 0 0 > 2001:db8:3:6::2 > 2001:db8:2:ff3::2 4 512 > 0 ^$ 0 0 > 2001:db8:3:3::1 > 2001:db8:3:6::2 66 5588 > 0 ^$ 0 0 > 2001:db8:3:3::3 > 2001:db8:1:1::2 6 720 > 0 ^$ 0 2001:db8:3:3::1 > 2001:db8:3:3::3 > 2001:db8:3:3::1 399 117569 > 65533 65530_65533 65530 2001:db8:1:1::2 > 2001:db8:3:6::2 > 2001:db8:2:ff3::2 4 512 > 0 ^$ 0 0 > 2001:db8:3:3::1 > 2001:df0:232:eea0::fff3 1024 98304 > 0 ^$ 0 0 > 2001:db8:3:3::1 > fe80::42:acff:fe11:2 10 640 > 65533 65530_65533 65530 2001:db8:1:1::2 > 2001:db8:3:3::3 > 2001:db8:2:ff3::2 3 384 > 0 ^$ 0 0 > 2001:db8:3:3::1 > 2001:db8:3:3::3 467 36683 > > > > > > > 2015-04-09 12:29 GMT+09:00 Paolo Lucente <[email protected]>: > > > Hi Noriyuki-san, > > > > peer_src_as is zero because you have to explicitely define the > > value of the 'bgp_peer_src_as_type' configuration directive. For > > a quick test you can set bgp_peer_src_as_type to 'bgp' in order > > to check you can populate with success the peer_src_as primitive. > > > > However note that 'bgp_peer_src_as_type: bgp' performs a reverse > > BGP lookup, which is not ideal due to the asymmetric nature of > > internet traffic: you may want to set it to 'map' and determine > > peer_src_as basing on the source MAC address for example. > > > > Regards, > > Paolo > > > > On Thu, Apr 09, 2015 at 11:46:17AM +0900, Maoke wrote: > > > I reviewed the mail message about the issue of lacking BGP-related > > > information in the memory dump, and make following configuration for my > > > pmacctd: > > > > > > >> pmacctd.conf > > > ! > > > daemonize: true > > > imt_path[inbound]: /tmp/collect.pipe-eth0-in > > > imt_path[outbound]: /tmp/collect.pipe-eth0-out > > > pidfile: /var/run/pmacctd.pid > > > logfile: /var/log/pmacctd.log > > > interface: eth0 > > > ! > > > aggregate[inbound]: tag, tag2, src_host, dst_host, src_port, dst_port, > > > peer_src_as > > > aggregate[outbound]: tag, tag2, src_host, dst_host, src_port, dst_port, > > > peer_dst_as > > > aggregate_filter[inbound]: dst net 2001:db8:3::/48 > > > aggregate_filter[outbound]: src net 2001:db8:3::/48 > > > ! > > > plugins: memory[inbound], memory[outbound], nfprobe[inbound], > > > nfprobe[outbound] > > > ! > > > pmacctd_as: bgp > > > bgp_daemon: true > > > bgp_daemon_ip: 2001:db8:3:3::3 > > > bgp_daemon_id: 172.17.0.2 > > > bgp_agent_map: .../maps/agent_to_peer.map-eth0 > > > !bgp_daemon_port: 179 > > > !bgp_daemon_msglog: false > > > ! > > > plugin_pipe_size: 2000000 > > > plugin_buffer_size: 10000 > > > imt_mem_pools_number: 0 > > > ! > > > bgp_table_dump_file: /tmp/bgp-$peer_src_ip.txt > > > bgp_table_dump_refresh_time: 300 > > > ! > > > > > > >> maps/agent_to_peer.map-eth0 > > > bgp_ip=2001:db8:3:3::1 ip=0.0.0.0/0 > > > > > > while the bgp router is at 2001:d8:3:3::1 and the pmacct is running on > > > 2001:db8:3:3::3. > > > > > > then i got the result as follows: why the outbound part has the > > peer_dst_as > > > correctly while the inbound part shows only AS 0 for the peer_src_as > > field? > > > > > > ---- > > > ~/pmacct_work$ pmacct -s -p /tmp/collect.pipe-eth0-in > > > TAG TAG2 PEER_SRC_AS SRC_IP > > > DST_IP SRC_PORT DST_PORT > > > PACKETS BYTES > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 53428 33449 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 54045 33446 > > 1 > > > 80 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:db8:3:6::2 39475 22 > > > 88 7300 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 55216 33443 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 46928 33443 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 36485 33448 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 52681 33444 > > 1 > > > 80 > > > 0 0 0 fe80::42:acff:fe11:2 > > > 2001:db8:3:3::1 0 0 > > > 62 10224 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 43491 33449 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 60351 33447 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 60558 33447 > > 1 > > > 80 > > > 0 0 0 2001:db8:3:3::3 > > > 2001:db8:3:3::1 179 38546 > > > 29 2624 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:db8:3:3::3 58451 22 > > > 2247 172324 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 33840 33443 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 35861 33447 > > 1 > > > 80 > > > 0 0 0 fe80::1 > > > 2001:db8:3:3::3 0 0 > > > 14 1008 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:db8:3:3::3 38546 179 > > > 52 4586 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 43479 33445 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 38198 33443 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 41602 33448 > > 1 > > > 80 > > > 0 0 0 2001:db8:3:6::2 > > > 2001:db8:3:3::1 22 39475 > > > 71 10564 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 49961 33444 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 35605 33446 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 59543 33447 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 38170 33449 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:2::1 > > > 2001:db8:3:6::2 22 35279 > > > 104 14176 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 49707 33446 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 44551 33448 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 51801 33445 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 34869 33445 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 37776 33448 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 48220 33446 > > 1 > > > 80 > > > 0 0 0 2001:db8:3:3::3 > > > 2001:db8:3:3::1 22 58451 > > > 2805 3024392 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 59607 33449 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 41632 33445 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 40006 33444 > > 1 > > > 80 > > > 0 0 0 2001:db8:2:ff3::2 > > > 2001:db8:3:6::2 48786 33444 > > 1 > > > 80 > > > > > > For a total of: 37 entries > > > > > > ~/pmacct_work$ pmacct -s -p /tmp/collect.pipe-eth0-out > > > TAG TAG2 PEER_DST_AS SRC_IP > > > DST_IP SRC_PORT DST_PORT > > > PACKETS BYTES > > > 0 0 65530 2001:db8:3:3::3 > > > 2001:db8:2:ff3::2 0 0 > > > 12 1536 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:db8:3:6::2 39475 22 > > > 88 7300 > > > 0 0 0 2001:db8:3:3::3 > > > fe80::1 0 0 > > > 14 896 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:df0:232:eea0::fff4 123 123 > > > 1024 98304 > > > 0 0 0 2001:db8:3:3::3 > > > 2001:db8:3:3::1 179 38546 > > > 29 2624 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:db8:3:3::3 58451 22 > > > 2290 175912 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:db8:3:3::3 38546 179 > > > 52 4586 > > > 0 0 65530 2001:db8:3:6::2 > > > 2001:db8:2:ff3::2 0 0 > > > 16 2048 > > > 0 0 0 2001:db8:3:6::2 > > > 2001:db8:3:3::1 22 39475 > > > 71 10564 > > > 0 0 65530 2001:db8:3:6::2 > > > 2001:db8:2:2::1 35279 22 > > > 108 8740 > > > 0 0 0 2001:db8:3:3::1 > > > 2001:df0:232:eea0::fff3 123 123 > > > 1024 98304 > > > 0 0 0 2001:db8:3:3::3 > > > 2001:db8:3:3::1 22 58451 > > > 2838 3035204 > > > 0 0 0 2001:db8:3:3::1 > > > fe80::42:acff:fe11:2 0 0 > > > 14 896 > > > > > > For a total of: 13 entries > > > > > > ~/pmacct_work$ sudo cat /tmp/bgp-2001_db8_3_3__1.txt > > > {"timestamp": "2015-04-09 02:40:01.687531", "peer_ip_src": > > > "2001:db8:3:3::1", "event_type": "dump_init"} > > > {"peer_ip_src": "2001:db8:3:3::1", "bgp_nexthop": "2001:db8:1:1::2", > > > "event_type": "dump", "ip_prefix": "2001:db8:2::/48", "as_path": "65530", > > > "origin": 0, "local_pref": 100} > > > {"peer_ip_src": "2001:db8:3:3::1", "bgp_nexthop": "2001:db8:1:1::2", > > > "event_type": "dump", "ip_prefix": "2001:db8:2:f00::/56", "as_path": > > "65530 > > > 65533", "origin": 0, "local_pref": 100} > > > {"peer_ip_src": "2001:db8:3:3::1", "bgp_nexthop": "2001:db8:3:3::1", > > > "event_type": "dump", "ip_prefix": "2001:db8:3::/48", "as_path": "", > > > "origin": 0, "local_pref": 100} > > > {"timestamp": "2015-04-09 02:40:01.687531", "peer_ip_src": > > > "2001:db8:3:3::1", "event_type": "dump_close"} > > > > > > ---- > > > > > > best regards, > > > maoke (Noriyuki Arai, Dr.) > > > > > _______________________________________________ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
