Hi Moftah In the docs [1] it’s stated that for networks_mask to work you have to include the mask as an aggregation primitive. The config you posted earlier doesn't seem to have that. It should probably read something like that
aggregate[out6]: src_net, src_mask Regards Felix [1] https://github.com/pmacct/pmacct/blob/master/CONFIG-KEYS Von: pmacct-discussion <[email protected]> im Auftrag von moftah moftah <[email protected]> Antworten an: "[email protected]" <[email protected]> Datum: Montag, 14. Oktober 2019 um 03:02 An: Paolo Lucente <[email protected]> Cc: "[email protected]" <[email protected]> Betreff: Re: [pmacct-discussion] getting IPv6 traffic per /64 subnet I upgraded to latest version from github and the error is no longer showing and it do write to the db but the problem is that it write the ipv6 src_net and dst_net as '::' so all ipv6 are the same tried to understand why it do that but failed (seems it is not able to understand that the ip belong to /64 subnet) and I though using network_mask directive is basically telling it that each ipv6 is part of /64 subnet but that didnt do it So here is my question We have /32 ipv6 subnet and we give each customer one /64 subnet I want to log the traffic per each /64 subnet out of the /32 subnet that we have I can do that using network file but this is unrealistic to write every /64 subnet we have in the file So is there any easier way to do that ? Thanks On Fri, Oct 11, 2019 at 2:41 PM Paolo Lucente <mailto:[email protected]> wrote: Hi, Thank you for the extra insight. Unfortunately this is happening on a packaged binary (ie. not self compiled) and very old, ie. > 4 years old code. Any chance you can switch to something more current, ie. 1.7.3 or master code on GitHub? Paolo On Fri, Oct 11, 2019 at 09:30:00AM -0400, moftah moftah wrote: > Just noticed i didnt send reply to the group > > I just enabled both debug and mysql query debug > > the query sent to mysql is really wrong > > here is the line before the line with error from sys log > Oct 10 20:45:01 ALM02 pmacctd[10723]: DEBUG ( out6/mysql ): 1 VALUES > statements sent to the MySQL server. > Oct 10 20:45:01 ALM02 pmacctd[10723]: ERROR ( out6/mysql ): Column 'ip_src' > specified twice > > Oct 10 20:45:01 ALM02 pmacctd[10722]: DEBUG ( in6/mysql ): 1 VALUES > statements sent to the MySQL server. > Oct 10 20:45:01 ALM02 pmacctd[10722]: ERROR ( in6/mysql ): Column 'ip_dst' > specified twice > > which is not that useful but when checking what query sent to db it looks > like this > > > 2019-10-11T01:07:01.018311Z 7 Query UPDATE `acct6_out` SET > packets=packets+6460, bytes=bytes+1590631 WHERE ip_src='' AND ip_src='::' > AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' > AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:07:01.029075Z 8 Query UPDATE `acct6_in` SET > packets=packets+9576, bytes=bytes+13235525 WHERE ip_dst='::' AND dst_port=0 > AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND > ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:07:01.113495Z 8 Query INSERT INTO `acct6_in` > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets, > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', > '0.0.0.0', 9576, 13235525) > 2019-10-11T01:07:01.117569Z 7 Query INSERT INTO `acct6_out` > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0', > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 6460, 1590631) > 2019-10-11T01:08:01.015998Z 11 Query UPDATE `acct6_in` SET > packets=packets+244378, bytes=bytes+335595260 WHERE ip_dst='::' AND > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:08:01.016375Z 12 Query UPDATE `acct6_out` SET > packets=packets+164730, bytes=bytes+51500942 WHERE ip_src='' AND > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND > ip_dst='0.0.0.0' > 2019-10-11T01:08:01.127378Z 12 Query INSERT INTO `acct6_out` > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0', > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 164730, 51500942) > 2019-10-11T01:08:01.127854Z 11 Query INSERT INTO `acct6_in` > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets, > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', > '0.0.0.0', 244378, 335595260) > 2019-10-11T01:09:01.012185Z 15 Query UPDATE `acct6_in` SET > packets=packets+276526, bytes=bytes+387029255 WHERE ip_dst='::' AND > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:09:01.027795Z 16 Query UPDATE `acct6_out` SET > packets=packets+177194, bytes=bytes+44384720 WHERE ip_src='' AND > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND > ip_dst='0.0.0.0' > 2019-10-11T01:09:01.114225Z 16 Query INSERT INTO `acct6_out` > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0', > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 177194, 44384720) > 2019-10-11T01:09:01.224697Z 15 Query INSERT INTO `acct6_in` > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets, > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', > '0.0.0.0', 276526, 387029255) > 2019-10-11T01:10:01.017359Z 18 Query UPDATE `acct6_in` SET > packets=packets+294880, bytes=bytes+412303624 WHERE ip_dst='::' AND > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:10:01.017829Z 20 Query UPDATE `acct6_out` SET > packets=packets+186580, bytes=bytes+48111642 WHERE ip_src='' AND > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND > ip_dst='0.0.0.0' > 2019-10-11T01:10:01.110120Z 18 Query INSERT INTO `acct6_in` > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets, > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', > '0.0.0.0', 294880, 412303624) > 2019-10-11T01:10:01.146967Z 20 Query INSERT INTO `acct6_out` > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0', > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 186580, 48111642) > 2019-10-11T01:11:01.016057Z 23 Query UPDATE `acct6_out` SET > packets=packets+167580, bytes=bytes+44638648 WHERE ip_src='' AND > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND > ip_dst='0.0.0.0' > 2019-10-11T01:11:01.016832Z 22 Query UPDATE `acct6_in` SET > packets=packets+261858, bytes=bytes+363933056 WHERE ip_dst='::' AND > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:11:01.107406Z 22 Query INSERT INTO `acct6_in` > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets, > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', > '0.0.0.0', 261858, 363933056) > 2019-10-11T01:11:01.133611Z 23 Query INSERT INTO `acct6_out` > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0', > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 167580, 44638648) > 2019-10-11T01:12:01.014452Z 30 Query UPDATE `acct6_in` SET > packets=packets+242934, bytes=bytes+335519993 WHERE ip_dst='::' AND > dst_port=0 AND ip_proto='ip' AND mac_src='0:0:0:0:0:0' AND > mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND ip_dst='0.0.0.0' > 2019-10-11T01:12:01.015199Z 29 Query UPDATE `acct6_out` SET > packets=packets+162488, bytes=bytes+44937541 WHERE ip_src='' AND > ip_src='::' AND src_port=0 AND dst_port=0 AND ip_proto='ip' AND > mac_src='0:0:0:0:0:0' AND mac_dst='0:0:0:0:0:0' AND ip_src='0.0.0.0' AND > ip_dst='0.0.0.0' > 2019-10-11T01:12:01.132584Z 29 Query INSERT INTO `acct6_out` > (ip_src, ip_src, src_port, dst_port, ip_proto, mac_src, mac_dst, ip_src, > ip_dst, packets, bytes) VALUES ('', '::', 0, 0, 'ip', '0:0:0:0:0:0', > '0:0:0:0:0:0', '0.0.0.0', '0.0.0.0', 162488, 44937541) > 2019-10-11T01:12:01.141778Z 30 Query INSERT INTO `acct6_in` > (ip_dst, dst_port, ip_proto, mac_src, mac_dst, ip_src, ip_dst, packets, > bytes) VALUES ('::', 0, 'ip', '0:0:0:0:0:0', '0:0:0:0:0:0', '0.0.0.0', > '0.0.0.0', 242934, 335519993) > > > do you know why it behave like this > the version i have is > pmacct -V > pmacct, pmacct client 1.5.2 (20150907-00) > > which is the one by default in ubuntu 18 repo > > > Thanks > > On Thu, Oct 10, 2019 at 7:29 PM Paolo Lucente <mailto:[email protected]> wrote: > > > > > Hi, > > > > Thank you for reporting this. Can show the integral error message you > > get back from MySQL? It may give relevant additional info; feel free to > > anonimize any confidential data it may contain (ie. IP addresses). > > > > Paolo > > > > > > On Thu, Oct 10, 2019 at 12:34:12PM -0400, moftah moftah wrote: > > > Hi All, > > > I have issue in making pmacct aggregate traffic for all ipv6 to be per > > /64 > > > not individual ip > > > > > > I am logging ipv4 and ipv6 and i made special plugin for ipv6 so i can > > use > > > network_mask but it does not work > > > > > > attached is my config > > > > > > with this config the error i get is > > > column ip_src specified twice > > > column ip_dst specified twice > > > > > > what i am doing wring here > > > > > > the table is v1 table with added 2 fields > > > net_dst > > > net_src > > > > > > can anyone understand why i get column specified twice error > > > > > > > > > Thanks > > > > > > my config > > > aggregate[in]: dst_host > > > aggregate[out]: src_host > > > aggregate[in6]: dst_net > > > aggregate[out6]: src_net > > > aggregate_filter[in]: dst net xxx.xxx.xxx.0/22 or dst net > > ttt.ttt.ttt.0/22 > > > aggregate_filter[out]: src net xxx.xxx.xxx.0/22 or src net > > ttt.ttt.ttt.0/22 > > > aggregate_filter[in6]: dst net zzzz:zzzz::/36 > > > aggregate_filter[out6]: src net zzzz:zzzz::/36 > > > networks_mask[in6]: 64 > > > networks_mask[out6]: 64 > > > interface: ens1f0 > > > plugins: , mysql[in], mysql[out] , mysql[in6], mysql[out6] > > > sql_multi_values: 102400000 > > > sql_history_roundoff: h > > > sql_refresh_time: 60 > > > sql_table[in]: acct_in > > > sql_table[out]: acct_out > > > sql_table[in6]: acct6_in > > > sql_table[out6]: acct6_out > > > !sql_host: localhost > > > sql_passwd:aaaaa > > > sql_user:aaaaa > > > plugin_buffer_size: 10240 > > > plugin_pipe_size: 2048000 > > > !sql_table_version: 9 > > > > > _______________________________________________ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
