In my environment setting nfacctd_time_new to true works better because sometimes router/firewall clocks seems not synchronized. Routers and firewalls are managed by another group of people so that I never chance to find out the clock issue. Using collector's clock is much better in this case; and if your time scale is 1s I think several milliseconds error is acceptable. My two cents. ________________________________________ 從: pmacct-discussion <pmacct-discussion-boun...@pmacct.net> 代表 Paolo Lucente <pa...@pmacct.net> 寄件日期: 2015年10月22日 23:58 至: pmacct-discussion@pmacct.net 主旨: Re: [pmacct-discussion] multiple nfacctd files being written
Hi Edward, Mario is right. Plus you can set nfacctd_time_new to true to make nfacctd use the time of arrival at the collector (rather than individual flow start times) for time binning. This approach will be less precise than using flow start times; a few considerations at this propo: 1) if flow timers at the router side are set low, it will only be slightly less precise; then again if this is acceptable or not depends on the use-case; 2) it does guarantee only the last/current file is going to be updated. Cheers, Paolo On Thu, Oct 22, 2015 at 07:51:23AM +0000, Jentsch, Mario wrote: > Hey Edward, > > each file contains the data of one timebin. Flows spread over a timeframe > longer than one timebin cause pmacct to create/update multiple files. > Depending on how long your Netflow exporter keeps the flow records before it > flushes them to the collector, the created/updated files will be more or > less far in the past. > > Regards, > Mario > > From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On > Behalf Of Edward Henigin > Sent: Wednesday, October 21, 2015 10:34 PM > To: pmacct-discuss. > Subject: Re: [pmacct-discussion] multiple nfacctd files being written > > And sometimes the filenames look correct but again they all come out at the > same time: > > (root) packet1:/opt/pmacct/data# ls -lt | head > total 6670424 > -rw------- 1 root root 3005237 Oct 21 15:28 nfacct-20151021-1525.csv > -rw------- 1 root root 1461133 Oct 21 15:28 nfacct-20151021-1527.csv > -rw------- 1 root root 2292406 Oct 21 15:28 nfacct-20151021-1526.csv > -rw------- 1 root root 3505033 Oct 21 15:27 nfacct-20151021-1524.csv > -rw------- 1 root root 2178942 Oct 21 15:26 nfacct-20151021-1523.csv > -rw------- 1 root root 2551924 Oct 21 15:24 nfacct-20151021-1522.csv > -rw------- 1 root root 3633218 Oct 21 15:23 nfacct-20151021-1521.csv > -rw------- 1 root root 2407956 Oct 21 15:22 nfacct-20151021-1520.csv > -rw------- 1 root root 2669403 Oct 21 15:21 nfacct-20151021-1519.csv > (root) packet1:/opt/pmacct/data# perl -le 'foreach (qw{1519 1520 1521 1522 > 1523 1524 1525 1526 1527}){print("$_: ". > localtime((stat("nfacct-20151021-$_.csv"))[9]))}' > 1519: Wed Oct 21 15:21:46 2015 > 1520: Wed Oct 21 15:22:50 2015 > 1521: Wed Oct 21 15:23:53 2015 > 1522: Wed Oct 21 15:24:58 2015 > 1523: Wed Oct 21 15:26:02 2015 > 1524: Wed Oct 21 15:27:06 2015 > 1525: Wed Oct 21 15:28:10 2015 > 1526: Wed Oct 21 15:28:10 2015 > 1527: Wed Oct 21 15:28:10 2015 > > Seems related? > > > On Wed, Oct 21, 2015 at 3:28 PM, Edward Henigin > <e...@eaohana.com<mailto:e...@eaohana.com>> wrote: > Hi Paolo, > > Running pmacct 1.5.2, simply using the print plugin, I'm getting multiple > files coming out at the same time with filenames suggesting they should be > coming out at different times, and sometimes very strange filenames like the > data is very old. > > ls output: > > (root) packet1:/opt/pmacct/data# ls -lt | head > total 6649800 > -rw------- 1 root root 1785873 Oct 21 15:20 nfacct-20151021-1450.csv > -rw------- 1 root root 2740509 Oct 21 15:20 nfacct-20151021-1518.csv > -rw------- 1 root root 2597403 Oct 21 15:20 nfacct-20151021-1519.csv > -rw------- 1 root root 2778987 Oct 21 15:19 nfacct-20151021-1517.csv > -rw------- 1 root root 3017902 Oct 21 15:18 nfacct-20151021-1516.csv > -rw------- 1 root root 2860626 Oct 21 15:17 nfacct-20151021-1515.csv > -rw------- 1 root root 3013418 Oct 21 15:16 nfacct-20151021-1514.csv > -rw------- 1 root root 3433555 Oct 21 15:15 nfacct-20151021-1513.csv > -rw------- 1 root root 2752513 Oct 21 15:14 nfacct-20151021-1512.csv > > and timestamps to the second: > > (root) packet1:/opt/pmacct/data# perl -le 'foreach (qw{1512 1513 1514 1515 > 1516 1517 1518 1450}){print("$_: ". > localtime((stat("nfacct-20151021-$_.csv"))[9]))}' > 1512: Wed Oct 21 15:14:18 2015 > 1513: Wed Oct 21 15:15:22 2015 > 1514: Wed Oct 21 15:16:26 2015 > 1515: Wed Oct 21 15:17:30 2015 > 1516: Wed Oct 21 15:18:34 2015 > 1517: Wed Oct 21 15:19:38 2015 > 1518: Wed Oct 21 15:20:42 2015 > 1450: Wed Oct 21 15:20:42 2015 > > Where is filename "...-1450" coming from, and why is it coming out at the > same time as -1518? > > Configuration: > > ! nfacctd configuration file > aggregate: peer_src_ip,in_iface,dst_host,dst_mask > plugins: print > plugin_buffer_size: 10240 > imt_buckets: 157 > imt_mem_pools_number: 256 > imt_mem_pools_size: 32768 > syslog: daemon > daemonize: true > ! > print_refresh_time: 64 > print_history: 1m > print_output: csv > print_output_file: /opt/pmacct/data/nfacct-%Y%m%d-%H%M.csv > print_output_file_append: true > ! > nfacctd_port: 2055 > > Syslog: > > Oct 21 15:15:21 packet1 nfacctd[558]: INFO ( default/print ): *** Purging > cache - START (PID: 558) *** > Oct 21 15:15:22 packet1 nfacctd[558]: INFO ( default/print ): *** Purging > cache - END (PID: 558, QN: 80643/80847, ET: 1) *** > Oct 21 15:16:25 packet1 nfacctd[593]: INFO ( default/print ): *** Purging > cache - START (PID: 593) *** > Oct 21 15:16:26 packet1 nfacctd[593]: INFO ( default/print ): *** Purging > cache - END (PID: 593, QN: 74092/74503, ET: 1) *** > Oct 21 15:17:29 packet1 nfacctd[651]: INFO ( default/print ): *** Purging > cache - START (PID: 651) *** > Oct 21 15:17:30 packet1 nfacctd[651]: INFO ( default/print ): *** Purging > cache - END (PID: 651, QN: 74211/74621, ET: 1) *** > Oct 21 15:18:33 packet1 nfacctd[690]: INFO ( default/print ): *** Purging > cache - START (PID: 690) *** > Oct 21 15:18:34 packet1 nfacctd[690]: INFO ( default/print ): *** Purging > cache - END (PID: 690, QN: 71470/72233, ET: 1) *** > Oct 21 15:19:37 packet1 nfacctd[739]: INFO ( default/print ): *** Purging > cache - START (PID: 739) *** > Oct 21 15:19:38 packet1 nfacctd[739]: INFO ( default/print ): *** Purging > cache - END (PID: 739, QN: 69195/73637, ET: 1) *** > Oct 21 15:20:41 packet1 nfacctd[779]: INFO ( default/print ): *** Purging > cache - START (PID: 779) *** > Oct 21 15:20:42 packet1 nfacctd[779]: INFO ( default/print ): *** Purging > cache - END (PID: 779, QN: 67848/83867, ET: 1) *** > > > Ed > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ==================================================================== 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
