Paolo, thanks for your work!
> 1) 01011970 corresponds to a UNIX timestamp of zero. The easiest way to
> mitigate it is to use 'nfacctd_time_new: true' so to consider the
> time of arrival at the collector (that is, nfacctd) as timestamp for
This parameter is set in the config, but the table is still created with
timestamp of zero.
> bucketing instead of the flow start time; alternatively - or also in
> addition - you can look in your flow data in case there are some with
> a flow start time of zero (and if not and you can pin point what gets
> purged to that table, we may be facing a bug or maybe an explanation
> for it ..).
Except for the standard fields IPFIX in the flow there is nothing. template
from router in attach.
How can I make collector on the information from the template when creating a
table?
I do not insist, but it seems to me that when using the parameter
nfacctd_time_new, the table should be created based on the current date. and
it's still some kind of bug. :(
> 2) It is a simplistic yet harmless approach to blindly send a CREATE
> TABLE, maybe the message back should be made a warning rather than an
> error. It's harmless because as you see, data is purged anyway; it's
> simplistic because it saves the effort to probe if the table exists.
i replace CREATE TABLE in schema file to CREATE TABLE IF NOT EXISTS and get
NOTICE , not ERROR. it's good :)
also encountered an unknown problem. on every purge flowdata place to table,
but in postgreesql log there are such error messages
2018-03-16 10:32:20.694 EET [4736] pmacct@pmacct ERROR: invalid input syntax
for type inet: ""
2018-03-16 10:32:20.694 EET [4736] pmacct@pmacct CONTEXT: COPY
traffic_01011970, line 91719, column ip_src: ""
2018-03-16 10:32:20.694 EET [4736] pmacct@pmacct STATEMENT: COPY
traffic_01011970 (ip_src, ip_dst, as_src, iface_in, iface_out, as_dst,
port_src, port_dst, ip_proto, timestamp_start, timestamp_start_residual,
timestamp_end, timestamp_end_residual, packets, bytes) FROM STDIN DELIMITER ','
2018-03-16 10:32:30.045 EET [4738] pmacct@pmacct ERROR: invalid input syntax
for type inet: ""
2018-03-16 10:32:30.045 EET [4738] pmacct@pmacct CONTEXT: COPY
traffic_01011970, line 103998, column ip_src: ""
2018-03-16 10:32:30.045 EET [4738] pmacct@pmacct STATEMENT: COPY
traffic_01011970 (ip_src, ip_dst, as_src, iface_in, iface_out, as_dst,
port_src, port_dst, ip_proto, timestamp_start, timestamp_start_residual,
timestamp_end, timestamp_end_residual, packets, bytes) FROM STDIN DELIMITER ','
2018-03-16 10:32:39.692 EET [4741] pmacct@pmacct ERROR: invalid input syntax
for type inet: ""
2018-03-16 10:32:39.692 EET [4741] pmacct@pmacct CONTEXT: COPY
traffic_01011970, line 113034, column ip_src: ""
2018-03-16 10:32:39.692 EET [4741] pmacct@pmacct STATEMENT: COPY
traffic_01011970 (ip_src, ip_dst, as_src, iface_in, iface_out, as_dst,
port_src, port_dst, ip_proto, timestamp_start, timestamp_start_residual,
timestamp_end, timestamp_end_residual, packets, bytes) FROM STDIN DELIMITER ','
each time a different line number, but the same column. is this of a problem?
> Paolo
> On Tue, Mar 13, 2018 at 04:32:52PM +0200, Andrey Koblyuk wrote:
>> Hello!
>> my config
>> aggregate[storage] :
>> src_host,dst_host,src_port,dst_port,proto,src_as,dst_as,in_iface,out_iface,timestamp_start,timestamp_end
>> sql_optimize_clauses[storage] : true
>> sql_table[storage] : traffic_%d%m%Y
>> sql_table_schema[storage] : /etc/pmacct/nfacct_schema.psql
>> sql_table_type[storage] : bgp
>> sql_dont_try_update[storage] : true
>> sql_use_copy[storage] : true
>> /etc/pmacct/nfacct_schema.psql:
>> CREATE TABLE traffic_%d%m%Y (
>> id_key BIGSERIAL PRIMARY KEY,
>> as_src BIGINT NOT NULL DEFAULT 0,
>> as_dst BIGINT NOT NULL DEFAULT 0,
>> ip_src inet NOT NULL DEFAULT '0.0.0.0',
>> ip_dst inet NOT NULL DEFAULT '0.0.0.0',
>> port_src INT NOT NULL DEFAULT 0,
>> port_dst INT NOT NULL DEFAULT 0,
>> ip_proto SMALLINT NOT NULL DEFAULT 0,
>> iface_in INT NOT NULL,
>> iface_out INT NOT NULL,
>> packets INT NOT NULL,
>> bytes BIGINT NOT NULL,
>> timestamp_start timestamp without time zone NOT NULL DEFAULT
>> '0001-01-01 00:00:00',
>> timestamp_start_residual INT NOT NULL DEFAULT 0,
>> timestamp_end timestamp without time zone NOT NULL DEFAULT
>> '0001-01-01 00:00:00',
>> timestamp_end_residual INT NOT NULL DEFAULT 0,
>> stamp_inserted timestamp without time zone NOT NULL DEFAULT
>> CURRENT_TIMESTAMP(0)
>> );
>> GRANT SELECT, INSERT, UPDATE, DELETE ON traffic_%d%m%Y TO pmacct;
>> 1) first "purge" create table traffic_01011970, but current date 13032018.
>> Why table create on 01011970?
>> 2) next "purge" print this error in log
>> INFO ( storage/pgsql ): *** Purging cache - START (PID: 11285) ***
>> ERROR ( storage/pgsql ): ERROR: relation "traffic_01011970" already exists
>> INFO ( storage/pgsql ): *** Purging cache - END (PID: 11285, QN:
>> 137084/137084, ET: 1) ***
>> Why does the table try to be recreated with each purge? what parameter is
>> responsible for this?
>> --
>> ANK32-RIPE
>> _______________________________________________
>> pmacct-discussion mailing list
>> http://www.pmacct.net/#mailinglists
--
ANK32-RIPE
DEBUG ( default/core ): NfV10 template type : flow
DEBUG ( default/core ): NfV10 template ID : 256
DEBUG ( default/core ):
-------------------------------------------------------------
DEBUG ( default/core ): | pen | field type | offset |
size |
DEBUG ( default/core ): | 0 | IPv4 src addr [8 ] | 0 |
4 |
DEBUG ( default/core ): | 0 | IPv4 dst addr [12 ] | 4 |
4 |
DEBUG ( default/core ): | 0 | tos [5 ] | 8 |
1 |
DEBUG ( default/core ): | 0 | L4 protocol [4 ] | 9 |
1 |
DEBUG ( default/core ): | 0 | L4 src port [7 ] | 10 |
2 |
DEBUG ( default/core ): | 0 | L4 dst port [11 ] | 12 |
2 |
DEBUG ( default/core ): | 0 | icmp type [32 ] | 14 |
2 |
DEBUG ( default/core ): | 0 | input snmp [10 ] | 16 |
4 |
DEBUG ( default/core ): | 0 | IPv4 src mask [9 ] | 20 |
1 |
DEBUG ( default/core ): | 0 | IPv4 dst mask [13 ] | 21 |
1 |
DEBUG ( default/core ): | 0 | src as [16 ] | 22 |
4 |
DEBUG ( default/core ): | 0 | dst as [17 ] | 26 |
4 |
DEBUG ( default/core ): | 0 | IPv4 next hop [15 ] | 30 |
4 |
DEBUG ( default/core ): | 0 | tcp flags [6 ] | 34 |
1 |
DEBUG ( default/core ): | 0 | output snmp [14 ] | 35 |
4 |
DEBUG ( default/core ): | 0 | in bytes [1 ] | 39 |
8 |
DEBUG ( default/core ): | 0 | in packets [2 ] | 47 |
8 |
DEBUG ( default/core ): | 0 | 152 [152 ] | 55 |
8 |
DEBUG ( default/core ): | 0 | 153 [153 ] | 63 |
8 |
DEBUG ( default/core ): | 0 | 136 [136 ] | 71 |
1 |
DEBUG ( default/core ):
-------------------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 72
DEBUG ( default/core ):
DEBUG ( default/core ): Received NetFlow/IPFIX packet from
[94.125.120.65:54276] version [10] seqno [107972]
DEBUG ( default/core ): NfV10 agent : 94.125.120.65:0
DEBUG ( default/core ): NfV10 template type : options
DEBUG ( default/core ): NfV10 template ID : 512
DEBUG ( default/core ):
-------------------------------------------------------------
DEBUG ( default/core ): | pen | field type | offset |
size |
DEBUG ( default/core ): | 0 | 144 [144 ] | 0 |
4 |
DEBUG ( default/core ): | 0 | 160 [160 ] | 4 |
8 |
DEBUG ( default/core ): | 0 | 130 [130 ] | 12 |
4 |
DEBUG ( default/core ): | 0 | 214 [214 ] | 16 |
1 |
DEBUG ( default/core ): | 0 | 215 [215 ] | 17 |
1 |
DEBUG ( default/core ):
-------------------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 18
DEBUG ( default/core ):
DEBUG ( default/core ): Received NetFlow/IPFIX packet from
[94.125.120.65:54276] version [10] seqno [107972]
DEBUG ( default/core ): Received NetFlow/IPFIX packet from
[94.125.120.65:54276] version [10] seqno [107972]
DEBUG ( default/core ): NfV10 agent : 94.125.120.65:1
DEBUG ( default/core ): NfV10 template type : options
DEBUG ( default/core ): NfV10 template ID : 512
DEBUG ( default/core ):
-------------------------------------------------------------
DEBUG ( default/core ): | pen | field type | offset |
size |
DEBUG ( default/core ): | 0 | 144 [144 ] | 0 |
4 |
DEBUG ( default/core ): | 0 | 160 [160 ] | 4 |
8 |
DEBUG ( default/core ): | 0 | 130 [130 ] | 12 |
4 |
DEBUG ( default/core ): | 0 | 214 [214 ] | 16 |
1 |
DEBUG ( default/core ): | 0 | 215 [215 ] | 17 |
1 |
DEBUG ( default/core ):
-------------------------------------------------------------
DEBUG ( default/core ): Netflow V9/IPFIX record size : 18
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
