Re: [pmacct-discussion] Timestamps in RabbitMQ/JSON output

2014-06-05 Thread Chris Wilson
Hi Paolo, On Thu, 5 Jun 2014, Paolo Lucente wrote: Thanks for the patch and the feedback. Let me review so that we see if we have something for the mainstream release. Great, thanks! I would probably not commit it as is, because I removed the stamp_inserted and stamp_updated fields from the

Re: [pmacct-discussion] Timestamps in RabbitMQ/JSON output

2014-06-05 Thread Chris Wilson
Hi Paolo, On Thu, 5 Jun 2014, Paolo Lucente wrote: DEBUG ( default/amqp ): publishing [E=pmacct RK=acct DM=0]: {"timestamp_start": "2014-06-03 22:42:00.202820", "ip_dst": "196.223.145.xxx", "ip_proto": "tcp", "tos": 0, "ip_src": "86.30.131.xxx", "bytes": 142, "port_dst": 36363, "packets": 1,

Re: [pmacct-discussion] Timestamps in RabbitMQ/JSON output

2014-06-03 Thread Chris Wilson
Hi Paolo, On Tue, 3 Jun 2014, Paolo Lucente wrote: What you describe for timestamps seems a good match for NetFlow, ie. cast packets into flows and handle these via a flow-aware cache (so active/passive expiration timers, max lifetime, etc.). All described is already part of the nfprobe plugi

[pmacct-discussion] Timestamps in RabbitMQ/JSON output

2014-06-02 Thread Chris Wilson
Hi Paolo, I've started work on a pure JavaScript replacement for pmGraph, so it should be easier to install and more lightweight: https://github.com/qris/nodeflow I have a small server-side component in Node which connect to RabbitMQ, receives JSON flow data from pmacct, and relays it to conne

Re: [pmacct-discussion] Newbie

2014-04-05 Thread Chris Wilson
Hi Mike, On Sat, 5 Apr 2014, Mike Hammett wrote: The OfficialConfigKeys is very verbose and no doubt holds the key (no pun intended) to every possible configuration, but all config examples I've found seem drastically simplistic or seemingly incomplete. Try this one: daemonize: false debug:

Re: [pmacct-discussion] HTTP traffic classification

2014-03-24 Thread Chris Wilson
Hi Karl, On Mon, 24 Mar 2014, Karl O. Pinc wrote: On 03/24/2014 06:31:30 AM, Stathis Gkotsis wrote: Concerning HTTP: I guess the thing to output would be hostname, since you can have multiple HTTP requests to different URLs inside one TCP Session.About DNS, what should be outputted? I guess th

Re: [pmacct-discussion] HTTP traffic classification

2014-03-22 Thread Chris Wilson
Hi all, On Sat, 22 Mar 2014, Viacheslav Dubrovskyi wrote: 22.03.2014 21:20, Stathis Gkotsis пишет: First, I would like to thank you for the great product, pmacct has proven very useful to me, which brings me to my question :) I see that it is possible to enable traffic classification, which is

Re: [pmacct-discussion] Error: 'sql_multi_values' is too small (100). Try with a larger value

2013-09-03 Thread Chris Wilson
Hi Paolo, On Tue, 3 Sep 2013, Paolo Lucente wrote: Maybe a bug in documentation in the release you are using? CONFIG-KEYS says: "The value of the directive is intended to be the size (in bytes) of the multi-values buffer.". So 100 bytes is on the low side, and by default MySQL comes with a 1M

Re: [pmacct-discussion] Error: 'sql_multi_values' is too small (100). Try with a larger value

2013-09-03 Thread Chris Wilson
On Tue, 3 Sep 2013, Edward van Kuik wrote: No, it should definitely batch the data into inserts of 1000 values each. Then why would it give me this error message? The error doesn't make sense if pmacct does break inserts into smaller batches. Sep  2 17:59:01 microserver pmacctd[

Re: [pmacct-discussion] Error: 'sql_multi_values' is too small (100). Try with a larger value

2013-09-03 Thread Chris Wilson
Hi Edward, On Tue, 3 Sep 2013, Edward van Kuik wrote: Sep  2 17:59:01 microserver pmacctd[17603]: ERROR ( summary/mysql ): 'sql_multi_values' is too small (100). Try with a larger value. I set mine to 1000. OK, so 1000 might work for you now. But it seems that pmacct can't split the insert

[pmacct-discussion] Error: 'sql_multi_values' is too small (100). Try with a larger value

2013-09-02 Thread Chris Wilson
Hi all, I tried to enable the sql_multi_values option, but setting it to a reasonable number of rows to insert at once (100) to avoid hitting the MySQL packet size limit. But I get these errors in the logs: Sep 2 17:59:01 microserver pmacctd[17603]: ERROR ( summary/mysql ): 'sql_multi_value

Re: [pmacct-discussion] Build fails to find libmysqlclient on 64-bit CentOS

2013-06-26 Thread Chris Wilson
Hi Paolo, On Tue, 25 Jun 2013, Paolo Lucente wrote: Sure, thanks for the tip: makes sense, will do. Also please find attached an RPM spec file to help build rpms for pmacct. It would be great if you could include this in the tarball. Cheers, Chris. -- Aptivate | http://www.aptivate.org | Ph

[pmacct-discussion] Build fails to find libmysqlclient on 64-bit CentOS

2013-06-25 Thread Chris Wilson
Hi Paolo, Configure fails to find /usr/lib64/mysql/libmysqlclient.so on 64-bit CentOS. You might want to add that to the list of search directories in configure.in? Cheers, Chris. -- Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838 Citylife House, Sturton Street, Cambridge, CB1 2Q

Re: [pmacct-discussion] Duplicate entry for key 1 (primary key violations)

2012-07-18 Thread Chris Wilson
Hi Paolo, On Wed, 18 Jul 2012, Paolo Lucente wrote: On an unrelated note, how hard would it be to get the log message from ULOG stored in the database, for example in the classification field? I had a look through the code but I couldn't see any way to store this field from the received packe

Re: [pmacct-discussion] Duplicate entry for key 1 (primary key violations)

2012-07-18 Thread Chris Wilson
Hi Paolo, On Fri, 29 Jun 2012, Paolo Lucente wrote: On Tue, Jun 26, 2012 at 10:13:30AM +0100, Chris Wilson wrote: OK, testing now. Would it be possible for pmacctd to log a warning if it exceeds any of these thresholds, to help with tuning without wasting memory? In a way you reckon things

Re: [pmacct-discussion] Duplicate entry for key 1 (primary key violations)

2012-06-26 Thread Chris Wilson
Hi Paolo, On Wed, 20 Jun 2012, Paolo Lucente wrote: I'm thinking to the possibility that given the aggregation method the SQL cache configured by default is not sufficient to keep all the aggregates over the time period - although the time period is very short. Can you as matter of test add t

Re: [pmacct-discussion] Duplicate entry for key 1 (primary key violations)

2012-06-18 Thread Chris Wilson
Bump? Cheers, Chris. On Tue, 12 Jun 2012, Chris Wilson wrote: > Hi all, > > We get many of these errors in our system logs: > > Jun 12 10:01:01 fen-fw2 pmacctd[2153]: ERROR ( short/mysql ): Duplicate entry > '72.232.223.58-82.68.244.70-80-46802-tcp-2012-06-12 09:56:

[pmacct-discussion] Duplicate entry for key 1 (primary key violations)

2012-06-12 Thread Chris Wilson
Hi all, We get many of these errors in our system logs: Jun 12 10:01:01 fen-fw2 pmacctd[2153]: ERROR ( short/mysql ): Duplicate entry '72.232.223.58-82.68.244.70-80-46802-tcp-2012-06-12 09:56:00' for key 1 They usually happen in batches. E.g. we had a few hundred at 07:27, then another few

Re: [pmacct-discussion] nfacctd

2012-04-01 Thread Chris Wilson
Hi Johan, your nfacctd is compiled without mysql support, so it's not logging to the database, only the memory plugin. Please fix that and try again. Cheers, Chris. -Original Message- From: johan lotter Sender: Chris Wilson Date: Sun, 1 Apr 2012 16:33:50 To: ; ; Cc: Su

Re: [pmacct-discussion] pmacct-discussion Digest, Vol 83, Issue 1

2012-02-02 Thread Chris Wilson
Hi Johan, On Thu, 2 Feb 2012, johan lotter wrote: Yet when I configure and run with mysql plugin I get no data... Does that mean that you get nothing in the database, or nothing graphed? I notice that you mentioned pmgraph later, which is a different project (that uses pmacct). If you get

Re: [pmacct-discussion] Infinite loop in sql_cache_insert

2011-12-02 Thread Chris Wilson
Hi Paolo, On Mon, 28 Nov 2011, Paolo Lucente wrote: Would be great if: 1) you can upgrade to something more recent than that, ie. issue could be related to timestamps and fix might well be in some other parts of the code (pkt_handlers.c pops to mind) I will probably do this soon as I'm intend

[pmacct-discussion] Infinite loop in sql_cache_insert

2011-11-27 Thread Chris Wilson
Hi all, I just caught my copy of pmacct on my laptop in an infinite loop in sql_cache_insert, burning 100% CPU. Unfortunately I don't know what traffic could have caused this. I have the process in a debugger right now, but without debug symbols I can't tell very much. It's 0.12.1 from Ubunt

Re: [pmacct-discussion] pmacct.conf for aggregated ip based accounting and vlan information

2011-08-03 Thread Chris Wilson
Hi Bernd, On Tue, 2 Aug 2011, Bernd Bornkessel wrote: > To lower the number of db records you could split aggregation: > > plugins: mysql[in], mysql[out] > aggregate[in]:vlan,dst_host > aggregate[out]:vlan,src_host > > aggregate_filter[in]:vlan and net > aggregate_filter[out]:vlan and net I'

Re: [pmacct-discussion] Broken aggregate Filter

2011-06-09 Thread Chris Wilson
Hi Bernd, On Thu, 9 Jun 2011, Bernd Bornkessel wrote: The working filter is: vlan and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19) The non-working are

Re: [pmacct-discussion] Broken aggregate Filter

2011-06-09 Thread Chris Wilson
Hi Bernd, On Thu, 9 Jun 2011, Bernd Bornkessel wrote: It works if I use: vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0

Re: [pmacct-discussion] Aggregate not working?

2010-11-11 Thread Chris Wilson
Hi Lockywolf, On Thu, 11 Nov 2010, Lockywolf __ wrote: aggregate[in]: dst_host aggregate[out]: src_host aggregate_filter[in]: dst net 192.168.88.0/16 aggregate_filter[out]: src net 192.168.88.0/16 plugins: mysql[in], mysql[out] Still, in MySQL i have (a lot of) lines like the following: | 0:0

Re: [pmacct-discussion] Source port column name depends on database

2010-10-06 Thread Chris Wilson
Hi Paolo, On Wed, 6 Oct 2010, Paolo Lucente wrote: Yes, that's intended for a couple of reasons: 1) don't expect to release any more table versions: you see that already happening with recently introduced primitives; idea is to stick to a table version (or style nowadays) and then customize i

Re: [pmacct-discussion] Source port column name depends on database

2010-10-06 Thread Chris Wilson
Hi Paolo, On Wed, 6 Oct 2010, Paolo Lucente wrote: To say this work (as agreed in the shape of sql table version 8) has been just committed to the CVS. Please give it a try and let me know if it seems to work to your eyes. Thanks for this. I haven't compiled it yet, but I noticed this line:

Re: [pmacct-discussion] Source port column name depends on database

2010-09-16 Thread Chris Wilson
Hi Karl and all, On Wed, 15 Sep 2010, Karl O. Pinc wrote: > I too don't like having a config switch. But note that changing the > schema in this fashion breaks backwards compatibility in anything that's > querying the data. Agreed, I hadn't thought of that. Is this the first time that column

Re: [pmacct-discussion] Source port column name depends on database

2010-09-15 Thread Chris Wilson
Hi Paolo, On Wed, 15 Sep 2010, Paolo Lucente wrote: > On Tue, Sep 14, 2010 at 09:16:37AM +0200, Chris Wilson wrote: > > > I'm not sure about adding a new config switch, do we actually need it? > > Funnily enough, and that was my perspective, in this case a configuration

Re: [pmacct-discussion] Source port column name depends on database

2010-09-14 Thread Chris Wilson
Hi Paolo, On Tue, 14 Sep 2010, Paolo Lucente wrote: > Agree. I seem to reckon this legacy issue is limited to the TCP/UDP > ports only and i'm thinking perhaps the best way to approach it is to > issue a true/false config switch, ie. sql_table_compat, for the purpose. > But for consistency wit

[pmacct-discussion] Source port column name depends on database

2010-09-13 Thread Chris Wilson
Hi all, We just had a bug report in pmGraph because it assumed that the source port database column was called "src_port" always, as it is in MySQL. The user is using a postgres database, and it appears that the column is called "port_src" there instead: if (!strcmp(config.type, "mysql") |

Re: [pmacct-discussion] MySQL index performance

2010-05-14 Thread Chris Wilson
Hi Paolo, On Thu, 13 May 2010, Paolo Lucente wrote: I think it would make no semantic difference, but would increase MySQL performance with these table types, if the primary key listed stamp_inserted first instead of last. The change you propose, as you say, would not be impacting - but wou

[pmacct-discussion] MySQL index performance

2010-05-12 Thread Chris Wilson
Hi all, I just noticed a factor that may be affecting the performance of pmacct when using a MySQL database in the default configuration. MySQL's InnoDB storage engine clusters records by the primary key, and it's recommended to make it a monotonically-increasing value: > As alluded to above,

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Chris Wilson
Hi Karl, On Fri, 19 Feb 2010, Karl O. Pinc wrote: > On 02/19/2010 07:42:08 AM, Chris Wilson wrote: > > > I deleted the primary key from that table because it should not be > > necessary (there should not be any duplicates if everything is > > configured correct

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Chris Wilson
Sorry, I realised just after I hit Send (as usual): On Fri, 19 Feb 2010, Chris Wilson wrote: > > I also wonder: how does the primary key of the 1 min table look like? > > Is it any different from the 1 hour table? With the sql_don_try_update > > turned on and the default in

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Chris Wilson
Hi Paolo and Daniel, (please allow me to jump in as I may be able to help here, despite currently being in country working on a project.) On Fri, 19 Feb 2010, Paolo Lucente wrote: > I also wonder: how does the primary key of the 1 min table look like? Is > it any different from the 1 hour tabl

Re: [pmacct-discussion] Missing information in mysql table

2010-02-09 Thread Chris Wilson
Hi Jonas, On Tue, 9 Feb 2010, Jonas Nylund wrote: > In logfile i see this > > Feb 9 16:47:46 oam02 sfacctd[14477]: DEBUG ( default/mysql ): INSERT INTO > `acct_v5_06` (stamp_updated, stamp_inserted, vlan, src_port, dst_port, tos, > ip_proto, agent_id, class_id, mac_src, mac_dst, ip_src, ip_dst,

Re: [pmacct-discussion] NAT question

2009-11-11 Thread Chris Wilson
Hi JF, On Thu, 12 Nov 2009, JF Cliche wrote: > I am behind two NAT routers (Linksys running DD-WRT) with port > forwarding up to the machine running pmacct, and yet pmacct reports SSH > traffic to the forwarded port with the public (external, non-NATed) > addresses. I thought all traffic shoul

Re: [pmacct-discussion] nfacct and MySQL

2009-10-14 Thread Chris Wilson
Hi Vitaliy, On Wed, 14 Oct 2009, Vitaliy Vladimirovich wrote: > 1. I use ng_netflow + nfacctd + bwstat on my firewall box (FreeBSD). How > can I specify nfacctd to use UNIX-socket instead inet for communicate > with MySQL. I'm not sure if it's possible at the monent. I would have suggested re

Re: [pmacct-discussion] packet logger

2009-09-30 Thread Chris Wilson
Hi Mada, On Tue, 22 Sep 2009, Mada R Perdhana wrote: > I'm want to use pmacct as a packet logger. The scenario is, I will put > pmacct at the main backbone line, then I want to log ALL the packet on > that line, then save it on a database backend. > > log all the packet like wireshark did, but

Re: [pmacct-discussion] timestamp rounding bug

2009-08-04 Thread Chris Wilson
Hi Paolo, On Mon, 3 Aug 2009, Paolo Lucente wrote: > Didn't act on it yet, being focused on some new features. My goal is to > do something about it in 0.12.0rc2. Basically it would be a fix for who > doesn't use an UTC clock on the system running pmacct. If there is > general interest around

Re: [pmacct-discussion] reloading config & accuracy

2009-06-22 Thread Chris Wilson
Hi Paolo, On Mon, 22 Jun 2009, Paolo Lucente wrote: > > 2. Accuracy of stats ? There have been quite a few emails to the list > > on how accurate netflow stats are and there have been instances where > > people have been getting twice the numbers they should have and other > > where there was

Re: [pmacct-discussion] Flexible aggregation

2009-06-14 Thread Chris Wilson
Hi Paolo and Karl, On Sat, 13 Jun 2009, Paolo Lucente wrote: > On Sat, Jun 13, 2009 at 03:07:01PM -0500, Karl O. Pinc wrote: > > >> We are only interested in a single table. > > > > Why can't two separate sql plugins write to the same table? > > What Karl is proposing here might really result i

Re: [pmacct-discussion] Flexible aggregation

2009-06-13 Thread Chris Wilson
Hi Paolo, On Sat, 13 Jun 2009, Paolo Lucente wrote: > > minb = 1, zero_dstip, minb = 1, zero_dstport, minb = 1, > > zero_srcport, minb = 1, zero_srcip > > > > Then any flows which together do not add up to enough bytes to pass > > the minb filters, even after aggregation, end u

Re: [pmacct-discussion] Flexible aggregation

2009-06-13 Thread Chris Wilson
Hi Paolo, On Sat, 13 Jun 2009, Paolo Lucente wrote: > Good pointer. From a brief scan of the Aguri homepage, please feel free > to correct whether i'm wrong, i see many similarities between pmacct and > Aguri. I guess so; I was thinking that Aguri seems to store its output in text files rathe

[pmacct-discussion] Flexible aggregation

2009-06-11 Thread Chris Wilson
Hi all, Has anyone heard of Aguri? "Aguri is an aggregation-based traffic profiler targeted for near real-time, long-term, and wide-area traffic monitoring. Aguri adapts itself to spatial traffic distribution by aggregating small volume flows into aggregates, and achieves temporal aggregation

Re: [pmacct-discussion] Ordering of packets in DB

2009-06-11 Thread Chris Wilson
Hi Suraj, On Thu, 11 Jun 2009, Suraj Nellikar (snellika) wrote: > I am analyzing the netflow packets in the mysql database. I see that > they are not ordered according to the flow. What is the criteria for > ordering the packets in the table? The "stamp updated" field shows the > same time for

Re: [pmacct-discussion] timestamp rounding bug

2009-04-20 Thread Chris Wilson
Hi Paolo, On Sun, 19 Apr 2009, Karl O. Pinc wrote: what makes sense to me is to collect timestamps in UTC, store them in UTC when storing them in a database, and let whatever's pulling the data out of the db present the data to the user in whatever fashion makes sense. Any other approach, i.

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Chris Wilson
Hi Johannes, On Mon, 13 Apr 2009, Johannes Formann wrote: Apr 13 17:20:01 server1 pmacctd[12419]: ERROR ( default/mysql ): Duplicate entry '0-00:1b:8f:61:55:c9-00:1c:c0:ab:8a:48-0-91.22.172.35-84.38.74.24' for key 1 >>> >>> As this crash is so early, perhaps the thread isn't i

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Chris Wilson
Hi Johannes, On Mon, 13 Apr 2009, Johannes Formann wrote: >> Paolo, this looks weird to me. pthread_getspecific() should not crash, >> that makes me think that the heap has been trashed (stack looks >> generally OK as the backtrace is OK). Perhaps a Valgrind is in order? >> Any static or fixed

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Chris Wilson
Hi Johannes, On Mon, 13 Apr 2009, Johannes Formann wrote: I'm not sure why "flows" is in your aggregate set since flows are already aggregated into flows in all cases by pmacctd, as far as I know (please correct me if I'm wrong). >>> >>> flow isn't in the primary key. >> >> I did

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Chris Wilson
Hi Johannes, On Mon, 13 Apr 2009, Johannes Formann wrote: >>> Apr 13 15:27:15 server kernel: pmacctd[1341]: segfault at f7002991 ip >>> f7bfa9ca sp ffb88334 error 4 in >>> libpthread-2.3.6.so[f7bf2000+e000] >> > I think I got it (using a written coredump): Yes, that's it, thank

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Chris Wilson
Hi Johannes, On Mon, 13 Apr 2009, Johannes Formann wrote: >>> I'll get this kind of error: Apr 13 15:27:15 server pmacctd[1341]: >>> ERROR ( default/mysql ): Duplicate entry >>> '0-00:1c:c0:74:5b:e9-00:00:0c:07:ac:6a-0-84.38.64.216-79.221.203.' for >>> key 1 >> >> I get this as well. It means

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Chris Wilson
Hi Johannes, On Mon, 13 Apr 2009, Johannes Formann wrote: > I'll get this kind of error: > Apr 13 15:27:15 server pmacctd[1341]: ERROR ( default/mysql ): Duplicate > entry '0-00:1c:c0:74:5b:e9-00:00:0c:07:ac:6a-0-84.38.64.216-79.221.203.' > for key 1 I get this as well. It means that the primary

Re: [pmacct-discussion] pmacct weird counters

2009-03-14 Thread Chris Wilson
Hi Paolo, On Sat, 14 Mar 2009, Paolo Lucente wrote: > Any signs of massive packet drops on any port throughout your switches? > I ask because the traffic reported might not have been actually > delivered to the end host. The switch has been up for 12.25 days, and in that time has recorded 2,0

Re: [pmacct-discussion] pmacct weird counters

2009-03-14 Thread Chris Wilson
Hi Karl, On Sat, 14 Mar 2009, Chris Wilson wrote: >> sum_net gets you a all the traffic to and from each network you list in >> your networks file, plus to and from anywhere else. The cross product. >> In your case, if you put only 192.168.0.0/24 in your networks file you

Re: [pmacct-discussion] pmacct weird counters

2009-03-14 Thread Chris Wilson
Hi Karl, On Sat, 14 Mar 2009, Karl O. Pinc wrote: >> Sorry, what is an aggregate on sum_net? I'm aggregating on ip_src and >> ip_dst respectively in two different plugins. > > sum_net gets you a all the traffic to and from each network you list in > your networks file, plus to and from anywhere

Re: [pmacct-discussion] pmacct weird counters

2009-03-14 Thread Chris Wilson
Hi Paolo, On Sat, 14 Mar 2009, Paolo Lucente wrote: > About the SQL INSERT conflict, are you by any chance making use of the > "sql_dont_try_update" directive in your configuration? Yes I am, because it's much more efficient. > And are you using 32bit counters? I think so, yes. I compiled with

Re: [pmacct-discussion] pmacct weird counters

2009-03-14 Thread Chris Wilson
Hi Karl, On Sat, 14 Mar 2009, Karl O. Pinc wrote: >> Do you have any ideas what might be going on here? > > Have you bound to an interface with 'interface'? > > Could be you're picking up, say, a file transfer to your gateway. > You'd want to monitor your external interface, or filter out traffi

[pmacct-discussion] pmacct weird counters

2009-03-14 Thread Chris Wilson
Hi Paolo, I'm running pmacctd 0.11.5 on a small network for traffic accounting. Generally it's behaving well, but occasionally I can see weird data being inserted: 17190 Query INSERT INTO `acct_v7` (stamp_updated, stamp_inserted, vlan, ip_dst, as_src, as_dst, src_port, dst_port, tcp_flags, tos

Re: [pmacct-discussion] HTTP Virtual Hosts classification

2009-02-18 Thread Chris Wilson
Hi all, On Wed, 18 Feb 2009, Paolo Lucente wrote: > In concept, and as documentation says, what you want to achieve is > feasible and your understanding of the classifier() is correct - you > only have to write down your own patterns: re-phrased, regular > expressions are typically employed to

Re: [pmacct-discussion] pmacct wildly inaccurate stats sometimes

2009-01-28 Thread Chris Wilson
Hi Mitch, On Wed, 28 Jan 2009, Mitch Pope wrote: > I really hope one of you can help, we're absolutely dieing with this > problem. We're an ISP and we run pmacct on a dedicated server that > connects to a distribution switch on the edge of our network. We use > pmacct strictly for data traffic

Re: [pmacct-discussion] multiple interfaces

2009-01-23 Thread Chris Wilson
Hi Mariano, On Fri, 23 Jan 2009, Mariano Spadaccini wrote: > Now the problem is only on the tagged port. But I have tried others > probe, with the same error (only unidirectional flows). > > However I have resolved with one pmacctd/one interface (untagged port). Have you tried using "any" as t

Re: [pmacct-discussion] multiple interfaces

2009-01-07 Thread Chris Wilson
Hi Anil and Juan, On Wed, 7 Jan 2009, Juan Rivera wrote: > My understanding is that any one instance of the daemon can only bind to > a single interface. I think that a workaround would be to run more than > one instance of the daemon, one per interface, and use a different > configuration fi

Re: [pmacct-discussion] pNRG and graphing

2008-10-21 Thread Chris Wilson
Hi Gregory, On Tue, 21 Oct 2008, Gregory Machin wrote: > Im trying to configure pmacctd to graph traffic passing through the > public interface of a firewall .. The public interface is connected to > an adsl router .. they share a dedicated private lan . The firewall's ip > is 192.168.42.1 and

Re: [pmacct-discussion] MySQL and Duplicate Primary Keys

2008-10-08 Thread Chris Wilson
Hi Paolo, On Wed, 8 Oct 2008, Paolo Lucente wrote: > Also, i see two different PIDs logging the duplication issue in your > email; whereas disabling the primary key the same tuple is written three > times; is it possible that there are multiple (3) concurrent pmacctd > instances running by mis

[pmacct-discussion] MySQL and Duplicate Primary Keys

2008-10-02 Thread Chris Wilson
Hi all, I always get a lot of errors like this when using pmacct on a MySQL database: Oct 2 06:26:01 fen-fw pmacctd[16237]: ERROR ( default/mysql ): Duplicate entry '00-0-0-217.160.76.21-10.0.156.226-4949-33730-tcp-0-2008-10-0' for key 1 Oct 2 06:26:01 fen-fw pmacctd[16239]: ERROR ( d

Re: [pmacct-discussion] How does pmacct divide between in and outbound traffic?

2008-08-04 Thread Chris Wilson
Hi Dennis, Dennis Kempin wrote: > I am currently trying to set up pmacct to account traffic between my host and > the internet. > > I account src and dst hosts without any filtering. > aggregate[out]: dst_host,src_host > aggregate[in]: dst_host,src_host > > Looking at the results i wondered h

Re: [pmacct-discussion] mysql plugin connect problem

2008-07-23 Thread Chris Wilson
Hi anil, Anil wrote: > ( default/mysql ) *** Purging cache - START *** > ERROR ( default/mysql ): PRIMARY 'mysql' backend trouble. > ERROR ( default/mysql ): The SQL server says: Access denied for user > 'admin'@'%.domain.com' to database 'bandwidth_db' > > ( default/mysql ) *** Purging cache -

Re: [pmacct-discussion] pmacct and nat ?

2008-07-07 Thread Chris Wilson
Hi Sebastien, Sébastien CRAMATTE wrote: >>> I'm running pmacctd on a natted network. >>> pmacctd account properply local traffic. My problem is that when I visit >>> a website o any other thing that is after the nat router (I'm >>> connected with cable modem) traffic is never accounted ! >>>

Re: [pmacct-discussion] pmacct and nat ?

2008-07-07 Thread Chris Wilson
Hi Sebastien, Sébastien CRAMATTE wrote: > I'm running pmacctd on a natted network. > pmacctd account properply local traffic. My problem is that when I visit > a website o any other thing that is after the nat router (I'm > connected with cable modem) traffic is never accounted ! > > Does th

Re: [pmacct-discussion] Measurement accuracy issues

2008-06-10 Thread Chris Wilson
Hi Ahmed, On Tue, 10 Jun 2008, Ahmed Kamal wrote: > I have setup pmacct with your help, and it's been running like a champ. I > have also installed darkstat for comparison. I am seeing a big error (around > 30%) between the 2 tools! ... > Here's what I am seeing: > > IP START

Re: [pmacct-discussion] pmacctd & transparent proxy

2006-12-21 Thread Chris Wilson
ken into two connections in the pmacct database (which just means that it is reflecting reality); and your pmacct client software needs to be modified to take advantage of the new table. Cheers, Chris. -- (aidworld) chris wilson | chief engineer ([EMAIL PROTECTED]) ___

Re: [pmacct-discussion] Performance survey

2006-11-19 Thread Chris Wilson
e due to load from pmacctd threads, but that was when it was running a 2.4 kernel on FC1, and it has since been upgraded to FC2. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Classification

2006-11-19 Thread Chris Wilson
gt; index based on date_trunc(), so you even don't need it for that purpose. > So I suggest just first and last packet as time stamps. Do you mean the first 'n' characters of the date field? That wouldn't be able to represent e.g. a 5-minute roundoff. > BTW.: I think it's really a mistake to use local time as timestamps. Why > not using seconds since 1970/1/1 0:00 UTC? This is standard and > unambiguous. I agree that this should be changed. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Classification

2006-11-17 Thread Chris Wilson
my RRD > related stuff. Have you considered making cacti templates to go along with > your scripts? Cacti templates are easily exported as XML for everyone to > use... This graph is way beyond what you could do with Cacti. I'm only using RRD because it draws pretty graphs, I proc

Re: [pmacct-discussion] Classification

2006-11-17 Thread Chris Wilson
lish these scripts when they're ready, and hope that they will be useful to other pmacct users. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Classification

2006-11-15 Thread Chris Wilson
l_refresh_time, you simply get fewer updates/inserts, no data loss except temporal resolution. What do you think about that idea? Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Classification

2006-11-12 Thread Chris Wilson
it will degrade gracefully under load, unlike the current situation. Paolo, please could you help us to do something about this? It appears to be a real problem with pmacct that affects several users. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) __

Re: [pmacct-discussion] Classification

2006-11-07 Thread Chris Wilson
tand what you mean by a time slot? For me, the relevant information is the start and end times of the flow, which I can use to draw graphs, etc. Ideally, I would like more detailed information about the flow at various points during its life (e.g. status every minute) and I'm not sure if I can get that using pmacctd, or how. I'm still working on it. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] Classification

2006-11-07 Thread Chris Wilson
Netflow. I'm considering implementing some of this functionality for the pmacct suite. I'm still concerned about the performance of the MySQL plugin with threading, so I'm considering providing an option to disable the extra threads, and run updates synchronously. I'd be ver

Re: [pmacct-discussion] Large number of threads

2006-10-18 Thread Chris Wilson
ly won't be able to even try pmacct-fe if I can't get Horde working, because it looks good and I really want to try it. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

[pmacct-discussion] Large number of threads

2006-10-18 Thread Chris Wilson
ers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org) ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

[pmacct-discussion] Classification

2006-10-18 Thread Chris Wilson
sified packets from the kernel with ULOG (or divert sockets on BSD), find a way to include the netfilter mark/connmark in the exported packets, and replace the pcap capture code with ULOG/divert capture code. Does anyone have any thoughts on this? Cheers, Chris. -- (aidworld) chris

[pmacct-discussion] Quiet mode and clearing counters

2006-10-18 Thread Chris Wilson
t;); prints a message but does nothing else. That doesn't seem very useful. Cheers, Chris. -- (aidworld) chris wilson | chief engineer (http://www.aidworld.org)Only in pmacct-chris: config.cache Only in pmacct-chris: config.log Only in pmacct-chris: config.status Only in pmacct-chris: Makefile