Hi Andras, Inline:
On Wed, Jul 03, 2013 at 01:29:38PM +0200, Andras Horvai wrote: > So in mongodb I have the historical data of connection and in mysql I have > the data what I can use for accounting (how many bytes a host receives and > sends). I would like to use one database backands but I did not find info > how to do this with mongodb only. What precisely (which behaviour or so) you do manage to achieve with MySQL that you cannot with MongoDB? Was reading your config and though it was hourly counters (sql_history: 1h) refreshing them every 5 mins (sql_refresh_time: 300) which would make sense: we do not support the update of records in MongoDB (but you can indeed always make sums). But then i spotted in your config a sql_dont_try_update set to true .. so i'm a bit puzzled. > Another thing is that I restart the pmacct everyday at midnight and create > a separate table (naming with the actual date) in every database. So for > every day I have the data in a separate table. This you can avoid by using dynamic table names, ie. to recall your MongoDB table name "acctv5.netflowsender_%Y_%m_%d". With MongoDB this is it since it's schemaless. With MySQL you can achieve the same but please read docs on how to feed the schema accordingly. Please refer to sql_table and sql_table_schema entries of the CONFIG-KEYS doc. Cheers, Paolo > On Tue, Jun 18, 2013 at 6:16 PM, Andras Horvai <andras.hor...@gmail.com>wrote: > > > Thanks Edward, > > > > I will check triggers.sql. > > > > Andras > > > > > > On Tue, Jun 18, 2013 at 5:04 PM, Edward van Kuik <edw...@nitric.co.za>wrote: > > > >> Hi > >> > >> What I do is I use triggers which create some summary tables for me as it > >> inserts. > >> I've attached my trigger sql in case you find it helpful. > >> > >> I insert it into tables with partitioning by range. But the system is > >> generally still quite slow. (I currently have 1.3 billion rows) > >> I am thinking of inserting the data and summary data into another type of > >> database, like mongo. > >> > >> Edward > >> > >> > >> > >> > >> On 18/06/2013 16:34, Andras Horvai wrote: > >> > >> Hello, > >> > >> > >> This is what I would like to achieve: > >> I would like to have a history of ip connections of my network, let's > >> say > >> back to 90 days. So to achieve this I understand that I have to aggregate > >> in > >> nfacctd. > >> Here is the aggregate what I tried: > >> > >> aggregate: src_host,dst_host,src_port,dst_port,proto > >> > >> The problem is when I do this type of aggregate my system slows > >> down very much. (I cannot run a simple query in mysql so I cannot > >> get the information what I need... so the collection is useless). > >> > >> I tried to distinguish the in and out traffic with plugins, > >> but did not help me regarding the performance. > >> The only way what does not slows down my system if I aggregate only > >> src or dst addresses (and separate these into different tables). > >> But with this only aggregate, I lost the information about the src_port, > >> dst_port and protocol (per ip pairs) > >> > >> I would like to monitor the traffic of 400 hosts. > >> Is it possible with a server with > >> 4 cores (Intel(R) Xeon(R) CPU E5603 @ 1.60GHz) > >> and with 4 GB of rams. > >> > >> Flow exporter is a L3 cisco siwtch in the core of the network. It has > >> vlans defined to them. I am interested in the traffic between local > >> vlanand local > >> vlan, and between local vlan and Internet (or external network). > >> > >> Thanks, > >> > >> Andras > >> > >> > >> _______________________________________________ > >> pmacct-discussion mailing listhttp://www.pmacct.net/#mailinglists > >> > >> > >> > >> -- > >> Edward van Kuik > >> Nitric Software Laboratory > >> 78 Strand Street > >> Tel: 021 300 1073 > >> Cell: 083 645 6443http://nitric.co.za/ > >> > >> > >> _______________________________________________ > >> pmacct-discussion mailing list > >> http://www.pmacct.net/#mailinglists > >> > > > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
