Michael you have ~92 flows/sec that is a bit too much for sqlite in its current implementation. Thanks for the feed back, we have some homework on our side
Luca On 13 Oct 2014, at 23:32, Michael Pollock <[email protected]> wrote: > I retrieved the following from the Cacti FlowView Summary screen. I put in > the timeframe going from 14:00 to 14:05 today. > > Total Flows 27,611 > Total Packets 5,851,868 > Duration of data (realtime) 343 > Average flow time (1/1000 secs) 315,311,552 > Average flow size (octets) 129,865 > Average flows / second (flow) 28 > Average Kbits / second (flow) 29,391 > Total Octets 3,585,696,719 > Total Time (1/1000 secs) (flows) 8,706,066,882,992 > Duration of data (1/1000 secs) 976,000 > Average packet size (octets) 613 > Average packets per flow 212 > Average flows / second (real) 80 > Average Kbits / second (real) 83,631 > > > > Michael Pollock > Systems Engineer > > <image001.gif> > 4400 River Green Parkway, Suite 100 > Duluth, GA 30096 > > Mobile: 678-520-0672 > [email protected] > www.possiblenow.com > > > From: [email protected] > [mailto:[email protected]] On Behalf Of Michael Pollock > Sent: Monday, October 13, 2014 5:07 PM > To: [email protected] > Cc: Lorenzo Mangani > Subject: Re: [Ntop] How to reduce size of historical interface database > > How could I determine that? Do I need to enable the historical interface > temporarily? > > I added the Cacti FlowView plugin as another destination from nProbe (along > with -a so both tools get all the data) and I can get the data from there, > but I’d prefer a more user-friendly interface like ntopng for others to be > able to view the data. > > > > Michael Pollock > Systems Engineer > > <image001.gif> > 4400 River Green Parkway, Suite 100 > Duluth, GA 30096 > > Mobile: 678-520-0672 > [email protected] > www.possiblenow.com > > > From: [email protected] > [mailto:[email protected]] On Behalf Of Luca Deri > Sent: Monday, October 13, 2014 2:59 PM > To: [email protected] > Cc: Lorenzo Mangani > Subject: Re: [Ntop] How to reduce size of historical interface database > > Michael, > just to size the problem, how many flows ntopng generates every 5 minutes? > > What you are asking is close to what we had in mind. i think that with some > further iteration we can come up with a precise list of requirements > > Cheers Luca > > On 13 Oct 2014, at 18:22, Michael Pollock <[email protected]> wrote: > > > Say I click on the interface (tcp://*:5556). If I then click the tab for > “historical activity”, I see a graph of how much was being used over a period > of time, which is what I expect. But next to that is “Protocols”. I’d love > to keep that data somehow, too. Also, you added a magnifying glass next to > each protocol. If I click that, I only get taken to “active flows”. I have > some that have many TB of data over time. I’d like to be able to go back and > see what was using that data and when. > > For example, we have a minor bit of slowness in our development network > around 2:00pm every day. I can’t figure out exactly how to find what was > using all the bandwidth during that time without using a historical > interface, but unfortunately that takes up so much data that it’s basically > unusable for me. > > Thanks! > > > Michael Pollock > Systems Engineer > > <image001.gif> > 4400 River Green Parkway, Suite 100 > Duluth, GA 30096 > > Mobile: 678-520-0672 > [email protected] > www.possiblenow.com > > From: [email protected] > [mailto:[email protected]] On Behalf Of Luca Deri > Sent: Monday, October 13, 2014 7:16 AM > To: [email protected] > Subject: Re: [Ntop] How to reduce size of historical interface database > > Michael, > if you have this traffic increase then you have a lot of traffic to monitor. > is this the case? What would you like to save exactly? > > We have started to add ElasticSearch support in ntopng, and this is another > option to use instead of SQlite. But all these are experiments until we find > a solution we like. > > Does anyone has a suggestion perhaps? > > Regards Luca > > On 10/10/2014 05:44 PM, Michael Pollock wrote: > I enabled the historical interface, but the database was growing at about 1 > GB every 5 minutes. Is there any way to reduce this? > > It looks like SQLite (which I am not familiar with, unfortunately) is just a > series of plain text files. Is there any way to compress this or just store > the overall usage data? Without historical data, this utility isn’t nearly > as useful. > > Thanks for anyone who can offer a suggestion! > > > Michael Pollock > Systems Engineer > > <image001.gif> > 4400 River Green Parkway, Suite 100 > Duluth, GA 30096 > > Mobile: 678-520-0672 > [email protected] > www.possiblenow.com > > The information contained in this communication is confidential, is intended > only for the use of the recipient named above, and may be legally privileged. > If the reader of this message is not the intended recipient, you are hereby > notified that any dissemination, distribution or copying of this > communication is strictly prohibited. If you have received this communication > in error, please resend this communication to the sender and delete the > original message or any copy of it from your computer system > > > > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > The information contained in this communication is confidential, is intended > only for the use of the recipient named above, and may be legally privileged. > If the reader of this message is not the intended recipient, you are hereby > notified that any dissemination, distribution or copying of this > communication is strictly prohibited. If you have received this communication > in error, please resend this communication to the sender and delete the > original message or any copy of it from your computer system > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > The information contained in this communication is confidential, is intended > only for the use of the recipient named above, and may be legally privileged. > If the reader of this message is not the intended recipient, you are hereby > notified that any dissemination, distribution or copying of this > communication is strictly prohibited. If you have received this communication > in error, please resend this communication to the sender and delete the > original message or any copy of it from your computer system > > The information contained in this communication is confidential, is intended > only for the use of the recipient named above, and may be legally privileged. > If the reader of this message is not the intended recipient, you are hereby > notified that any dissemination, distribution or copying of this > communication is strictly prohibited. If you have received this communication > in error, please resend this communication to the sender and delete the > original message or any copy of it from your computer system > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
