Hi, Could you please share ntopng configuration used? I think your setup doesn't allow ntopng to be quick enough. Remember that there is one thread per monitored interface and that thread has to 1. capture packets / receive flows 2. handle them 3. export to ES
Simone On Wed, Jun 29, 2016 at 12:33 AM, Andris Bjornson <and...@everylayer.com> wrote: > Hello, > > Hoping to find a little help here after unsuccessfully googling quite a > bit. > > I've successfully setup the latest dev version of ntopng to dump flows > using --dump-flows into elasticsearch. Elasticsearch is on the same > machine as ntopng. My total traffic volume to process is about ~130Mbps > peak. > > It works very well, but i think i am losing a lot of flows in the export > process. > > My ntopng log file is rapidly filled (at the rate of ~600 per second) like > the following: > > 29/Jun/2016 01:26:02 [ElasticSearch.cpp:64] WARNING: [ES] Message dropped. > Total messages dropped: 2799026 > > However, I don't think this is an elasticsearch capacity problem, because > I am not seeing the errors in elasticsearch.log that would normally > accompany elasticsearch running out of capacity. I'm monitoring iostat, > system load, and elasticsearch performance via marvel - and those all look > good. > > I'm not sure where to look next for more information about what might be > causing the "message dropped" logs. > > Any help much appreciated! > > Andris > > > > --- > Andris Bjornson | EveryLayer <http://www.everylayer.com/> > skype: andris.bjornson > > _______________________________________________ > Ntop mailing list > Ntop@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop >
_______________________________________________ Ntop mailing list Ntop@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop
