Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-07-11 Thread David Notivol
Hi Luca, Sorry for the delay replying the list. Thanks for your advice. I tested reducing the template to just the minimal fields, and the performance was way better. Right now, I left the template with ~50 fields (instead of ~150), and leaving zbalance in a dedicated core (not sharing threads w

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-27 Thread Luca Deri
Hi David your template is huge. Can you please omit (just for troubleshooting) "--flow-templ….” and report if you see changes in load? Thanks Luca > On 27 Jun 2018, at 08:43, David Notivol wrote: > > Hi, > And now: > - 1.log = scenario in your point 1, including top, zbalance output, and > np

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-27 Thread David Notivol
Hi, And now: - 1.log = scenario in your point 1, including top, zbalance output, and nprobe stats. El mié., 27 jun. 2018 a las 17:41, David Notivol () escribió: > Hi Alfredo, > > Sorry, I forgot to attach the files as you said. I sent them awhile ago, > but it seems the mail size is over the limi

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-27 Thread David Notivol
Hi Alfredo, Sorry, I forgot to attach the files as you said. I sent them awhile ago, but it seems the mail size is over the limit and get held for approval. I'm trying now deleting some info from my first email and pasting one file at a time. - 0.log = top output for the scenario in my fist email

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-27 Thread Alfredo Cardigliano
Hi David > On 27 Jun 2018, at 14:20, David Notivol wrote: > > Hi Alfredo, > Thanks for your recommendations. > > I tested using core affinity as you suggested, and the in drops disappeared > in zbalance. The output drops persist, but the absolute drops are less than > before. > Actually I ha

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-27 Thread David Notivol
Hi Alfredo, Thanks for your recommendations. I tested using core affinity as you suggested, and the in drops disappeared in zbalance. The output drops persist, but the absolute drops are less than before. Actually I had tested the core affinity, but I didn't have in mind the physical cores. Now I

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-27 Thread Alfredo Cardigliano
Hi David it seems that you have packet loss both on zbalance and nprobe, I recommend you to: 1. set the core affinity for both zbalance_ipc and the nprobe instances, trying to use a different core for each (at least do not share the zbalance_ipc physical core with nprobe instances) 2. did you try

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-26 Thread David Notivol
Hi Alfredo, Thanks for replying. This is an excerpt of the zbalance and nprobe statistics: 26/Jun/2018 17:29:58 [zbalance_ipc.c:265] = 26/Jun/2018 17:29:58 [zbalance_ipc.c:266] Absolute Stats: Recv 1'285'430'239 pkts (1'116'181'903 drops) - Forwarded 1'266'272'285 pkts (19'

Re: [Ntop-misc] nProbe performance, zbalance packet drops

2018-06-26 Thread Alfredo Cardigliano
Hi David please also provide statistics from zbalance_ipc (output or log file) and nprobe (you can get live stats from /proc/net/pf_ring/stats/) Thank you Alfredo > On 26 Jun 2018, at 15:32, David Notivol wrote: > > Hello list, > > We're using nProbe to export flows information to kafka. We're

[Ntop-misc] nProbe performance, zbalance packet drops

2018-06-26 Thread David Notivol
Hello list, We're using nProbe to export flows information to kafka. We're listening from two 10Gb interfaces that we merge with zbalance_ipc, and we split them into 16 queues to have 16 nprobe instances. The problem is we are seeing about 40% packet drops reported by zbalance_ipc, so it looks li