In case it helps anyone else reading this, my startup script needed some corrections, ending up with:
for i in `seq 0 1 15`; do snort -q -u snort -g snort --pid-path /var/run --create-pidfile -D -c /etc/snort/snort.conf -l /logs/snort/eth4_eth5/instance-$i --daq-dir=/usr/local/lib/daq --daq pfring_zc --daq-mode passive -i zc:eth4@ $i,zc:eth5@$i --daq-var clusterid=$i --daq-var bindcpu=$i done i.e. I needed to remove idsbridge=1 (might be a mistake that the IDS with multiqueue example does have this set in README.1st I linked earlier?) and I needed to change my variable to be $i instead of $1. However when I run this script it only start's 4 "daemon child", then gives a "bus error" on the next 4 and then returns to the command line with no mention of the other 8. /var/log/messages tells me: snort[4888]: FATAL ERROR: Can't initialize DAQ pfring_zc (-1) - pfring_zc_daq_initialize: Cluster failed: No buffer space available (error 105) ZC[4897]: error mmap'ing hugepage /mnt/huge/pfring_zc_14: Cannot allocate memory ZC[4897]: error mmap'ing 128 hugepages of 2048 KB If it's relevant, when I run the ZC load_driver script (I took the MQ=1,1,1,1 out as advised, so that just has "insmod ./ixgbe.ko RSS=16,16,16,16") that says: Warning: 512 hugepages available, 1024 requested Things I have checked are: sudo more /sys/kernel/mm/transparent_hugepage/enabled always madvise [never] sudo more /proc/sys/vm/nr_hugepages 1024 sudo more /proc/meminfo MemTotal: 32748700 kB MemFree: 27563604 kB <snip> AnonHugePages: 0 kB HugePages_Total: 1024 HugePages_Free: 1024 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Any ideas on how I can fix this please? Thanks J. On 2 December 2015 at 15:10, Alfredo Cardigliano <[email protected]> wrote: > > On 02 Dec 2015, at 16:08, James <[email protected]> wrote: > > Ah.. so if only 16 queues, I should go back to only 16 copies of snort? > > > Yes, or you can consider using zbalance_ipc for load balance in software > to more queues, > but I do not what is the performance you can reach with 48 queues, you > should run some test. > > Alfredo > > > On 2 December 2015 at 15:06, Alfredo Cardigliano <[email protected]> > wrote: > >> Sorry, forgot to tell you RSS on ixgbe supports up to 16 queues. >> >> Alfredo >> >> On 02 Dec 2015, at 16:06, Alfredo Cardigliano <[email protected]> >> wrote: >> >> It looks fine, you can omit MQ. >> >> Alfredo >> >> On 02 Dec 2015, at 16:04, James <[email protected]> wrote: >> >> Many thanks for the help Alfredo. So I'll crank things up to use all >> CPU's and that gives me (I've converted it to a for loop): >> >> for i in `seq 0 1 48`; do >> snort -q --pid-path /var/run --create-pidfile -D -c /etc/snort/snort.conf >> -l /logs/snort/eth4_eth5/instance-$1 --daq-dir=/usr/local/lib/daq --daq >> pfring_zc --daq-mode passive -i zc:eth4@$1,zc:eth5@$1 --daq-var >> clusterid=$1 --daq-var idsbridge=1 --daq-var bindcpu=$1 >> done >> >> Is this the correct load_driver.sh setting to match? I'm not sure about >> the MQ values? >> insmod ./ixgbe.ko MQ=1,1,1,1 RSS=48,48,48,48 >> >> On 2 December 2015 at 14:15, Alfredo Cardigliano <[email protected]> >> wrote: >> >>> Please use README.1st as reference. >>> What you need to know: >>> 1. Use --daq-var clusterid=K where K is a unique number per snort >>> instance, used for resource allocation >>> 2. Use --daq-var bindcpu=K where K is the core id for affinity, please >>> ignore interrupts affinity with ZC >>> 3. Use “,” in -i in please of “+” for interfaces aggregation, “+” is >>> used for IPS/IDS-bridge mode >>> 4. We usually recommend using only the CPU where the NIC is connected, >>> however since snort is (likely) the bottleneck, feel free to use all the >>> cores available, setting RSS=N,N where N is the number of cores and the >>> number of snort instances. >>> >>> Alfredo >>> >>> On 02 Dec 2015, at 15:08, James <[email protected]> wrote: >>> >>> Follow-up question - should I use the cluster-id parameter? >>> >>> This uses it: >>> >>> https://svn.ntop.org/svn/ntop/trunk/attic/PF_RING/userland/snort/pfring-daq-module-zc/README.1st >>> >>> But this does not: >>> http://www.ntop.org/pf_ring/accelerating-snort-with-pf_ring-dna/ >>> >>> On 2 December 2015 at 14:01, James <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> I posted a few weeks ago and have since got pf_ring with ZC working. >>>> I'm now trying to decide how best to configure snort (in IDS mode). My >>>> server has 4 X 12 core CPU's and two NIC's which are being fed one half >>>> each of a 10Gb connection. >>>> >>>> I have a few key questions: >>>> - Within the ixgbe zc load_drive.sh script, would the default 16 queue >>>> option do, or would you choose something different: insmod ./ixgbe.ko >>>> MQ=1,1,1,1 RSS=16,16,16,16 >>>> >>>> - Assuming the choice of 16 above, should I start 16 copies of Snort >>>> like this (variation on the example from ntop website)? >>>> snort -q --pid-path /var/run --create-pidfile -D -c >>>> /etc/snort/snort.conf -l /var/log/snort/eth4_eth5/instance-1 >>>> --daq-dir=/usr/local/lib/daq --daq pfring_zc --daq-mode passive -i >>>> zc:eth4@0 >>>> +zc:eth5@0 --daq-var idsbridge=1 --daq-var bindcpu=0 >>>> >>>> The information on http://www.metaflows.com/features/pf_ring about CPU >>>> affinity and interrupts has confused me somewhat. >>>> >>>> Thanks >>>> J. >>>> >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >> >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> >> >> >> >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc >
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
