OK loaded with just "insmod ixgbe.ko RSS=1,1 mtu=1522", and it works for "-n 12". If I use my custom pfdnacluster_master that duplicates traffic to another 12 queues, then it fails again (I guess I should try the original pfdnacluster_master with "-n 24").
It would be useful to know what the memory limitations are! Anyway, I'm running Suricata on 12 cores at the moment, which is the important thing. Best Wishes, Chris On 08/10/12 13:00, Alfredo Cardigliano wrote: > > On Oct 8, 2012, at 1:56 PM, Chris Wakelin <[email protected]> wrote: > >> Actually I'm wondering about the FdirPballoc; I'm not actually using >> hardware filters at the moment (though may want to in the future), so I >> can probably have FdirMode=0? > > Yes > > Alfredo > >> >> Best Wishes, >> Chris >> >> On 08/10/12 12:51, Alfredo Cardigliano wrote: >>> >>> On Oct 8, 2012, at 1:46 PM, Chris Wakelin <[email protected]> wrote: >>> >>>> I've got 32GB, but it's a 3.2.0 kernel (Ubuntu 12.04 64-bit). I tried >>>> >>>> insmod ixgbe.ko RSS=1,1 FdirPballoc=3,3 mtu=1522 num_rx_slots=2048 >>>> num_tx_slots=0 >>> >>> Ok, I will check the same kernel version/driver options. >>> >>>> >>>> but it didn't seem to make any difference. >>>> >>>> pf_ring is loaded with "transparent_mode=2 min_num_slots=4096" >>> >>> These settings are ignored with DNA. >>> >>> Alfredo >>> >>>> >>>> Best Wishes, >>>> Chris >>>> >>>> On 08/10/12 12:35, Alfredo Cardigliano wrote: >>>>> Christ >>>>> How much memory do you have on this system? >>>>> On my system (kernel 2.6.36, 4Gb of RAM, ixgbe) it is working fine. >>>>> Please try decreasing the num_rx_slots (actually memory allocation >>>>> happens at pfdnacluster_master boot, but it's worth giving it a try >>>>> because it's possible you are running out of memory). >>>>> >>>>> Regards >>>>> Alfredo >>>>> >>>>> On Oct 8, 2012, at 1:17 PM, Chris Wakelin <[email protected]> >>>>> wrote: >>>>> >>>>>> Any ideas on this? Is this related to the 32768 num_rx_slots on ixgbe? >>>>>> >>>>>> I'd like to get somewhere with this as with just 8 cores, I'm dropping >>>>>> packets. >>>>>> >>>>>> Best Wishes, >>>>>> Chris >>>>>> >>>>>> On 01/10/12 17:50, Chris Wakelin wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've got a strange problem when trying to increase the number of cores >>>>>>> used by my Suricata+Bro IDS(+ARGUS) setup. When I try to get Suricata to >>>>>>> use more than 8 devices, it fails on my Ubuntu 12.04 (kernel 3.2.0) >>>>>>> boxes. However on my Ubuntu 10.04 (kernel 2.6.32) box it seems OK. >>>>>>> >>>>>>> Strangely, there's no problem running Suricata, using libpfring, on >>>>>>> dnacluster:1@0 .. 1@7 and Bro IDS, using lipcap, on dnacluster:1@8 .. >>>>>>> 1@15 (with my custom pfdnacluster_master that duplicates traffic). >>>>>>> >>>>>>> I can reproduce the problem with pfcount_aggregator (with "#define >>>>>>> MAX_NUM_DEVS 16" rather than 8):- >>>>>>> >>>>>>>> pfdnacluster_master -i dna0 -c 1 -n 12 >>>>>>> >>>>>>>> pfcount_aggregator_cdw -i >>>>>>>> dnacluster:1@0+dnacluster:1@1+dnacluster:1@2+dnacluster:1@3+dnacluster:1@4+dnacluster:1@5+dnacluster:1@6+dnacluster:1@7+dnacluster:1@8+dnacluster:1@9+dnacluster:1@10+dnacluster:1@11 >>>>>>>> -l 1522 >>>>>>>> Using PF_RING v.5.4.5 >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> Impossible to know the device address >>>>>>>> # Device RX channels: 1 >>>>>>>> pfring_set_direction returned [rc=-7][direction=0] >>>>>>>> pfring_open error [Cannot allocate memory] (pf_ring not loaded or >>>>>>>> perhaps you use quick mode and have already a socket bound to >>>>>>>> dnacluster:1@8 ?) >>>>>>> >>>>>>> >>>>>>> The 12.04 boxes have ixgbe devices loaded with "insmod ixgbe.ko RSS=1,1 >>>>>>> FdirPballoc=3,3 mtu=1522". >>>>>>> >>>>>>> The 10.04 box has e1000e (DNA demo license) loaded with just "insmod >>>>>>> e1000e". >>>>>>> >>>>>>> Is this expected? >>>>>>> >>>>>>> Best Wishes, >>>>>>> Chris >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- >>>>>> Christopher Wakelin, [email protected] >>>>>> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908 >>>>>> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094 >>>>>> _______________________________________________ >>>>>> Ntop-misc mailing list >>>>>> [email protected] >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>> >>>>> _______________________________________________ >>>>> Ntop-misc mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>> >>>> >>>> >>>> -- >>>> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- >>>> Christopher Wakelin, [email protected] >>>> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908 >>>> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094 >>>> _______________________________________________ >>>> Ntop-misc mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >> >> >> -- >> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- >> Christopher Wakelin, [email protected] >> IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908 >> Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094 >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > -- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, [email protected] IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908 Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094 _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
