Hi Jesse there is something strange actually, with RSS=1,1 I would expect to see a single queue per interface, but it seems you have multiple queues on the second interface enabled. Please let me see the output of cat /proc/net/pf_ring/dev/p2p1/info
Alfredo > On 11 Nov 2014, at 16:06, Jesse Bowling <[email protected]> wrote: > > Anything else I should be trying on this? SVN version? Seems more like a > configuration error, but willing to try whatever at this point... > > Cheers, > > Jesse > On Nov 5, 2014, at 9:57 AM, Jesse Bowling <[email protected]> wrote: > >> Hi Alfredo, >> >> Thank you for the speedy response. I did use modprobe rather than insmod, >> but that should be fine, right? I was using a .conf file in /etc/modprobe.d, >> but removed that and used the below load_driver.sh; the output was different >> this time but the result was the same (works with no BPF filter, doesn’t >> work with a filter)...These cards have one port per card... >> >> # egrep -v '^[ ]*#|^[ ]*$' load_driver.sh >> FAMILY=ixgbe >> modprobe -r ixgbe >> modprobe -r pf_ring >> echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages >> mkdir /mnt/huge >> mount -t hugetlbfs nodev /mnt/huge >> modprobe pf_ring >> modprobe ixgbe RSS=1,1 >> sleep 1 >> killall irqbalance >> INTERFACES=$(cat /proc/net/dev|grep ':'|grep -v 'lo'|grep -v 'sit'|awk -F":" >> '{print $1}'|tr -d ' ') >> for IF in $INTERFACES ; do >> TOCONFIG=$(ethtool -i $IF|grep $FAMILY|wc -l) >> if [ "$TOCONFIG" -eq 1 ]; then >> printf "Configuring %s\n" "$IF" >> ifconfig $IF up >> sleep 1 >> bash /root/set_irq_affinity $IF >> ethtool -G $IF rx 32768 >> ethtool -G $IF tx 32768 >> fi >> done >> >> # ./load_driver.sh >> mkdir: cannot create directory `/mnt/huge': File exists >> irqbalance: no process killed >> Configuring p1p1 >> no rx vectors found on p1p1 >> no tx vectors found on p1p1 >> p1p1 mask=1 for /proc/irq/61/smp_affinity >> Configuring p2p1 >> no rx vectors found on p2p1 >> no tx vectors found on p2p1 >> p2p1 mask=1 for /proc/irq/63/smp_affinity >> p2p1 mask=2 for /proc/irq/64/smp_affinity >> p2p1 mask=4 for /proc/irq/65/smp_affinity >> p2p1 mask=8 for /proc/irq/66/smp_affinity >> p2p1 mask=10 for /proc/irq/67/smp_affinity >> p2p1 mask=20 for /proc/irq/68/smp_affinity >> p2p1 mask=40 for /proc/irq/69/smp_affinity >> p2p1 mask=80 for /proc/irq/70/smp_affinity >> p2p1 mask=100 for /proc/irq/71/smp_affinity >> p2p1 mask=200 for /proc/irq/72/smp_affinity >> p2p1 mask=400 for /proc/irq/73/smp_affinity >> p2p1 mask=800 for /proc/irq/74/smp_affinity >> p2p1 mask=1000 for /proc/irq/75/smp_affinity >> p2p1 mask=2000 for /proc/irq/76/smp_affinity >> p2p1 mask=4000 for /proc/irq/77/smp_affinity >> p2p1 mask=8000 for /proc/irq/78/smp_affinity >> >> On Nov 5, 2014, at 5:26 AM, Alfredo Cardigliano <[email protected]> wrote: >> >>> Hi Jesse >>> it seems the script is loading the driver with RSS enabled, this means that >>> your application is capturing from the first queue only. >>> Can I see your load_driver.sh? I want to make sure you are passing RSS=1,1 >>> (as many 1 as the number of interfaces) to the driver. >>> >>> Alfredo >>> >>>> On 05 Nov 2014, at 03:16, Jesse Bowling <[email protected]> wrote: >>>> >>>> Hi, >>>> >>>> I’m having trouble using ZC drivers. I’ll include the relevant details of >>>> loading, etc at the bottom. Essentially I can’t get my usual apps (such as >>>> argus) which I compile against libpcap to recognize interfaces in the >>>> usual combinations, in ZC or non-ZC modes. I’m falling back to just >>>> testing the tcpdump distributed with PF_RING, and still having some >>>> issues. There’s a good chance these are based on my own misconceptions, so >>>> please bear with me. :) >>>> >>>> Initially I tried doing a tcpdump with a similar interface call as used to >>>> work in the 5.X series: >>>> >>>> # ldd /usr/local/sbin/tcpdump >>>> linux-vdso.so.1 => (0x00007fff4ff8b000) >>>> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003959e00000) >>>> libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x000000395b200000) >>>> libc.so.6 => /lib64/libc.so.6 (0x0000003959a00000) >>>> /lib64/ld-linux-x86-64.so.2 (0x0000003959600000) >>>> >>>> # /usr/local/sbin/tcpdump -i 'p1p1;p2p1' -nn -c 100 >>>> <snip> >>>> 100 packets captured >>>> 100 packets received by filter >>>> 0 packets dropped by kernel >>>> >>>> So far so good. After playing a bit, I could not figure out how to do this >>>> with the ZC drivers directly; as this may be part of the issue with using >>>> argus, I’d like to know if it’s possible to call tcpdump in this fashion >>>> and invoke the ZC drivers...Could you provide an example if this is >>>> possible? >>>> >>>> The next issue I ran into was related to using BPF filters; I could not >>>> get this to work using non-ZC drivers...For instance, this tcpdump will >>>> capture no packets: >>>> >>>> # /usr/local/sbin/tcpdump -i 'p1p1;p2p1' -nn -c 100 "ip" >>>> tcpdump: WARNING: SIOCGIFADDR: p1p1;p2p1: No such device >>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >>>> listening on p1p1;p2p1, link-type EN10MB (Ethernet), capture size 8192 >>>> bytes >>>> ^C >>>> 0 packets captured >>>> 0 packets received by filter >>>> 0 packets dropped by kernel >>>> >>>> Since this shouldn’t be using ZC at all, and this invocation worked on >>>> previous versions, it seems this must be a bug? >>>> >>>> Another area I’m having trouble understanding is the use of the z* example >>>> programs...It would seem that I should be using zbalance_ipc to put the >>>> two streams together and then share them to multiple processes...Trying to >>>> do this to a single process exhibits the same issues with filters: >>>> >>>> # ./zbalance_ipc -i p1p1,p2p1 -c 99 -g 1 >>>> <switch windows> >>>> #/usr/local/sbin/tcpdump -i zc:99@0 -nn -c 10 >>>> <snip> 10 packets are displayed >>>> #/usr/local/sbin/tcpdump -i zc:99@0 -nn -c 10 “ip” >>>> <nothing> >>>> >>>> I’d like to resolve the BPF issue to start, and perhaps get some >>>> confirmation that I should use zbalance_ipc to get ZC’ed data to my >>>> applications...And if you can see I’m already heading down a road to >>>> inefficient usage, please let me know. :) >>>> >>>> Thanks for the time, and reading this far, >>>> >>>> Jesse >>>> >>>> >>>> # cat /etc/modprobe.d/pfring.conf >>>> options min_num_slots=8096 transparent_mode=2 >>>> >>>> # cat /proc/net/pf_ring/info >>>> PF_RING Version : 6.0.1 ($Revision: exported$) >>>> Total rings : 0 >>>> >>>> Standard (non DNA) Options >>>> Ring slots : 4096 >>>> Slot version : 15 >>>> Capture TX : Yes [RX+TX] >>>> IP Defragment : No >>>> Socket Mode : Standard >>>> Transparent mode : Yes [mode 0] >>>> Total plugins : 0 >>>> Cluster Fragment Queue : 0 >>>> Cluster Fragment Discard : 0 >>>> >>>> # ./load_driver.sh >>>> mkdir: cannot create directory `/mnt/huge': File exists >>>> Configuring p1p1 >>>> no rx vectors found on p1p1 >>>> no tx vectors found on p1p1 >>>> p1p1 mask=1 for /proc/irq/61/smp_affinity >>>> p1p1 mask=2 for /proc/irq/62/smp_affinity >>>> p1p1 mask=4 for /proc/irq/63/smp_affinity >>>> p1p1 mask=8 for /proc/irq/64/smp_affinity >>>> p1p1 mask=10 for /proc/irq/65/smp_affinity >>>> p1p1 mask=20 for /proc/irq/66/smp_affinity >>>> p1p1 mask=40 for /proc/irq/67/smp_affinity >>>> p1p1 mask=80 for /proc/irq/68/smp_affinity >>>> p1p1 mask=100 for /proc/irq/69/smp_affinity >>>> p1p1 mask=200 for /proc/irq/70/smp_affinity >>>> p1p1 mask=400 for /proc/irq/71/smp_affinity >>>> p1p1 mask=800 for /proc/irq/72/smp_affinity >>>> p1p1 mask=1000 for /proc/irq/73/smp_affinity >>>> p1p1 mask=2000 for /proc/irq/74/smp_affinity >>>> p1p1 mask=4000 for /proc/irq/75/smp_affinity >>>> p1p1 mask=8000 for /proc/irq/76/smp_affinity >>>> Configuring p2p1 >>>> no rx vectors found on p2p1 >>>> no tx vectors found on p2p1 >>>> p2p1 mask=1 for /proc/irq/78/smp_affinity >>>> p2p1 mask=2 for /proc/irq/79/smp_affinity >>>> p2p1 mask=4 for /proc/irq/80/smp_affinity >>>> p2p1 mask=8 for /proc/irq/81/smp_affinity >>>> p2p1 mask=10 for /proc/irq/82/smp_affinity >>>> p2p1 mask=20 for /proc/irq/83/smp_affinity >>>> p2p1 mask=40 for /proc/irq/84/smp_affinity >>>> p2p1 mask=80 for /proc/irq/85/smp_affinity >>>> p2p1 mask=100 for /proc/irq/86/smp_affinity >>>> p2p1 mask=200 for /proc/irq/87/smp_affinity >>>> p2p1 mask=400 for /proc/irq/88/smp_affinity >>>> p2p1 mask=800 for /proc/irq/89/smp_affinity >>>> p2p1 mask=1000 for /proc/irq/90/smp_affinity >>>> p2p1 mask=2000 for /proc/irq/91/smp_affinity >>>> p2p1 mask=4000 for /proc/irq/92/smp_affinity >>>> >>>> # modinfo pf_ring >>>> filename: >>>> /lib/modules/2.6.32-431.23.3.el6.x86_64/kernel/net/pf_ring/pf_ring.ko >>>> alias: net-pf-27 >>>> description: Packet capture acceleration and analysis >>>> author: Luca Deri <[email protected]> >>>> license: GPL >>>> srcversion: 9205E6179CCDF3C754F2122 >>>> depends: >>>> vermagic: 2.6.32-431.23.3.el6.x86_64 SMP mod_unload modversions >>>> parm: min_num_slots:Min number of ring slots (uint) >>>> parm: perfect_rules_hash_size:Perfect rules hash size (uint) >>>> parm: transparent_mode:0=standard Linux, >>>> 1=direct2pfring+transparent, 2=direct2pfring+non transparentFor 1 and 2 >>>> you need to use a PF_RING aware driver (uint) >>>> parm: enable_debug:Set to 1 to enable PF_RING debug tracing into >>>> the syslog (uint) >>>> parm: enable_tx_capture:Set to 1 to capture outgoing packets >>>> (uint) >>>> parm: enable_frag_coherence:Set to 1 to handle fragments (flow >>>> coherence) in clusters (uint) >>>> parm: enable_ip_defrag:Set to 1 to enable IP >>>> defragmentation(only rx traffic is defragmentead) (uint) >>>> parm: quick_mode:Set to 1 to run at full speed but with upto one >>>> socket per interface (uint) >>>> >>>> >>>> >>>> _______________________________________________ >>>> Ntop-misc mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
