Lots of info here... IMO the key is definitely in the NIC and driver(s) - that's how Sniffer got so popular so fast - it was the only thing that could keep up! Intel makes good hardware, but maybe their drivers suck? At least for the OS you're running. 64bits doesn't mean it's twice as fast.
I haven't worked with high speed capture much, but if you can't capture at least 75% of wire speed with a normal mix of packet sizes at zero loss, something is wrong. Maybe search around and see what OS/driver/NIC combo works best. Surely someone has this working somewhere! Gary ________________________________ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jorge Cuevas Sent: Tuesday, April 22, 2008 1:51 PM To: [email protected] Subject: [Ntop] Some help with PF_ring issues Hi all: We are now implementing OSSIM (www.ossim.net) on a clients network (snort, pads, p0f, ntop, and arpwatch running), and we have an issue with the amount of traffic we can monitor. The actual phase of the project is about measuring that amount using the only server we have right now: Dell PE2950 Xeon 5110 1.6GHz/4MB 1066FSB with 4GB FB 667MHz Memory and an Intel Pro/1000 PT NIC PCIe Card. There is a Debian etch 64bits running. We need another NIC only for administering purposes. The traffic in our production environment is injected from a Cisco 6500 core, and we have four trunked links of 1Gbit each, which is being inserted (Spanned) 1 to 1, 2 to 1, 3 to 1 and 4 to 1 during the tests. In the 1 to 1 spanning we have a 17,5 Mb peaks and 80 kbits average traffic, and in the 2 to 1 spanning we have 63 Mb peaks and 273 kbits average. Doesn't seem to much for PF_RING (even though the system will be overloaded with so many apps: all the sensors + database + correllation engine ...). We have a testing environment, with a Intel(R) Pentium(R) D CPU 3.00GHz, 1 GB memory, a Realtek RTL-8169 NIC, where we want to reproduce the improvement of PF_RING functionality. It also has a separated NIC for administration. There is a Debian etch 32bits running. We will use tcpreplay for injecting traffic (using a real capture from the real environment) In our testing environment, following the howtos about PF_RING, we have downloaded the code, edited and run mkpatch... We have a comment here... this script does the patch itself?? You don't need to execute "patch" afterwards as the instructions say. Just a point. We execute make menuconfig and select the options (rx and tx polling for r8169 as a module, PF_RING sockets as a module) and generate the new kernel which we boot (Debian like). No problem We insert the LKM ring.ko -> insmod ring.ko bucket_len=1500 num_slots=8192 Success: "Welcome to PF_RING 3.7.8 (C) 2004-08 L.Deri <[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED]> NET: Registered protocol family 27 [PF_RING] Bucket length 1500 bytes [PF_RING] Ring slots 8192 [PF_RING] Slot version 9 [PF_RING] Capture TX Yes [RX+TX] [PF_RING] IP Defragment No [PF_RING] initialized correctly [PF_RING] registered /proc/net/pf_ring/" Next we compile libpfing and libpcap, generating the new dynamic libraries in /usr/local/lib. Seems a success... We go to ntop: we use ./autogen.sh LDFLAGS="-L/usr/local/lib -lpfring -lpcap -lpthread" as the options an seems a success, the execution gives PF_RING loading options... But, executing ldd against the binary: where is the libpcap linking option?? # ldd /usr/local/bin/ntop linux-gate.so.1 => (0xffffe000) libpfring.so => /usr/local/lib/libpfring.so (0xb7ee7000) libntopreport-3.3.5.so => /usr/local/lib/libntopreport-3.3.5.so (0xb7e3e000) libntop-3.3.5.so => /usr/local/lib/libntop-3.3.5.so (0xb781e000) libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb780c000) libcrypt.so.1 => /lib/tls/i686/cmov/libcrypt.so.1 (0xb77de000) libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb76ad000) libssl.so.0.9.8 => /usr/lib/i686/cmov/libssl.so.0.9.8 (0xb766e000) libcrypto.so.0.9.8 => /usr/lib/i686/cmov/libcrypto.so.0.9.8 (0xb7534000) librrd_th.so.2 => /usr/lib/librrd_th.so.2 (0xb74ed000) libgdbm.so.3 => /usr/lib/libgdbm.so.3 (0xb74e7000) libz.so.1 => /usr/lib/libz.so.1 (0xb74d3000) /lib/ld-linux.so.2 (0xb7ef4000) libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb74cf000) libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xb7465000) libpng12.so.0 => /usr/lib/libpng12.so.0 (0xb7441000) libart_lgpl_2.so.2 => /usr/lib/libart_lgpl_2.so.2 (0xb742b000) libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7406000) Then we go to snort (maybe you have nothing to do with it...): #./configure --enable-dynamicplugin #export LDFLAGS="-L/usr/local/lib -lpcap" Compiling seems successfull... But the execution takes up to 100% CPU and 85% RAM... And again, executing ldd against the binary: where is the libpcap linking option?? Any ideas?? # ldd /usr/local/bin/snort linux-gate.so.1 => (0xffffe000) libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7f03000) libpcre.so.3 => /usr/lib/libpcre.so.3 (0xb7edd000) libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7eb7000) libnsl.so.1 => /lib/tls/i686/cmov/libnsl.so.1 (0xb7ea1000) libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7e9d000) libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7d6c000) /lib/ld-linux.so.2 (0xb7f1f000) Also, executing any of theses binaries gives some strange messages (dmesg), any ideas?? [PF_RING] successfully allocated 50925568 bytes at 0xf8afd000 [PF_RING] allocated 32770 slots [slot_len=1554][tot_mem=50925568] [PF_RING] removed /proc/net/pf_ring/25491-lo.0 --> STRANGE!!! [PF_RING] successfully allocated 50925568 bytes at 0xf8afd000 [PF_RING] allocated 32770 slots [slot_len=1554][tot_mem=50925568] [PF_RING] removed /proc/net/pf_ring/25491-eth5.1 [PF_RING] successfully allocated 50925568 bytes at 0xf8afd000 [PF_RING] allocated 32770 slots [slot_len=1554][tot_mem=50925568] [PF_RING] removed /proc/net/pf_ring/25491-eth0.2 [PF_RING] removed /proc/net/pf_ring/25491.0 Then we go to our tests (previously done with the standard Debian packages and kernel), and there is very little improvement: We have changed the module parameters num_slots: 8192 and 32768 These are our results: Tests Dropped by Libpcap Standard Sockets PF_ring: 8192 PF_Ring: 32768 tcpreplay -C -t -l 60 -i eth0 capt_50MB (485.37 Mbps/sec) 189,50% 118,00% 130,00% tcpreplay -C -t -l 200 -i eth0 capt_50MB () 151,90% 106,00% 110,00% tcpreplay -C -M 20 -l 10 -i eth0 capt_50MB () 0,20% 0,90% 0,00% tcpreplay -C -M 70 -l 20 -i eth0 capt_50MB () 15,80% 3,70% 2,10% tcpreplay -C -t -l 60 -i eth0 capt_50MB () 289,20% 171,30% 161,10% tcpreplay -C -M 0.5 -i eth0 capt_50MB () 0,00% 0,00% 0,00% Something to do with the BFP filters? Now, do you think this is enough? We did nothing about RT_IRQ and real-time patches to the kernel, should we? Should we go and do the same on the production environment? Should it go better just for having a better NIC and the driver support (intel)? Any change to these tests we should make? Thanx a lot just for reading... Regards Jorge Cuevas <font size="1"> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'> </div> "This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." </font>
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
