Hi Alfredo, Thanks and sorry, I haven't had the chance to look at this. I will try to replicate this on my home network and capture a pcap for you. Thanks YM From: [email protected] Date: Fri, 28 Feb 2014 11:56:24 +0100 To: [email protected] Subject: Re: [Ntop-misc] PF_RING Packet Distribution - After Upgrade
Hi YMcould you provide a pcap we can use to reproduce this issue and check the difference between 5.6.2. and 5.6.3? ThanksAlfredo On 27 Feb 2014, at 06:59, Y M <[email protected]> wrote:We have a very modest box with PF_RING and 2 Snort (2.9.5.6 with daq 2.0.1) processes monitoring a link (one interface) that peaks at 50MB. This was an initial test setup (installation, performance, upgrades, etc...) to an upcoming hardware upgrade. The box was running PF_RING version 5.6.2 (no revision was showing) with the PF_RING aware NIC driver e1000e. One of the major reasons this sensor was setup is to monitor P2P. At peak times and during ongoing P2P, the box was crunching packets with almost zero drops (~ 6000 rules). This has changed after we upgraded to PF_RING from SVN (version 5.6.3 Revision 7331) and Snort (2.9.6.0 with daq 2.0.2). After the upgrade, PF_RING starts dropping packets when P2P happens. I do not have the statistics before the upgrade (sorry), configuration wise, it is the same for both setups. Running PF_RING in transparent_mode 2 : cat /proc/net/pf_ring/info PF_RING Version : 5.6.3 ($Revision: 7331$) Total rings : 2 Standard (non DNA) Options Ring slots : 65536 Slot version : 15 Capture TX : No [RX only] IP Defragment : No Socket Mode : Standard Transparent mode : No [mode 2] Total plugins : 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 1385 Everything runs fine until a P2P "session" kicks in. The distribution of the packets seems to favor on process over the other. Clearly the Num Free Slots of the first process is 0: /proc/net/pf_ring/14815-eth1.6 Appl. Name : snort-cluster-11-socket-0 Tot Packets : 295195537 Tot Pkt Lost : 73299631 TX: Send Errors : 0 Reflect: Fwd Errors: 0 Min Num Slots : 78028 Num Free Slots : 0 /proc/net/pf_ring/14826-eth1.7 Appl. Name : snort-cluster-11-socket-0 Tot Packets : 89023033 Tot Pkt Lost : 0 TX: Send Errors : 0 Reflect: Fwd Errors: 0 Min Num Slots : 78028 Num Free Slots : 78010 /proc/net/pf_ring/14815-eth1.6 Appl. Name : snort-cluster-11-socket-0 Tot Packets : 298773670 Tot Pkt Lost : 74293208 TX: Send Errors : 0 Reflect: Fwd Errors: 0 Min Num Slots : 78028 Num Free Slots : 0 /proc/net/pf_ring/14826-eth1.7 Appl. Name : snort-cluster-11-socket-0 Tot Packets : 89169841 Tot Pkt Lost : 0 TX: Send Errors : 0 Reflect: Fwd Errors: 0 Min Num Slots : 78028 Num Free Slots : 77928 And from snort stats: /var/snort/snort-1.stats pkt_drop_percent as 34.181 /var/snort/snort-2.stats pkt_drop_percent as 0.000 After the P2P session ends (note the value of [Num Free Slots] of the first snort process). The value of the [Tot Pkt Lost] is right after the P2P session ends: /proc/net/pf_ring/14815-eth1.6 Appl. Name : snort-cluster-11-socket-0 Tot Packets : 350584381 Tot Pkt Lost : 94985249 TX: Send Errors : 0 Reflect: Fwd Errors: 0 Min Num Slots : 78028 Num Free Slots : 77927 /proc/net/pf_ring/14826-eth1.7 Appl. Name : snort-cluster-11-socket-0 Tot Packets : 96783586 Tot Pkt Lost : 0 TX: Send Errors : 0 Reflect: Fwd Errors: 0 Min Num Slots : 78028 Num Free Slots : 77961 /var/snort/snort-1.stats pkt_drop_percent as 0.000 /var/snort/snort-2.stats pkt_drop_percent as 0.000 Currently we are not binding processes to cores, that is something I am going to test soon to see the difference. The command running snort is wrapped by a loop and using the following: /usr/local/snort/bin/snort -c /usr/local/snort/etc/snort.conf -i eth1 -l /var/log/snort/snort-1 --perfmon-file /var/snort/snort-1.stats -g snort -u snort --pid-path=/var/run -R -1 /usr/local/snort/bin/snort -c /usr/local/snort/etc/snort.conf -i eth1 -l /var/log/snort/snort-2 --perfmon-file /var/snort/snort-2.stats -g snort -u snort --pid-path=/var/run -R -2 The PF_RING configurations are stored in the daq configuration section within snort.conf file: more /usr/local/snort/etc/snort.conf | grep daq # Configure DAQ related options for inline operation. For more information, see README.daq config daq: pfring config daq_dir: /usr/local/lib/daq config daq_mode: passive config daq_var: clusterid=11 I know that the modest hardware (CPU) may have a major role in this but the fact that such drops were not observed in the previous setup/before the upgrade makes me wonder if I missed something in the middle or if there were new changes I am not aware of. Any leads or help are appreciated. Thanks. YM _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc _______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
