Hi Alfredo, Sorry for the delay on response.
I have tried the SVN code, and I'm afraid that it did not make a discernible difference on my test box. Anything else I can do to help troubleshoot? Cheers, Jesse On Sun, Apr 6, 2014 at 7:21 PM, Alfredo Cardigliano <[email protected]>wrote: > Hi Jesse > I reworked a bit the fragment hash, could you run some test with latest > code from svn? > Please let us know. > > Alfredo > > On 05 Apr 2014, at 02:31, Jesse Bowling <[email protected]> wrote: > > I was reminded that my filter filter would catch all but the last > fragmented packet, so I ran a new capture with: > > time tcpdump -i 'p6p1;p6p2' -nn -c 1000 'ip[6] & 0x20 !=0 or ip[6:2] & > 0x1fff != 0' > /dev/null > > (Thanks Judy for the filter!) > > It took just over 61 minutes to collect 1000 fragmented packets on a link > that averaged 1.3 Gb/s during that hour. I wouldn't consider that heavily > fragmented traffic (although that's obviously a small sample time). Could > there be an issue with removing fragments from the cluster queue? > > Cheers, > > Jesse > > > On Fri, Apr 4, 2014 at 1:34 PM, Jesse Bowling <[email protected]>wrote: > >> Hi Luca, >> >> Thank you for your response. I'll look into getting a traffic capture, >> but I'm not hopeful. I'd argue against the fragmentation issue for two >> reasons: >> >> *) The change in packet loss was consistent with the change in PF_RING >> versions, and previous versions did not exhibit this behavior >> *) Filtering current traffic with tcpdump -i 'p6p1;p6p2' -nn 'ip[6] & >> 0x20 != 0' does not seem to reveal any unusual amount of fragmented packets >> (a dozen or so over 3 minutes). >> >> If I'm unable to provide pcaps, what's next best? Any debug logging I can >> turn on, or attach to debugger, etc? >> >> Cheers, >> >> Jesse >> >> >> On Fri, Apr 4, 2014 at 11:03 AM, Luca Deri <[email protected]> wrote: >> >>> Jesse >>> I am traveling with my team this week, sorry for not being responsive. >>> Would you be able to provide us a traffic sample you have? It looks you >>> have strongly frgmented traffic >>> >>> Regards Luca >>> >>> On 02 Apr 2014, at 11:42, Jesse Bowling <[email protected]> wrote: >>> >>> I've not seen any comments or updates on this issue. I'm still >>> experiencing this issue and would appreciate any advice on how to >>> troubleshoot, or at least some confirmation from other sites that they see >>> this issue with 5.6.2 PF_RING? >>> >>> Thanks, >>> >>> Jesse >>> >>> >>> On Tue, Mar 25, 2014 at 2:14 PM, Jesse Bowling >>> <[email protected]>wrote: >>> >>>> Hello, >>>> >>>> Recently I updated my sensors to use PF_RING 5.6.2, after a good long >>>> while on an SVN version of 5.6.1. Initial results on a lightly loaded test >>>> box were good, however when the same setup was loaded onto more heavily >>>> loaded boxes we started seeing a substantial number of dropped packets. >>>> >>>> We use the ixgbe PF_RING aware drivers on RHEL 6. The >>>> /proc/net/pf_ring/info reports: >>>> >>>> PF_RING Version : 5.6.2 ($Revision: exported$) >>>> Total rings : 28 >>>> >>>> Standard (non DNA) Options >>>> Ring slots : 16384 >>>> Slot version : 15 >>>> Capture TX : No [RX only] >>>> IP Defragment : No >>>> Socket Mode : Standard >>>> Transparent mode : No [mode 2] >>>> Total plugins : 0 >>>> Cluster Fragment Queue : 32720 >>>> Cluster Fragment Discard : 69173446 >>>> >>>> The Cluster Fragment Discard numbers are much larger than usual. >>>> Additionally the NIC started reporting drops (less than 1%, but compared to >>>> the previous setup which dropped 0 packets at the NIC). Finally, the snort >>>> and argus processes running on top of this setup now report drops at a much >>>> higher rate than before. Previously snort ran with 1% or less drops, argus >>>> typically dropped only a few thousandths of a percent. Currently, snort is >>>> reporting drops of 15-20%, and argus is seeing roughly 15% drops as well. >>>> This is on a 10 Gb link with peaks of 1.8 Gb/sec with daytime averages of >>>> 1.4 Gb/sec. Similar, although smaller, increases in dropped packets was >>>> seen on another link that is much less loaded (300-500 Mb/sec, loss >>>> typically in the 5-10% range). >>>> >>>> Overall this is a substantial loss of performance. While I also >>>> upgraded snort and argus to newer versions at the same time as PF_RING, I >>>> find it less likely that both programs made changes to adversely affect >>>> packet capture rates at the same time. >>>> >>>> Are there any other reports of this happening? Anything I can do to >>>> help troubleshoot where this packet loss might be happening? >>>> >>>> I will file this on the bugzilla as well for tracking. >>>> >>>> Cheers, >>>> >>>> Jesse >>>> >>>> -- >>>> Jesse Bowling >>>> >>>> >>> >>> >>> -- >>> Jesse Bowling >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> >> >> >> -- >> Jesse Bowling >> >> > > > -- > Jesse Bowling > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > -- Jesse Bowling
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
