On May 21, 2004, at 7:26 AM, [EMAIL PROTECTED] wrote:
The way I see it in snort's implementation of the statistics it's doing ps_drop/(ps_recv + ps_drop). So I believe that part is accurate.
As far as I can tell, that'd be *wrong* on BSDs and Linux 2.4 and later.
The BPF in the BSDs I've looked at keeps a count of packets handed to "bpf_tap()" and "bpf_mtap()"; both of those call "bpf_filter()" to see whether the packet in question should be discarded and, if not, call "catchpacket()" to attempt to provide the packet to userland. If that attempt fails because there's no buffer space for the packet, the count of dropped packets is incremented. This means that all packets counted as dropped are also counted as received, so using "ps_recv + ps_drop" as the denominator counts dropped packets twice in the denominator.
A comment I put into "pcap-linux.c" says
* In "linux/net/packet/af_packet.c", at least in the
* 2.4.9 kernel, "tp_packets" is incremented for every
* packet that passes the packet filter *and* is
* successfully queued on the socket; "tp_drops" is
* incremented for every packet dropped because there's
* not enough free space in the socket buffer.
*
* When the statistics are returned for a PACKET_STATISTICS
* "getsockopt()" call, "tp_drops" is added to "tp_packets",
* so that "tp_packets" counts all packets handed to
* the PF_PACKET socket, including packets dropped because
* there wasn't room on the socket buffer - but not
* including packets that didn't pass the filter.
so, in the statistics returned if the PACKET_STATISTICS socket option is supported on PF_PACKET sockets (which I think first officially appeared in 2.4, although Alexey Kuznetzov's turbopacket patches, which were, I think, incorporated in 2.4, might work the same way), the count of packets received also includes dropped packets.
You might want to pass that on to the Snort developers. (Well, actually, it appears you already said that in
http://marc.theaimsgroup.com/?l=snort-users&m=108508394000910&w=2
"In Linux it seems ps_recv already contains the number dropped, so adding
ps_drop to them could increase the counter." Note, though, that with my recent libpcap changes, it should also make "ps_recv" contain the number dropped on Solaris as well, so with current-CVS main-branch or 0.8-branch libpcap using "ps_recv+ps_drop" as the denominator will also be wrong on Solaris.)
However, that doesn't explain how the heck a percentage >100 can possibly be reported, unless "ps_recv" is (as someone noted in the original snort-users thread) is negative.
I did some tests last night and this morning with two versions of snort, one
compiled with libpcap 0.7.2 and the other with libpcap 0.8.3. I injected
1,000,000 packets to both of them over a cross-over cable. Snort with
0.7.2 reported capturing roughly 970k out of 970k packets, meaning it was
definitely dropping packets, just didn't know about it.
So it sounds as if the packets are, indeed, being dropped somewhere in the STREAMS stack for the DLPI connection other than in "bufmod", so that "bufmod" can't report the drops...
Snort with libpcap 0.8.3 reported more accurate statistics,
...and 0.8.3 causes the same dropping to happen in "bufmod" (rather than, say, at the stream head, dropping packets because the queue is full, rather than "bufmod" checking whether it can queue stuff up above it and dropping the packet, and counting it, if it can't), so that the dropped packets are counted...
and consistenly dropped less packets.
...and the other changes reduced the number of packets being dropped (as I think the person who submitted the SourceForge libpcap bug in question said they would).
BTW, on the RH vs. FreeBSD drop issue in that thread:
http://marc.theaimsgroup.com/?l=snort-users&m=108503075830508&w=2
http://marc.theaimsgroup.com/?l=snort-users&m=108507260320540&w=2
I'm a little skeptical of that being a libpcap 0.7[.x] vs. 0.8[.x] issue, as I don't remember doing anything to pcap-bpf.c that would cause such a problem (the code path might be a little longer on *all* platforms with the calls through function pointers in the pcap_t, but I wouldn't expect that to make a significant difference) - there shouldn't be much of *any* difference in the packet drop reporting. Now, perhaps the libpcap being used on Red Hat wasn't reporting drops, either because the RH-based sensors were running a kernel that didn't report drops to libpcap or because an older version of libpcap that didn't get packet statistics from the kernel was being used.
- This is the tcpdump-workers list. Visit https://lists.sandelman.ca/ to unsubscribe.
