Hi.
I am quite astonished by the interest aroused by our tests a year after their
publication on the web. We (the Italians, i.e. the network group of the Politecnico di
Torino) performed these tests in November 1999 in order to obtain quantitative results
for my graduation thesis that was focused on the development of WinPcap. Since, as far
as I know, BPF is the best public capture device in the Unix world, we used the latest
freebsd distribution of that period (3.3) and compared the results obtained. Since
then, we never updated that page. I know that freebsd and its BPF implementation have
been updated meanwhile, but winpcap as well has been improved and optimized. This
means that the results are outdated, but the comparison is acceptable, because it
takes in consideration the best implementations of that period. FYI, we have done
other tests in September 2000 comparing the new WinPcap 2.1 with BPF under freebsd
4.1, and the results were quite interesting, but I will speak of this later.
Our goal was to create a suite of tests able to:
- measure the performance of the various components (although this is difficult
because of the impossibility to isolate each component from interacting one with the
others and with the OS).
- measure capture performance "as a whole", from the NIC to the hard disk.
I know perfectly that the second point involves measuring the performance of the whole
operating system, and not only of the capture driver, but we included it because (as
we wrote commenting the tests) in our opinion it is the most interesting from the
end-user standpoint. I know also that better capture performance can be obtained
tuning the OS (eliminating the unnecessary protocol stacks, setting bigger buffers,
choosing a fast file system for the dumps, etc), but we were interested in measuring
the usage that the standard tcpdump/windump user makes of these tools, so we installed
a fresh win2k professional and a fresh freebsd on the same disk of a P2 400 and
launched the tests without modifying anything.
In my opinion, those tests demonstrate that:
- BPF for freebsd 3.3 is more efficient in the CPU usage. On the other side, winpcap
2.02 is more responsive, because it delivers the packets to userland as soon as the
application is ready to receive them. Both the problems have been solved by recent
releases: BPF now has an 'immediate mode' that can be used to pass the packets to
user-mode without the need to fill the hold buffer, winpcap has the 'delayed write'
capability that optimizes the CPU usage.
- The filtering and pre-filtering process has about the same impact in Windows and
freebsd. However, since the Win32 version of the BPF virtual processor is nearly
identical to the original, the Windows kernels (and in particular the one of win NTx)
seem sensibly faster in the pre-filtering (interrupt+dma(or equivalent)+nic driver),
in particular when al lot of packets are discarded by the BPF filter. This was very
surprising for us when we performed the test, because comparing the linear and elegant
implementation of the bsd networking with the enormous and complex NDIS we expected
exactly the opposite situation. However our recent tests seems to confirm this
situation. I think I have an explanation for this, and I will write about it only if
somebody is interested.
- When the packets are dumped to disk, the file system plays a very important role.
This is quite obvious. Notice for example that, according to our tests, a fat32
partition under Windows 2k is able to save 15-20% more packets than a NFTS partition
of the same disk: fat is a very poor file system, but for this reason it is very fast,
and not only in Windows. However, Win2k seems to have a very efficient storage
management. In my opinion, the block size of the stdio library is not the only
parameter that influences the dump perormance. Don't forget that the data that tcpdump
saves to disk is already buffered by libpcap (with a 32k buffer in bsd and a 256k
buffer in win), so a further buffering is not so essential. FS kernel-level cache
(that in Win2k is quite large) is very important too, because it is the last buffer
before the physical write. Moreover, the method used by the application to save the
data has to be taken in consideration: libpcap requires pcap_dispatch() to be invoked
for every saved packet, and is able to save with pcap_dump() only a packet at a time.
This is very inefficient, because requires a lot of cpu and doesn't make a good use of
the libpcap packet buffer. In my opinion, a pcap_write() function able to directly
save the content of the packet buffer would be noticeably faster.
Now, some words on our new tests. We performed them in order to measure the
performance of the new version of winpcap (2.1) that currently is in beta on our web
site. We compared it to BPF for freebsd 4.1, trying to obtain more accurate and
objective results. In particular, we saved every dump (under Win98,Win2k and freebsd)
on the same fat32 partition, and tested tcpdump also with a kernel-buffer size of
2*512K. Moreover, we stressed the capture architectures with a more sustained traffic
(until 83kpps). I don't attach the results to this message because they are in an 88k
excel file, but I have not problems to send them to the people interested.
They show that the cpu performance of winpcap for winNTx is now comparable with the
one of freebsd and that freebsd has still lower dump performance, but saving on fat32
it is able to obtain better results and in a case (22000 500bytes packets per second)
it is better than win2k. The really strange thing is that BPF seems not to perform
noticeably better if a 1 megabyte buffer is used instead of the standard 64k buffer,
and in some cases the 32k buffer performs even better. I wrote a message to the
tcpdump mailing list (http://www.tcpdump.org/lists/workers/2000/msg01135.html) to ask
about this behaviour, but I obtained no satisfactory answer.
Finally, I would like to say that since I know we can't be considered very impartial,
we would like to see other tests and evaluations on this subject, because we think it
is a very important matter. We had to make a comparison with bpf because we didn't
find any quantitative value on this subject before. If anyone obtained different
results, please let us know.
Loris.
From: Dragos Ruiu <[EMAIL PROTECTED]>
To : <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>;
<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Subject: [Ethereal-dev] Fwd: kyxtech: freebsd outsniffed by wintendo !!?!?
>
> (Hurm.... Wintendo outperforming unix???!?? Something's
> improper about this, and it ought to be fixed... :-)
> Comments? Other OS numbers: more recent
> FreeBSD versions? Solaris? Tru64? Optimization
> patches? Can those OO MSDN lobotomies actually
> be good things? Hurm... The Italian gauntlet has
> been thrown down.... --dr :-)
>
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
