On Aug 20, 2005, at 1:57 PM, Christoph Lameter wrote:
On Fri, 19 Aug 2005, Andi Kleen wrote:
Hmm - but is a 9k or 16k packet on the wire not equivalent to a micro
burst?
(actually it is not that micro compared to 1.5k packets). At least
against
burstiness they don't help and make things even worse because the
bursts
cannot be split up anymore.
9k or 16k is just a very puny little size these days. That will not
give anyone much benefit. We need to be able to transfer large amounts
of data. This is going to be measured in megabytes or gigabytes of data
and not in kilobytes.
I think you are really missing something fundamental here. The
processing costs for TCP can be split in to per-byte costs and
per-packet costs. (This is a slight oversimplification, but good
enough for this discussion.) The per-packet costs include things like
memory allocation, protocol processing, and device interrupt handling.
The per-byte costs are bus transfer times, calculating checksums, etc.
Using larger packets helps reduce the overall per-packet costs. There
are diminishing returns here in using larger packets, and the marginal
benefit of using packet much larger than 16k isn't all that great. (As
an aside, there are other benefits to using larger packets than just
processing speed. For example, it makes congestion control easier.
But that's another discussion.)
TSO and TOE both help significantly with the per-packet costs. They
are effectively equivalent here to using larger packets. Doing
zero-copy and checksum offloading helps with the per-byte costs, and is
possible today with stock Linux, and I believe most TOE implementations
do. But TOE and TSO in and of themselves *do not* help with the
per-byte costs. TOE currently has an advantage over TSO because it
reduces the receive path costs in both ack and data processing.
Moreover jumbo packets essentially not standardized and most network
devices switch jumbo packets off or do not support it because of that
fact. Devices that send jumbo frames may cause other devices on
the network to malfunction.
This is certainly a concern. Fixing these issues IMHO is globally more
important (and architecturally more desirable) than TOEs. Some may
disagree. :-)
We are discussing something that is not useful for todays network load
and not standardized. TOE is the only answer to offloading transfers of
data encountered in contemporary networks.
Most people who are involved in this area see multiple solutions with
different trade-offs. TOE is just one option.
-John
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
