I'm submitting the following fast-track on behalf of Paul Durrant at
SolarFlare, who has done some interesting performance work with 10G ethernet.
Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI
This information is Copyright 2008 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
xesballoc: enhanced esballoc
1.2. Name of Document Author/Supplier:
Author: Paul Durrant
1.3 Date of This Document:
21 August, 2008
4. Technical Description
PROBLEM
A high performance NIC driver needs to avoid copying data wherever
possible. Thus, it is common on the receive path to encapsulate DMA
buffers in a STREAMS block so that they may be passed directly up the
stack. The primitive normally used to do this is desballoc(9F). When
such a block is freed a callback into the driver is made. This
callback is specified by the frtn_t structure:
typedef struct free_rtn {
void (*free_func)();
caddr_t free_arg;
} frtn_t;
that was passed to desballoc(9F), and can be used to recover the block
that was encapsulated and recycle it. Unfortunately the encapsulating
STREAMS block is always freed; so the first action of the driver is
usually to allocate a new one, which leads to extra CPU overhead and
kmem cache thrashing.
SOLUTION
We propose a new primitive xesballoc(), which is a variant of
desballoc(9F). The semantics of the function itself are identical
however the function specified in the frtn_t structure is subtley
different. For blocks allocated using desballoc(9f)
frtn_t.free_func() is passed a single argumnent: frtn_t.free_arg,
effectively giving it a type of:
void (*free_func)(caddr_t)
For blocks allocated using xesballoc() frtn_t.free_func will be passed
a second argument, a pointer to a boolean_t, effectively giving it a
type of:
void (*free_func)(caddr_t, boolean_t *)
If the driver does not modify the boolean then it remains B_FALSE. In
this case the STREAMS block will be freed when free_func() returns, as
is the case with blocks allocated using desballoc(9F). However, if the
driver sets the boolean to B_TRUE then the STREAMS block will not be
freed and it is assumed that the driver will recycle this too.
NOTES
It should be noted that drivers using xesballoc() or desballoc(9F)
must be careful not to assume references to the original mblk_t
returned by function will be valid when frtn_t.free_func() is called
since there is no guarantee that the STREAMS block has not been
dupb()ed and the original mblk_t freed. It is guaranteed that the
dblk_t will be the same and so the mblk_t should always be accessed
via the db_mblk field of the dblk_t to ensure a valid reference.
Clearly this point is not so crucial for consumers of desballoc(9F)
since the STREAMS block was always freed immediately after the call to
frtn_t.free_func().
6. Resources and Schedule
6.4. Steering Committee requested information
6.4.1. Consolidation C-team Name:
ON
6.5. ARC review type: FastTrack
6.6. ARC Exposure: open
