Andrew Gallatin wrote:
Garrett D'Amore writes:
> There's also the whole problem of reference counting and DR.... a loaned
> up packet has to be freed using a special function as part of
> esballoc/desballoc, but that means you might have to reject a DDI detach
> in the face of outstanding loaned up packets still held elsewhere in the
> stack. (You need the dip to be valid to do ddi_dma_unbind... you also
> need your driver to be kept from unloading from kernel memory so that
> the function pointer associated with the free routine is still valid.)
Maintaining a free pool is not rocket science. This is more
of a documentation issue than anything else. FWIW, this is
an issue that driver writers coming from Windows will be familiar
with, at least.
Do windows driver developers have to deal with dynamic reconfiguration?
> All this is being done to reduce/eliminate the cost of
> ddi_dma_addr_bind_handle/ddi_dma_unbind_handle(). I'm starting to think
> that the cost of those two routines is sufficiently low on modern
> systems to not be worth the risk and complication that loan up offers.
> The "new" DDI DMA routines -- ddi_dma_addr_bind_handle and friends --
I think you'd need to benchmark a 1G or 10G driver on a modern
system with an IOMMU and see how much difference there really
is..
Actually, its easy enough to compare with a 1G or even a 100Mbp system
with an IOMMU and a slower processor...
To properly handle loan up offers its own costs. And the question, to
my mind, is where is the trade-off?
There may be other approaches that can help at extreme ends as well...
such as LSO, page flipping into userland, etc.
I do like the idea of figuring out how to "prebind" buffers that you
described earlier, and I've been giving it a lot of thought. The
question, to my mind, is whether or not we really need to have the full
"generic" support that the DDI DMA layer gives us. By that I mean, ddi
can consider things like memory locality, different caching behaviors
for different regions of memory, different addressility, etc. But if
(and I believe on most systems this is true) the memory space is more or
less uniform, then maybe we could make some assumptions that would allow
us to preallocate and stash a physical address/cookie in an mblk-like
structure somewhere... something to tell NIC drivers that "hey, this
buffer is already prepped and ready to go, just plug it in."
Of course there are other problems too... like address alignment, page
boundary considerations, etc. Most modern bus masters are pretty
flexible here (at least on the tx path), but you never know....
- Garrett
Drew
_______________________________________________
networking-discuss mailing list
[email protected]
