James Carlson wrote:
> Garrett D'Amore writes:
>
>> I've needed such a function in my own work. So in the past I've either
>> restructured my code to make use of the side effect, or implemented my
>> own version of mcopymsg. mcopymsg is useful when copying tx mblks into
>> a DMA buffer, where you know you can toss the mblk when its done, but
>> there are cases that come up where you can't or don't want to do that.
>>
>
> I think you could do better for that special case. For instance, it'd
> be very nice to discard the db_credp references (so that you're not
> pinning down cred_ts and thus thus zones and thus rootvps) as you do
> this copy. That way, the driver wouldn't have to know anything about
> this odd and annoying detail of the system.
>
> Doing that requires having a function that is more specific to this
> particular purpose, than a general-purpose "static buffer" unloading
> function. It means that the unload function needs to know that the
> source mblks are at the end of the line (output side of a driver), and
> not just being processed by some intermediate module.
>
>
>> For example, you may wish to copy the msg into an aligned DMA region,
>> and need to do that (for performance) *before* acquiring the locks
>> associated with a tx ring. After acquiring such locks, you might find
>> that the ring itself is full, so you need to re-queue the msg for later
>> delivery (in Nemo, this is done by m_tx() returning the original
>> message.) mcopymsg()'s side effect here is toxic.
>>
>
> That one case seems pretty narrow to me. How is it that you're able
> to unload somewhere, but you've run out of tx ring resources? And why
> would copying the same packet multiple times during a period of low
> resources be a good solution? I'd think that's a design level
> mismatch of some sort ... though I guess it might be possible.
>
I had more DMA resources available than ring entries. The reason for
this was to avoid tying up the ring lock during bcopy. Evidence has
shown that the tx ring lock is the single most limiting factor in many
NIC drivers. (Specific examples include nxge, e1000g.)
The multiple-copying solution is admittedly less than ideal, but the
assumption here was that such situations of low resources are infrequent
enough that it shouldn't happen. (And in this particular case, two
checks were done... one at the time the DMA buffer was acquired, and
then again at the time the ring resource was acquired. In the middle
the tx ring lock was dropped, which was the race that needed to be
protected against.)
In the particular case I've cited here, I actually gave up, as this was
for a 10/100 driver where the additional complexity frankly wasn't
warranted. But approaches like this could help reduce the tx lock
contention that is hurting performance of other more modern NICs.
(Pretty much all of them I've looked at suffer from tx lock contention.)
> Mostly what I'm worried about here is adding functions that appear to
> be generic but that have no clear and good usage cases. In all the
> times we've done something like that in the past, we've ended up
> getting bitten -- either the functions are ignored, or they're misused
> for the undocumented side-effects.
>
> In other words, if someone wants to add these functions as specified,
> that's great, but I'd like to see the proposal be made _complete_ by
> including information about a consumer that'll integrate along with
> the new functions.
>
Fair enough. I don't disagree with your general reservations, but I
think at least the mcopymsg-without-side-effects case may be useful
enough to warrant use.
-- Garrett
_______________________________________________
networking-discuss mailing list
[email protected]
