Andrew Gallatin wrote:
> Hi,
>
> When testing single-stream 10GbE netperf receive performance on both
> S10U4 and S11b84, I found an odd performance quirk. As I increased
> the receiver's socket buffer size, I noticed a decrease in performance
> as the socket buffer sizes approached 512KB, with a crash in
> performance above 512KB. I also noticed that mpstat was reporting an
> increasing number of xcals as I increased the socket buffer size.
> ...
> Looking at the source of kstrgetmsg(), there is a heuristic which
> tries to do a pullup(mp, -1) if the first mblk has a length of less
> than mblk_pull_len, or 64 bytes. In this case, it is pulling up about
> 460KB.
>
> If I disable this by setting mblk_pull_len to 0, then the decrease in
> performance with an increase in socket buffer size goes away on S10U4
> (haven't looked at b84 yet).
>
> Can somebody explain the "performance degredation from copyout
> overhead" talked about by the comment near the declaration of
> mblk_pull_len in common/os/streamio.c. I thought copyout was the
> normal copy routine plus a trap handler, and should cost the same as
> the copy in the pullup.
>
For reference, the two bugs that tie in with this are:
4226443
4347047
As 4347047 introduced this behaviour, I've copied a section from the
Comments to the
Description (nothing confidential or going to cause anyone pain) so you
should be able
to check that out when the bug database syncs up. As for 4226443, I
need to review
that one more carefully, but I'll add that the behaviour you're seeing
was introduced by
'047, while the general problem it was solving goes back to '443.
Since you're experimenting with this, can you try changing the if() from
if (MBLKL(bp) < mblk_pull_len) {
to
if ((bp->b_cont != NULL) && (MBLKL(bp) < mblk_pull_len) &&
(MBLKL(bp->b_cont) < mblk_pull_len)) {
It's not going to be as good as the "0" case, but it should be better
than what it
is today for your case.
This is one of those times when it would be nice if there was a packet
structure
that "owned" the mblks and where you track the number of chained mblks,
thus changing this "guessing game" heuristic into some simple math, e.g:
if (pkt->dsize / pkt->mblkcnt < mblk_pull_len)
...
Has anyone opened a bug for you on this or mentioned if there is a known
bug associated with this?
Darren
_______________________________________________
networking-discuss mailing list
[email protected]
