Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/12038#issuecomment-203521210
> will only introduce exactly one copy if the small buffer number is lager
than 16
That's better, but is it needed at all? I don't see any comments about why
consolidating the buffers is a win in the source for CompositeByteBuf.
Traversing the single buffer should be slightly faster because there's less
bookkeeping, but there's the cost of copying that data in the first place.
When testing this code, I remember that during large transfers packets
would arrive in 64k chunks at the most, so that means that once you're
transferring more than 1MB, you'd have to copy things.
Have you tried not consolidating to see whether there's any negative
side-effect?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
