We're working on a patch that monkeys with the TCP buffers because we're seeing slow downs with big transfers as well. It might be related...
On 1/28/10 9:25 AM, "stephen mulcahy" <stephen.mulc...@deri.org> wrote: > Jay Booth wrote: >> Did you set io.file.buffer.size (or whatever the property is) to a large >> value? > > Just re-ran the benchmark with that bumped to 65536 (as proposed in > http://www.cloudera.com/blog/tag/configuration/). The benchmark is still > slower with jumbo frames than without (but difference was reduced a > little by bumping io.file.buffer.size. > >> Also, even if you do set that, I'm not 100% sure that it will lead to use of >> the jumboframes, I'd have to take a look through the code to be sure. >> Another issue is that you may lose any networking performance gains as a >> result of increased buffer allocation costs and garbage collection. > > I guess there must be something going on anyways - as there is clearly a > performance drop-off, which surprised me, as lots of apps benefit > significantly from jumbo frames. > > Thanks for the feedback, > > -stephen
