tomaswolf commented on issue #485:
URL: https://github.com/apache/mina-sshd/issues/485#issuecomment-2060039475
After some analysis, here's what's going on:
transferTo/transferFrom, as well as the FileChannel.write() operations, are
_positional_ operations. `readableChannel.transferTo(0, length,
writeableChannel)` will essentially read 8kB `ByteBuffer`s from the file and
then call `writeableChannel.write()` for each buffer.
However, `SftpRemotePathChannel.write()` doesn't know that it is being
called essentially for a sequential copy operation, and so it doesn't employ a
number of optimizations. The result is the slow transfer.
If you change the logic and use `writeableChannel.transferFrom()`, then the
`SftpRemotePathChannel` drives the operation, and it knows that it is going to
sequentially read buffers. Hence it can employ these optimizations.
When you use OutputStream/InputStream as in my `Files.copy()` examples, then
it is known that a sequential data transfer occurs, and the SFTP implementation
can employ its optimizations unconditionally.
Finally, transferTo/transferFrom by default copy data in 8kB chunks. With
streams, the chunks are about 32kB. This difference causes the 25% slowdown.
Hence:
- In general, using streams is the simplest for downloading and uploading
files, and gives good performance.
- If you want to use `FileChannel`s:
- Always let the remote channel drive the operation. Use transferFrom for
uploading and transferTo for downloading.
- Execute transferTo/From in a loop until all data has been transferred.
- Increase the transfer buffer size via
`SftpModuleProperties.COPY_BUF_SIZE.set(session, 32 * 1024);`
It might be possible to improve our implementation to handle the case you
stumbled upon better, but I'm not sure yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
