Hi Carlos
I already implemented a basic API some time ago, which is used in
production. Maybe it can serve you as a starting point. The API is very
raw, but it works. It allows you to trigger an arbitrary number of
concurrent async SFTP reads and writes and to fetch the results later
using the request id. Look for the sftp_async_*() functions:
https://github.com/limes-datentechnik-gmbh/libssh/blob/limes/src/sftp.c
The hard parts have to be implemented by user code which is why I did
not submit it to upstream. The user code has to manage lists of
in-transit requests, handle write ACKs, implement a suitable read-ahead
strategy (depends on read pattern, e.g. lengthy sequential reads vs.
heavy seeking), proper seeks (you need to track file offsets from user
perspective for working relative seeks and discard already requested
blocks). Those parts should be handled by libssh in a final
implementation because it's easy to produce memory leaks, lose track of
the correct file offset or have an underperforming read-ahead algorithm.
The most difficult aspect is the read-ahead strategy. There is no one
size fits all. If you optimize for maximum throughput by using a large
read-ahead window, you will waste a lot of bandwidth/time waiting for
data you no longer need if you seek a lot because there is no way in
SFTP to cancel sent read requests. If you optimize for seek, you will
have slower throughput.
The current SFTP API could be re-implemented using the async methods and
a configurable per-file read-ahead strategy which would considerably
speed up performance. Libssh users that don't write async applications
but want reasonable throughput and a simple API would no longer be bound
by network latency as is the case with the current implementation.
Regards,
Tilo
Am 02.06.2020 um 20:32 schrieb Carlos Cabanero:
I couldn’t find any async_write method, but some thought was put into it
from this 2014 thread:
https://www.libssh.org/archive/libssh/2014-06/0000014.html. Is this
somewhere? Are we still interested to have it and are there any suggestions?
Thanks!