On Wednesday, 31 May 2023 09:38:49 CEST Eshan Kelkar wrote: > Libssh needs a high level api for uploading and > downloading files. Functions like sftp_put() and > sftp_get() need to be introduced which the users > can call simply to upload and download files instead > of having to write their own functions to perform uploads > and downloads using the low level read/write api's. > > This mail suggests approaches that can be followed > to develop that api. > > Some terminology : > 1. local_fd - is the file descriptor [obtained using open() > or creat()] of a file on the local machine of the user. > > 2. remote_file - is a sftp file handle [obtained using > sftp_open()] of a file on the connected server. > > 3. concurrent_requests - This is the number of requests > we initially issue before trying to get the responses. > > Say you are downloading and you have 20 concurrent > requests, this means that first we have issued 20 read > requests, and then when we are trying to get/wait for the > response of the first request, the server may be processing > or may have processed the other 19 requests (This is the > advantage of asynchronous request-response pattern over > synchronous request-response where we would have issued > the next request [and server would have processed it] only > after getting a response of the first request). > > And after getting the response of the first request, we issue > another request if needed and then try to get the responses > of the other outstanding requests and repeat this procedure > for them too. > > Though you can see this name "concurrent_requests" is not > the best as the requests are not being issued in a concurrent > manner here, neither are they being processed concurrently by > the server. So if a better name comes to your mind, kindly suggest. > > 4. chunk_size - The number of bytes we usually issue a > read/write request for, if the number of bytes to read/write > is less than this chunk_size then the request is issued for > those many number of bytes and not chunk_size number > of bytes. > > How the api can look from User's perspective : > > Approach 1 : > ---------------------------------------------- > The user has to pass each of the 4 required things > for the transfer to the put, get functions. > > For the default values of concurrent_requests and > chunk_size we can provide macros (which expand > to default values suggested by libssh) which the user > can use if he/she is not interested in setting some > custom values. > > int sftp_put(int local_fd, > sftp_file remote_file, > int concurrent_requests, > size_t chunk_size); > > int sftp_get(int local_fd, > sftp_file remote_file, > int concurrent_requests, > size_t chunk_size);
Thanks for the write up. If you use these function which look simple to use you want to also add a progress callback passed down and a userdata pointer. This way a UI (also cmdline) could display some progress. -- Andreas Schneider a...@cryptomilk.org GPG-ID: 8DFF53E18F2ABC8D8F3C92237EE0FC4DCC014E3D
