On 20/05/14 01:23, Jure Grabnar wrote:
2) Try to figure out file size with Content-Length header. If size is unknown fallback to a single thread download. Would it be sensible to allow user to specify file size with some switch?
Probably. Sometimes it's available from the download page even if the server does not provide it.
3) The main thread maintains a pool of available servers. It spawns at most N threads if N< M or at most M threads if M< N, where M is number of available mirrors. Every thread downloads each own chunk from each own mirror using current implementation of concurrent download for Metalink. If some mirror becomes unavailable during download from i-th thread, that threads terminates and notifies the main thread. The main thread spawns a new thread from available mirrors; if none is available at the moment, it waits until some mirror becomes available (whenever some other thread finishes downloading its chunk).
What should happen if there's no mirror left?
4) A file would be downloaded to a single temporary file as described here: http://lists.gnu.org/archive/html/bug-wget/2014-05/msg00025.html I'm still fixing the patch, because at least one memory corruption bug is still lurking around which is yet to be found.
Try using valgrind. I find it invaluable for these kind of bugs.
b) Front end What would be a good way to specify mirror list? Specifying a switch and listing all mirrors could be quite awkward. Should we introduce some sort of a simple file format?
A plain text file with one url per line?
I believe we should take into consideration number 2: downloading multiple files from multiple servers. Do we want to apply different switches (options) to different files? What about if we want to combine 1. and 2.: multiple files from multiple mirror list? The simplest way would be to use Metalink file for such purpose but is it the most elegant? All your suggestions are greatly appreciated. Best Regards, Jure Grabnar