Thanks, pushed. Tim
On Tuesday 01 September 2015 23:37:46 Hubert Tarasiuk wrote: > Here is a small fix to Metalink support in main function. > The bug was that if the Metalink file parsing failed, there would be a > free() call for non-initialized memory pointer. > > W dniu 24.06.2015 o 08:06, Hubert Tarasiuk pisze: > > I have prepared a Metalink patch for Wget. The two main features are: > > > > - support Metalink v3 and Metalink v4 XML files: > > https://tools.ietf.org/html/rfc5854 > > using libmetalink parser: https://launchpad.net/libmetalink > > > > - support Metalink in HTTP headers: http://tools.ietf.org/html/rfc6249 > > > > > > Specifically what the patch implements (for both XML and HTTP): > > - keep downloading from consecutive resource URLs until a successful > > retrieval > > - verify SHA256 digest (this digest is mandatory for Metalink, thus it > > should be included in all Metalink documents) > > - verify OpenPGP signatures (using public keys from user's keyring) > > using GPGME and GnuPG > > > > Checksum mismatch means download failure and it proceeds to try with > > another resource (if available). > > If the signature cannot be verified (missing public key would be the > > most common reason), we do NOT assume download failure. > > If the signature can be verified and the verification fails (ie. data > > does not match signature), we assume download failure. > > > > Please note that the PGP signatures are only working for > > Metalink-over-HTTP at this time due to a bug in libmetalink. > > > > > > Following options were added to Wget: > > > > --input-metalink=FILE - download files described in Metalink file FILE > > (like --input-file) > > > > --metalink-over-http - when downloading from HTTP URLs: > > -> issue a HEAD request and check for Metalink metadata in reponse > > -> if found: switch to Metalink-mode > > -> if not found: fall back to ordinary HTTP download > > > > _Test suite_ > > I have made two modifications to Python test suite: > > - allow multiple SendHeaders with same name by using a Python list as > > dictionary value > > - do not start the HTTP test in constructor; do it in the begin() method > > instead (as the method name would suggest); original behaviour was to > > run the test in object constructor and the begin() method would just > > return the result > > > > Please let me know what do you think about the patches. Some test cases > > are included. If you would like to test it on actual servers, here is > > what I found: > > - Metalink files with PGP signatures: http://curl.haxx.se/download.html > > - Metalink in HTTP headers: > > https://download.gnome.org/apps/3.0/3.0.0/sources/ > > > > The commits are also available via Github interface: > > https://github.com/jy987321/Wget/commits/metalink > > > > Hubert > > > > W dniu 28.05.2015 o 00:49, Hubert Tarasiuk pisze: > >> I have talked with Giuseppe and he suggested that we might not do TCP > >> Fast Open support for FTP at this time (he argued that FTP is slow > >> either way :). > >> > >> Instead I might focus on implementing some basics of Metalink protocol > >> for HTTP and FTP resources in Wget. > >> > >> Do you have any thoughts about that?
