On 2017-04-28, Enrico Zini wrote: > On Thu, Apr 27, 2017 at 08:26:13AM -0700, Vagrant Cascadian wrote: > >> I want to keep an eye to a patch that would be acceptible to the release >> team for stretch at this point, so maybe only refactoring this new code >> for a smaller, more readable diff would be best. > >> Below is a crude implementation using python3-urllib3. Maybe it's good >> enough. > > I would be tempted to rebase the download code on > urllib.request.urlretrieve, because it avoids an external dependency and > it does it all in one shot: > https://docs.python.org/3/library/urllib.request.html#urllib.request.urlretrieve > > The only thing it does not do is verifying the Last-Modified header. > However, since we already have extrafiles, we could use the checksums in > there to see if the files need redownloading. That would avoid the > worries about parsing timestamps, too.
Sounds good! >> It also supports using a proxy, although it seems to require a bit of >> code duplication. More elegant support for proxies would be nice. > > The standard library's urllib is supposed to look at $http_proxy and use > it, according to https://docs.python.org/3/library/urllib.request.html > so at least in theory proxies are a non issues. I wasn't quite sure how to pass around environment variables if set from the configuration (e.g. profiles/*.conf). I guess we could check for the *_proxy environment variables with env.get("http_proxy") and set with os.environ (if not already present)? >> Ironically, there are no remaining calls to wget in mirror_wget.py. >> Probably should deprecate and/or remove wget_debian_mirror and other >> variables that reference wget... > > I would do that, leaving the variable so that nothing explodes on old > config files, but marking it as deprecated/ignored. I could even add an > "ignored" flag to variables that makes it throw an exception if the code > uses them. > > Would you like me to try and provide a version which uses urlretrieve? All these ideas sound good to me, so please take a shot at it! live well, vagrant
signature.asc
Description: PGP signature
