On 18Oct2021 21:25, Tom P <[email protected]> wrote:
>In the context of building Docker images, it is often required to download
>stuff. If curl/wget is available, great, but often slim images don't
>include that. The urllib could provide a very simple download functionality
>(like http offers a simple server):
>
>from urllib.request import urlopen
> data = urlopen('https://.../install-poetry.py').read()
> # print or save data
Well, it could I suppose, but it is probably better to reach straight
for the "requests" module (get it with "pip install requests").
It offers much functionality, in particular its response object has an
iter_content() method for fetching payload content iteratively.
The problem with a "download()" method is that it is almost never what
you need. There are too many ways to want to do it, and one almost
_never_ wants to suck the download itself into memory as you do above
with read() because downloads are often large, sometimes very large.
You also don't always want to put it into a file.
As a result, a "download()" method would rapidly grow a bunch of options
trying to accomodate many simple but differing uses. Instead you might
be better with a method which could return the contents in a loop to
process as you see fit, which is what iter_content() does.
Untested example (but based on some real world download code I've got
right here):
rsp = requests.get(url)
with open("filename","wb") as f:
for chunk in rsp.iter_content():
f.write(chunk)
Now, if you find yourself doing that _specific_ variation often, write
yourself a function to do it and keep it in a module of your own.
Cheers,
Cameron Simpson <[email protected]>
_______________________________________________
Python-ideas mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/RSXNKXFVVIXIL7DXDW65AYOUKZQBR3S3/
Code of Conduct: http://python.org/psf/codeofconduct/
