The pulp 3 alpha included two solutions for providing downloading
support in the plugin API. Each solution was based on different
concurrency technologies, protocol support libs and abstractions. The
goal was to test drive each, get community feedback and make a second
pass at documenting the /minimal/ requirements and characteristics
deemed desirable for download support in the API. This criteria appears
at the end of this email.
Agreement has been reached to proceed with the /asyncio/based solution.
The underlying asyncio/coroutine technology has several advantages and
none of the disadvantages imposed by python threading. The abstractions
in the proposed solution are sufficient to meet the minimal criteria
we've documented.
Unless there are objections, the next steps will include writing stories to:
* Remove the /futures/ solution.
* Move (promote) the /asyncio/ solution to the /download/ package.
* Update the ChangeSet to work with /asyncio/ based downloaders.
* Update the File plugin to work with /asyncio/ downloaders.
* Update the Example plugin imports.
The feature set and a few characteristics of the /asyncio/ solution need
(some) further discussion and may necessitate follow up stories. But,
this set of stories will get us most of the way there.
--------------------------------------------
_Requirements_
* support concurrent downloading by changesets
* usesa /Downloader/
* support concurrent downloading by streamer
* uses a /Downloader/
* importer provides the right, configured downloader(defined interface)
* streamer and changeset does not choose
___Desirable_
* Platform supply downloading support for known protocols.
* Support concurrent downloading
* single file
* groups of files
* HTTP downloader
* size verification
* hash verification
* use the importer settings
* automatically retry
* automatically follow HTTP redirects
* raise 400+ HTTP responses as foreground errors
* exposes underlying lib settings
* connection pooling (concurrent)
* connection reuse (concurrent)
* connection keep-alive support (concurrent)
* extensible
* must signal that headers are available prior to flowing data
* raises fatal exceptions
* FILE downloader
* size verification
* hash verification
* use the importer settings
* extensible
* raises fatal Exceptions
* Factory to build a downloader from the importer settings
* Leverage library/language as much as possible
/Downloader//- /characteristics
* Start the download
* Do something with the downloaded content.
* Write to file
* Write to output stream (Eg: streamer)
* Calculate and providethesize and digests
* Verify size and/or digest(s) values (requirement)
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev
