Hi, Fetcher was originally designed to fetch CommandInfo::URIs (e.g., executor binary) for executors/tasks. A recent refactor (MESOS-336 <https://issues.apache.org/jira/browse/MESOS-336>) added caching support to the fetcher. The recent work on filesystem isolation/unified containerizer ( MESOS-2840 <https://issues.apache.org/jira/browse/MESOS-2840>) requires Mesos to fetch filesystem images (e.g., APPC/DOCKER images) as well. The natural question is: can we leverage the fetcher to fetch those filesystem images (and cache them accordingly)? Unfortunately, the existing fetcher interface is tightly coupled with CommandInfo::URIs for executors/tasks, making it very hard to be re-used to fetch/cache filesystem images.
Another motivation for the refactor is that we want to extend the fetcher to support more types of schemes. For instance, we want to support magnet URI to enable p2p fetching. This is in fact quite important for operating a large cluster (MESOS-3596 <https://issues.apache.org/jira/browse/MESOS-3596>). The goal here is to allow fetcher to be extended (e.g., using modules) so that operators can add custom fetching support. I proposed a solution in this doc <https://docs.google.com/document/d/1p8tmQrGtxG6keZVo19gvHPr9WHxeny6PHTVofcLWqco/edit?usp=sharing>. The main idea is to decouple artifacts fetching from artifacts cache management. We can make artifacts fetching extensible (e.g. to support p2p fetching), and solve the cache management part later. Let me know your thoughts! Thanks! - Jie
