[ https://issues.apache.org/jira/browse/MESOS-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marco Massenzio updated MESOS-2057: ----------------------------------- Story Points: 8 (was: 13) > Concurrency control for fetcher cache > ------------------------------------- > > Key: MESOS-2057 > URL: https://issues.apache.org/jira/browse/MESOS-2057 > Project: Mesos > Issue Type: Improvement > Components: fetcher, slave > Reporter: Bernd Mathiske > Assignee: Bernd Mathiske > Original Estimate: 96h > Remaining Estimate: 96h > > Having added a URI flag to CommandInfo messages (in MESOS-2069) that > indicates caching, caching files downloaded by the fetcher in a repository, > now ensure that when a URI is "cached", it is only ever downloaded once for > the same user on the same slave as long as the slave keeps running. > This even holds if multiple tasks request the same URI concurrently. If > multiple requests for the same URI occur, perform only one of them and reuse > the result. Make concurrent requests for the same URI wait for the one > download. > Different URIs from different CommandInfos can be downloaded concurrently. > No cache eviction, cleanup or failover will be handled for now. Additional > tickets will be filed for these enhancements. (So don't use this feature in > production until the whole epic is complete.) > Note that implementing this does not suffice for production use. This ticket > contains the main part of the fetcher logic, though. See the epic MESOS-336 > for the rest of the features that lead to a fully functional fetcher cache. > The proposed general approach is to keep all bookkeeping about what is in > which stage of being fetched and where it resides in the slave's > MesosContainerizerProcess, so that all concurrent access is disambiguated and > controlled by an "actor" (aka libprocess "process"). > Depends on MESOS-2056 and MESOS-2069. -- This message was sent by Atlassian JIRA (v6.3.4#6332)