[
https://issues.apache.org/jira/browse/MESOS-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190205#comment-17190205
]
Andrei Sekretenko commented on MESOS-10169:
-------------------------------------------
Master:
{noformat}
commit a4a2937218c654c4b1490820d0b1d119ba0ae16d
Author: Andrei Sekretenko <[email protected]>
Date: Thu Aug 20 17:02:51 2020 +0200
Deduplicated concurrent image pulls by docker store.
This patch makes the docker store reuse a pending pull if asked for
an image that is already being pulled.
The pull caching follows the same approach as the earlier attempt in
https://reviews.apache.org/r/39331 . However, handing out futures to
the store callers and handling their discards are performed differently,
using a form of reference counting, so that the pull itself is discarded
only if all the futures returned by `Store::get()` have been discarded.
Review: https://reviews.apache.org/r/72790
commit d7d6498bd4cd88d9d929ef89441de80c54118193
Author: Andrei Sekretenko <[email protected]>
Date: Thu Aug 20 17:49:13 2020 +0200
Reverted removal of the PullingSameImageSimutanuously test.
Now that the docker store triggers pull at most once per multiple
simultaneous requests to the store, removal of the
`PullingSameImageSimutanuously` test in
33c61a1907129126f3b2e37b1f53827a04e89a34 can be reverted.
Review: https://reviews.apache.org/r/72791
commit f6791097995f0e736349a852fdb7fef11e809e63
Author: Andrei Sekretenko <[email protected]>
Date: Thu Aug 20 18:45:44 2020 +0200
Fixed typos in the name of `PullingSameImageSimutanuously` test.
Review: https://reviews.apache.org/r/72792
commit 3ee545c4f039000adc80d4bac3550e40616999db
Author: Andrei Sekretenko <[email protected]>
Date: Thu Aug 20 18:16:39 2020 +0200
Added discarding a future returned by get() into the docker store test.
This patch extends the `PullingSameImageSimultaneously` test
with discarding one of the futures returned by `store->get()`
and making sure that the pull still completes.
Review: https://reviews.apache.org/r/72793
commit 5274e973c9d9ff06f9c80af9c80efd510aa492b7 (HEAD -> master, origin/master,
origin/HEAD, ci/ci/asekretenko/deduplicated_pull, deduplicated_pull,
ci/asekretenko/deduplicated_pull)
Author: Andrei Sekretenko <[email protected]>
Date: Thu Aug 20 18:36:40 2020 +0200
Added a test for discarding image pull on discard of getting image.
This test checks that when all futures returned by docker store's
`get()` that are pending image pull are discarded by the callers,
the pull future (returned by the puller to the store) is discarded
by the store as well.
Review: https://reviews.apache.org/r/72794
{noformat}
> Reintroduce image fetch deduplication while keeping it possible to destroy
> UCR containers in PROVISIONING state
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MESOS-10169
> URL: https://issues.apache.org/jira/browse/MESOS-10169
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.7.3, 1.8.2, 1.9.1, 1.10.0, 1.7.4, 1.11.0, 1.10.1
> Reporter: Andrei Sekretenko
> Assignee: Andrei Sekretenko
> Priority: Major
>
> The fix for https://issues.apache.org/jira/browse/MESOS-9964 had to drop the
> image fetch deduplication that was introduced long ago in
> https://issues.apache.org/jira/browse/MESOS-3736. We should reintroduce the
> deduplication back.
> UCR containers pulling the same image at the same time should pull the image
> only once.
> *This deduplication should not prevent killing some of the PROVISIONING
> containers, and such killing should not interfere with provisioning of other
> containers.*
> I would argue that it is OK for all the containers depending on a single
> image pull to hang if this single pull hangs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
