[
https://issues.apache.org/jira/browse/MESOS-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13972194#comment-13972194
]
Benjamin Hindman commented on MESOS-336:
----------------------------------------
I'm not sure I understand you correctly. If the fetcher is asked to download
the URI 'protocol://path/to/foobar-0.4.5.tar.gz' with checksum 'ABC' and later
asked to download the URI
'anotherprotocol://some/other/path/to/foobar-0.4.5.tar.gz' with checksum 'ABC'
why couldn't (wouldn't) we treat these as the same file? This seems pretty
standard, i.e., you might put a file up at location X and then mirror it at
location Y, Z, and W and as long as the checksum's match you're confident you
got the right file.
Alternatively, using the entire URI name doesn't seem to be out of the question
either, we'd just have to write a mapping from URI to how we put it in the
directory cache (i.e., maybe have a directory of protocols first like 'http'
then match the path to the file). I'm not sure this is necessary though as I
think that the above semantics are OKAY and won't be surprising to users.
> Mesos slave should cache executors
> ----------------------------------
>
> Key: MESOS-336
> URL: https://issues.apache.org/jira/browse/MESOS-336
> Project: Mesos
> Issue Type: Improvement
> Components: slave
> Reporter: brian wickman
> Assignee: Bernd Mathiske
> Labels: newbie
>
> The slave should be smarter about how it handles pulling down executors. In
> our environment, executors rarely change but the slave will always pull it
> down from regardless HDFS. This puts undue stress on our HDFS clusters, and
> is not resilient to reduced HDFS availability.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
