[ https://issues.apache.org/jira/browse/MESOS-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101576#comment-15101576 ]
haosdent commented on MESOS-1887: --------------------------------- I saw the new fetcher module add --hadoop_client to specify the hadoop client path. No sure whether this flag is passed when runtime or when start agent in the future. If it is passed when runtime, I think this issue have already been resolved. > Allow to specify programs to use in the fetcher > ----------------------------------------------- > > Key: MESOS-1887 > URL: https://issues.apache.org/jira/browse/MESOS-1887 > Project: Mesos > Issue Type: Improvement > Components: fetcher, slave > Reporter: Tobias Weingartner > > In environments with multiple different hadoop versions, the hadoop version > on the slave's {{$PATH}}, or found via {{$HADOOP_HOME}} may not be the always > globally correct version of hadoop. IE: > * The slave may be fetching executors from hadoop1 > * The execution may be against/for a hadoop2 > Unfortunately, {{hdfs://}} urls are not specific enough to be able to > dis-ambiguate the particular version of hadoop you should use to access the > hadoop/hdfs cluster given in the url. > Off hand, I can see two possible solutions: > * extra information from the framework in the task struct to help identify > the fetchers environment for fetching this particular executor (different > from the slave's environment, different from a task's environment). > * fetchers global environment (different from slave and task's > environemnt), allowing a different hadoop/hdfs cluster to be used to fetch > _all_ executors from within hdfs > * a method to replace the fetcher with site specific scripts/programs > Other? -- This message was sent by Atlassian JIRA (v6.3.4#6332)