[
https://issues.apache.org/jira/browse/YARN-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806273#comment-15806273
]
Miklos Szegedi commented on YARN-5714:
--------------------------------------
Thank you for the comment [~templedf].
That option raises the question to prepend or append to the client list in
{{sanitizeEnv()}} and in the code that populates it for the context and whether
the assumption is true all the time. The serialization code in
{{addEnvToProto()}} needs to be changed to iterate a LinkedHashMap order and
the deserialization code in {{initEnv()}} to create a LinkedHashMap. Moreover,
since it relies on the client and the application to give the right order all
the time, so it probably involves changes outside Yarn.
Do you also suggest to change the container launch context interface to accept
a linked hash map or iterator instead of a map? Is the plan to keep the
interface but handle it correctly, if it is a linked hash map?
A positive effect of the LinkedHashMap is that the algorithm does not need to
run at every container launch.
> ContainerExecutor does not order environment map
> ------------------------------------------------
>
> Key: YARN-5714
> URL: https://issues.apache.org/jira/browse/YARN-5714
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.4.1, 2.5.2, 2.7.3, 2.6.4, 3.0.0-alpha1
> Environment: all (linux and windows alike)
> Reporter: Remi Catherinot
> Assignee: Remi Catherinot
> Priority: Trivial
> Labels: oct16-medium
> Attachments: YARN-5714.001.patch, YARN-5714.002.patch,
> YARN-5714.003.patch, YARN-5714.004.patch, YARN-5714.005.patch,
> YARN-5714.006.patch
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> when dumping the launch container script, environment variables are dumped
> based on the order internally used by the map implementation (hash based). It
> does not take into consideration that some env varibales may refer each
> other, and so that some env variables must be declared before those
> referencing them.
> In my case, i ended up having LD_LIBRARY_PATH which was depending on
> HADOOP_COMMON_HOME being dumped before HADOOP_COMMON_HOME. Thus it had a
> wrong value and so native libraries weren't loaded. jobs were running but not
> at their best efficiency. This is just a use case falling into that bug, but
> i'm sure others may happen as well.
> I already have a patch running in my production environment, i just estimate
> to 5 days for packaging the patch in the right fashion for JIRA + try my best
> to add tests.
> Note : the patch is not OS aware with a default empty implementation. I will
> only implement the unix version on a 1st release. I'm not used to windows env
> variables syntax so it will take me more time/research for it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
