[
https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373202#comment-16373202
]
Mridul Muralidharan commented on YARN-7935:
-------------------------------------------
Hi [~tgraves],
Initial expectation was that there will be no change in client application
with introduction of docker support in yarn.
Unfortunately, in our experiments, we find the need to make application changes
- the scope of which is still not know.
For example, with 'host' networking mode, there is actually no change required
and things work seemlessly ! (and give us package isolation we so badly require)
But with bridge or overlay, YARN-7935 will be required (among other things -
which is still under investigation).
>From spark side of things, I am still unsure about what all other changes are
>needed and we are still investigating there too.
We use "--hostname" not just to specify where to bind to, but also what to
report back to driver.
Driver uses this for comm as well as locality computation (HOST_LOCAL and
RACK_LOCAL).
Hence it is not simple case of replacing --hostname with CONTAINER_HOSTNAME (if
available) - that will break HOST locality computation.
I wanted to create an a jira in spark once the scope was more concrete, but I
can do that sooner if it helps to collaborate better.
If you are doing some experiments, would be great to touch base !
> Expose container's hostname to applications running within the docker
> container
> -------------------------------------------------------------------------------
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch
>
>
> Some applications have a need to bind to the container's hostname (like
> Spark) which is different from the NodeManager's hostname(NM_HOST which is
> available as an env during container launch) when launched through Docker
> runtime. The container's hostname can be exposed to applications via an env
> CONTAINER_HOSTNAME. Another potential candidate is the container's IP but
> this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
