[
https://issues.apache.org/jira/browse/YARN-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590143#comment-16590143
]
Shane Kumpf commented on YARN-8675:
-----------------------------------
Looking back over the history here, I think we made the wrong decision in
setting the YARN defined hostname when {{\-\-net=host}}. I think we should make
{{\-\-net=host}} return the NM hostname, even if Registry DNS is enabled, as
you originally proposed in YARN-7797 via your early patches, [~eyang]. While
using the YARN defined hostname was nice for testing, it breaks several aspects
of running both Services and "native" Hadoop frameworks, such as MR and Spark,
side by side, which is a core goal of the containerization effort.
The problem isn't domain, it is that the "ctr" hostname we are setting won't
exist in DNS for these containers. Concretely, the NM will set
{{\-\-hostname=ctr-e111-11111111111-111111-01-000006.domain.site}} even though
that entry will never be available via DNS, since the Spark job is not running
as a YARN Service, and not writing any entries to ZK. Anything related to that
container that relies on DNS lookups will fail.
> Setting hostname of docker container breaks with "host" networking mode for
> Apps which do not run as a YARN service
> -------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-8675
> URL: https://issues.apache.org/jira/browse/YARN-8675
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Yesha Vora
> Assignee: Suma Shivaprasad
> Priority: Major
> Labels: Docker
>
> Applications like the Spark AM currently do not run as a YARN service and
> setting hostname breaks driver/executor communication if docker version
> >=1.13.1 , especially with wire-encryption turned on.
> YARN-8027 sets the hostname if YARN DNS is enabled. But the cluster could
> have a mix of YARN service/native Applications.
> The proposal is to not set the hostname when "host" networking mode is
> enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
