[ https://issues.apache.org/jira/browse/YARN-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589066#comment-16589066 ]
Eric Yang commented on YARN-8675: --------------------------------- Further analysis of the problem indicated that the registry DNS is set to mycluster.com, while the host level domain is example.com. If the hostname is host1.example.com, spark on yarn cluster workload will start the container as host1.mycluster.com. Host1.mycluster.com is unresolvable because no registryDNS entry is written to zookeeper. Without using YARN service API, there is no AM logic that handles registration of hostname to IP mapping. This is the reason that it failed. For handling net=host situation properly without using YARN service API, the registry DNS must set to same as host level domain, which is example.com. System administrator must configure registryDNS domain properly to permit application to use host level domain. This is to ensure that decouple of infrastructure cluster (YARN), and workload cluster (YARN apps). The application does not try to impersonate infrastructure cluster unless explicitly allowed. This is a feature, not a bug. > Setting hostname of docker container breaks with "host" networking mode for > Apps which do not run as a YARN service > ------------------------------------------------------------------------------------------------------------------- > > Key: YARN-8675 > URL: https://issues.apache.org/jira/browse/YARN-8675 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Yesha Vora > Assignee: Suma Shivaprasad > Priority: Major > Labels: Docker > > Applications like the Spark AM currently do not run as a YARN service and > setting hostname breaks driver/executor communication if docker version > >=1.13.1 , especially with wire-encryption turned on. > YARN-8027 sets the hostname if YARN DNS is enabled. But the cluster could > have a mix of YARN service/native Applications. > The proposal is to not set the hostname when "host" networking mode is > enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org