Avoiding using hostname for YARN nodemanagers

2017-12-05 Thread Alvaro Brandon
Hello: I'm using Docker images to build a YARN cluster. I have a problem when the node managers register with the resource manager. Since they are containers they use the hash that the Docker engine assigns to them as the hostname. *17/12/05 14:56:16 INFO nodemanager.NodeStatusUpdaterImpl: Regis

Re: Avoiding using hostname for YARN nodemanagers

2017-12-05 Thread Vinayakumar B
Hi Alvaro, I think you can configure to use custom hostname for docker containers as well. Hostname should be provided durin launch of containers using -h parameter. And with user created docker network DNS resolution of these hostnames among the containers is possible. provide --network-alias pa

Re: Avoiding using hostname for YARN nodemanagers

2017-12-05 Thread Alvaro Brandon
Thanks for your answer Vinay: The thing is that I'm using Marathon and not the Docker engine per se. I don't want to set a -h parameter to each instance that is launched, since this is the responsibility of the container orchestrator platform. That's why I need an option like the HDFS one. Alvaro

Re: Avoiding using hostname for YARN nodemanagers

2017-12-05 Thread Susheel Kumar Gadalay
Check properties yarn.nodemanager.hostname, yarn.resourcemanager.hostname under yarn-site.xml. On 12/5/17, Alvaro Brandon wrote: > Thanks for your answer Vinay: > > The thing is that I'm using Marathon and not the Docker engine per se. I > don't want to set a -h parameter to each instance that i