Yes, Yan. But that communication is initiated by the AM. Whether an
application's AM does it or not, the NM always heartbeats the status of
its containers to the RM.
On Jul 30, 2015 6:40 PM, "Yan Fang" <yanfang...@gmail.com> wrote:
> Just one point to add:
>
> {quote}
> AM gets notified of container status from the RM.
> {quote}
>
> I think this is not 100% correct. AM can communicate with NM through
> NMClientAsync
> <
> https://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/yarn/client/api/async/NMClientAsync.html
> >
> to
> get container status, though Samza does not implement the CallbackHandler.
>
> Thanks,
>
> Fang, Yan
> yanfang...@gmail.com
>
> On Thu, Jul 30, 2015 at 6:06 PM, Navina Ramesh
> <nram...@linkedin.com.invalid
> > wrote:
>
> > The NM (and hence, by extension the container) heartbeats to the RM, not
> > the AM. AM gets notified of container status from the RM.
> > The AM starts / stops /releases a container process by communicating to
> the
> > NM.
> >
> > Navina
> >
> >
> > On Thu, Jul 30, 2015 at 5:55 PM, Thomas Becker <tobec...@tivo.com>
> wrote:
> >
> > > Ok, I thought there was some communication from the container to the
> AM,
> > > it sounds like you're saying it's in the other direction only? Don't
> > > containers heartbeat to the AM? Regardless, even if we can't get a
> > better
> > > address for the AM from YARN, we could at least filter the addresses we
> > get
> > > back from the JVM to exclude loopbacks.
> > >
> > > -Tommy
> > > ________________________________________
> > > From: Navina Ramesh [nram...@linkedin.com.INVALID]
> > > Sent: Thursday, July 30, 2015 8:40 PM
> > > To: dev@samza.apache.org
> > > Subject: Re: Coordinator URL always 127.0.0.1
> > >
> > > Hi Tommy,
> > > Yi is right. Container start is coordinated by the AppMaster using an
> > > NMClient. Container host name and port is provided by the RM during
> > > allocation.
> > > In Yarn (at least, afaik), when the node joins a cluster, the NM
> > registers
> > > itself with the RM. So, the NM might still be using
> > > getLocalhost.getAddress().
> > >
> > > I don't know of any other way to programmatically fetch the machine's
> > > hostname (apart from some hacky shell commands).
> > >
> > > Cheers,
> > > Navina
> > >
> > > On Thu, Jul 30, 2015 at 5:23 PM, Yi Pan <nickpa...@gmail.com> wrote:
> > >
> > > > Hi, Tommy,
> > > >
> > > > Yeah, I agree that the current implementation is not bullet-proof to
> > any
> > > > different networking configuration on the host. As for the AM <->
> > > container
> > > > communication, if I am not mistaken, it is through the NMClient and
> the
> > > > node HTTP address is wrapped within the Container object returned
> from
> > > RM.
> > > > I am not very familiar with that part of source code. Navina may be
> > able
> > > to
> > > > help more here.
> > > >
> > > > -Yi
> > > >
> > > > On Thu, Jul 30, 2015 at 4:27 PM, Thomas Becker <tobec...@tivo.com>
> > > wrote:
> > > >
> > > > > Hi Yi,
> > > > > Thanks a lot for your reply. I don't doubt we can get it to work
> by
> > > > > mucking with the networking configuration, but to me this feels
> like
> > a
> > > > > workaround, not a solution.
> > > InetAddress.getLocalHost().getHostAddress()
> > > > is
> > > > > not a reliable way of obtaining an IP that other machines can
> connect
> > > to.
> > > > > Just today I tested on several Linux distros and it did not work on
> > any
> > > > of
> > > > > them. Can we do something more robust here? How does the
> container
> > > > > communicate status to the AM?
> > > > >
> > > > > -Tommy
> > > > >
> > > > > ________________________________________
> > > > > From: Yi Pan [nickpa...@gmail.com]
> > > > > Sent: Thursday, July 30, 2015 6:48 PM
> > > > > To: dev@samza.apache.org
> > > > > Subject: Re: Coordinator URL always 127.0.0.1
> > > > >
> > > > > Hi, Tommy,
> > > > >
> > > > > I think that it might be a commonly asked question regarding to
> > > multiple
> > > > > IPs on a single host. A common trick w/o changing code is (copied
> > from
> > > > SO:
> > > > >
> > > > >
> > > >
> > >
> >
> http://stackoverflow.com/questions/2381316/java-inetaddress-getlocalhost-returns-127-0-0-1-how-to-get-real-ip
> > > > > )
> > > > >
> > > > > {code}
> > > > >
> > > > > 1.
> > > > >
> > > > > Find your host name. Type: hostname. For example, you find your
> > > > hostname
> > > > > is mycomputer.xzy.com
> > > > > 2.
> > > > >
> > > > > Put your host name in your hosts file. /etc/hosts . Such as
> > > > >
> > > > > 10.50.16.136 mycomputer.xzy.com
> > > > >
> > > > >
> > > > > {code}
> > > > >
> > > > > -Yi
> > > > >
> > > > > On Thu, Jul 30, 2015 at 11:35 AM, Tommy Becker <tobec...@tivo.com>
> > > > wrote:
> > > > >
> > > > > > We are testing some jobs on a YARN grid and noticed they are
> often
> > > not
> > > > > > starting up properly due to being unable to connect to the job
> > > > > coordinator.
> > > > > > After some investigation it seems as if the jobs are always
> > getting a
> > > > > > coordinator URL of http://127.0.0.1:<port> But my understanding
> > is
> > > > that
> > > > > > the coordinator runs only in the AM, so I'd expect these URLs to
> > more
> > > > > often
> > > > > > than not be to some other machine. Looking at the code however,
> > I'm
> > > > not
> > > > > > sure how that would ever happen since the URL for the coordinator
> > > > always
> > > > > > comes from InetAddress.getLocalHost().getHostAddress() in
> > > > > > org.apache.samza.coordinator.server.HttpServer#getUrl
> > > > > >
> > > > > > Am I off base here? Because I don't see how this is ever going
> to
> > > work
> > > > > in
> > > > > > scenarios where the AM is on a different node than the
> containers.
> > > > > >
> > > > > > --
> > > > > > Tommy Becker
> > > > > > Senior Software Engineer
> > > > > >
> > > > > > Digitalsmiths
> > > > > > A TiVo Company
> > > > > >
> > > > > > www.digitalsmiths.com<http://www.digitalsmiths.com>
> > > > > > tobec...@tivo.com<mailto:tobec...@tivo.com>
> > > > > >
> > > > > > ________________________________
> > > > > >
> > > > > > This email and any attachments may contain confidential and
> > > privileged
> > > > > > material for the sole use of the intended recipient. Any review,
> > > > copying,
> > > > > > or distribution of this email (or any attachments) by others is
> > > > > prohibited.
> > > > > > If you are not the intended recipient, please contact the sender
> > > > > > immediately and permanently delete this email and any
> attachments.
> > No
> > > > > > employee or agent of TiVo Inc. is authorized to conclude any
> > binding
> > > > > > agreement on behalf of TiVo Inc. by email. Binding agreements
> with
> > > TiVo
> > > > > > Inc. may only be made by a signed written agreement.
> > > > > >
> > > > >
> > > > > ________________________________
> > > > >
> > > > > This email and any attachments may contain confidential and
> > privileged
> > > > > material for the sole use of the intended recipient. Any review,
> > > copying,
> > > > > or distribution of this email (or any attachments) by others is
> > > > prohibited.
> > > > > If you are not the intended recipient, please contact the sender
> > > > > immediately and permanently delete this email and any attachments.
> No
> > > > > employee or agent of TiVo Inc. is authorized to conclude any
> binding
> > > > > agreement on behalf of TiVo Inc. by email. Binding agreements with
> > TiVo
> > > > > Inc. may only be made by a signed written agreement.
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Navina R.
> > >
> > > ________________________________
> > >
> > > This email and any attachments may contain confidential and privileged
> > > material for the sole use of the intended recipient. Any review,
> copying,
> > > or distribution of this email (or any attachments) by others is
> > prohibited.
> > > If you are not the intended recipient, please contact the sender
> > > immediately and permanently delete this email and any attachments. No
> > > employee or agent of TiVo Inc. is authorized to conclude any binding
> > > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> > > Inc. may only be made by a signed written agreement.
> > >
> >
> >
> >
> > --
> > Navina R.
> >
>
