Re: Securely discovering Application Master's metadata or sending a secret to Application Master at submission

2016-06-10 Thread Mingyu Kim
Ah, I see. I thought the RPC port was used for YARN’s own communication, but that field in ApplicationReport is exactly there for exposing some interfaces to the client, which is what I was looking for. I’ll take a look at MRAppMaster for an example. Thanks a lot for help! Mingyu From: S

view job counters in real time

2016-06-10 Thread Joseph Naegele
Hi all, Is it possible to monitor MR Job counters while a job is running? I'm using Hadoop 2.7.1 and YARN. I can view counters for past jobs using the history server, but can't find a way to view them in real time. Thanks

Re: Verifying the authenticity of submitted AM

2016-06-10 Thread Sunil Govind
HI Mingyu, May be you can take a look at below link https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/yarn.html It will give a fair idea about the security you can get for an application - Sunil On Fri, Jun 10, 2016 at 3:54 AM Mingyu Kim wrote: > // forking for clarify >

Re: ResourceManager API

2016-06-10 Thread Sunil Govind
Hi Kishore Below command may help you to get some basic information which you are looking for. https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#logs Further to this, some more enhancements are happening as part of YARN-4904, but its not part of any release as

Re: Securely discovering Application Master's metadata or sending a secret to Application Master at submission

2016-06-10 Thread Sunil Govind
Hi Mike Adding to what Rohith has mentioned, you can refer to below interface to know what all information which you can get from Yarn w.r.t one application. https://hadoop.apache.org/docs/r2.7.0/api/org/apache/hadoop/yarn/api/records/ApplicationReport.html This has RPC port ApplicationMaster, an

AW: Looking for documentation/guides on Hadoop 2.7.2

2016-06-10 Thread Mike Wenzel
Hi Anu and Johny, first of all I want to thank both of you for replying. I didn't wanted to look that frustrated as it seems to look. I'm pretty fine and my interests in Hadoop are huge :) @Anu: I did an free-online course by cloudera / udacity which helped me a lot to make things clear regard