RE: Securely discovering Application Master's metadata or sending a secret to Application Master at submission

Hi Basically I see you have multiple questions 1. How to get AM RPC port ? >>> This you can get it via YarnClient# getApplicationReport(). This gives >>> common/generic application specific details. Note that RM does not maintain >>> any custom details for applications. 2. How can

Re: Failed to start namenode.

Good to hear the issue is resolved and able to continue with your setup! Best Regards, Rakesh On Fri, Jun 10, 2016 at 9:34 AM, Hafiz Mujadid wrote: > Thanks Anu and Rakesh for your response. The problem was that principal > name was not added to database, I was trying to connect to hadoop throu

Re: Failed to start namenode.

Thanks Anu and Rakesh for your response. The problem was that principal name was not added to database, I was trying to connect to hadoop through. The keytab file permissions were not enough.So by changing the permission and adding relative principal name solved the issue. Thanks On Thu, Jun 9, 2

ResourceManager API

Hi Experts, Is there a way to get the logs from resourcemanager api for running job ? please help me. -- Sincere Regards, A.Kishore Kumar,

Re: CapacityScheduler vs. FairScheduler

I have the same question. Thanks! *Alvin Chyan*Lead Software Engineer, Data 901 Marshall St, Suite 200, Redwood City, CA 94063 turn.com | @TurnPlatform This message is Turn Confidential, except for information included that is alr

Re: SaslException: No common protection layer between client and server ?

Thanks, Chris. Basically, I've tried all 3 values for 'hadoop.rpc.authentication'. I was getting the same error as before for 'authentication' and 'integrity'. But for 'privacy', I started getting a different error (quoted below). The recommendation I've heard so far is this: "The new error is

Verifying the authenticity of submitted AM

// forking for clarify Related to the question I had below, I’m wondering how I can verify the authenticity of the submitted AM. (For example, when I’m making a call to AM, I’d like to verify that I’m talking to the AM that I submitted, not someone else who hijacked my network traffic. Also,

Re: Securely discovering Application Master's metadata or sending a secret to Application Master at submission

Hi Rohith, Thanks for the pointers. I checked the Hadoop documentation you linked, but it’s not clear how I can expose client interface for providing metadata. By “YARN internal communications”, I was referring to the endpoints that are exposed by AM on the RPC port as reported in Applicatio

Re: SaslException: No common protection layer between client and server ?

Hello Dmitry, For a successful SASL authentication, both the client and server need to negotiate and come to agreement on a quality of protection. The Hadoop configuration property hadoop.rpc.protection supports a comma-separate list of values that map to the SASL QoP values. hadoop.rpc.pr

Re: Looking for documentation/guides on Hadoop 2.7.2

Mike, Here is a guide on how to do some of the work but, this is using Ambari and not just the tar.gz This can help you understand how to piece certain things together. https://cwiki.apache.org/confluence/display/AMBARI/Start+Guide+Using+Centos+6.x this help me understand more when I was in

Re: Looking for documentation/guides on Hadoop 2.7.2

Hi Mike, I am sorry your experience with setting up Hadoop has been frustrating and mysterious. I will try to give partial answers / pointers to where you should be looking. Please be patient with me. Ø After reading the book I had a first idea of how components work together, but for me the

Re: Failed to start namenode.

Hi Hafiz, All suggestions from Rakesh are great ways to debug your current situation. However, it is hard to answer this question without specifics of the distro. The issue is that different vendors and apache seems to have slightly different recommendation. If you are working with apache – he

Re: Failed to start namenode.

Hi, Could you please check kerberos principal name is specified correctly in "hdfs-site.xml", which is used to authenticate against Kerberos. If using _HOST variable in hdfs-site.xml, ensure that hostname is getting resolved and it matches with the principal name. If keytab file defined in "hdfs-

Reg: Yarn resourcemanager logs alert

Hi Experts, My requirement is to get the alert from yarn resourcemanager logs to mail, if specific pattern occurred in the log, which is the best way ? please help me. -- Sincere Regards, KishoreKumar.

Failed to start namenode.

Hi, I have setup kerbores with hadoop and I am facing following exception when i start hadoop. ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. java.io.IOException: Login failure for admin/admin@queryiorealm from keytab /usr/local/var/krb5kdc/kadm5.keytab: javax.se

Looking for documentation/guides on Hadoop 2.7.2

Hey everyone. I just started some weeks ago to learn about Hadoop. I got the task to understand the Hadoop Ecosystem, and be able to answer some questions. First of all I started reading a book "OReilly - Hadoop The Definitive Guide". After reading the book I had a first idea of how components w