Issuse to get oozi/spark to read from a local cluster and write back to a remote cluster

Hi there, I am having issues to get oozie spark action to read from hdfs  on one secured cluster and write it back to an hdfs in a remote secured  cluster with kerberos . However ooze/spark can read from the hdfs in a local cluster and write back to hdfs in the same cluster . the oozie job

yarn groups issue with ambari/hortonworks

through ambari and open a new shell , I can see that yarn user no longer belongs to the new group. My question is whether ambari is removoig the yarn user from any group other than it's primary one. Any response to this will be appreciated. Thanks Reyane OUKPEDJO

Yarn Localization failure on hadoop

/ command is working fine as a user sumbitting the Application to make sure kerberos credentials for this user is not to be blamed.  a mapreduce  pi example could run successfully. Any idea what could cause this kind of issue? Thanks Reyane OUKPEDJO NODE MANAGER LOGS HERE 2015-04-27 22:30:48,682

keberos issue

Hi Everyone, I setup a kerberos enabled cluster using HDP-2.2 and I am facing the following issue : 2015-03-04 16:24:43,419 WARN  security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleDTRenewerAppSubmitEvent(785)) - Unable to add the application to the delegation token

I was wondering what could make these 2 variables different: HADOOP_CONF_DIR vs YARN_CONF_DIR

Can someone explain what makes the above variable different  ? Most of the time they are set pointing to the same directory. Thanks  Reyane OUKPEDJO

issue with defaut yarn.application.classpath value from yarn-default.xml for hadoop-2.3.0

of that parameter. When I checked the same file on hadoop-2.2.0, I can see that a default value is set. Can anyone explain why this happens ? should I open a bug report? or was this intentional ? Thanks Reyane OUKPEDJO   property     descriptionDOOP_CONF_DIR,           $HADOOP_COMMON_HOME/share/hadoop

Re: Yarn - avoid Launching Client and Application manager every time

the application masters. But it is doable. As for making the application master run like a server , I am not sure if that makes much sense since that will make him look like a job tracker. And it is because of the above limitations that the JobTracker no longer exists on yarn. Thanks Reyane OUKPEDJO

Re: what happens to a client attempting to get a new app when the resource manager is already down

Thank you very much. Reyane OUKPEDJO On Monday, February 3, 2014 10:28 AM, Jian He j...@hortonworks.com wrote: For NMs talking with RM, you need to restart the NMs to take the configs into effect. For the client talking with RM, you don't need to restart the RM. Jian On Mon, Feb 3, 2014

Re: what happens to a client attempting to get a new app when the resource manager is already down

Thank you very much.  One last question: Do you think those parameters setting require restarting the cluster? Thanks  Reyane OUKPEDJO On Sunday, February 2, 2014 11:06 AM, Jian He j...@hortonworks.com wrote: Hi  All entities including NM and client etc. use the same RMProxy to talk

what happens to a client attempting to get a new app when the resource manager is already down

. However it just stay in a loops and sleep after 10 iteration and continue the same loops. Below you can find the logs. Any idea how to leave this loop ? is there any parameter that control the number of seconds before giving up. Thanks Reyane OUKPEDJO logs 14/01/31 10:48:05 INFO ipc.Client

Re: Container's completion issue

? Or your own custom application? +Vinod Hortonworks Inc. http://hortonworks.com/ On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO r.oukpe...@yahoo.com wrote: Hi there, I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more

Container's completion issue

the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed. Thanks  Reyane OUKPEDJO 

Re: C++ example for hadoop-2.2.0

I will suggest getting the source and compiling in your machine. If your machine is 64 bits you will get the 64 bits native libraries. That will solve the problem you have .Please get hadoop-2.2.0-src.tar.gz. Thanks  Reyane OUKPEDJO On Monday, November 4, 2013 7:56 AM, Amr Shahin amrnab

how to build hadoop native libraries in debug mode

library but with a mix of both modes. But I need it to be completely in debug mode . Any help?  Thanks Reyane OUKPEDJO

How to compile and build hadoop native libraries in debug mode

library but with a mix of both modes. But I need it to be completely in debug mode . Any help?  Thanks Reyane OUKPEDJO

Re: Hadoop Jobtracker heap size calculation and OOME

Hi there, I had a similar issue with hadoop-1.2.0 JobTracker keep crashing until I set HADOOP_HEAPSIZE=2048 I did not have this kind of issue with previous versions. But you can try this if you have memory and see. In my case the issue was gone after I set as above. Thanks Reyane OUKPEDJO

Hadoop-2.0.1 log files deletion

property on mapred-site.xml. some of the job logs were deleted after 4 hours. Can this be a bug or if not is there any other property that override this ? Thank you. Reyane OUKPEDJO

Re: Hadoop-2.0.1 log files deletion

Thanks problem solved. Reyane OUKPEDJO On 10 October 2013 11:10, Krishna Kishore Bonagiri write2kish...@gmail.comwrote: Hi Reyane, Did you try yarn.nodemanager.log.retain-seconds? increasing that might help. The default value is 10800 seconds, that means 3 hours. Thanks