how can i get job start and finish time?

[ch huang]

hi,maillist:
   my web ui is not available, i can use "yarn application -list"
,my question is how can i get job start and finish time?

Re: how to caculate a HDFS directory size ?

[Jing Zhao]

You may want to check the fs shell command COUNT and DU.

On Wed, Jan 8, 2014 at 4:57 PM, ch huang  wrote:
> hi,maillist:
>   i want to caculate the size of a HDFS directory ,how to do it ?

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

how to caculate a HDFS directory size ?

[ch huang]

hi,maillist:
  i want to caculate the size of a HDFS directory ,how to do it ?

Hadoop-streaming : PYTHONPATH not working when mapper runs

[Dhanasekaran Anbalagan]

Hi Guys,

I have problem when I ran hadoop streaming PYTHONPATH environment variable
not properly getting value. when I try to extend sys.path code level  not
working. It's says KeyError:'PYTHONPATH' In my case I already exported
PYTHONPATH in /etc/bashrc

mapper logs:
==
stderr logs
Traceback (most recent call last):
  File
"/disk1/mapred/local/taskTracker/tech/jobcache/job_201312121801_0053/attempt_201312121801_0053_m_00_0/work/./cmapper.py",
line 14, in 
*sys.path.extend(os.environ['PYTHONPATH'].split(os.path.pathsep))*
  File "/usr/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
*KeyError: 'PYTHONPATH'*


I tried /etc/sudoers file I added PYTHONPATH

Defaultsenv_keep ="PYTHONPATH"

also not working.

can you please guide me How to fix this.

-Dhanasekaran.

Did I learn something today? If not, I wasted it.

org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException

[xeon Mailinglist]

When I try to launch the namenode and the datanode in MRv2, the datanode
can't connect to the namenode, giving me the error below. I also put the
core-site file that I use below.

The Firewall in the hosts is disabled. I don't have excluded nodes defined.
Why the datanodes can't connect to the namenode?  Any help to solve this
problem?


org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException):
Datanode denied communication with namenode: DatanodeRegistrati
on(0.0.0.0, storageID=DS-1449645935-172.16.1.10-50010-1389224474955,
infoPort=50075, ipcPort=50020,
storageInfo=lv=-40;cid=CID-9a8571a3-17ae-49b2-b957-b009e88b9f9a;nsid=9
34416283;c=0)
at
org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:631)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3398)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:881)
at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90)
at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:18295)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1741)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1737)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1735)

at org.apache.hadoop.ipc.Client.call(Client.java:1235)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
at com.sun.proxy.$Proxy9.registerDatanode(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
at com.sun.proxy.$Proxy9.registerDatanode(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.registerDatanode(DatanodeProtocolClientSideTranslatorPB.java:146)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.register(BPServiceActor.java:623)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:225)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664)
at java.lang.Thread.run(Thread.java:701)

I set the core-site.xml


   fs.default.name
hdfs://10.103.0.17:9000

   hadoop.tmp.dir /tmp/hadoop-temp


hadoop.proxyuser.root.hosts*

hadoop.proxyuser.root.groups*

Re: /home/r9r/hadoop-2.2.0/bin/hadoop: line 133: /usr/java/default/bin/java: No such file or directory

[Chris Mawata]

What is on the system path?
(what do you get at the command console type)
$PATH
perhaps you have

/usr/java/default/bin

in there

On 1/8/2014 3:12 PM, Allen, Ronald L. wrote:

Hello again,

I'm trying to install Hadoop 2.2.0 on Redhat 2.6.32-358.23.2.el6.x86_64.  I 
have untar-ed hadoop-2.2.0.tar.gz and set my path variables as below.

export HADOOP_INSTALL=/home/r9r
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path"=$HADOOP_INSTALL/lib

export JAVA_HOME=/usr/lib/jvm/java-1.7.0/jre/

When I type hadoop version, I get the following:

/home/r9r/hadoop-2.2.0/bin/hadoop: line 133: /usr/java/default/bin/java: No 
such file or directory
/home/r9r/hadoop-2.2.0/bin/hadoop: line 133: exec: /usr/java/default/bin/java: 
cannot execute: No such file or directory

I've checked and rechecked my JAVA_HOME.  I feel like it is correct.  I've 
checked hadoop-env.sh and it is set to export JAVA_HOME=${JAVA_HOME}

I am stuck and do not know what to try from here.  Does anyone have any ideas?

Thanks!
Ronnie.

Re: /home/r9r/hadoop-2.2.0/bin/hadoop: line 133: /usr/java/default/bin/java: No such file or directory

[Hardik Pandya]

your java home is not set correctly - its still looking under
usr/java/default/bin/java

in your hadoop-env.sh
JAVA_HOME should be /usr/lib/jvm/java-1.7.0/jre/

does your $PATH includes correct ${JAVA_HOME}?





On Wed, Jan 8, 2014 at 3:12 PM, Allen, Ronald L.  wrote:

> Hello again,
>
> I'm trying to install Hadoop 2.2.0 on Redhat 2.6.32-358.23.2.el6.x86_64.
>  I have untar-ed hadoop-2.2.0.tar.gz and set my path variables as below.
>
> export HADOOP_INSTALL=/home/r9r
> export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
> export HADOOP_COMMON_HOME=$HADOOP_INSTALL
> export HADOOP_HDFS_HOME=$HADOOP_INSTALL
> export YARN_HOME=$HADOOP_INSTALL
> export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
> export HADOOP_OPTS="-Djava.library.path"=$HADOOP_INSTALL/lib
>
> export JAVA_HOME=/usr/lib/jvm/java-1.7.0/jre/
>
> When I type hadoop version, I get the following:
>
> /home/r9r/hadoop-2.2.0/bin/hadoop: line 133: /usr/java/default/bin/java:
> No such file or directory
> /home/r9r/hadoop-2.2.0/bin/hadoop: line 133: exec:
> /usr/java/default/bin/java: cannot execute: No such file or directory
>
> I've checked and rechecked my JAVA_HOME.  I feel like it is correct.  I've
> checked hadoop-env.sh and it is set to export JAVA_HOME=${JAVA_HOME}
>
> I am stuck and do not know what to try from here.  Does anyone have any
> ideas?
>
> Thanks!
> Ronnie

enabling priority jobs to Hadoop with Yarn

[ricky l]

Hi,

Can someone give advice for the following scenario?
I have a MR pool with Yarn; the total number of available containers is
100. At a given time, a lower priority job is submitted and consumes all
100 containers. Later, I submit a higher priority job and it is not running
as the all containers are taken by the lower priority job. I tried to
change the job priority by having -Dmapred.job.priority=HIGH and change the
priority using CLI - mapred job priority job-id HIGH. Both solutions do not
seem to work. Can any one give advice? thx.

Re: A question about Hadoop 1 job user id used for group mapping, which could lead to performance degradatioin

[Jian Fang]

Looked a bit deeper and seems this code was introduced by the following
JIRA.

https://issues.apache.org/jira/browse/MAPREDUCE-1457

There is another related JIRA, i.e.,
https://issues.apache.org/jira/browse/MAPREDUCE-4329.

Perhaps, the warning message is a side effect of JIRA MAPREDUCE-1457 when
the cluster is running in non-secured mode. There should be some code path
that caused the job id was treated as user name in task tracker or job
tracker. Then the job id was passed to HDFS name node. This is definitely a
big problem since the heavy warning logs alone degraded the system
performance in a relatively big cluster.

This behavior is very easy to reproduce by simply running terasort on a
cluster.

Any suggestion to fix this problem?




On Wed, Jan 8, 2014 at 11:18 AM, Jian Fang wrote:

> Thanks Vinod for your quick response. It is running in non-secure mode.
>
> I still don't get what is the purpose to use job id in UGI. Could you
> please explain a bit more?
>
> Thanks,
>
> John
>
>
> On Wed, Jan 8, 2014 at 10:11 AM, Vinod Kumar Vavilapalli <
> vino...@hortonworks.com> wrote:
>
>> It just seems like lazy code. You can see that, later, there is this:
>>
>> {code}
>>
>> for(Token token :
>> UserGroupInformation.getCurrentUser().getTokens()) {
>>   childUGI.addToken(token);
>> }
>>
>> {code}
>>
>> So eventually the JobToken is getting added to the UGI which runs
>> task-code.
>>
>> >  WARN org.apache.hadoop.security.UserGroupInformation (IPC Server
>> handler 63 on 9000): No groups available for user job_201401071758_0002
>>
>> This seems to be a problem. When the task tries to reach the NameNode, it
>> should do so as the user, not the job-id. It is not just logging, I'd be
>> surprised if jobs pass. Do you have permissions enabled on HDFS?
>>
>> Oh, or is this in non-secure mode (i.e. without kerberos)?
>>
>> +Vinod
>>
>>
>> On Jan 7, 2014, at 5:14 PM, Jian Fang 
>> wrote:
>>
>> > Hi,
>> >
>> > I looked at Hadoop 1.X source code and found some logic that I could
>> not understand.
>> >
>> > In the org.apache.hadoop.mapred.Child class, there were two UGIs
>> defined as follows.
>> >
>> > UserGroupInformation current =
>> UserGroupInformation.getCurrentUser();
>> > current.addToken(jt);
>> >
>> > UserGroupInformation taskOwner
>> >  =
>> UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
>> > taskOwner.addToken(jt);
>> >
>> > But it is the taskOwner that is actually passed as a UGI to task
>> tracker and then to HDFS. The first one was not referenced any where.
>> >
>> > final TaskUmbilicalProtocol umbilical =
>> >   taskOwner.doAs(new
>> PrivilegedExceptionAction() {
>> > @Override
>> > public TaskUmbilicalProtocol run() throws Exception {
>> >   return
>> (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
>> >   TaskUmbilicalProtocol.versionID,
>> >   address,
>> >   defaultConf);
>> > }
>> > });
>> >
>> > What puzzled me is that the job id is actually passed in as the user
>> name to task tracker. On the Name node side, when it tries to map the
>> non-existing user name, i.e., task id, to a group, it always returns empty
>> array. As a result, we always see annoying warning messages such as
>> >
>> >  WARN org.apache.hadoop.security.UserGroupInformation (IPC Server
>> handler 63 on 9000): No groups available for user job_201401071758_0002
>> >
>> > Sometimes, the warning messages were thrown so fast, hundreds or even
>> thousands per second for a big cluster, the system performance was degraded
>> dramatically.
>> >
>> > Could someone please explain why this logic was designed in this way?
>> Any benefit to use non-existing user for the group mapping? Or is this a
>> bug?
>> >
>> > Thanks in advance,
>> >
>> > John
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>

Re: Why none AppMaster node seeks IPCServer on itself.

[Saeed Adel Mehraban]

There is no firewall.


On Wed, Jan 8, 2014 at 9:58 PM, Vinod Kumar Vavilapalli <
vino...@hortonworks.com> wrote:

> Checked the firewall rules?
>
> +Vinod
>
> On Jan 8, 2014, at 3:22 AM, Saeed Adel Mehraban 
> wrote:
>
> Hi all.
> I have an installation on Hadoop on 3 nodes, namely master, slave1 and
> slave2. When I try to run a job, assuming appmaster be on slave1, every map
> and reduce tasks which take place on slave2 will fail due to
> ConnectException.
> I checked the port which slave2 wants to connect to. It differs randomly
> each time, but when I look for it in slave1 logs, I can see this line:
> "2014-01-08 02:14:25,206 INFO [Socket Reader #1 for port 38226]
> org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 38226"
> So there is a process on slave1 listening to this port, but slave2 tasks
> want to connect to this port on slave2.
>
> Do you know why is this happening?
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

/home/r9r/hadoop-2.2.0/bin/hadoop: line 133: /usr/java/default/bin/java: No such file or directory

[Allen, Ronald L.]

Hello again,

I'm trying to install Hadoop 2.2.0 on Redhat 2.6.32-358.23.2.el6.x86_64.  I 
have untar-ed hadoop-2.2.0.tar.gz and set my path variables as below.

export HADOOP_INSTALL=/home/r9r
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path"=$HADOOP_INSTALL/lib

export JAVA_HOME=/usr/lib/jvm/java-1.7.0/jre/

When I type hadoop version, I get the following:

/home/r9r/hadoop-2.2.0/bin/hadoop: line 133: /usr/java/default/bin/java: No 
such file or directory
/home/r9r/hadoop-2.2.0/bin/hadoop: line 133: exec: /usr/java/default/bin/java: 
cannot execute: No such file or directory

I've checked and rechecked my JAVA_HOME.  I feel like it is correct.  I've 
checked hadoop-env.sh and it is set to export JAVA_HOME=${JAVA_HOME}

I am stuck and do not know what to try from here.  Does anyone have any ideas?

Thanks!
Ronnie

Re: A question about Hadoop 1 job user id used for group mapping, which could lead to performance degradatioin

[Jian Fang]

Thanks Vinod for your quick response. It is running in non-secure mode.

I still don't get what is the purpose to use job id in UGI. Could you
please explain a bit more?

Thanks,

John


On Wed, Jan 8, 2014 at 10:11 AM, Vinod Kumar Vavilapalli <
vino...@hortonworks.com> wrote:

> It just seems like lazy code. You can see that, later, there is this:
>
> {code}
>
> for(Token token :
> UserGroupInformation.getCurrentUser().getTokens()) {
>   childUGI.addToken(token);
> }
>
> {code}
>
> So eventually the JobToken is getting added to the UGI which runs
> task-code.
>
> >  WARN org.apache.hadoop.security.UserGroupInformation (IPC Server
> handler 63 on 9000): No groups available for user job_201401071758_0002
>
> This seems to be a problem. When the task tries to reach the NameNode, it
> should do so as the user, not the job-id. It is not just logging, I'd be
> surprised if jobs pass. Do you have permissions enabled on HDFS?
>
> Oh, or is this in non-secure mode (i.e. without kerberos)?
>
> +Vinod
>
>
> On Jan 7, 2014, at 5:14 PM, Jian Fang 
> wrote:
>
> > Hi,
> >
> > I looked at Hadoop 1.X source code and found some logic that I could not
> understand.
> >
> > In the org.apache.hadoop.mapred.Child class, there were two UGIs defined
> as follows.
> >
> > UserGroupInformation current = UserGroupInformation.getCurrentUser();
> > current.addToken(jt);
> >
> > UserGroupInformation taskOwner
> >  =
> UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> > taskOwner.addToken(jt);
> >
> > But it is the taskOwner that is actually passed as a UGI to task tracker
> and then to HDFS. The first one was not referenced any where.
> >
> > final TaskUmbilicalProtocol umbilical =
> >   taskOwner.doAs(new
> PrivilegedExceptionAction() {
> > @Override
> > public TaskUmbilicalProtocol run() throws Exception {
> >   return
> (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
> >   TaskUmbilicalProtocol.versionID,
> >   address,
> >   defaultConf);
> > }
> > });
> >
> > What puzzled me is that the job id is actually passed in as the user
> name to task tracker. On the Name node side, when it tries to map the
> non-existing user name, i.e., task id, to a group, it always returns empty
> array. As a result, we always see annoying warning messages such as
> >
> >  WARN org.apache.hadoop.security.UserGroupInformation (IPC Server
> handler 63 on 9000): No groups available for user job_201401071758_0002
> >
> > Sometimes, the warning messages were thrown so fast, hundreds or even
> thousands per second for a big cluster, the system performance was degraded
> dramatically.
> >
> > Could someone please explain why this logic was designed in this way?
> Any benefit to use non-existing user for the group mapping? Or is this a
> bug?
> >
> > Thanks in advance,
> >
> > John
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Ways to manage user accounts on hadoop cluster when using kerberos security

[Jay Vyas]

I recently found a pretty simple and easy way to set ldap up for my machines on 
rhel and wrote it up using jumpbox and authconfig.

If you are in the cloud and only need a quick easy ldap idh and nssswitch 
setup, this is I think the easiest / cheapest way to do it.

I know rhel and fedora come with authconfig not sure about the other Linux 
distros:

http://jayunit100.blogspot.com/2013/12/an-easy-way-to-centralize.html?m=1





> On Jan 8, 2014, at 1:22 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> 
>> On Jan 7, 2014, at 2:55 PM, Manoj Samel  wrote:
>> 
>> I am assuming that if the users are in a LDAP, can using the PAM for LDAP 
>> solve the issue.
> 
> 
> That's how I've seen this issue addressed. 
> 
> +Vinod
> 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader of 
> this message is not the intended recipient, you are hereby notified that any 
> printing, copying, dissemination, distribution, disclosure or forwarding of 
> this communication is strictly prohibited. If you have received this 
> communication in error, please contact the sender immediately and delete it 
> from your system. Thank You.

Re: Why none AppMaster node seeks IPCServer on itself.

[Vinod Kumar Vavilapalli]

Checked the firewall rules?

+Vinod

On Jan 8, 2014, at 3:22 AM, Saeed Adel Mehraban  wrote:

> Hi all.
> I have an installation on Hadoop on 3 nodes, namely master, slave1 and 
> slave2. When I try to run a job, assuming appmaster be on slave1, every map 
> and reduce tasks which take place on slave2 will fail due to ConnectException.
> I checked the port which slave2 wants to connect to. It differs randomly each 
> time, but when I look for it in slave1 logs, I can see this line:
> "2014-01-08 02:14:25,206 INFO [Socket Reader #1 for port 38226] 
> org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 38226"
> So there is a process on slave1 listening to this port, but slave2 tasks want 
> to connect to this port on slave2.
> 
> Do you know why is this happening?


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Ways to manage user accounts on hadoop cluster when using kerberos security

[Vinod Kumar Vavilapalli]

On Jan 7, 2014, at 2:55 PM, Manoj Samel  wrote:

> I am assuming that if the users are in a LDAP, can using the PAM for LDAP 
> solve the issue.


That's how I've seen this issue addressed. 

+Vinod
-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: A question about Hadoop 1 job user id used for group mapping, which could lead to performance degradatioin

[Vinod Kumar Vavilapalli]

It just seems like lazy code. You can see that, later, there is this:

{code}

for(Token token : UserGroupInformation.getCurrentUser().getTokens()) 
{
  childUGI.addToken(token);
}

{code}

So eventually the JobToken is getting added to the UGI which runs task-code.

>  WARN org.apache.hadoop.security.UserGroupInformation (IPC Server handler 63 
> on 9000): No groups available for user job_201401071758_0002

This seems to be a problem. When the task tries to reach the NameNode, it 
should do so as the user, not the job-id. It is not just logging, I'd be 
surprised if jobs pass. Do you have permissions enabled on HDFS?

Oh, or is this in non-secure mode (i.e. without kerberos)?

+Vinod


On Jan 7, 2014, at 5:14 PM, Jian Fang  wrote:

> Hi,
> 
> I looked at Hadoop 1.X source code and found some logic that I could not 
> understand. 
> 
> In the org.apache.hadoop.mapred.Child class, there were two UGIs defined as 
> follows.
> 
> UserGroupInformation current = UserGroupInformation.getCurrentUser();
> current.addToken(jt);
> 
> UserGroupInformation taskOwner 
>  = 
> UserGroupInformation.createRemoteUser(firstTaskid.getJobID().toString());
> taskOwner.addToken(jt);
> 
> But it is the taskOwner that is actually passed as a UGI to task tracker and 
> then to HDFS. The first one was not referenced any where.
> 
> final TaskUmbilicalProtocol umbilical = 
>   taskOwner.doAs(new PrivilegedExceptionAction() {
> @Override
> public TaskUmbilicalProtocol run() throws Exception {
>   return 
> (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
>   TaskUmbilicalProtocol.versionID,
>   address,
>   defaultConf);
> }
> });
> 
> What puzzled me is that the job id is actually passed in as the user name to 
> task tracker. On the Name node side, when it tries to map the non-existing 
> user name, i.e., task id, to a group, it always returns empty array. As a 
> result, we always see annoying warning messages such as
> 
>  WARN org.apache.hadoop.security.UserGroupInformation (IPC Server handler 63 
> on 9000): No groups available for user job_201401071758_0002
> 
> Sometimes, the warning messages were thrown so fast, hundreds or even 
> thousands per second for a big cluster, the system performance was degraded 
> dramatically. 
> 
> Could someone please explain why this logic was designed in this way? Any 
> benefit to use non-existing user for the group mapping? Or is this a bug?
> 
> Thanks in advance,
> 
> John


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: what all can be done using MR

[Chris Mawata]

Yes.
Check out, for example, 
http://packtlib.packtpub.com/library/hadoop-mapreduce-cookbook/ch06lvl1sec66#



On 1/8/2014 2:41 AM, unmesha sreeveni wrote:

Can we do aggregation with in Hadoop MR
like find min,max,sum,avg of a column in a csv file.

--
/Thanks & Regards/
/
/
Unmesha Sreeveni U.B/
/
Junior Developer

http://www.unmeshasreeveni.blogspot.in/

/
/

Re: Distributing the code to multiple nodes

[Chris Mawata]

2 nodes and replication factor of 2 results in a replica of each block
present on each node. This would allow the possibility that a single node
would do the work and yet be data local.  It will probably happen if that
single node has the needed capacity.  More nodes than the replication
factor are needed to force distribution of the processing.
Chris
On Jan 8, 2014 7:35 AM, "Ashish Jain"  wrote:

> Guys,
>
> I am sure that only one node is being used. I just know ran the job again
> and could see that CPU usage only for one server going high other server
> CPU usage remains constant and hence it means other node is not being used.
> Can someone help me to debug this issue?
>
> ++Ashish
>
>
> On Wed, Jan 8, 2014 at 5:04 PM, Ashish Jain  wrote:
>
>> Hello All,
>>
>> I have a 2 node hadoop cluster running with a replication factor of 2. I
>> have a file of size around 1 GB which when copied to HDFS is replicated to
>> both the nodes. Seeing the block info I can see the file has been
>> subdivided into 8 parts which means it has been subdivided into 8 blocks
>> each of size 128 MB.  I use this file as input to run the word count
>> program. Some how I feel only one node is doing all the work and the code
>> is not distributed to other node. How can I make sure code is distributed
>> to both the nodes? Also is there a log or GUI which can be used for this?
>> Please note I am using the latest stable release that is 2.2.0.
>>
>> ++Ashish
>>
>
>

Re: Distributing the code to multiple nodes

[Ashish Jain]

Guys,

I am sure that only one node is being used. I just know ran the job again
and could see that CPU usage only for one server going high other server
CPU usage remains constant and hence it means other node is not being used.
Can someone help me to debug this issue?

++Ashish


On Wed, Jan 8, 2014 at 5:04 PM, Ashish Jain  wrote:

> Hello All,
>
> I have a 2 node hadoop cluster running with a replication factor of 2. I
> have a file of size around 1 GB which when copied to HDFS is replicated to
> both the nodes. Seeing the block info I can see the file has been
> subdivided into 8 parts which means it has been subdivided into 8 blocks
> each of size 128 MB.  I use this file as input to run the word count
> program. Some how I feel only one node is doing all the work and the code
> is not distributed to other node. How can I make sure code is distributed
> to both the nodes? Also is there a log or GUI which can be used for this?
> Please note I am using the latest stable release that is 2.2.0.
>
> ++Ashish
>

Distributing the code to multiple nodes

[Ashish Jain]

Hello All,

I have a 2 node hadoop cluster running with a replication factor of 2. I
have a file of size around 1 GB which when copied to HDFS is replicated to
both the nodes. Seeing the block info I can see the file has been
subdivided into 8 parts which means it has been subdivided into 8 blocks
each of size 128 MB.  I use this file as input to run the word count
program. Some how I feel only one node is doing all the work and the code
is not distributed to other node. How can I make sure code is distributed
to both the nodes? Also is there a log or GUI which can be used for this?
Please note I am using the latest stable release that is 2.2.0.

++Ashish

Why none AppMaster node seeks IPCServer on itself.

[Saeed Adel Mehraban]

Hi all.
I have an installation on Hadoop on 3 nodes, namely master, slave1 and
slave2. When I try to run a job, assuming appmaster be on slave1, every map
and reduce tasks which take place on slave2 will fail due to
ConnectException.
I checked the port which slave2 wants to connect to. It differs randomly
each time, but when I look for it in slave1 logs, I can see this line:
"2014-01-08 02:14:25,206 INFO [Socket Reader #1 for port 38226]
org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 38226"
So there is a process on slave1 listening to this port, but slave2 tasks
want to connect to this port on slave2.

Do you know why is this happening?

Re: Running Hadoop v2 clustered mode MR on an NFS mounted filesystem

[Atish Kathpal]

Figured out 1. The output of the reduce was going to the slave node, while
I was looking for it in the master node. Which is perfectly fine.
Need guidance for 2. though!

Thanks
Atish


On Wed, Jan 8, 2014 at 3:30 PM, Atish Kathpal wrote:

> Hi
>
> By giving the complete URI, the MR jobs worked across both nodes. Thanks a
> lot for the advice.
>
> *Two issues though*:
> 1. On completion of the MR job, I see only the "_SUCCESS" file in the
> output directory, but no part-r file containing the actual results of the
> wordcount job. However I am seeing the correct output on running MR over
> HDFS. What is going wrong? Any place I can find logs for the MR job. I see
> no errors on the console.
> Command used:
> hadoop jar
> /home/hduser/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
> wordcount file:///home/hduser/testmount/ file:///home/hduser/testresults/
>
>
> 2. I am observing that the mappers seem to be accessing files
> sequentially, splitting the files across mappers, and then reading data in
> parallelel, then moving on to the next file. What I want instead is that,
> files themselves should be accessed in parallel, that is, if there are 10
> files to be MRed, then MR should ask for each of these files in parallel in
> one go, and then work on the splits of these files in parallel.
> *Why do I need this?* Some of the data coming from the NFS mount point is
> coming from offline media (which takes ~5-10 seconds of time before first
> bytes are received). So I would like all required files to be asked at the
> onset itself from the NFS mount point. This way several offline media will
> be spun up parallely and as the data from these media gets available MR can
> process them.
>
> Would be glad to get inputs on these points!
>
> Thanks
> Atish
>
> Tip for those who are trying similar stuff::
> In my case. after a while the jobs would fail, complaining of 
> "java.lang.OutOfMemoryError:
> Java heap 
> space",
> but I was able to rectify this with help from:
> http://stackoverflow.com/questions/13674190/cdh-4-1-error-running-child-java-lang-outofmemoryerror-java-heap-space
>
>
>
>
>
> On Sun, Dec 22, 2013 at 2:47 PM, Atish Kathpal wrote:
>
>> Thanks Devin, Yong, and Chris for your replies and suggestions. I will
>> test the suggestions made by Yong and Devin and get back to you guys.
>>
>> As on the bottlenecking issue, I agree, but  I am trying to run few MR
>> jobs on a traditional NAS server. I can live with a few bottlenecks, so
>> long as I don't have to move the data to a dedicated HDFS cluster.
>>
>>
>> On Sat, Dec 21, 2013 at 8:06 AM, Chris Mawata wrote:
>>
>>>  Yong raises an important issue:  You have thrown out the I/O
>>> advantages of HDFS and also thrown out the advantages of data locality. It
>>> would be interesting to know why you are taking this approach.
>>> Chris
>>>
>>>
>>> On 12/20/2013 9:28 AM, java8964 wrote:
>>>
>>> I believe the "-fs local" should be removed too. The reason is that even
>>> you have a dedicated JobTracker after removing "-jt local", but with "-fs
>>> local", I believe that all the mappers will be run sequentially.
>>>
>>>  "-fs local" will force the mapreducer run in "local" mode, which is
>>> really a test mode.
>>>
>>>  What you can do is to remove both "-fs local -jt local", but give the
>>> FULL URI of the input and output path, to tell Hadoop that they are local
>>> filesystem instead of HDFS.
>>>
>>>  "hadoop jar
>>> /hduser/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
>>> wordcount file:///hduser/mount_point file:///results"
>>>
>>>  Keep in mind followings:
>>>
>>>  1) The NFS mount need to be available in all your Task Nodes, and
>>> mounted in the same way.
>>> 2) Even you can do that, but your sharing storage will be your
>>> bottleneck. NFS won't work well for scalability.
>>>
>>>  Yong
>>>
>>>  --
>>> Date: Fri, 20 Dec 2013 09:01:32 -0500
>>> Subject: Re: Running Hadoop v2 clustered mode MR on an NFS mounted
>>> filesystem
>>> From: dsui...@rdx.com
>>> To: user@hadoop.apache.org
>>>
>>> I think most of your problem is coming from the options you are setting:
>>>
>>>  "hadoop jar
>>> /hduser/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
>>> wordcount *-fs local -jt local* /hduser/mount_point/  /results"
>>>
>>>  You appear to be directing your namenode to run jobs in the *LOCAL* job
>>> runner and directing it to read from the *LOCAL* filesystem. Drop the
>>> *-jt* argument and it should run in distributed mode if your cluster is
>>> set up right. You don't need to do anything special to point Hadoop towards
>>> a NFS location, other than set up the NFS location properly and make sure
>>> if you are directing to it by name that it will resolve to the right
>>> address. Hadoop doesn't care where it is, as long as it can read from and
>>> write to it. The

Re: Running Hadoop v2 clustered mode MR on an NFS mounted filesystem

[Atish Kathpal]

Hi

By giving the complete URI, the MR jobs worked across both nodes. Thanks a
lot for the advice.

*Two issues though*:
1. On completion of the MR job, I see only the "_SUCCESS" file in the
output directory, but no part-r file containing the actual results of the
wordcount job. However I am seeing the correct output on running MR over
HDFS. What is going wrong? Any place I can find logs for the MR job. I see
no errors on the console.
Command used:
hadoop jar
/home/hduser/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
wordcount file:///home/hduser/testmount/ file:///home/hduser/testresults/


2. I am observing that the mappers seem to be accessing files sequentially,
splitting the files across mappers, and then reading data in parallelel,
then moving on to the next file. What I want instead is that, files
themselves should be accessed in parallel, that is, if there are 10 files
to be MRed, then MR should ask for each of these files in parallel in one
go, and then work on the splits of these files in parallel.
*Why do I need this?* Some of the data coming from the NFS mount point is
coming from offline media (which takes ~5-10 seconds of time before first
bytes are received). So I would like all required files to be asked at the
onset itself from the NFS mount point. This way several offline media will
be spun up parallely and as the data from these media gets available MR can
process them.

Would be glad to get inputs on these points!

Thanks
Atish

Tip for those who are trying similar stuff::
In my case. after a while the jobs would fail, complaining of
"java.lang.OutOfMemoryError:
Java heap 
space",
but I was able to rectify this with help from:
http://stackoverflow.com/questions/13674190/cdh-4-1-error-running-child-java-lang-outofmemoryerror-java-heap-space





On Sun, Dec 22, 2013 at 2:47 PM, Atish Kathpal wrote:

> Thanks Devin, Yong, and Chris for your replies and suggestions. I will
> test the suggestions made by Yong and Devin and get back to you guys.
>
> As on the bottlenecking issue, I agree, but  I am trying to run few MR
> jobs on a traditional NAS server. I can live with a few bottlenecks, so
> long as I don't have to move the data to a dedicated HDFS cluster.
>
>
> On Sat, Dec 21, 2013 at 8:06 AM, Chris Mawata wrote:
>
>>  Yong raises an important issue:  You have thrown out the I/O advantages
>> of HDFS and also thrown out the advantages of data locality. It would be
>> interesting to know why you are taking this approach.
>> Chris
>>
>>
>> On 12/20/2013 9:28 AM, java8964 wrote:
>>
>> I believe the "-fs local" should be removed too. The reason is that even
>> you have a dedicated JobTracker after removing "-jt local", but with "-fs
>> local", I believe that all the mappers will be run sequentially.
>>
>>  "-fs local" will force the mapreducer run in "local" mode, which is
>> really a test mode.
>>
>>  What you can do is to remove both "-fs local -jt local", but give the
>> FULL URI of the input and output path, to tell Hadoop that they are local
>> filesystem instead of HDFS.
>>
>>  "hadoop jar
>> /hduser/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
>> wordcount file:///hduser/mount_point file:///results"
>>
>>  Keep in mind followings:
>>
>>  1) The NFS mount need to be available in all your Task Nodes, and
>> mounted in the same way.
>> 2) Even you can do that, but your sharing storage will be your
>> bottleneck. NFS won't work well for scalability.
>>
>>  Yong
>>
>>  --
>> Date: Fri, 20 Dec 2013 09:01:32 -0500
>> Subject: Re: Running Hadoop v2 clustered mode MR on an NFS mounted
>> filesystem
>> From: dsui...@rdx.com
>> To: user@hadoop.apache.org
>>
>> I think most of your problem is coming from the options you are setting:
>>
>>  "hadoop jar
>> /hduser/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
>> wordcount *-fs local -jt local* /hduser/mount_point/  /results"
>>
>>  You appear to be directing your namenode to run jobs in the *LOCAL* job
>> runner and directing it to read from the *LOCAL* filesystem. Drop the
>> *-jt* argument and it should run in distributed mode if your cluster is
>> set up right. You don't need to do anything special to point Hadoop towards
>> a NFS location, other than set up the NFS location properly and make sure
>> if you are directing to it by name that it will resolve to the right
>> address. Hadoop doesn't care where it is, as long as it can read from and
>> write to it. The fact that you are telling it to read/write from/to a NFS
>> location that happens to be mounted as a local filesystem object doesn't
>> matter - you could direct it to the local /hduser/ path and set the -fs
>> local option, and it would end up on the NFS mount, because that's where
>> the NFS mount actually exists, or you could direct it to the absolute
>> network location of the fold

Re: issue about how to assiging map output to reducer?

[Chris Mawata]

Depends on the distribution of the keys and how the partitioner is
assigning keys to reducers.  (Remember that pairs with the same key have to
go to the same reducer).
Chris
On Jan 8, 2014 2:33 AM, "ch huang"  wrote:

> hi,maillist:
> i look the containers log from " hadoop fs -cat
> /var/log/hadoop-yarn/apps/root/logs/application_1388730279827_2770/CHBM221_50853"
>
> and log say it get 25 map output , and assiging 7 to fetcher 5, assiging 7
> to fetcher 4 and assiging 11 to fetcher 3,my question is why not
> " assiging 8 to fetcher 5, assiging 8 to fetcher 4 and assiging 9 to
> fetcher 3 " ?
>
> 2014-01-08 11:28:00,346 INFO [EventFetcher for fetching Map Completion
> Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher:
> attempt_1388730279827_2770_r_00_0: Got 25 new map-outputs
> 2014-01-08 11:28:00,348 INFO [fetcher#5]
> org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
> CHBM223:8080 with 7 to fetcher#5
> 2014-01-08 11:28:00,349 INFO [fetcher#5]
> org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7
> to CHBM223:8080 to fetcher#5
> 2014-01-08 11:28:00,349 INFO [fetcher#4]
> org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
> CHBM222:8080 with 7 to fetcher#4
> 2014-01-08 11:28:00,349 INFO [fetcher#4]
> org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7
> to CHBM222:8080 to fetcher#4
> 2014-01-08 11:28:00,352 INFO [fetcher#3]
> org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
> CHBM221:8080 with 11 to fetcher#3
> 2014-01-08 11:28:00,352 INFO [fetcher#3]
> org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 11 of 11
> to CHBM221:8080 to fetcher#3
>

XML to TEXT

[Ranjini Rathinam]

Hi,

As suggest i tried with the code , but in the result.txt i got output only
header. Nothing else was printing.

After debugging i came to know that while parsing , there is no value.

The problem is in line given below which is bold. While putting SysOut i
found no value printing in this line.

String xmlContent = value.toString();

InputStream is = new ByteArrayInputStream(xmlContent.getBytes());
DocumentBuilderFactory factory =
DocumentBuilderFactory.newInstance();
DocumentBuilder builder;
try {
builder = factory.newDocumentBuilder();
 *   Document doc = builder.parse(is);*


*String ed=doc.getDocumentElement().getNodeName();*
out.write(ed.getBytes());
DTMNodeList list = (DTMNodeList) getNode("/Company/Employee",
doc,XPathConstants.NODESET);


When iam printing

out.write(xmlContent.getBytes):- the whole xml is being printed.

then i wrote for Sysout for list ,nothing printed.
out.write(ed.getBytes):- nothing is being printed.

Please suggest where i am going wrong. Please help to fix this.

Thanks in advance.

I have attached my code.Please review.


Mapper class:-

public class XmlTextMapper extends Mapper {
private static final XPathFactory xpathFactory =
XPathFactory.newInstance();
@Override
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String resultFileName = "/user/task/Sales/result.txt";

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create(resultFileName), conf);
FSDataOutputStream out = fs.create(new Path(resultFileName));
InputStream resultIS = new ByteArrayInputStream(new byte[0]);
String header = "id,name\n";
out.write(header.getBytes());
String xmlContent = value.toString();

InputStream is = new ByteArrayInputStream(xmlContent.getBytes());
DocumentBuilderFactory factory =
DocumentBuilderFactory.newInstance();
DocumentBuilder builder;
try {
builder = factory.newDocumentBuilder();
Document doc = builder.parse(is);

   String ed=doc.getDocumentElement().getNodeName();
   out.write(ed.getBytes());
DTMNodeList list = (DTMNodeList) getNode("/Company/Employee",
doc,XPathConstants.NODESET);
int size = list.getLength();
for (int i = 0; i < size; i++) {
Node node = list.item(i);
String line = "";
NodeList nodeList = node.getChildNodes();
int childNumber = nodeList.getLength();
for (int j = 0; j < childNumber; j++)
{
line += nodeList.item(j).getTextContent() + ",";
}
if (line.endsWith(","))
line = line.substring(0, line.length() - 1);
line += "\n";
out.write(line.getBytes());
}
} catch (ParserConfigurationException e) {
 e.printStackTrace();
} catch (SAXException e) {
 e.printStackTrace();
} catch (XPathExpressionException e) {
 e.printStackTrace();
}
IOUtils.copyBytes(resultIS, out, 4096, true);
out.close();
}
public static Object getNode(String xpathStr, Node node, QName
retunType)
throws XPathExpressionException {
XPath xpath = xpathFactory.newXPath();
return xpath.evaluate(xpathStr, node, retunType);
}
}



Main class
public class MainXml {
public static void main(String[] args) throws Exception {

Configuration conf = new Configuration();

if (args.length != 2) {
System.err
.println("Usage: XMLtoText  ");
System.exit(-1);
}

  String output="/user/task/Sales/";
   Job job = new Job(conf, "XML to Text");
job.setJarByClass(MainXml.class);
   // job.setJobName("XML to Text");

FileInputFormat.addInputPath(job, new Path(args[0]));

   // FileOutputFormat.setOutputPath(job, new Path(args[1]));
  Path outPath = new Path(output);
  FileOutputFormat.setOutputPath(job, outPath);
  FileSystem dfs = FileSystem.get(outPath.toUri(), conf);
  if (dfs.exists(outPath)) {
  dfs.delete(outPath, true);
  }
job.setMapperClass(XmlTextMapper.class);

job.setNumReduceTasks(0);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}



My xml file



100
ranjini
IT1
123456
nextlevel1

Chennai1
Navallur1



1001
ranjinikumar
IT
1234516
nextlevel

Chennai
Navallur





Thanks in advance
Ranjini. R

passing Xml using hive

[Ranjini Rathinam]

Hi,

This is my xml file



100
ranjini
IT1
123456
nextlevel1

Chennai1
Navallur1



1001
ranjinikumar
IT
1234516
nextlevel

Chennai
Navallur





In hive shell , i use xpath

 hive> select xpath(str,'/Company/Employee/ename/text()') from hivexml;
Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is
set to 0 since there's no reduce operator Starting Job =
job_201401081034_0019, Tracking URL =
http://localhost:50030/jobdetails.jsp?jobid=job_201401081034_0019 Kill
Command = /usr/local/hadoop/bin/../bin/hadoop job
-Dmapred.job.tracker=localhost:4441 -kill job_201401081034_0019 2014-01-08
14:18:18,199 Stage-1 map = 0%, reduce = 0%
2014-01-08 14:18:45,294 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201401081034_0019 with errors
 FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask hive>

When check in the URL of tasktracker and jobtracker i did not find any
error log there.


Please help to fix this , What is correct step to solve.

Thanks in advance.

Ranjini R

Re: Authentication issue on Hadoop 2.2.0

[Silvina Caíno Lores]

Found out that building my code with Maven (along with the Pipes examples)
worked. Any clues why?

Thanks,
Silvina


On 19 December 2013 13:16, Silvina Caíno Lores wrote:

> And that it is caused by this exception, as I've found out
>
> 2013-12-19 13:14:28,237 ERROR [pipe-uplink-handler]
> org.apache.hadoop.mapred.pipes.BinaryProtocol: java.io.EOFException
> at java.io.DataInputStream.readByte(DataInputStream.java:267)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:308)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:329)
> at
> org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:125)
>
>
> On 18 December 2013 10:52, Silvina Caíno Lores wrote:
>
>> I forgot to mention that the wordcount pipes example runs successfully.
>>
>>
>> On 18 December 2013 10:50, Silvina Caíno Lores wrote:
>>
>>> Hi everyone,
>>>
>>> I'm working with a single node cluster and a Hadoop Pipes job that
>>> throws the following exception on execution:
>>>
>>> 13/12/18 10:44:55 INFO mapreduce.Job: Running job: job_1387359324416_0002
>>> 13/12/18 10:45:03 INFO mapreduce.Job: Job job_1387359324416_0002 running
>>> in uber mode : false
>>> 13/12/18 10:45:03 INFO mapreduce.Job: map 0% reduce 0%
>>> 13/12/18 10:45:08 INFO mapreduce.Job: Task Id :
>>> attempt_1387359324416_0002_m_00_0, Status : FAILED
>>> Error: java.io.IOException
>>> at
>>> org.apache.hadoop.mapred.pipes.OutputHandler.waitForAuthentication(OutputHandler.java:186)
>>> at
>>> org.apache.hadoop.mapred.pipes.Application.waitForAuthentication(Application.java:195)
>>> at
>>> org.apache.hadoop.mapred.pipes.Application.(Application.java:150)
>>> at
>>> org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:69)
>>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
>>> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:171)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at javax.security.auth.Subject.doAs(Subject.java:415)
>>> at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1515)
>>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:166)
>>>
>>> /// same exception several times ///
>>>
>>> 13/12/18 10:45:25 INFO mapreduce.Job: map 93% reduce 0%
>>> 13/12/18 10:45:26 INFO mapreduce.Job: map 100% reduce 100%
>>> 13/12/18 10:45:26 INFO mapreduce.Job: Job job_1387359324416_0002 failed
>>> with state FAILED due to: Task failed task_1387359324416_0002_m_00
>>> Job failed as tasks failed. failedMaps:1 failedReduces:0
>>>
>>>
>>> I don't really get why the job is at 100% if it actually failed by the
>>> way.
>>>
>>> Any ideas? Thanks in advance!
>>>
>>> Best,
>>> Silvina
>>>
>>>
>>>
>>>
>>>
>>>
>>
>

Re: Ways to manage user accounts on hadoop cluster when using kerberos security

[bc Wong]

LDAP/AD is pretty much it. You can also have Kerberos authenticate directly
to AD, or set up one-way trust between AD and MIT Kerberos. There are other
identity management systems that basically implement the same. At the end
of the day, you need to have (1) users in KDC (2) users on the nodes, and
(3) user-group mapping. And it makes sense for all three to come from the
same system.

Cheers,
bc


On Tue, Jan 7, 2014 at 2:55 PM, Manoj Samel  wrote:

> Hi,
>
> From the documentation + code,  "when kerberos is enabled, all tasks are
> run as the end user (e..g as user "joe" and not as hadoop user "mapred")
> using the task-controller (which is setuid root and when it runs, it does a
> setuid/setgid etc. to Joe and his groups ). For this to work, user "joe"
> linux account has to be present on all nodes of the cluster."
>
> In a environment with large and dynamic user population; it is not
> practical to add every end user to every node of the cluster (and drop user
> when end user is deactivated etc.)
>
> What are other options get this working ?
>
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP
> solve the issue.
>
> Any other suggestions?
>
> --
> Thanks,
>
> Manoj
>

Re: Cannot see MR logs in Hadoop 2.x

[Kyle Lin]

As this link(
http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/)
said, Hortonworks' HDP change the user-logs rule.



2014/1/8 Kyle Lin 

>
> Sorry for my rough asking. Refine my question.
>
> When using hadoop 1.x, I can see the MR job logs under "userlogs"
> folder in local file system. But I cannot find the MR log in hadoop 2.x
> environment. There is no "userlogs" folder.
>
> Can anybody tell me where to find the log files?
>
> Kyle
>
>
> 2014/1/8 Kyle Lin 
>
>> Hey guys
>>
>> My Environment: HDP 2 (Hadoop 2.2.0).
>>
>> When using Hadoop 1.x, I write log imformation in my MapReduce code,
>> then see the log message in TaskTracker log. But In YARN, I cannot find the
>> log message in any log files(historyserver, nodemanager, resourcemanager),
>> even using System.out.println.
>>
>> I also try to see app log through the address of
>> yarn.resourcemanager.webapp.address. Still cannot see my log messages.
>>
>>
>> Kyle
>>
>
>