this line: 2013-10-11 10:24:53,033 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred (auth:SIMPLE) cause:java.io.IOException: java.lang.NullPointerException
is imho indicating that i am using the user "mapred" for executing (fyi: submitting the job from the CLI ( hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar wordcount hdfs_input_path hdfs_output_path) from another node) ... the file permissions for this file are: -rwxr-x--x 1 mapred hadoop 1382 Oct 10 15:02 topology.py* i temporarly had set the permissions to 777 to see if something changes, but it didn't ... I checked only the jobtracker, are the other nodes important for this as well? thx already in advance, especially for the quick response! Wolli 2013/10/11 DSuiter RDX <dsui...@rdx.com> > The user running the job (might not be your username depending on your > setup) does not appear to have executable permissions on the jobtracker > cluster topology python script - I'm basing this on the lines: > > 2013-10-11 10:24:53,035 WARN org.apache.hadoop.net.ScriptBasedMapping: > Exception running > /run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py > 10.160.25.249 > java.io.IOException: Cannot run program > "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py" (in > directory "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER"): > java.io.IOException: error=13, Permission denied > > So checking on the permissions for that file, determining what user is > kicking off your job, which depends on how you submit it, and making sure > that user has the execute permission on that file will probably fix this. > > If you are using a management console, such as Cloudera SCM, when you > submit jobs, they are run as an application user, so, Flume services run > under the "Flume" user, HBase jobs will typically run under the HBase user, > and so on. It can cause some surprises if you do not expect it. > > *Devin Suiter* > Jr. Data Solutions Software Engineer > 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212 > Google Voice: 412-256-8556 | www.rdx.com > > > On Fri, Oct 11, 2013 at 7:59 AM, fab wol <darkwoll...@gmail.com> wrote: > >> Hey everyone, I've got supplied with a decent ten node CDH 4.4 cluster, >> only 7 days old, and someone tried some HBase stuff on it. Now I wanted to >> try some MR Stuff on it, but starting a Job is already not possible (even >> the wordcount example). The error log of the jobtracker produces a log 700k >> lines long but it consists mainly of these lines repeatedly: >> >> 2013-10-11 10:24:53,033 INFO org.apache.hadoop.mapred.JobTracker: Lost >> tracker 'tracker_z-asanode02:localhost/127.0.0.1:53712' >> 2013-10-11 10:24:53,033 ERROR >> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException >> as:mapred (auth:SIMPLE) cause:java.io.IOException: >> java.lang.NullPointerException >> 2013-10-11 10:24:53,034 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 22 on 8021, call >> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@13b31acd, true, >> true, true, -1), rpc version=2, client version=32, >> methodsFingerPrint=-159967141 from 10.160.25.250:44389: error: >> java.io.IOException: java.lang.NullPointerException >> java.io.IOException: java.lang.NullPointerException >> at >> org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2751) >> at >> org.apache.hadoop.mapred.JobTracker.addNewTracker(JobTracker.java:2731) >> at >> org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3227) >> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2931) >> at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474) >> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002) >> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1751) >> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1747) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1745) >> 2013-10-11 10:24:53,035 WARN org.apache.hadoop.net.ScriptBasedMapping: >> Exception running >> /run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py >> 10.160.25.249 >> java.io.IOException: Cannot run program >> "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py" (in >> directory "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER"): >> java.io.IOException: error=13, Permission denied >> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460) >> at org.apache.hadoop.util.Shell.runCommand(Shell.java:206) >> at org.apache.hadoop.util.Shell.run(Shell.java:188) >> at >> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) >> at >> org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResolveCommand(ScriptBasedMapping.java:242) >> at >> org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.resolve(ScriptBasedMapping.java:180) >> at >> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:119) >> at >> org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2750) >> at >> org.apache.hadoop.mapred.JobTracker.addNewTracker(JobTracker.java:2731) >> at >> org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3227) >> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2931) >> at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at >> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474) >> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002) >> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1751) >> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1747) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1745) >> Caused by: java.io.IOException: java.io.IOException: error=13, Permission >> denied >> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148) >> at java.lang.ProcessImpl.start(ProcessImpl.java:65) >> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453) >> ... 21 more >> >> it doesn't matter if it is a pure hadoop job or a oozie submitted job. >> there seems to be something wrong in the basic configuration. Anyone an >> idea? >> >> Cheers >> Wolli >> > >
