It looks like you are correct, and I did not have the right solution, I apologize. I'm not sure if the other nodes need to be involved either. Now I'm hoping someone with deeper knowledge will step in, because I'm curious also! Some of the most knowledgeable people on here are on US Pacific Time, so you will probably get more responses in a few hours. Sorry I couldn't be of more assistance.
Sincerely, *Devin Suiter* Jr. Data Solutions Software Engineer 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212 Google Voice: 412-256-8556 | www.rdx.com On Fri, Oct 11, 2013 at 8:52 AM, fab wol <darkwoll...@gmail.com> wrote: > this line: > > 2013-10-11 10:24:53,033 ERROR org.apache.hadoop.security.UserGroupInformation: > PriviledgedActionException as:mapred (auth:SIMPLE) > cause:java.io.IOException: java.lang.NullPointerException > > is imho indicating that i am using the user "mapred" for executing (fyi: > submitting the job from the CLI ( hadoop jar > /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/hadoop-examples.jar > wordcount hdfs_input_path hdfs_output_path) from another node) ... the file > permissions for this file are: > > -rwxr-x--x 1 mapred hadoop 1382 Oct 10 15:02 topology.py* > > i temporarly had set the permissions to 777 to see if something changes, > but it didn't ... I checked only the jobtracker, are the other nodes > important for this as well? > > thx already in advance, especially for the quick response! > Wolli > > > 2013/10/11 DSuiter RDX <dsui...@rdx.com> > >> The user running the job (might not be your username depending on your >> setup) does not appear to have executable permissions on the jobtracker >> cluster topology python script - I'm basing this on the lines: >> >> 2013-10-11 10:24:53,035 WARN org.apache.hadoop.net.ScriptBasedMapping: >> Exception running >> /run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py >> 10.160.25.249 >> java.io.IOException: Cannot run program >> "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py" (in >> directory "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER"): >> java.io.IOException: error=13, Permission denied >> >> So checking on the permissions for that file, determining what user is >> kicking off your job, which depends on how you submit it, and making sure >> that user has the execute permission on that file will probably fix this. >> >> If you are using a management console, such as Cloudera SCM, when you >> submit jobs, they are run as an application user, so, Flume services run >> under the "Flume" user, HBase jobs will typically run under the HBase user, >> and so on. It can cause some surprises if you do not expect it. >> >> *Devin Suiter* >> Jr. Data Solutions Software Engineer >> 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212 >> Google Voice: 412-256-8556 | www.rdx.com >> >> >> On Fri, Oct 11, 2013 at 7:59 AM, fab wol <darkwoll...@gmail.com> wrote: >> >>> Hey everyone, I've got supplied with a decent ten node CDH 4.4 cluster, >>> only 7 days old, and someone tried some HBase stuff on it. Now I wanted to >>> try some MR Stuff on it, but starting a Job is already not possible (even >>> the wordcount example). The error log of the jobtracker produces a log 700k >>> lines long but it consists mainly of these lines repeatedly: >>> >>> 2013-10-11 10:24:53,033 INFO org.apache.hadoop.mapred.JobTracker: Lost >>> tracker 'tracker_z-asanode02:localhost/127.0.0.1:53712' >>> 2013-10-11 10:24:53,033 ERROR >>> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException >>> as:mapred (auth:SIMPLE) cause:java.io.IOException: >>> java.lang.NullPointerException >>> 2013-10-11 10:24:53,034 INFO org.apache.hadoop.ipc.Server: IPC Server >>> handler 22 on 8021, call >>> heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@13b31acd, true, >>> true, true, -1), rpc version=2, client version=32, >>> methodsFingerPrint=-159967141 from 10.160.25.250:44389: error: >>> java.io.IOException: java.lang.NullPointerException >>> java.io.IOException: java.lang.NullPointerException >>> at >>> org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2751) >>> at >>> org.apache.hadoop.mapred.JobTracker.addNewTracker(JobTracker.java:2731) >>> at >>> org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3227) >>> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2931) >>> at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at >>> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474) >>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002) >>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1751) >>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1747) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:396) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1745) >>> 2013-10-11 10:24:53,035 WARN org.apache.hadoop.net.ScriptBasedMapping: >>> Exception running >>> /run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py >>> 10.160.25.249 >>> java.io.IOException: Cannot run program >>> "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER/topology.py" (in >>> directory "/run/cloudera-scm-agent/process/556-mapreduce-JOBTRACKER"): >>> java.io.IOException: error=13, Permission denied >>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:460) >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:206) >>> at org.apache.hadoop.util.Shell.run(Shell.java:188) >>> at >>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) >>> at >>> org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResolveCommand(ScriptBasedMapping.java:242) >>> at >>> org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.resolve(ScriptBasedMapping.java:180) >>> at >>> org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:119) >>> at >>> org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2750) >>> at >>> org.apache.hadoop.mapred.JobTracker.addNewTracker(JobTracker.java:2731) >>> at >>> org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3227) >>> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2931) >>> at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at >>> org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474) >>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002) >>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1751) >>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1747) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:396) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1745) >>> Caused by: java.io.IOException: java.io.IOException: error=13, >>> Permission denied >>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148) >>> at java.lang.ProcessImpl.start(ProcessImpl.java:65) >>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:453) >>> ... 21 more >>> >>> it doesn't matter if it is a pure hadoop job or a oozie submitted job. >>> there seems to be something wrong in the basic configuration. Anyone an >>> idea? >>> >>> Cheers >>> Wolli >>> >> >> >
