Based on the logs your system dir is set to > hdfs://devubuntu05:9000/data/hadoop/tmp/hadoop-mapred/mapred/system
what is your fs.default.name and hadoop.tmp.dir in core-site.xml set to? -- Arpit Gupta Hortonworks Inc. http://hortonworks.com/ On Apr 30, 2013, at 7:39 AM, "Kevin Burton" <rkevinbur...@charter.net> wrote: > > Thank you. > > mapred.system.dir is not set. I am guessing that it is whatever the default > is. What should I set it to? > > /tmp is already 777 > > kevin@devUbuntu05:~$ hadoop fs -ls /tmp > Found 1 items > drwxr-xr-x - hdfs supergroup 0 2013-04-29 15:45 /tmp/mapred > kevin@devUbuntu05:~$ hadoop fs -ls -d /tmp > Found 1 items > drwxrwxrwt - hdfs supergroup 0 2013-04-29 15:45 /tmp > > But notice that the mapred folder in the /tmp folder is 755. > So I changed it: > > kevin@devUbuntu05 $ hadoop fs -ls -d /tmp > drwxrwxrwt - hdfs supergroup 0 2013-04-29 15:45 /tmp > > kevin@devUbuntu05 $ hadoop fs -ls -R /tmp > drwxr-xr-x - mapred supergroup 0 2013-04-29 15:45 /tmp/mapred > drwxr-xr-x - mapred supergroup 0 2013-04-29 15:45 > /tmp/mapred/system > > I still get the errors in the log file: > > 2013-04-30 09:35:11,609 WARN org.apache.hadoop.mapred.JobTracker: Failed to > operate on mapred.system.dir > (hdfs://devubuntu05:9000/data/hadoop/tmp/hadoop-mapred/mapred/system) because > of permissions. > 2013-04-30 09:35:11,609 WARN org.apache.hadoop.mapred.JobTracker: This > directory should be owned by the user 'mapred (auth:SIMPLE)' > 2013-04-30 09:35:11,609 WARN org.apache.hadoop.mapred.JobTracker: Bailing out > ... > . . . . . > org.apache.hadoop.security.AccessControlException: Permission denied: > user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x > . . . . . > Caused by: > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): > Permission denied: user=mapred, access=WRITE, > inode="/":hdfs:supergroup:drwxrwxr-x > 2013-04-30 09:35:11,610 FATAL org.apache.hadoop.mapred.JobTracker: > org.apache.hadoop.security.AccessControlException: Permission denied: > user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x > . . . . . . > > > From: Arpit Gupta [mailto:ar...@hortonworks.com] > Sent: Tuesday, April 30, 2013 9:25 AM > To: user@hadoop.apache.org > Subject: Re: Permission problem > > what is your mapred.system.dir set to in mapred-site.xml? > > By default it will write to /tmp on hdfs. > > So you can do the following > > create /tmp on hdfs and chmod it to 777 as user hdfs and then restart > jobtracker and tasktrackers. > > In case its set to /mapred/something then create /mapred and chown it to user > mapred. > > > -- > Arpit Gupta > Hortonworks Inc. > http://hortonworks.com/ > > On Apr 30, 2013, at 6:36 AM, "Kevin Burton" <rkevinbur...@charter.net> wrote: > > > To further complicate the issue the log file in > (/var/log/hadoop-0.20-mapreduce/hadoop-hadoop-jobtracker-devUbuntu05.log) is > owned by mapred:mapred and the name of the file seems to indicate some other > lineage (hadoop,hadoop). I am out of my league in understanding the > permission structure for hadoop hdfs and mr. Ideas? > > From: Kevin Burton [mailto:rkevinbur...@charter.net] > Sent: Tuesday, April 30, 2013 8:31 AM > To: user@hadoop.apache.org > Cc: 'Mohammad Tariq' > Subject: RE: Permission problem > > That is what I perceive as the problem. The hdfs file system was created with > the user ‘hdfs’ owning the root (‘/’) but for some reason with a M/R job the > user ‘mapred’ needs to have write permission to the root. I don’t know how to > satisfy both conditions. That is one reason that I relaxed the permission to > 775 so that the group would also have write permission but that didn’t seem > to help. > > From: Mohammad Tariq [mailto:donta...@gmail.com] > Sent: Tuesday, April 30, 2013 8:20 AM > To: Kevin Burton > Subject: Re: Permission problem > > user?"ls" shows "hdfs" and the log says "mapred".. > > Warm Regards, > Tariq > https://mtariq.jux.com/ > cloudfront.blogspot.com > > > On Tue, Apr 30, 2013 at 6:22 PM, Kevin Burton <rkevinbur...@charter.net> > wrote: > I have relaxed it even further so now it is 775 > > kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$ hadoop fs -ls -d / > Found 1 items > drwxrwxr-x - hdfs supergroup 0 2013-04-29 15:43 / > > But I still get this error: > > 2013-04-30 07:43:02,520 FATAL org.apache.hadoop.mapred.JobTracker: > org.apache.hadoop.security.AccessControlException: Permission denied: > user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x > > > From: Mohammad Tariq [mailto:donta...@gmail.com] > Sent: Monday, April 29, 2013 5:10 PM > To: user@hadoop.apache.org > Subject: Re: Incompartible cluserIDS > > make it 755. > > Warm Regards, > Tariq > https://mtariq.jux.com/ > cloudfront.blogspot.com > > >