Thank you.
mapred.system.dir is not set. I am guessing that it is whatever the default is. What should I set it to? /tmp is already 777 kevin@devUbuntu05:~$ hadoop fs -ls /tmp Found 1 items drwxr-xr-x - hdfs supergroup 0 2013-04-29 15:45 /tmp/mapred kevin@devUbuntu05:~$ hadoop fs -ls -d /tmp Found 1 items drwxrwxrwt - hdfs supergroup 0 2013-04-29 15:45 /tmp But notice that the mapred folder in the /tmp folder is 755. So I changed it: kevin@devUbuntu05 $ hadoop fs -ls -d /tmp drwxrwxrwt - hdfs supergroup 0 2013-04-29 15:45 /tmp kevin@devUbuntu05 $ hadoop fs -ls -R /tmp drwxr-xr-x - mapred supergroup 0 2013-04-29 15:45 /tmp/mapred drwxr-xr-x - mapred supergroup 0 2013-04-29 15:45 /tmp/mapred/system I still get the errors in the log file: 2013-04-30 09:35:11,609 WARN org.apache.hadoop.mapred.JobTracker: Failed to operate on mapred.system.dir (hdfs://devubuntu05:9000/data/hadoop/tmp/hadoop-mapred/mapred/system) because of permissions. 2013-04-30 09:35:11,609 WARN org.apache.hadoop.mapred.JobTracker: This directory should be owned by the user 'mapred (auth:SIMPLE)' 2013-04-30 09:35:11,609 WARN org.apache.hadoop.mapred.JobTracker: Bailing out ... . . . . . org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x . . . . . Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessContr olException): Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x 2013-04-30 09:35:11,610 FATAL org.apache.hadoop.mapred.JobTracker: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x . . . . . . From: Arpit Gupta [mailto:ar...@hortonworks.com] Sent: Tuesday, April 30, 2013 9:25 AM To: user@hadoop.apache.org Subject: Re: Permission problem what is your mapred.system.dir set to in mapred-site.xml? By default it will write to /tmp on hdfs. So you can do the following create /tmp on hdfs and chmod it to 777 as user hdfs and then restart jobtracker and tasktrackers. In case its set to /mapred/something then create /mapred and chown it to user mapred. -- Arpit Gupta Hortonworks Inc. http://hortonworks.com/ On Apr 30, 2013, at 6:36 AM, "Kevin Burton" <rkevinbur...@charter.net> wrote: To further complicate the issue the log file in (/var/log/hadoop-0.20-mapreduce/hadoop-hadoop-jobtracker-devUbuntu05.log) is owned by mapred:mapred and the name of the file seems to indicate some other lineage (hadoop,hadoop). I am out of my league in understanding the permission structure for hadoop hdfs and mr. Ideas? From: Kevin Burton [mailto:rkevinburton@ <http://charter.net> charter.net] Sent: Tuesday, April 30, 2013 8:31 AM To: <mailto:user@hadoop.apache.org> user@hadoop.apache.org Cc: 'Mohammad Tariq' Subject: RE: Permission problem That is what I perceive as the problem. The hdfs file system was created with the user 'hdfs' owning the root ('/') but for some reason with a M/R job the user 'mapred' needs to have write permission to the root. I don't know how to satisfy both conditions. That is one reason that I relaxed the permission to 775 so that the group would also have write permission but that didn't seem to help. From: Mohammad Tariq [ <mailto:donta...@gmail.com> mailto:donta...@gmail.com] Sent: Tuesday, April 30, 2013 8:20 AM To: Kevin Burton Subject: Re: Permission problem user?"ls" shows "hdfs" and the log says "mapred".. Warm Regards, Tariq <https://mtariq.jux.com/> https://mtariq.jux.com/ <http://cloudfront.blogspot.com> cloudfront.blogspot.com On Tue, Apr 30, 2013 at 6:22 PM, Kevin Burton < <mailto:rkevinbur...@charter.net> rkevinbur...@charter.net> wrote: I have relaxed it even further so now it is 775 <mailto:kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$> kevin@devUbuntu05:/var/log/hadoop-0.20-mapreduce$ hadoop fs -ls -d / Found 1 items drwxrwxr-x - hdfs supergroup 0 2013-04-29 15:43 / But I still get this error: 2013-04-30 07:43:02,520 FATAL org.apache.hadoop.mapred.JobTracker: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxrwxr-x From: Mohammad Tariq [mailto: <mailto:donta...@gmail.com> donta...@gmail.com] Sent: Monday, April 29, 2013 5:10 PM To: <mailto:user@hadoop.apache.org> user@hadoop.apache.org Subject: Re: Incompartible cluserIDS make it 755. Warm Regards, Tariq <https://mtariq.jux.com/> https://mtariq.jux.com/ <http://cloudfront.blogspot.com> cloudfront.blogspot.com
