[
https://issues.apache.org/jira/browse/IGNITE-218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393537#comment-14393537
]
Ivan Veselovsky commented on IGNITE-218:
----------------------------------------
I'm now testing Ignite BigTop packages , and It looks like I was able to
reproduce this issue.
The problem is that the user that ran mapred job (this is the user the Hadoop
client was running on behalf of, "root" in your case) is not the same as the
user GG (or Ignite) node is running on behalf of ("gridgain" in your case).
In Ignite trunk we already have fix for that: the secondary filesystem bean
constructor now has 3rd parameter "userName", that helps to set the user name
client will interact with the 2ndary file system on behalf of. So, in your case
setting userName to "root" should solve the problem.
Unfortunately, this fix is available in Ignite trunk only, so for GG the only
workaround I see is to start the jobs under the same user that runs GG node
(that is, "gridgain").
> Wrong staging permissions while running MR job under hadoop accelerator
> -----------------------------------------------------------------------
>
> Key: IGNITE-218
> URL: https://issues.apache.org/jira/browse/IGNITE-218
> Project: Ignite
> Issue Type: Bug
> Components: hadoop
> Affects Versions: sprint-1
> Environment: Hadoop 2.4.1
> gridgain-hadoop-6.5.6 (Bigtop trunk) no later than
> d90b89dcc4ac7895d902fed034793c5fcb332e8b
> Reporter: Konstantin Boudnik
> Assignee: Ivan Veselovsky
> Fix For: sprint-3
>
>
> I am trying to run an MR example on gridgain-hadoop setup in Bigtop. The ggfs
> is configured to be backed by HDFS with
> {code}
> <property name="secondaryFileSystem">
> <bean
> class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
> <constructor-arg
> value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
> <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
> </bean>
> </property>
> {code}
> set for {{ggfsConfiguration}}
> While all usual filesystem operations work properly, running an MR job
> produces the following exception in the beginning of the run
> {noformat}
> HADOOP_CONF_DIR=/etc/hadoop/gridgain.client.conf hadoop jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 2 2
> Number of Maps = 2
> Samples per Map = 2
> Wrote input for Map #0
> Wrote input for Map #1
> Starting Job
> java.io.IOException: The ownership on the staging directory
> /tmp/hadoop-yarn/staging/root/.staging is not as expected. It is owned by
> gridgain. The directory must be owned by the submitter root or by root
> at
> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:112)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
> at
> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}
> However the permissions seem to be correct.
> {noformat}
> [root@vmhost05-hbase2 gridgain.client.conf]# sudo -u hdfs hadoop fs -ls
> /tmp/hadoop-yarn/staging/root
> Found 1 items
> drwx------ - root root 0 2015-02-10 19:31
> /tmp/hadoop-yarn/staging/root/.staging
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
