[
https://issues.apache.org/jira/browse/IGNITE-218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365836#comment-14365836
]
Ivan Veselovsky commented on IGNITE-218:
----------------------------------------
Installing BigTop 0.8.0 on top of Ubuntu in general I followed these
instructions:
http://www.thecloudavenue.com/2014/01/InstallationAndConfigurationOfApacheBigtop.html
.
Installing of Ignite node is just several steps (unless otherwise stated we
work under "bigtop" non-root account ):
{code}
cd ~/
unzip -q ..../ignite-hadoop-{version}.zip
cd ignite-hadoop-{version}/
mkdir cfg0/
cp ./config/hadoop/core-site.ignite.xml ./cfg0/
cp ./config/hadoop/mapred-site.ignite.xml ./cfg0/
IGH=`pwd -P`
# Create custom hadoop clinet script for Ignite:
echo >~/bin/hadoop-ignited <<<EOF
# This is needed to resolve ignite logging configuration:
export IGNITE_HOME=${IGH}
export
HADOOP_CLASSPATH=\${IGNITE_HOME}/libs/ignite-core-1.0.0-RC3-SNAPSHOT.jar:\${IGNITE_HOME}/libs/ignite-hadoop/ignite-hadoop-1.0.0-RC3-SNAPSHOT.jar
hadoop --config \${IGNITE_HOME}/cfg0 "\${@}"
EOF
chmod +x ~/bin/hadoop-ignited
# now start the Ignite node:
cd bin/
./ignite.sh -v &
# run a Hadoop sample (assume ~/bin is in the PATH):
hadoop-ignited jar ..../hadoop-mapreduce-examples.jar pi 5 5
{code}
Note : if you configure secondary Fs (IGFS over HDFS), you don't need
{code}<constructor-arg name="cfgPath" value="/home/ivan/empty.xml"/> {code} any
more -- now it is optional.
> Wrong staging permissions while running MR job under hadoop accelerator
> -----------------------------------------------------------------------
>
> Key: IGNITE-218
> URL: https://issues.apache.org/jira/browse/IGNITE-218
> Project: Ignite
> Issue Type: Bug
> Components: hadoop
> Affects Versions: sprint-1
> Environment: Hadoop 2.4.1
> gridgain-hadoop-6.5.6 (Bigtop trunk) no later than
> d90b89dcc4ac7895d902fed034793c5fcb332e8b
> Reporter: Konstantin Boudnik
> Assignee: Ivan Veselovsky
> Fix For: sprint-3
>
>
> I am trying to run an MR example on gridgain-hadoop setup in Bigtop. The ggfs
> is configured to be backed by HDFS with
> {code}
> <property name="secondaryFileSystem">
> <bean
> class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
> <constructor-arg
> value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
> <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
> </bean>
> </property>
> {code}
> set for {{ggfsConfiguration}}
> While all usual filesystem operations work properly, running an MR job
> produces the following exception in the beginning of the run
> {noformat}
> HADOOP_CONF_DIR=/etc/hadoop/gridgain.client.conf hadoop jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 2 2
> Number of Maps = 2
> Samples per Map = 2
> Wrote input for Map #0
> Wrote input for Map #1
> Starting Job
> java.io.IOException: The ownership on the staging directory
> /tmp/hadoop-yarn/staging/root/.staging is not as expected. It is owned by
> gridgain. The directory must be owned by the submitter root or by root
> at
> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:112)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
> at
> org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}
> However the permissions seem to be correct.
> {noformat}
> [root@vmhost05-hbase2 gridgain.client.conf]# sudo -u hdfs hadoop fs -ls
> /tmp/hadoop-yarn/staging/root
> Found 1 items
> drwx------ - root root 0 2015-02-10 19:31
> /tmp/hadoop-yarn/staging/root/.staging
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
