Xiang Zhang created HDFS-15475:
----------------------------------
Summary: -D mapreduce.framework.name CLI parameter for miniCluster
not working
Key: HDFS-15475
URL: https://issues.apache.org/jira/browse/HDFS-15475
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.7.0
Reporter: Xiang Zhang
I am running miniCluster using doc here:
[https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/CLIMiniCluster.html.|https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/CLIMiniCluster.html]
I notice that by default mapreduce jobs do not run on YARN and I understand
that setting mapreduce.framework.name to yarn can make this work.
{code:java}
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>{code}
So I tried to add this in etc/hadoop/mapreduces-site.xml and managed to run a
wordcount example through this:
{code:java}
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar
wordcount hdfs://localhost:8020/user/iamabug/input
hdfs://localhost:8020/user/iamabug/output
{code}
However, according to the doc, this parameter should also be available to be
set through -D parameter, i.e.,
{code:java}
bin/hadoop jar
./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.0-tests.jar
minicluster -format -D mapreduce.framework.name=yarn -writeConfig 2.txt
{code}
Notice that I write config to 2.txt and the parameter can be found in this file:
{code:java}
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
<source>programatically</source>
</property>
{code}
I submitted a wordcount example again and it didn't run on YARN according to
the logs and YARN Web UI ([http://localhost:8088).|http://localhost:8088)./]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
