[
https://issues.apache.org/jira/browse/MAPREDUCE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Joseph Evans resolved MAPREDUCE-4077.
Resolution: Not A Problem
The issue is with the order of your arguments. I know this is ugly, but it is
difficult to fix for various backwards compatibility reasons. The -archives
and -files need to come before the -input argument. They need to be at the
beginning like with the -D arguments.
Issues while using Hadoop Streaming job
---
Key: MAPREDUCE-4077
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4077
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
When we use -file option it says deprecated and use -files.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/3 -mapper cat -reducer wc -file hadoop
02/02/19 10:55:51 WARN streaming.StreamJob: -file option is deprecated,
please use generic option -files instead.
{code}
But when we use -files option, it says unrecognized option.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output
/test/output/3 -mapper cat -reducer wc -files hadoop
02/02/19 10:56:42 ERROR streaming.StreamJob: Unrecognized option: -files
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
{code}
When we use -archives option, it says unrecognized option.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output
/test/output/3 -mapper cat -reducer wc -archives testarchive.rar
02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
{code}
But in the options it will display the usage of the -archives.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output
/test/output/3 -mapper cat -reducer wc -archives testarchive.rar
02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
..
..
-libjars comma separated list of jarsspecify comma separated jar files
to include in the classpath.
-archives comma separated list of archivesspecify comma separated
archives to be unarchived on the compute machines.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
