Author: aw Date: Wed Aug 13 19:34:26 2014 New Revision: 1617806 URL: http://svn.apache.org/r1617806 Log: MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm (Akira AJISAKA via aw)
Added: hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm Modified: hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt Modified: hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt?rev=1617806&r1=1617805&r2=1617806&view=diff ============================================================================== --- hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt (original) +++ hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt Wed Aug 13 19:34:26 2014 @@ -154,6 +154,9 @@ Trunk (Unreleased) MAPREDUCE-5944. Remove MRv1 commands from CommandsManual.apt.vm (Akira AJISAKA via aw) + MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm + (Akira AJISAKA via aw) + Release 2.6.0 - UNRELEASED INCOMPATIBLE CHANGES Added: hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm?rev=1617806&view=auto ============================================================================== --- hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm (added) +++ hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm Wed Aug 13 19:34:26 2014 @@ -0,0 +1,224 @@ +~~ Licensed to the Apache Software Foundation (ASF) under one or more +~~ contributor license agreements. See the NOTICE file distributed with +~~ this work for additional information regarding copyright ownership. +~~ The ASF licenses this file to You under the Apache License, Version 2.0 +~~ (the "License"); you may not use this file except in compliance with +~~ the License. You may obtain a copy of the License at +~~ +~~ http://www.apache.org/licenses/LICENSE-2.0 +~~ +~~ Unless required by applicable law or agreed to in writing, software +~~ distributed under the License is distributed on an "AS IS" BASIS, +~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +~~ See the License for the specific language governing permissions and +~~ limitations under the License. + + --- + MapReduce Commands Guide + --- + --- + ${maven.build.timestamp} + +MapReduce Commands Guide + +%{toc|section=1|fromDepth=2|toDepth=4} + +* Overview + + MapReduce commands are invoked by the <<<bin/mapred>>> script. Running the + script without any arguments prints the description for all commands. + + Usage: <<<mapred [--config confdir] COMMAND>>> + + MapReduce has an option parsing framework that employs parsing generic + options as well as running classes. + +*-------------------------+---------------------------------------------------+ +|| COMMAND_OPTIONS || Description | +*-------------------------+---------------------------------------------------+ +| --config confdir | Overwrites the default Configuration directory. Default +| | is $\{HADOOP_PREFIX\}/conf. +*-------------------------+---------------------------------------------------+ +| COMMAND COMMAND_OPTIONS | Various commands with their options are described +| | in the following sections. The commands have been +| | grouped into {{User Commands}} and +| | {{Administration Commands}}. +*-------------------------+---------------------------------------------------+ + +* User Commands + + Commands useful for users of a hadoop cluster. + +** <<<pipes>>> + + Runs a pipes job. + + Usage: <<<mapred pipes [-conf <path>] [-jobconf <key=value>, <key=value>, + ...] [-input <path>] [-output <path>] [-jar <jar file>] [-inputformat + <class>] [-map <class>] [-partitioner <class>] [-reduce <class>] [-writer + <class>] [-program <executable>] [-reduces <num>]>>> + +*----------------------------------------+------------------------------------+ +|| COMMAND_OPTION || Description +*----------------------------------------+------------------------------------+ +| -conf <path> | Configuration for job +*----------------------------------------+------------------------------------+ +| -jobconf <key=value>, <key=value>, ... | Add/override configuration for job +*----------------------------------------+------------------------------------+ +| -input <path> | Input directory +*----------------------------------------+------------------------------------+ +| -output <path> | Output directory +*----------------------------------------+------------------------------------+ +| -jar <jar file> | Jar filename +*----------------------------------------+------------------------------------+ +| -inputformat <class> | InputFormat class +*----------------------------------------+------------------------------------+ +| -map <class> | Java Map class +*----------------------------------------+------------------------------------+ +| -partitioner <class> | Java Partitioner +*----------------------------------------+------------------------------------+ +| -reduce <class> | Java Reduce class +*----------------------------------------+------------------------------------+ +| -writer <class> | Java RecordWriter +*----------------------------------------+------------------------------------+ +| -program <executable> | Executable URI +*----------------------------------------+------------------------------------+ +| -reduces <num> | Number of reduces +*----------------------------------------+------------------------------------+ + +** <<<job>>> + + Command to interact with Map Reduce Jobs. + + Usage: <<<mapred job + | [{{{../../hadoop-project-dist/hadoop-common/CommandsManual.html#Generic_Options}GENERIC_OPTIONS}}] + | [-submit <job-file>] + | [-status <job-id>] + | [-counter <job-id> <group-name> <counter-name>] + | [-kill <job-id>] + | [-events <job-id> <from-event-#> <#-of-events>] + | [-history [all] <jobOutputDir>] | [-list [all]] + | [-kill-task <task-id>] | [-fail-task <task-id>] + | [-set-priority <job-id> <priority>]>>> + +*------------------------------+---------------------------------------------+ +|| COMMAND_OPTION || Description +*------------------------------+---------------------------------------------+ +| -submit <job-file> | Submits the job. +*------------------------------+---------------------------------------------+ +| -status <job-id> | Prints the map and reduce completion + | percentage and all job counters. +*------------------------------+---------------------------------------------+ +| -counter <job-id> <group-name> <counter-name> | Prints the counter value. +*------------------------------+---------------------------------------------+ +| -kill <job-id> | Kills the job. +*------------------------------+---------------------------------------------+ +| -events <job-id> <from-event-#> <#-of-events> | Prints the events' details + | received by jobtracker for the given range. +*------------------------------+---------------------------------------------+ +| -history [all]<jobOutputDir> | Prints job details, failed and killed tip + | details. More details about the job such as + | successful tasks and task attempts made for + | each task can be viewed by specifying the + | [all] option. +*------------------------------+---------------------------------------------+ +| -list [all] | Displays jobs which are yet to complete. + | <<<-list all>>> displays all jobs. +*------------------------------+---------------------------------------------+ +| -kill-task <task-id> | Kills the task. Killed tasks are NOT counted + | against failed attempts. +*------------------------------+---------------------------------------------+ +| -fail-task <task-id> | Fails the task. Failed tasks are counted + | against failed attempts. +*------------------------------+---------------------------------------------+ +| -set-priority <job-id> <priority> | Changes the priority of the job. Allowed + | priority values are VERY_HIGH, HIGH, NORMAL, + | LOW, VERY_LOW +*------------------------------+---------------------------------------------+ + +** <<<queue>>> + + command to interact and view Job Queue information + + Usage: <<<mapred queue [-list] | [-info <job-queue-name> [-showJobs]] + | [-showacls]>>> + +*-----------------+-----------------------------------------------------------+ +|| COMMAND_OPTION || Description +*-----------------+-----------------------------------------------------------+ +| -list | Gets list of Job Queues configured in the system. + | Along with scheduling information associated with the job + | queues. +*-----------------+-----------------------------------------------------------+ +| -info <job-queue-name> [-showJobs] | Displays the job queue information and + | associated scheduling information of particular job queue. + | If <<<-showJobs>>> options is present a list of jobs + | submitted to the particular job queue is displayed. +*-----------------+-----------------------------------------------------------+ +| -showacls | Displays the queue name and associated queue operations + | allowed for the current user. The list consists of only + | those queues to which the user has access. +*-----------------+-----------------------------------------------------------+ + +** <<<classpath>>> + + Prints the class path needed to get the Hadoop jar and the required + libraries. + + Usage: <<<mapred classpath>>> + +** <<<distcp>>> + + Copy file or directories recursively. More information can be found at + {{{./DistCp.html}Hadoop DistCp Guide}}. + +** <<<archive>>> + + Creates a hadoop archive. More information can be found at + {{{./HadoopArchives.html}Hadoop Archives Guide}}. + +* Administration Commands + + Commands useful for administrators of a hadoop cluster. + +** <<<historyserver>>> + + Start JobHistoryServer. + + Usage: <<<mapred historyserver>>> + +** <<<hsadmin>>> + + Runs a MapReduce hsadmin client for execute JobHistoryServer administrative + commands. + + Usage: <<<mapred hsadmin + [-refreshUserToGroupsMappings] | + [-refreshSuperUserGroupsConfiguration] | + [-refreshAdminAcls] | + [-refreshLoadedJobCache] | + [-refreshLogRetentionSettings] | + [-refreshJobRetentionSettings] | + [-getGroups [username]] | [-help [cmd]]>>> + +*-----------------+-----------------------------------------------------------+ +|| COMMAND_OPTION || Description +*-----------------+-----------------------------------------------------------+ +| -refreshUserToGroupsMappings | Refresh user-to-groups mappings +*-----------------+-----------------------------------------------------------+ +| -refreshSuperUserGroupsConfiguration| Refresh superuser proxy groups mappings +*-----------------+-----------------------------------------------------------+ +| -refreshAdminAcls | Refresh acls for administration of Job history server +*-----------------+-----------------------------------------------------------+ +| -refreshLoadedJobCache | Refresh loaded job cache of Job history server +*-----------------+-----------------------------------------------------------+ +| -refreshJobRetentionSettings|Refresh job history period, job cleaner settings +*-----------------+-----------------------------------------------------------+ +| -refreshLogRetentionSettings | Refresh log retention period and log retention +| | check interval +*-----------------+-----------------------------------------------------------+ +| -getGroups [username] | Get the groups which given user belongs to +*-----------------+-----------------------------------------------------------+ +| -help [cmd] | Displays help for the given command or all commands if none is +| | specified. +*-----------------+-----------------------------------------------------------+