[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260082#comment-16260082 ]
ASF GitHub Bot commented on HADOOP-14971: ----------------------------------------- Github user ajfabbri commented on a diff in the pull request: https://github.com/apache/hadoop/pull/282#discussion_r152149374 --- Diff: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committers.md --- @@ -550,6 +551,102 @@ Conflict management is left to the execution engine itself. | `fs.s3a.committer.staging.tmp.path` | | X | X | Path in the cluster filesystem for temporary data | `tmp/staging` | +```xml +<property> + <name>fs.s3a.committer.name</name> + <value>file</value> + <description> + Committer to create for output to S3A, one of: + "file", "directory", "partitioned", "magic". + </description> +</property> + +<property> + <name>fs.s3a.committer.magic.enabled</name> + <value>false</value> + <description> + Enable support in the filesystem for the S3 "Magic" committer. + When working with AWS S3, S3Guard must be enabled for the destination + bucket, as consistent metadata listings are required. + </description> +</property> + +<property> + <name>fs.s3a.committer.threads</name> + <value>8</value> + <description> + Number of threads in committers for parallel operations on files + (upload, commit, abort, delete...) + </description> +</property> + +<property> + <name>fs.s3a.committer.staging.tmp.path</name> + <value>tmp/staging</value> + <description> + Path in the cluster filesystem for temporary data. + This is for HDFS, not the local filesystem. + It is only for the summary data of each file, not the actual + data being committed. + Using an unqualified path guarantees that the full path will be + generated relative to the home directory of the user creating the job, + hence private (assuming home directory permissions are secure). + </description> +</property> + +<property> + <name>fs.s3a.committer.staging.unique-filenames</name> + <value>true</value> + <description> + Option for final files to have a unique name through job attempt info, + or the value of fs.s3a.committer.staging.uuid + When writing data with the "append" conflict option, this guarantees + that new data will not overwrite any existing data. + </description> +</property> + +<property> + <name>fs.s3a.committer.staging.conflict-mode</name> + <value>fail</value> + <description> + Staging committer conflict resolution policy. + Supported: "fail", "append", "replace". + </description> +</property> + +<property> + <name>s.s3a.committer.staging.abort.pending.uploads</name> --- End diff -- typo here > Merge S3A committers into trunk > ------------------------------- > > Key: HADOOP-14971 > URL: https://issues.apache.org/jira/browse/HADOOP-14971 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.0.0 > Reporter: Steve Loughran > Assignee: Steve Loughran > Attachments: HADOOP-13786-040.patch, HADOOP-13786-041.patch > > > Merge the HADOOP-13786 committer into trunk. This branch is being set up as a > github PR for review there & to keep it out the mailboxes of the watchers on > the main JIRA -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org