steveloughran commented on a change in pull request #2530:
URL: https://github.com/apache/hadoop/pull/2530#discussion_r555247027
##########
File path:
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committer_architecture.md
##########
@@ -1312,6 +1312,16 @@ On `close()`, summary data would be written to the file
`/results/latest/__magic/job400_1/task_01_01/latest.orc.lzo.pending`.
This would contain the upload ID and all the parts and etags of uploaded data.
+A marker file is also created, so that code which verifies that a newly
created file
+exists does not fail.
+1. These marker files are zero bytes long.
+1. They declare the full length of the final file in the HTTP header
+ `x-hadoop-s3a-magic-marker`.
Review comment:
nicely spotted. will fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
