[
https://issues.apache.org/jira/browse/HADOOP-15421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458794#comment-16458794
]
Ryan Blue commented on HADOOP-15421:
------------------------------------
I think this makes sense. As long as there is a _SUCCESS file, it may as well
be used to pass the scope, i.e., which files were successful. What
statistics/metrics are you adding to the file?
> Stabilise/formalise the JSON _SUCCESS format used in the S3A committers
> -----------------------------------------------------------------------
>
> Key: HADOOP-15421
> URL: https://issues.apache.org/jira/browse/HADOOP-15421
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Priority: Major
>
> the S3A committers rely on an atomic PUT to save a JSON summary of the job to
> the dest FS, containing files, statistics, etc. This is for internal testing,
> but it turns out to be useful for spark integration testing, Hive, etc.
> IBM's stocator also generated a manifest.
> Proposed: come up with (an extensible) design that we are happy with as a
> long lived format.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
