HonahX commented on code in PR #11660:
URL: https://github.com/apache/iceberg/pull/11660#discussion_r1894512326
##########
format/spec.md:
##########
@@ -693,6 +686,64 @@ A snapshot's `first-row-id` is assigned to the table's
current `next-row-id` on
The snapshot's `first-row-id` is the starting `first_row_id` assigned to
manifests in the snapshot's manifest list.
+#### Snapshot Summary
+
+##### Required Field `operation`
+The snapshot summary's `operation` field is used by some operations, like
snapshot expiration, to skip processing certain snapshots. Possible `operation`
values are:
+
+* `append` -- Only data files were added and no files were removed.
+* `replace` -- Data and delete files were added and removed without changing
table data; i.e., compaction, changing the data file format, or relocating data
files.
+* `overwrite` -- Data and delete files were added and removed in a logical
overwrite operation.
+* `delete` -- Data files were removed and their contents logically deleted
and/or delete files were added to delete rows.
+
+##### Optional Metrics
+All metrics fields should have numeric string values (e.g., `"120"`).
+Some of them are also used to represent partition-level metrics, in [Optional
Partition-Level Summary](#optional-partition-level-summary).
+
+| Field | Description
| Used in Partition-Level Summary |
+|-------------------------------------|-------------------------------------------------------------------|---------------------------------|
+| **`added-data-files`** | Number of data files added in the
current snapshot | Yes |
+| **`deleted-data-files`** | Number of data files deleted in the
current snapshot | Yes |
+| **`total-data-files`** | Total number of data files in the
current snapshot | No |
+| **`added-delete-files`** | Number of delete files added in the
current snapshot | Yes |
+| **`added-equality-delete-files`** | Number of equality delete files added
in the current snapshot | Yes |
+| **`removed-equality-delete-files`** | Number of equality delete files
removed in the current snapshot | Yes |
+| **`added-position-delete-files`** | Number of position delete files added
in the current snapshot | Yes |
+| **`removed-position-delete-files`** | Number of position delete files
removed in the current snapshot | Yes |
+| **`added-dvs`** | Number of deletion vectors added in
the current snapshot | Yes |
+| **`removed-dvs`** | Number of deletion vectors removed in
the current snapshot | Yes |
+| **`removed-delete-files`** | Number of delete files removed in the
current snapshot | Yes |
+| **`total-delete-files`** | Total number of delete files in the
current snapshot | No |
+| **`added-records`** | Number of records added in the current
snapshot | Yes |
+| **`deleted-records`** | Number of records deleted in the
current snapshot | Yes |
+| **`total-records`** | Total number of records in the current
snapshot | No |
+| **`added-files-size`** | The size of files added in the current
snapshot | Yes |
+| **`removed-files-size`** | The size of files removed in the
current snapshot | Yes |
+| **`total-files-size`** | The size of all files in the current
snapshot | No |
+| **`added-position-deletes`** | Number of position delete records
added in the current snapshot | Yes |
+| **`removed-position-deletes`** | Number of position delete records
removed in the current snapshot | Yes |
+| **`total-position-deletes`** | Total number of position delete
records in the current snapshot | No |
+| **`added-equality-deletes`** | Number of equality delete records
added in the current snapshot | Yes |
+| **`removed-equality-deletes`** | Number of equality delete records
removed in the current snapshot | Yes |
+| **`total-equality-deletes`** | Total number of equality delete
records in the current snapshot | No |
+| **`deleted-duplicate-files`** | Number of duplicate files deleted in
the current snapshot | No |
Review Comment:
This comes from a check in ManifestFilterManager:
https://github.com/apache/iceberg/blob/90be5d7360bc7ff274e7d00cb7259afbf78f223b/core/src/main/java/org/apache/iceberg/ManifestFilterManager.java#L492-L497
"Duplicate" means there are 2 manifest entry pointing to the same data file
in one Manifest.
What do you think of
```
Number of duplicate files deleted, where duplicates are files recorded more
than once in the manifest
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
