findepi commented on code in PR #4945:
URL: https://github.com/apache/iceberg/pull/4945#discussion_r927697215
##########
format/spec.md:
##########
@@ -632,6 +633,30 @@ When expiring snapshots, retention policies in table and
snapshot references are
2. The snapshot is not one of the first `min-snapshots-to-keep` in the
branch (including the branch's referenced snapshot)
5. Expire any snapshot not in the set of snapshots to retain.
+#### Statistics file
+
+Statistics files are valid [Puffin files](../puffin-spec). Statistics are
informational. A reader can choose to
+ignore statistics information. Statistics support is not required to read the
table correctly.
+
+Statistics file's metadata within `statistics` table snapshot field is a
struct with the following fields:
+
+| v1 | v2 | Field name | Type
| Description
|
+|------------|------------|---------------------------------|-----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|
+| _required_ | _required_ | **`statistics-path`** | `string`
| Path of the statistics file. See [Puffin file
format](../puffin-spec).
|
+| _required_ | _required_ | **`file-size-in-bytes`** | `long`
| Size of the statistics file.
|
+| _required_ | _required_ | **`file-footer-size-in-bytes`** | `long`
| Total size of the statistics file's footer (not the footer
payload size). See [Puffin file format](../puffin-spec) for footer definition. |
+| _required_ | _required_ | **`blob-metadata`** | `list<blob
metadata>` (see below) | A list of the blob metadata for statistics contained
in the file with structure described below.
|
+
+Statistic metadata is a struct with the following fields:
+
+| v1 | v2 | Field name | Type
| Description
|
+|------------|------------|--------------------------------------|------------------------|--------------------------------------------------------------------------------------------------|
+| _required_ | _required_ | **`type`** | `string`
| Type of the statistic. Matches Blob type in the Puffin file.
|
+| _required_ | _required_ | **`fields`** |
`list<integer>` | Ordered list of fields, given by field ID, on which
the statistic was calculated. |
+| _required_ | _required_ | **`source-snapshot-id`** | `long`
| ID of the Iceberg table's snapshot the blob was computed from.
|
+| _required_ | _required_ | **`source-snapshot-sequence-numer`** | `long`
| Sequence number of the Iceberg table's snapshot the blob was
computed from. |
Review Comment:
if we attach stats file to a given snapshot (per
https://github.com/apache/iceberg/pull/4945#discussion_r905295067) then these
fields become redundant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
