rdblue commented on code in PR #12672:
URL: https://github.com/apache/iceberg/pull/12672#discussion_r2019428750
##########
core/src/main/java/org/apache/iceberg/SnapshotProducer.java:
##########
@@ -283,11 +285,26 @@ public Snapshot apply() {
throw new RuntimeIOException(e, "Failed to write manifest list file");
}
- Long addedRows = null;
- Long lastRowId = null;
- if (base.rowLineageEnabled()) {
- addedRows = calculateAddedRows(manifests);
- lastRowId = base.nextRowId();
+ Long assignedRows = null;
+ if (base.formatVersion() >= 3) {
+ assignedRows = writer.nextRowId() - base.nextRowId();
Review Comment:
This is different because the added rows may be different. This accumulates
the `added_row_count` for all manifests that were assigned a `first-row-id`,
not all manifests. That distinction is why I called this `assigned` instead of
`added`.
I also need to look into how we handle upgraded tables. When writing the
manifest list, we will always assign a `first-row-id` for manifests that do not
have one in v3. I think that leaves a gap where we will assign a `first-row-id`
to a manifest that has `existing` data files and could potentially skip them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
