amogh-jahagirdar commented on code in PR #11829:
URL: https://github.com/apache/iceberg/pull/11829#discussion_r1900307975
##########
core/src/main/java/org/apache/iceberg/FastAppend.java:
##########
@@ -157,12 +158,16 @@ public List<ManifestFile> apply(TableMetadata base,
Snapshot snapshot) {
}
@Override
- public Object updateEvent() {
+ public Object updateEvent(Snapshot committedSnapshot) {
long snapshotId = snapshotId();
- Snapshot snapshot = ops().current().snapshot(snapshotId);
- long sequenceNumber = snapshot.sequenceNumber();
+ ValidationException.check(
+ snapshotId == committedSnapshot.snapshotId(),
+ "Committed snapshotId %s does not match expected snapshotId %s",
+ committedSnapshot.snapshotId(),
+ snapshotId);
+ long sequenceNumber = committedSnapshot.sequenceNumber();
Review Comment:
Can we uplevel this logic, it's common to all the implementations?
##########
core/src/main/java/org/apache/iceberg/SnapshotProducer.java:
##########
@@ -475,10 +475,14 @@ public void commit() {
}
}
+ Object updateEvent(Snapshot committedSnapshot) {
Review Comment:
`SnapshotProducer` is package private, so I think we're OK in terms of
backwards compatibility since it's not like a public API is being broken.
##########
core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java:
##########
@@ -956,23 +952,16 @@ public List<ManifestFile> apply(TableMetadata base,
Snapshot snapshot) {
}
@Override
- public Object updateEvent() {
+ public Object updateEvent(Snapshot committedSnapshot) {
long snapshotId = snapshotId();
- Snapshot justSaved = ops().refresh().snapshot(snapshotId);
- long sequenceNumber = TableMetadata.INVALID_SEQUENCE_NUMBER;
- Map<String, String> summary;
- if (justSaved == null) {
- // The snapshot just saved may not be present if the latest metadata
couldn't be loaded due to
- // eventual
- // consistency problems in refresh.
- LOG.warn("Failed to load committed snapshot: omitting sequence number
from notifications");
- summary = summary();
- } else {
- sequenceNumber = justSaved.sequenceNumber();
- summary = justSaved.summary();
- }
-
- return new CreateSnapshotEvent(tableName, operation(), snapshotId,
sequenceNumber, summary);
+ ValidationException.check(
+ snapshotId == committedSnapshot.snapshotId(),
+ "Committed snapshotId %s does not match expected snapshotId %s",
+ committedSnapshot.snapshotId(),
+ snapshotId);
Review Comment:
Do we really need the validation? I feel like the principle of this change
is that the update event that is produced is always going to be derived from
the passed in committed snapshot. I think passing `committedSnapshot.id()` to
the event suffices
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
