After reading the problem that can happen it makes sense to me to unlink the table metadata's lastUpdatedMillis from the snapshot's lastUpdatedMillis.
On Fri, Nov 7, 2025 at 7:09 AM Yufei Gu <[email protected]> wrote: > Can we populate the Delta table commit time(9:58) for the first change? > > Yufei > > > On Thu, Nov 6, 2025 at 9:59 PM Steven Wu <[email protected]> wrote: > >> Let me add some additional context in addition to what Hongyue said. >> >> Imaging a metadata translation use case from Delta to Iceberg. If the >> Iceberg table metadata stamp is set to the snapshot timestamp (as today), >> we can get into the following anomaly state. >> >> - At 10:00, a schema change (committed at 9:58 at the source Delta >> table) was translated to the Iceberg metadata. Since DDL doesn't add a new >> snapshot to the table, the table metadata timestamp was set to 10:00 >> (system clock). >> - At 10:01, a DML change (committed at 9:59 in the source Delta >> table) was translated to the Iceberg metadata. The Iceberg snapshot >> timestamp should be set to 9:59 for consistent time travel behavior. >> Current Java implementation would set the table metadata timestamp to 9:59 >> (same as snapshot timestamp). >> >> Now, the table metadata timestamp went backwards in the 2nd commit above. >> We can avoid such timestamp anomaly behavior with the proposed change. With >> that, the 2nd commit would set the table timestamp to 10:01 and set the >> snapshot timestamp to 9:59. >> >> >> >> On Thu, Nov 6, 2025 at 3:24 PM Steve <[email protected]> wrote: >> >>> Hey all: >>> >>> I would like to reach out to the Iceberg community to seek input and >>> thoughts on PR 14504, which proposes changing the behavior of table >>> metadata's last updated timestamp. >>> >>> This was discussed yesterday on the Iceberg community sync. The current >>> Java SDK implementation ties the `last-updated-ms` field in TableMetadata >>> to the snapshot creation timestamp when the snapshot was added. Ryan shared >>> his initial motivation but also indicated that he's open to change. We also >>> confirmed that the last-updated-at timestamp in TableMetadata is purely >>> informational and provides no guarantee. >>> >>> Unlinking timestamps allows them to reflect their own state, >>> particularly given the separation of concerns between the engine's role of >>> authoring snapshots and the catalog's role of writing metadata.json. >>> >>> Looking forward to your feedback. >>> >>> https://github.com/apache/iceberg/pull/14504 >>> >>> Thanks, >>> Hongyue >>> >>
