> Can we populate the Delta table commit time(9:58) for the first change?
Yufei, it is possible but it would require us to update snapshot producer APIs for all DDL operations. That also includes the transaction APIs. Hence it is a quite disruptive change. Separation of timestamps also makes sense conceptually. In the context of REST catalog where snapshot file is generated at the client side and the metadata.json file is generated at server side. As Hongyue said, only snapshot timestamps are used for time travel. Otherwise, timestamp values are mainly for informational purposes. On Fri, Nov 7, 2025 at 6:55 AM Eduard Tudenhöfner <[email protected]> wrote: > After reading the problem that can happen it makes sense to me to unlink > the table metadata's lastUpdatedMillis from the snapshot's > lastUpdatedMillis. > > On Fri, Nov 7, 2025 at 7:09 AM Yufei Gu <[email protected]> wrote: > >> Can we populate the Delta table commit time(9:58) for the first change? >> >> Yufei >> >> >> On Thu, Nov 6, 2025 at 9:59 PM Steven Wu <[email protected]> wrote: >> >>> Let me add some additional context in addition to what Hongyue said. >>> >>> Imaging a metadata translation use case from Delta to Iceberg. If the >>> Iceberg table metadata stamp is set to the snapshot timestamp (as today), >>> we can get into the following anomaly state. >>> >>> - At 10:00, a schema change (committed at 9:58 at the source Delta >>> table) was translated to the Iceberg metadata. Since DDL doesn't add a >>> new >>> snapshot to the table, the table metadata timestamp was set to 10:00 >>> (system clock). >>> - At 10:01, a DML change (committed at 9:59 in the source Delta >>> table) was translated to the Iceberg metadata. The Iceberg snapshot >>> timestamp should be set to 9:59 for consistent time travel behavior. >>> Current Java implementation would set the table metadata timestamp to >>> 9:59 >>> (same as snapshot timestamp). >>> >>> Now, the table metadata timestamp went backwards in the 2nd commit >>> above. We can avoid such timestamp anomaly behavior with the proposed >>> change. With that, the 2nd commit would set the table timestamp to 10:01 >>> and set the snapshot timestamp to 9:59. >>> >>> >>> >>> On Thu, Nov 6, 2025 at 3:24 PM Steve <[email protected]> wrote: >>> >>>> Hey all: >>>> >>>> I would like to reach out to the Iceberg community to seek input and >>>> thoughts on PR 14504, which proposes changing the behavior of table >>>> metadata's last updated timestamp. >>>> >>>> This was discussed yesterday on the Iceberg community sync. The current >>>> Java SDK implementation ties the `last-updated-ms` field in TableMetadata >>>> to the snapshot creation timestamp when the snapshot was added. Ryan shared >>>> his initial motivation but also indicated that he's open to change. We also >>>> confirmed that the last-updated-at timestamp in TableMetadata is purely >>>> informational and provides no guarantee. >>>> >>>> Unlinking timestamps allows them to reflect their own state, >>>> particularly given the separation of concerns between the engine's role of >>>> authoring snapshots and the catalog's role of writing metadata.json. >>>> >>>> Looking forward to your feedback. >>>> >>>> https://github.com/apache/iceberg/pull/14504 >>>> >>>> Thanks, >>>> Hongyue >>>> >>>
