Hey Ryan: Thanks again for the earlier review and guidance on apache/iceberg#14504. I've addressed your previous comments and hope you can take another look.
https://github.com/apache/iceberg/pull/14504 Thanks, Steve On Fri, Nov 7, 2025 at 3:27 PM Steven Wu <[email protected]> wrote: > > > Can we populate the Delta table commit time(9:58) for the first change? > > Yufei, it is possible but it would require us to update snapshot producer > APIs for all DDL operations. That also includes the transaction APIs. Hence > it is a quite disruptive change. > > Separation of timestamps also makes sense conceptually. In the context of > REST catalog where snapshot file is generated at the client side and the > metadata.json file is generated at server side. > > As Hongyue said, only snapshot timestamps are used for time travel. > Otherwise, timestamp values are mainly for informational purposes. > > > On Fri, Nov 7, 2025 at 6:55 AM Eduard Tudenhöfner <[email protected]> > wrote: >> >> After reading the problem that can happen it makes sense to me to unlink the >> table metadata's lastUpdatedMillis from the snapshot's lastUpdatedMillis. >> >> On Fri, Nov 7, 2025 at 7:09 AM Yufei Gu <[email protected]> wrote: >>> >>> Can we populate the Delta table commit time(9:58) for the first change? >>> >>> Yufei >>> >>> >>> On Thu, Nov 6, 2025 at 9:59 PM Steven Wu <[email protected]> wrote: >>>> >>>> Let me add some additional context in addition to what Hongyue said. >>>> >>>> Imaging a metadata translation use case from Delta to Iceberg. If the >>>> Iceberg table metadata stamp is set to the snapshot timestamp (as today), >>>> we can get into the following anomaly state. >>>> >>>> At 10:00, a schema change (committed at 9:58 at the source Delta table) >>>> was translated to the Iceberg metadata. Since DDL doesn't add a new >>>> snapshot to the table, the table metadata timestamp was set to 10:00 >>>> (system clock). >>>> At 10:01, a DML change (committed at 9:59 in the source Delta table) was >>>> translated to the Iceberg metadata. The Iceberg snapshot timestamp should >>>> be set to 9:59 for consistent time travel behavior. Current Java >>>> implementation would set the table metadata timestamp to 9:59 (same as >>>> snapshot timestamp). >>>> >>>> Now, the table metadata timestamp went backwards in the 2nd commit above. >>>> We can avoid such timestamp anomaly behavior with the proposed change. >>>> With that, the 2nd commit would set the table timestamp to 10:01 and set >>>> the snapshot timestamp to 9:59. >>>> >>>> >>>> >>>> On Thu, Nov 6, 2025 at 3:24 PM Steve <[email protected]> wrote: >>>>> >>>>> Hey all: >>>>> >>>>> I would like to reach out to the Iceberg community to seek input and >>>>> thoughts on PR 14504, which proposes changing the behavior of table >>>>> metadata's last updated timestamp. >>>>> >>>>> This was discussed yesterday on the Iceberg community sync. The current >>>>> Java SDK implementation ties the `last-updated-ms` field in TableMetadata >>>>> to the snapshot creation timestamp when the snapshot was added. Ryan >>>>> shared his initial motivation but also indicated that he's open to >>>>> change. We also confirmed that the last-updated-at timestamp in >>>>> TableMetadata is purely informational and provides no guarantee. >>>>> >>>>> Unlinking timestamps allows them to reflect their own state, particularly >>>>> given the separation of concerns between the engine's role of authoring >>>>> snapshots and the catalog's role of writing metadata.json. >>>>> >>>>> Looking forward to your feedback. >>>>> >>>>> https://github.com/apache/iceberg/pull/14504 >>>>> >>>>> Thanks, >>>>> Hongyue
