> This is better handled by attaching a global transaction-id (e.g. UUID
that is monotonically increasing) to the snapshot metadata (iceberg allows
adding this to the summary)
I believe even if the client can provide metadata for a snapshot during
commit operation, you never know if that's the same
I should also add one more thing. The PR I linked to above is a good way to
introduce a clock, but it was pointed out in the sync that even if we had a
service that provided synchronized timestamps, there is no guarantee that
there isn't a race condition between committers getting timestamps and th
Thanks, Gautam! I think that's a good summary of the discussion.
On Thu, Sep 10, 2020 at 11:56 AM Gautam wrote:
> Wanted to circle back on this thread. Linear timestamps was discussed
> during the sync and the conclusion was that timestamp based incremental
> reading is generally discouraged as
Wanted to circle back on this thread. Linear timestamps was discussed
during the sync and the conclusion was that timestamp based incremental
reading is generally discouraged as that introduces correctness issues.
Even if a custom clock is available keeping timestamps atomic and
monotonically incre
A missing or corrupt version hint file shouldn't cause a failure. If the
hint is not available, then Iceberg should find the latest metadata version
by checking for the existence of metadata files. I think we just need to
add a warning and ignore this error. Can you open a PR?
rb
On Thu, Sep 10,
Hi Team,
I have been playing around with Iceberg tables, and I had some failing writes
(not really sure how it actually happened), and ended up with an empty
metadata/version-hint.text file.
When I have tried to read this table with Catalogs.loadTable(configuration,
properties) I got a NullPoin
