harnsit opened a new issue, #5960:
URL: https://github.com/apache/iceberg/issues/5960
### Apache Iceberg version
0.13.1
### Query engine
Spark
### Please describe the bug 🐞
We save the snapshot ID in another table and after every update (merge) in
an iceberg table, we query the latest snapshot ID for that table. I have
observed that although the snapshot is committed, the response of the SELECT
query on table.history doesn't return the snapshot ID immediately. After some
updates, it finally shows up
E.g.
```
************************************************** First Commit
*************************************************
7618:22/10/11 20:44:26 INFO BaseMetastoreTableOperations: Successfully
committed to table hive_prod.some_dlh.t_some_table in 555 ms
7619:22/10/11 20:44:26 INFO SnapshotProducer: Committed snapshot
8679062927306695788 (MergeAppend)
9044:SELECT * FROM hive_prod.some_dlh.t_some_table.history ORDER BY
made_current_at desc
********************** Printing snapshotDataset for table t_some_table
+--------------------+-------------------+-------------------+-------------------+
9097:| made_current_at| snapshot_id|
parent_id|is_current_ancestor|
9098:+--------------------+-------------------+-------------------+-------------------+
9099:|2022-10-11 20:43:...|2828537442614125919|2803735536595908931|
true|
9100:|2022-10-11 20:43:...|2803735536595908931| 927882461206457236|
true|
9101:|2022-10-11 20:42:...| 927882461206457236|3522558446433967482|
true|
9102:|2022-10-11 20:42:...|3522558446433967482| null|
true|
9103:+--------------------+-------------------+-------------------+-------------------+
************************************************** Second Commit
*************************************************
10317:22/10/11 20:44:46 INFO BaseMetastoreTableOperations: Successfully
committed to table hive_prod.some_dlh.t_some_table in 932 ms
10318:22/10/11 20:44:46 INFO SnapshotProducer: Committed snapshot
2711676855945504392 (MergeAppend)
11751:SELECT * FROM hive_prod.some_dlh.t_some_table.history ORDER BY
made_current_at desc
**********************Printing snapshotDataset for table t_some_table
+--------------------+-------------------+-------------------+-------------------+
11786:| made_current_at| snapshot_id|
parent_id|is_current_ancestor|
11787:+--------------------+-------------------+-------------------+-------------------+
11788:|2022-10-11 20:43:...|2828537442614125919|2803735536595908931|
true|
11789:|2022-10-11 20:43:...|2803735536595908931| 927882461206457236|
true|
11790:|2022-10-11 20:42:...| 927882461206457236|3522558446433967482|
true|
11791:|2022-10-11 20:42:...|3522558446433967482| null|
true|
11792:+--------------------+-------------------+-------------------+-------------------+
************************************************** Third Commit
(now shows all) *************************************************
13015:22/10/11 20:45:06 INFO BaseMetastoreTableOperations: Successfully
committed to table hive_prod.some_dlh.t_some_table in 260 ms
13016:22/10/11 20:45:06 INFO SnapshotProducer: Committed snapshot
2402450212603362919 (MergeAppend)
**********************Printing snapshotDataset for table t_some_table
+--------------------+-------------------+-------------------+-------------------+
14488:| made_current_at| snapshot_id|
parent_id|is_current_ancestor|
14489:+--------------------+-------------------+-------------------+-------------------+
14490:|2022-10-11 20:45:...|2402450212603362919|2711676855945504392|
true|
14491:|2022-10-11 20:44:...|2711676855945504392|8679062927306695788|
true|
14492:|2022-10-11 20:44:...|8679062927306695788|2828537442614125919|
true|
14493:|2022-10-11 20:43:...|2828537442614125919|2803735536595908931|
true|
14494:|2022-10-11 20:43:...|2803735536595908931| 927882461206457236|
true|
14495:|2022-10-11 20:42:...| 927882461206457236|3522558446433967482|
true|
14496:|2022-10-11 20:42:...|3522558446433967482| null|
true|
14497:+--------------------+-------------------+-------------------+-------------------+
```
E.g. snapshot ID 2711676855945504392 is not available immediately after it
is committed, but after another snapshot ID is committed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
