[DISCUSS] Handling of reference names in TableIdentifier in the context of Snapshot Tagging/Branching

2021-11-16 Thread Eduard Tudenhoefner
Hi everyone, In the context of the Snapshot Tagging and Branching feature we will eventually need to have a way to distinguish between a table that is being referenced as namespace.tbl (which means it's that table on whatever the current default

Re: Upcoming Iceberg Community Sync (11/17 9:00am PT)

2021-11-16 Thread OpenInx
The correct PR link contributed by Reo-LEI is: https://github.com/apache/iceberg/pull/3240, so let's update the following part: b. v2's extra meta columns messed up the flink's RowData pos: * https://github.com/apache/iceberg/pull/3240 (Thanks Reo-LEI ) * Another

Re: Upcoming Iceberg Community Sync (11/17 9:00am PT)

2021-11-16 Thread OpenInx
Let me give more inputs from my perspective. 1. Fixed few critical flink v2 reader bugs: a. The flink avro reader bug: https://github.com/apache/iceberg/pull/3540 b. v2's extra meta columns messed up the flink's RowData pos: *

Re: Spark Merge On Read Support

2021-11-16 Thread Sreeram Garlapati
This makes sense, thanks a lot @Ryan Blue . Are all building blocks for MOR support (features like - delta-based plans) fully available in Spark 3.2 - or is there any reason we would need Spark 3.3? Or is there more ongoing work needed to fully validate this? I am in need of this specific data

Upcoming Iceberg Community Sync (11/17 9:00am PT)

2021-11-16 Thread Sam Redai
Hey everyone, This is just a friendly reminder that the next Iceberg Community Sync is tomorrow (11/17) at 9:00AM PT. The meeting will be recorded and shared shortly after on the dev mailing list in case you can’t make it. Friendly reminder to add any highlights, agenda, or discussion items to