Hey Iceberg Nation, Here are the meeting minutes from last few meeting's minutes. I've had some adjustments after moving on from Tabular, thanks for bearing with me.
Transcription/Recording https://youtu.be/j1GncDMj8HY Summary 0:11 Several contributors have switched employers, but remain committed to the Iceberg community 6:44 Releases are planned for Iceberg Java 1.6, Rust, and Python 13:13 Discussions around OAuth 2.0 integration, REST catalog, and materialized views 25:52 Proposal to have separate sync meetings focused on REST catalog development Employer Changes Some contributors (e.g. from Tabular acquired by Databricks) have changed employers, they remain committed to the Iceberg community and open collaboration Release Updates Iceberg Java 1.6 7:00 Include Kafka commit code and Gradle/Rev API updates 7:22 JB to start the release process next week Rust 7:48 Focus on read support - file pruning, filter pushdown, Arrow integration 9:06 Waiting on fix for field ID bug before releasing Python 5:25 Making progress on snapshot management capabilities OAuth 2.0 Integration 13:13 Discussions around keeping/removing the tokens endpoint 13:25 Need to build shared understanding before deciding 17:46 Proposal to have a separate vote if consensus can't be reached REST Catalog 25:52 Proposal to have separate sync meetings focused just on REST catalog development 29:56 Move REST catalog spec to a separate repo? Concerns around breaking links 31:42 Have regular note-taking for sub-sync meetings Materialized Views 32:49 Good progress made, most functional questions answered 35:16 Some remaining naming/terminology discussions 34:05 Plan to start PR to adopt spec definitions soon Server Capabilities 37:28 Important for indicating support for endpoints like plan/pre-plan 42:30 Discussions around authentication, backward compatibility 45:04 Likely to bring up for a community vote Other Topics 32:35 Quick updates on SQL UDFs, rebranding, supporting more storage services, geospatial data types Notes * Highlights * Iceberg Summit [video’s are online]( https://www.youtube.com/@ApacheIceberg/videos), please check them out! * Java * Thanks Eduard for [forking the rev-api plugin]( https://plugins.gradle.org/plugin/io.github.nastra.revapi) to support Gradle greater than 8.1, thanks [JB for doing the actual work]( https://github.com/nastra/gradle-revapi/pull/11)! Alternative PR has been created to avoid use of unmaintained revapi gradle plugin ([#10386]( https://github.com/apache/iceberg/pull/10386)). * Thanks Manu Zhang for the [TLC]( https://github.com/apache/iceberg/pull/10374) [for]( https://github.com/apache/iceberg/pull/10394) [the]( https://github.com/apache/iceberg/pull/10397) [docs]( https://github.com/apache/iceberg/pull/10463) * Thanks Piotr for [working towards JDK21]( https://github.com/apache/iceberg/pull/10474) support and [fixing]( https://github.com/apache/iceberg/pull/10521) [a lot]( https://github.com/apache/iceberg/pull/10530) [along]( https://github.com/apache/iceberg/pull/10485) [the]( https://github.com/apache/iceberg/pull/10517) [way]( https://github.com/apache/iceberg/pull/10475). * Thanks Szehon for fixing [rewrite positional deletes files]( https://github.com/apache/iceberg/pull/10020) on tables with 1k+ columns. * Python * Thanks Chinmay Bhat for adding snapshot management; [creation of tags and branches](https://github.com/apache/iceberg-python/pull/728), and looking up [snapshots by datetime]( https://github.com/apache/iceberg-python/pull/748). * Thanks Sung for adding partitioned writes based for [temporal partitioned writes](https://github.com/apache/iceberg-python/pull/784). * Rust * Thanks Shabana for adding the PredicateVisitor for [filtering out manifests](https://github.com/apache/iceberg-rust/pull/367). * Go * Thanks Matt for adding [predicates and expressions]( https://github.com/apache/iceberg-go/pull/91)! * Upcoming releases * Iceberg 1.6.0 Release ([devlist]( https://lists.apache.org/thread/ymx4kbbfmndmhlrzfrpgzj3hmo6294pv), [milestone](https://github.com/apache/iceberg/milestone/44)) scheduled for June. Preparation started, plan to submit to vote next week (JB is release manager for this one). * Iceberg-rust 0.3.0 release ([devlist]( https://lists.apache.org/thread/x1kn3oq5lv6hllf1d50pbyrwcwthy4t1), [tracking issue](https://github.com/apache/iceberg-rust/issues/348), [project](https://github.com/orgs/apache/projects/339/views/1)) * PyIceberg heading to 0.7.0 ([tracking issue]( https://github.com/apache/iceberg-python/issues/736)) * Discussion ([all proposals]( https://github.com/apache/iceberg/labels/proposal)) * [Path forward](https://github.com/apache/iceberg/issues/10537) for the OAuth2 [concerns]( https://lists.apache.org/thread/twk84xx7v0xy5q5tfd9x5torgr82vv50) in the REST Catalog (Dmitri/Robert) * Iceberg Materialized Views (Jan) ([Spec]( https://docs.google.com/document/d/1UnhldHhe3Grz8JBngwXPA6ZZord1xMedY5ukEhZYF-A/edit#heading=h.5e2hiq3in5af), [Video of meeting]( https://drive.google.com/file/d/1DE09tYS28L3xL_NgnM9g0Olbe6aHza5G/view?usp=sharing )) * REST catalog things to another repo (iceberg-rest?) with separated sync meetings * (Left-overs) from the last meeting: * REST catalog improvements * (Pre)Plan Endpoints ([#9695]( https://github.com/apache/iceberg/pull/9695)) * Append Endpoints ([#10202]( https://github.com/apache/iceberg/pull/10202)) * Server Capabilities ([#9940]( https://github.com/apache/iceberg/pull/9940)) * Client-side purge in REST catalog ([#10089]( https://github.com/apache/iceberg/issues/10089)) * SQL UDFs [mailing list discussion]( https://lists.apache.org/thread/gzofnqqpq5m0cjpts8z5k9rkz4y8gv10) * Access decision exchange ([#10395]( https://github.com/apache/iceberg/issues/10395)) * Go Over Geo Proposal [#10260]( http://github.com/apache/iceberg/issues/10260) * Variant Data Type Proposal [#10392]( https://github.com/apache/iceberg/issues/10392) * Question: What will happen if iceberg-rust support more storage services than iceberg-java? Do we need to create a proposal first? (Xuanwo) * Rebrand iceberg to iceberg-java (Xuanwo) * inspired by [parquet’s recent renaming]( https://github.com/apache/parquet-java) * iceberg community is much bigger (go, python, rust) * After this rebrand, iceberg can be used to refer iceberg community as a whole or the format spec without confusion. * [Historical context]( https://lists.apache.org/thread/xps8fm9wkgjsk9onc08xdnlbxk9zmh9s)
