Re: [DISCUSSION] Preparing the Apache iceberg-rust 0.3.0 release

Hi, JB I'm not familiar with the Avro community. Do they release all packages including rust together, or do they release them separately? I have noticed that current apache-avro's version is v0.16.0 and the next one will be v0.17.0? On Thu, Jun 20, 2024, at 14:52, Jean-Baptiste Onofré wrote:

Re: [DISCUSSION] Preparing the Apache iceberg-rust 0.3.0 release

Hi Renjie As discussed together, we are working on Avro 1.11.4 and 1.12.0 releases. We are pretty close now, I think we can reasonably plan the releases to vote next week. I will keep you posted about that. Regards JB On Thu, Jun 20, 2024 at 8:37 AM Renjie Liu wrote: > > Hi, all: > >> we just

Re: [PROPOSAL] New REST Catalog Spec

+1 for TCK and a reference implementation. Once the reference implementation is ready, I suggest we also update the below quickstart to use it. https://iceberg.apache.org/spark-quickstart/?h=tabular#spark-and-iceberg-quickstart In my opinion, we should avoid directing open-source users to closed-s

Re: [DISCUSSION] Preparing the Apache iceberg-rust 0.3.0 release

Hi, all: we just have to make sure that we're not able to write metadata without > field-IDs because that would violate the spec (and cause potentially > compatibility issues down the road). We need to wait for a while for the avro community's release to fix it, but it's not expected to be too l

Re: Iceberg python library sync

Hi everyone, FYI, the next community python library sync meeting will be on Tuesday (06/25) at 9 AM (US/Pacific). Here is the meeting agenda for the topics to discuss: https://docs.google.com/document/d/1oMKodaZJrOJjPfc8PDVAoTdl02eGQKHlhwuggiw7s9U/edit?pli=1#bookmark=id.3gwejswztnw5. Please feel f

Re: [PROPOSAL] New REST Catalog Spec

Hi Dmitri Yes, it makes sense to me. As discussed today during the community meeting, where the "simple ref impl"/TCK are located is not a big deal. But, I think it would be great to actually have sref impl/TCK (in a separate repo or not). Regards JB On Wed, Jun 19, 2024 at 7:08 PM Dmitri Bourl

Re: [DISCUSSION] Preparing the Apache iceberg-rust 0.3.0 release

+1 all in for a release looking at the amount of great features that are staged. Thank you Renjie and Xuanwo for attending the community sync, especially looking at your time zone. Renjie raised the issue of not having field-IDs based resolution when reading Avro. I think we're fine for leaving th

Iceberg MV Refresh

Hey Guys, Great progress on the MV spec and thanks a ton to Jan and Walaa for driving this. One of our latest achievements was that we finalized the view lineage and materialization table refresh JSON so that we can definitively and concisely describe what data is in the materialization table. R

Re: [PROPOSAL] New REST Catalog Spec

Re: TCK and ref impl in the "spec" repo. I believe the reference implementation there does not have to be complete in the sense of actually managing Iceberg metadata. It only needs to be complete enough to allow TCK tests to pass. TCKs will be the normative checks for "real" catalog implementations

Re: [PROPOSAL] New REST Catalog Spec

Hi folks, I wanted to provide an update on this proposal and also extend discussion. I started to rework on the REST Catalog proposal by splitting in v1-improvements and v2-proposal (focusing on extensions), step by step. I would like to emphasize a few proposals from the doc: 1. In order to gua

Re: Agenda Community Sync 19th June

Hey everyone, Thanks for the input. I've collected everything in the notes , feel free to do suggestions or edits. Thanks Brian for running the recording. Looking forward to seeing everyone later today! Kind regards

Re: Agenda Community Sync 19th June

Hi Brian, Thanks ! See you later today then. Regards JB On Wed, Jun 19, 2024 at 3:59 PM Brian Olsen wrote: > > Hey all! > > So I just spoke with Fokko. I’ll be happy to hop on to continue the > recordings and I still owe you all some sync notes from the last few meetings > (those are still co

Re: Agenda Community Sync 19th June

Hey all! So I just spoke with Fokko. I’ll be happy to hop on to continue the recordings and I still owe you all some sync notes from the last few meetings (those are still coming). I’m not sure if Ryan will be joining given the holiday but if anything Fokko will be back up. On Wed, Jun 19, 2024

Re: Agenda Community Sync 19th June

Hi, all: I want to share progress about iceberg-rust and discuss about 0.3.0 release. On Wed, Jun 19, 2024 at 9:07 PM Jean-Baptiste Onofré wrote: > Hi Jan, > > Thanks for your message. > > The document has been updated. > > I will provide two major updates from my side: > 1. Gradle update and r

[DISCUSSION] Addressing security questions in the Iceberg REST specification

Alright, I've just created the issue: https://github.com/apache/iceberg/issues/10537 The proposal document is here: https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ/ Robert On 29.05.24 22:01, Robert Stupp wrote: Jack's proposal is pretty much along what we'v

Re: Proposal for REST APIs for Iceberg table scans

Hi Ryan, You are right: I can't access the document either. AFAIR, Jack did the doc, he will fix that soon I'm sure. Regards JB On Wed, Jun 19, 2024 at 1:17 AM Ryan Blue wrote: > It looks like the design doc from the original email is no longer > available. Could someone fix the permissions? >

Re: Agenda Community Sync 19th June

Hi Jan, Thanks for your message. The document has been updated. I will provide two major updates from my side: 1. Gradle update and revapi (both alternative and plugin fix) 2. Iceberg Java 1.6.0 release preparation (including some dependency updates) As it's Juneteenth today, if US part of the