Iceberg Delete Compaction Interface Design

Hi everyone, As there are more and more people adopting the v2 spec, we are seeing an increasing number of requests for delete compaction support. Here is a document discussing the use cases and basic interface design for it to get the community aligned around what compactions we would offer and

Re: [DISCUSS] Spark version support strategy

Hi OpenInx, I'm sorry I misunderstood the thinking of the Flink community. Thanks for the clarification. - Wing Yew On Tue, Sep 28, 2021 at 7:15 PM OpenInx wrote: > Hi Wing > > As we discussed above, we community prefer to choose option.2 or > option.3. So in fact, when we planned to upgrade t

Re: [DISCUSS] Spark version support strategy

Hi Wing As we discussed above, we community prefer to choose option.2 or option.3. So in fact, when we planned to upgrade the flink version from 1.12 to 1.13, we are doing our best to guarantee the master iceberg repo could work fine for both flink1.12 & flink1.13. More context please see [1], [2

Re: Iceberg disaster recovery and relative path sync-up

Also, in S3’s case, my understanding is that instead of write.object-storage.path/write.data.path, users must now make sure that the location prefix must be short to get the benefits of appending a hash to the data paths. For example, a large prefix like "s3://somebucket/region/timestamp/folder

Re: [DISCUSS] Spark version support strategy

In the last community sync, we spent a little time on this topic. For Spark support, there are currently two options under consideration: Option 2: Separate repo for the Spark support. Use branches for supporting different Spark versions. Main branch for the latest Spark version (3.2 to begin with

Re: Proposal: Support for views in Iceberg

Hi All, Please see the spec in markdown format at the PR here to facilitate adding/responding to comments. Please review. thanks, Anjali On Tue, Sep 7, 2021 at 9:31 PM Jack Ye wrote: > Hi everyone, > > I have been thinking about the view support du

Fwd: Data encryption in Iceberg

Hi all, The encryption sync is set for next Tue, October 5, at 9am PDT. Additional folks interested to join - let me know in a direct message. Cheers, Gidon -- Forwarded message - From: Gidon Gershinsky Date: Wed, Sep 1, 2021 at 9:10 PM Subject: Fwd: Data encryption in Iceberg