Re: Welcoming Yan Yan as a new committer!

Congratulations, Yan! Best, Yufei `This is not a contribution` On Tue, Mar 23, 2021 at 8:44 PM Russell Spitzer wrote: > Congratulations! > > On Mar 23, 2021, at 9:35 PM, OpenInx wrote: > > Congrats Yan ! You deserve it. > > On Wed, Mar 24, 2021 at 7:18 AM Miao Wang > wrote: > >> Congrats

Re: Welcoming Yan Yan as a new committer!

Congratulations! > On Mar 23, 2021, at 9:35 PM, OpenInx wrote: > > Congrats Yan ! You deserve it. > > On Wed, Mar 24, 2021 at 7:18 AM Miao Wang wrote: > Congrats @Yan Yan ! > > > > Miao > > > > From: Ryan Blue mailto:b...@apache.org>> > Reply-To: "dev@icebe

Re: Welcoming Yan Yan as a new committer!

Congrats Yan ! You deserve it. On Wed, Mar 24, 2021 at 7:18 AM Miao Wang wrote: > Congrats @Yan Yan ! > > > > Miao > > > > *From: *Ryan Blue > *Reply-To: *"dev@iceberg.apache.org" > *Date: *Tuesday, March 23, 2021 at 3:43 PM > *To: *Iceberg Dev List > *Subject: *Welcoming Yan Yan as a new c

Re: When is the next release of Iceberg ?

Hi Himanshu Thanks for the email, currently we flink+iceberg support writing CDC events into apache iceberg table by flink datastream API, besides the spark/presto/hive could read those events in batch job. But there are still some issues that we do not finish yet: 1. Expose the iceberg v2 to

Re: Single Reader Benchmarks on S3-like Storage

You can use S3FileIO with any catalog implementation including HadoopCatalog and HiveCatalog by setting the io-impl catalog property. Detail is described in https://iceberg.apache.org/custom-catalog/#custom-file-io-implementation It would be very interesting to see how it performs versus HadoopFil

RE: Single Reader Benchmarks on S3-like Storage

Dan, thanks for getting back to me! I’ve not experimented with S3FileIO, and you are right, I’m using HadoopFileIO through HadoopTables. I’ve seen some example usage of S3FileIO is the glue catalog implementation. Are there other catalogs that support S3FileIO? The in-memory implementation is j

Re: Welcoming Yan Yan as a new committer!

Congrats @Yan Yan! Miao From: Ryan Blue Reply-To: "dev@iceberg.apache.org" Date: Tuesday, March 23, 2021 at 3:43 PM To: Iceberg Dev List Subject: Welcoming Yan Yan as a new committer! Hi everyone, I'd like to welcome Yan Yan as a new Iceberg committer. Thanks for

Re: Single Reader Benchmarks on S3-like Storage

Hey Mayur, thanks for the detailed writeup. I would say that what you're looking at in terms of performance is very specific to the file system implementation (like you've already discovered by replacing the GHFS implementation). Within iceberg, this is scoped very specifically to the FileIO impl

Welcoming Yan Yan as a new committer!

Hi everyone, I'd like to welcome Yan Yan as a new Iceberg committer. Thanks for all your contributions, Yan! rb -- Ryan Blue

Re: Extending Apache Iceberg Encryption Module

Thanks for the feedback to the doc, we are also closely following the Parquet encryption work and would like to have that in Iceberg as soon as possible with the right architecture. Here are some brief thoughts for the points you mentioned in the email, I will add more details in the google doc:

Single Reader Benchmarks on S3-like Storage

Hi, I've been running performance benchmarks on core Iceberg readers on Google Cloud Storage (GCS). I would like to share some of my results and check whether there are ways to improve performance on S3-like storage in general. The details (including sample code) are listed below the question

When is the next release of Iceberg ?

We are planning for use Flink + Iceberg for syncing mysql binlog's via debezium and its seams of things are dependent on next release.

Re: Extending Apache Iceberg Encryption Module

Hi Jack, We're working on Parquet encryption, which is about to be released in the upcoming parquet-mr-1.12 version. Recently, we've started to look into its integration in Iceberg. It became immediately clear we need to take a wider view that covers other types of encryption in Iceberg (file stre