Hi everyone, As mentioned in the latest Iceberg sync, I'd love to do another Python release. I would like to reach out to the community to see if there is anything we should include.
I know that it was promised that the next release would have to write support, but so many features have already accumulated, and we want don't want to rush write support, I would suggest doing a release in between to get it out to the public. A summary of what's on master: - Add gzip metadata support <https://github.com/apache/iceberg/pull/7984> - PyArrow HDFS support <https://github.com/apache/iceberg/pull/7997> - Support serverless environments (AWS Lambda) <https://github.com/apache/iceberg/pull/8061> - Many fixes around Avro performance (PRs 1 <https://github.com/apache/iceberg/pull/8074>, 2 <https://github.com/apache/iceberg/pull/8075>, 3 <https://github.com/apache/iceberg/pull/8082>, 4 <https://github.com/apache/iceberg/pull/8084>) - Remove the upper bound of PyParsing dependency <https://github.com/apache/iceberg/pull/8116> (blocking a PR in Airflow <https://github.com/apache/airflow/pull/32786>) - Moving the reading of Avro to Cython <https://github.com/apache/iceberg/pull/8134> (10x speed improvement(!)) - Support for the SQLCatalog <https://github.com/apache/iceberg/pull/7921> (JDBC in Java) - Fix support for UUID columns <https://github.com/apache/iceberg/pull/8267> - A lot of bugfixes! What I think we should include (but please feel free to add anything to this list): - Support for adding columns <https://github.com/apache/iceberg/pull/8174> - Optimize concurrency <https://github.com/apache/iceberg/pull/8104> (follow up on the Support servless environments) - Bump Pydantic to v2 <https://github.com/apache/iceberg/pull/7782> (this will unblock PyIceberg Polars integration <https://github.com/pola-rs/polars/pull/10375>, and improve performance of the JSON (de)serialization) I've added those to the 0.5.0 milestone <https://github.com/apache/iceberg/milestone/30>. Feel free to reach out if you think anything is missing, otherwise we can proceed with the release. Also, everyone is invited to review the open PR's :) Cheers, Fokko
