singhpk234 opened a new pull request, #7531:
URL: https://github.com/apache/iceberg/pull/7531
### About the change
presently when using brotli as compression codec for parquet it fails with
```
Caused by: org.apache.parquet.hadoop.BadConfigurationException: Class
org.apache.hadoop.io.compress.BrotliCodec was not found
at
org.apache.parquet.hadoop.CodecFactory.getCodec(CodecFactory.java:243)
at
org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.<init>(CodecFactory.java:144)
at
org.apache.parquet.hadoop.CodecFactory.createCompressor(CodecFactory.java:208)
at
org.apache.parquet.hadoop.CodecFactory.getCompressor(CodecFactory.java:191)
```
this just makes sure BrotliCodec is available in class path so that it can
be loaded and doesn't fails with the error above.
Alternatively we could also mention in iceberg to install BrotliCodec when
selecting brotli as the compression, somewhat spark does
(https://spark.apache.org/docs/latest/sql-data-sources-parquet.html, ref:
brotli requires BrotliCodec to be installed) rather than taking a new
dependency (this repo for BrotliCodec was created by ryan himself :) ) .
cc @rdblue @jackye1995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
