On Tue, 29 Jun 2021 17:36:58 -0700 Micah Kornfield <[email protected]> wrote: > Unless someone has implemented the corresponding changes in Parquet-MR it > will not be compatible with hadoop (I haven't been paying close attention > but I don't recall seeing a PR adding support for parquet-mr).
Indeed, a JIRA is open about that: https://issues.apache.org/jira/browse/PARQUET-2032 I would encourage anyone interested to try and contribute the LZ4_RAW support in parquet-mr. I don't know how involved that is (ideally it should be relatively easy, but that depends on the state of Java compression libraries, which seems to be a thorny topic). Regards Antoine.
