Thank you, Wes. What about adding separate LZ4_BLOCK and LZ4_FRAME definitions in parquet-format, or at least tightening up the specification?
And is there any motivation in JVM parquet implementations to clear this up? michael > On Oct 23, 2019, at 5:28 PM, Wes McKinney <[email protected]> wrote: > > We are waiting on a volunteer to come forward and sort out the proper > implementation in Parquet C++ for LZ4. It is disabled in the meantime > I think > > On Wed, Oct 23, 2019 at 10:11 AM Michael Heuer <[email protected]> wrote: >> >> Hello, >> >> There are a few issues related to block-vs-frame LZ4 compression in >> Parquet/Arrow and related >> >> https://issues.apache.org/jira/browse/PARQUET-1241 >> <https://issues.apache.org/jira/browse/PARQUET-1241> >> https://issues.apache.org/jira/browse/PARQUET-1515 >> <https://issues.apache.org/jira/browse/PARQUET-1515> >> https://issues.apache.org/jira/browse/HADOOP-12990 >> <https://issues.apache.org/jira/browse/HADOOP-12990> >> https://github.com/dask/fastparquet/pull/315 >> <https://github.com/dask/fastparquet/pull/315> >> https://issues.apache.org/jira/browse/IMPALA-8617 >> <https://issues.apache.org/jira/browse/IMPALA-8617> >> >> Might anyone be able to summarize the current status and take a guess at the >> way forward? >> >> Thank you in advance, >> >> michael
