[ https://issues.apache.org/jira/browse/PARQUET-2222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730992#comment-17730992 ]
Xuwei Fu commented on PARQUET-2222: ----------------------------------- [~gszadovszky] Hi Gabor maybe I'm misleading. 1. RLE for Boolean is just implemented a few month ago by me. I guess Legacy writer cannot write use RLE. 2. For RL/DL, in parquet-cpp, only RLE can be used. In arrow-rs, I found the code below {code} /// RLE/Bit-Packing hybrid encoding for values. /// Currently is used only for data pages v2 and supports boolean types. pub struct RleValueEncoder<T: DataType> { // Buffer with raw values that we collect, // when flushing buffer they are encoded using RLE encoder encoder: Option<RleEncoder>, _phantom: PhantomData<T>, } {code} I guess it only supports Page V2 and format V2. [~tustvold] am I right? > [Format] RLE encoding spec incorrect for v2 data pages > ------------------------------------------------------ > > Key: PARQUET-2222 > URL: https://issues.apache.org/jira/browse/PARQUET-2222 > Project: Parquet > Issue Type: Bug > Components: parquet-format > Reporter: Antoine Pitrou > Assignee: Gang Wu > Priority: Critical > Fix For: format-2.10.0 > > > The spec > (https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3) > has this: > {code} > rle-bit-packed-hybrid: <length> <encoded-data> > length := length of the <encoded-data> in bytes stored as 4 bytes little > endian (unsigned int32) > {code} > But the length is actually prepended only in v1 data pages, not in v2 data > pages. -- This message was sent by Atlassian Jira (v8.20.10#820010)