> On May 9, 2018, 8:43 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java > > Lines 1076 (patched) > > <https://reviews.apache.org/r/67039/diff/1/?file=2018739#file2018739line1089> > > > > Positions are packed variable in protobuf. > > Also there is stream suppression. > > Same column in a stripe may have a stream suppressed (say all non-nulls > > values, isPresent gets suppressed) then its positions will not be recorded. > > Same column in another stripe might have all streams. So all this does is > > if the stream does not exist don't move on to next position. > > > > [[0,0,0],[0,0],[0,0]] -> positions for isPresent, Data, Length.. > > [[0,0],[0,0]] -> positions for Data, Length.. (isPresent suppressed) > > Sergey Shelukhin wrote: > I'm not sure what you mean... I'm moving this code outside of the > 0-length check, and moving into utility method. > All calls of the same method did the exact same thing before. > > Prasanth_J wrote: > > https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/impl/writer/TreeWriterBase.java#L248-L256 > > If a stream is suppressed, its positions are removed. > > So I am not sure if it is safe to advance the positions without checking > if the stream exists/available or not.
Actually we remove the positions only for isPresent streams. I think it should be ok if we advance positions first before seek for other streams. - Prasanth_J ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67039/#review202793 ----------------------------------------------------------- On May 9, 2018, 8:55 p.m., Sergey Shelukhin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/67039/ > ----------------------------------------------------------- > > (Updated May 9, 2018, 8:55 p.m.) > > > Review request for hive and Prasanth_J. > > > Repository: hive-git > > > Description > ------- > > see jira > > > Diffs > ----- > > > llap-server/src/java/org/apache/hadoop/hive/llap/io/decode/OrcEncodedDataConsumer.java > fc0c66a888 > ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedReaderImpl.java > 1d7eceb1ef > > ql/src/java/org/apache/hadoop/hive/ql/io/orc/encoded/EncodedTreeReaderFactory.java > 42532f9a0e > > > Diff: https://reviews.apache.org/r/67039/diff/2/ > > > Testing > ------- > > > Thanks, > > Sergey Shelukhin > >
