[ https://issues.apache.org/jira/browse/DRILL-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419544#comment-16419544 ]
ASF GitHub Bot commented on DRILL-5846: --------------------------------------- Github user sachouche commented on the issue: https://github.com/apache/drill/pull/1060 I have updated this pull request with the following changes: - Excluded the implicit column optimizations from this pull request (will be included as part of another Drill Jira) - Tuned a threshold (16bytes vs 1024) on when to use "putByte() vs copyMemory()" based on @vrozov findings when benchmarking using JMH - There were lingering questions on the use of byte arrays for intermediary processing; after a discussion with @vrozov we agreed to tackle such an analysis within a new JIRA ([DRILL-6301](https://issues.apache.org/jira/browse/DRILL-6301?filter=-1)) so that the Drill customer can immediately rip the benefits of the current enhancements - Moved the newly added MemoryUtils class to the Parquet flat reader project (made it package private); this is temporary till DRILL-6301 is resolved. > Improve Parquet Reader Performance for Flat Data types > ------------------------------------------------------- > > Key: DRILL-5846 > URL: https://issues.apache.org/jira/browse/DRILL-5846 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Parquet > Affects Versions: 1.11.0 > Reporter: salim achouche > Assignee: salim achouche > Priority: Major > Labels: performance > Fix For: 1.14.0 > > > The Parquet Reader is a key use-case for Drill. This JIRA is an attempt to > further improve the Parquet Reader performance as several users reported that > Parquet parsing represents the lion share of the overall query execution. It > tracks Flat Data types only as Nested DTs might involve functional and > processing enhancements (e.g., a nested column can be seen as a Document; > user might want to perform operations scoped at the document level that is no > need to span all rows). Another JIRA will be created to handle the nested > columns use-case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)