[ https://issues.apache.org/jira/browse/HIVE-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057875#comment-17057875 ]
Panagiotis Garefalakis commented on HIVE-23014: ----------------------------------------------- [~petertoth] thanks for reporting this. It seems that the particular regression is observed on Single Column Scans >From Wide Columns: [https://github.com/apache/spark/blob/f5118f81e395bde0cd8253dbef6a9e6455c3958a/sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala#L267] Did you observe similar ReaderImpl behaviour in other cases? > ORC reading performance > ----------------------- > > Key: HIVE-23014 > URL: https://issues.apache.org/jira/browse/HIVE-23014 > Project: Hive > Issue Type: Bug > Components: ORC > Affects Versions: 2.3.6 > Reporter: Peter Toth > Priority: Major > > Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version. > Some of the ORC reading benchmark shows that there is a huge performance > difference in ORC reading between the 2 versions. I measured that > {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar > is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar. > I'm not sure if more recent Hive versions still suffer from this performance > regression. > Please see some details here: SPARK-30565 -- This message was sent by Atlassian Jira (v8.3.4#803005)