[jira] [Commented] (SPARK-16060) Vectorized ORC reader
[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16841949#comment-16841949 ] Atif Sharif commented on SPARK-16060: - Thanks for this update. You can visit my website [CDHPL|https://www.cdhpl.com] for tech posts. > Vectorized ORC reader > - > > Key: SPARK-16060 > URL: https://issues.apache.org/jira/browse/SPARK-16060 > Project: Spark > Issue Type: New Feature > Components: SQL >Affects Versions: 1.6.3, 2.0.2, 2.1.2, 2.2.1 >Reporter: Liang-Chi Hsieh >Assignee: Dongjoon Hyun >Priority: Major > Labels: release-notes, releasenotes > Fix For: 2.3.0 > > > Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive > Orc already support vectorization, we should add this support to improve Orc > reading performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16060) Vectorized Orc reader
[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318658#comment-16318658 ] Apache Spark commented on SPARK-16060: -- User 'cloud-fan' has created a pull request for this issue: https://github.com/apache/spark/pull/20205 > Vectorized Orc reader > - > > Key: SPARK-16060 > URL: https://issues.apache.org/jira/browse/SPARK-16060 > Project: Spark > Issue Type: New Feature > Components: SQL >Reporter: Liang-Chi Hsieh >Assignee: Dongjoon Hyun > Fix For: 2.3.0 > > > Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive > Orc already support vectorization, we should add this support to improve Orc > reading performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16060) Vectorized Orc reader
[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286695#comment-16286695 ] Apache Spark commented on SPARK-16060: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/19943 > Vectorized Orc reader > - > > Key: SPARK-16060 > URL: https://issues.apache.org/jira/browse/SPARK-16060 > Project: Spark > Issue Type: New Feature > Components: SQL >Reporter: Liang-Chi Hsieh > > Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive > Orc already support vectorization, we should add this support to improve Orc > reading performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16060) Vectorized Orc reader
[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16203064#comment-16203064 ] Rajiv Chodisetti commented on SPARK-16060: -- Will this fix, this issue the below issue as well , where reading ORC metadata reading is very slow when input data storage is S3 ? https://forums.databricks.com/questions/12499/parquet-vs-orc-s3-metadata-read-performance.html > Vectorized Orc reader > - > > Key: SPARK-16060 > URL: https://issues.apache.org/jira/browse/SPARK-16060 > Project: Spark > Issue Type: New Feature > Components: SQL >Reporter: Liang-Chi Hsieh > > Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive > Orc already support vectorization, we should add this support to improve Orc > reading performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16060) Vectorized Orc reader
[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937711#comment-15937711 ] Liang-Chi Hsieh commented on SPARK-16060: - cc [~rxin] If the approach based on Hive package is not ok for you, shall we directly go to make presto's ORC reader work for Spark SQL? > Vectorized Orc reader > - > > Key: SPARK-16060 > URL: https://issues.apache.org/jira/browse/SPARK-16060 > Project: Spark > Issue Type: New Feature > Components: SQL >Reporter: Liang-Chi Hsieh > > Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive > Orc already support vectorization, we should add this support to improve Orc > reading performance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16060) Vectorized Orc reader
[ https://issues.apache.org/jira/browse/SPARK-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338925#comment-15338925 ] Apache Spark commented on SPARK-16060: -- User 'viirya' has created a pull request for this issue: https://github.com/apache/spark/pull/13775 > Vectorized Orc reader > - > > Key: SPARK-16060 > URL: https://issues.apache.org/jira/browse/SPARK-16060 > Project: Spark > Issue Type: New Feature > Components: SQL >Reporter: Liang-Chi Hsieh > > Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive > Orc already support vectorization, we should add this support to improve Orc > reading performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
