[
https://issues.apache.org/jira/browse/SPARK-34862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308504#comment-17308504
]
Cheng Su commented on SPARK-34862:
----------------------------------
Just FYI I am working on it now.
> Support nested column in Spark ORC vectorized readers
> -----------------------------------------------------
>
> Key: SPARK-34862
> URL: https://issues.apache.org/jira/browse/SPARK-34862
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Cheng Su
> Priority: Minor
>
> The task is to support nested column type in Spark ORC vectorized reader.
> Currently ORC vectorized reader does not support nested column type (struct,
> array and map). We implemented nested column vectorized reader for FB-ORC in
> our internal fork of Spark. We are seeing performance improvement compared to
> non-vectorized reader when reading nested columns. In addition, this can also
> help improve the non-nested column performance when reading non-nested and
> nested columns together in one query.
>
> ORC:
> [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala#L138]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
