[ https://issues.apache.org/jira/browse/SPARK-34863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528454#comment-17528454 ]
Apache Spark commented on SPARK-34863: -------------------------------------- User 'sadikovi' has created a pull request for this issue: https://github.com/apache/spark/pull/36366 > Support nested column in Spark Parquet vectorized readers > --------------------------------------------------------- > > Key: SPARK-34863 > URL: https://issues.apache.org/jira/browse/SPARK-34863 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.2.0 > Reporter: Cheng Su > Assignee: Chao Sun > Priority: Minor > Fix For: 3.3.0 > > > The task is to support nested column type in Spark Parquet vectorized reader. > Currently Parquet vectorized reader does not support nested column type > (struct, array and map). We implemented nested column vectorized reader for > FB-ORC in our internal fork of Spark. We are seeing performance improvement > compared to non-vectorized reader when reading nested columns. In addition, > this can also help improve the non-nested column performance when reading > non-nested and nested columns together in one query. > > Parquet: > [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala#L173] -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org