[ 
https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171868#comment-17171868
 ] 

Muhammad Samir Khan commented on SPARK-32550:
---------------------------------------------

[~maropu] added some benchmark results in 
[https://github.com/apache/spark/pull/29366]. For Avro, with a couple of 
benchmarks I am proposing to add in SPARK-32531 (PR: 
[https://github.com/apache/spark/pull/29352]) I get the following on master 
branch:



*Before:*
Nested Struct 75s average
Array of Structs 34s average

*After:*
Nested Struct 49s average
Array of Structs 19s average

> Make SpecificInternalRow constructors faster by using while loops instead of 
> maps
> ---------------------------------------------------------------------------------
>
>                 Key: SPARK-32550
>                 URL: https://issues.apache.org/jira/browse/SPARK-32550
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Muhammad Samir Khan
>            Priority: Major
>
> Two constructors in SpecificInternalRow can be made faster by using while 
> loops instead of maps. This was originally noticed while working on 
> SPARK-32532 and SPARK-32533 and will have impacts on performance of reading 
> ORC and Avro files. Profiled the change using added benchmarks in SPARK-32531 
> for nested and array'd structs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to