[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17171868#comment-17171868 ]
Muhammad Samir Khan commented on SPARK-32550: --------------------------------------------- [~maropu] added some benchmark results in [https://github.com/apache/spark/pull/29366]. For Avro, with a couple of benchmarks I am proposing to add in SPARK-32531 (PR: [https://github.com/apache/spark/pull/29352]) I get the following on master branch: *Before:* Nested Struct 75s average Array of Structs 34s average *After:* Nested Struct 49s average Array of Structs 19s average > Make SpecificInternalRow constructors faster by using while loops instead of > maps > --------------------------------------------------------------------------------- > > Key: SPARK-32550 > URL: https://issues.apache.org/jira/browse/SPARK-32550 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.0.0 > Reporter: Muhammad Samir Khan > Priority: Major > > Two constructors in SpecificInternalRow can be made faster by using while > loops instead of maps. This was originally noticed while working on > SPARK-32532 and SPARK-32533 and will have impacts on performance of reading > ORC and Avro files. Profiled the change using added benchmarks in SPARK-32531 > for nested and array'd structs. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org