[ https://issues.apache.org/jira/browse/SPARK-29033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Rosen resolved SPARK-29033. -------------------------------- Resolution: Won't Fix I'm resolving this as "Won't Fix" because we ended up doing the opposite in SPARK-29503, removing CreateNamedStructUnsafe because its sole use was associated with a correctness bug. > Always use CreateNamedStructUnsafe, the UnsafeRow-based version of the > CreateNamedStruct codepath > ------------------------------------------------------------------------------------------------- > > Key: SPARK-29033 > URL: https://issues.apache.org/jira/browse/SPARK-29033 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.0.0 > Reporter: Josh Rosen > Assignee: Josh Rosen > Priority: Major > > Spark 2.x has two separate implementations of the "create named struct" > expression: regular {{CreateNamedStruct}} and {{CreateNamedStructUnsafe}}. > The "unsafe" version was added in SPARK-9373 to support structs in > {{GenerateUnsafeProjection}}. These two expressions both extend the > {{CreateNameStructLike}} trait. > For Spark 3.0, I propose to always use the "unsafe" code path: this will > avoid object allocation / boxing inefficiencies in the "safe" codepath, which > is an especially big problem when generating Encoders for deeply-nested case > classes. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org