[
https://issues.apache.org/jira/browse/SPARK-29033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen resolved SPARK-29033.
--------------------------------
Resolution: Won't Fix
I'm resolving this as "Won't Fix" because we ended up doing the opposite in
SPARK-29503, removing CreateNamedStructUnsafe because its sole use was
associated with a correctness bug.
> Always use CreateNamedStructUnsafe, the UnsafeRow-based version of the
> CreateNamedStruct codepath
> -------------------------------------------------------------------------------------------------
>
> Key: SPARK-29033
> URL: https://issues.apache.org/jira/browse/SPARK-29033
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Josh Rosen
> Assignee: Josh Rosen
> Priority: Major
>
> Spark 2.x has two separate implementations of the "create named struct"
> expression: regular {{CreateNamedStruct}} and {{CreateNamedStructUnsafe}}.
> The "unsafe" version was added in SPARK-9373 to support structs in
> {{GenerateUnsafeProjection}}. These two expressions both extend the
> {{CreateNameStructLike}} trait.
> For Spark 3.0, I propose to always use the "unsafe" code path: this will
> avoid object allocation / boxing inefficiencies in the "safe" codepath, which
> is an especially big problem when generating Encoders for deeply-nested case
> classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
