Josh Rosen created SPARK-29033:
----------------------------------
Summary: Always use CreateNamedStructUnsafe codepath
Key: SPARK-29033
URL: https://issues.apache.org/jira/browse/SPARK-29033
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0
Reporter: Josh Rosen
Assignee: Josh Rosen
Spark 2.x has two separate implementations of the "create named struct"
expression: regular {{CreateNamedStruct}} and {{CreateNamedStructUnsafe}}. The
"unsafe" version was added in SPARK-9373 to support structs in
{{GenerateUnsafeProjection}}. These two expressions both extend the
{{CreateNameStructLike}} trait.
For Spark 3.0, I propose to always use the "unsafe" code path: this will avoid
object allocation / boxing inefficiencies in the "safe" codepath, which is an
especially big problem when generating Encoders for deeply-nested structs.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
