[
https://issues.apache.org/jira/browse/SPARK-49188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-49188.
---------------------------------
Fix Version/s: 4.0.0
Assignee: Nikola Mandic
Resolution: Fixed
> Internal error on concat_ws called on array of arrays of string
> ---------------------------------------------------------------
>
> Key: SPARK-49188
> URL: https://issues.apache.org/jira/browse/SPARK-49188
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Nikola Mandic
> Assignee: Nikola Mandic
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> Applying the following sequence of queries in *ANSI mode*
> (spark.sql.ansi.enabled=true):
> {code:java}
> create table test_table(dat array<string>) using parquet;
> select concat_ws(',', collect_list(dat)) FROM test_table; {code}
> yields an internal error:
> {code:java}
> org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase
> analysis failed with an internal error. You hit a bug in Spark or the Spark
> plugins you use. Please, report this bug to the corresponding communities or
> vendors, and provide the full stack trace. SQLSTATE: XX000
> ...
> Caused by: java.lang.NullPointerException: Cannot invoke
> "org.apache.spark.sql.types.AbstractDataType.simpleString()" because
> "<local3>" is null
> at
> org.apache.spark.sql.errors.DataTypeErrorsBase.toSQLType(DataTypeErrorsBase.scala:55)
> at
> org.apache.spark.sql.errors.DataTypeErrorsBase.toSQLType$(DataTypeErrorsBase.scala:51)
> at org.apache.spark.sql.catalyst.expressions.Cast$.toSQLType(Cast.scala:44)
> at
> org.apache.spark.sql.catalyst.expressions.Cast$.typeCheckFailureMessage(Cast.scala:426)
> at
> org.apache.spark.sql.catalyst.expressions.Cast.typeCheckFailureInCast(Cast.scala:501)
> at
> org.apache.spark.sql.catalyst.expressions.Cast.checkInputDataTypes(Cast.scala:525)
> at
> org.apache.spark.sql.catalyst.expressions.Cast.resolved$lzycompute(Cast.scala:551)
> at org.apache.spark.sql.catalyst.expressions.Cast.resolved(Cast.scala:550)
> ... {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
