[ https://issues.apache.org/jira/browse/SPARK-34714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450236#comment-17450236 ]
Manu Zhang commented on SPARK-34714: ------------------------------------ FYI, this was resolved by https://issues.apache.org/jira/browse/SPARK-34713 and https://issues.apache.org/jira/browse/SPARK-34749 > collect_list(struct()) fails when used with GROUP BY > ---------------------------------------------------- > > Key: SPARK-34714 > URL: https://issues.apache.org/jira/browse/SPARK-34714 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.1 > Environment: Databricks Runtime 8.0 > Reporter: Lauri Koobas > Priority: Major > Fix For: 3.1.2 > > > The following is failing in DBR8.0 / Spark 3.1.1, but works in earlier DBR > and Spark versions: > {quote}with step_1 as ( > select 'E' as name, named_struct('subfield', 1) as field_1 > ) > select name, collect_list(struct(field_1.subfield)) > from step_1 > group by 1 > {quote} > Fails with the following error message: > {quote}AnalysisException: cannot resolve > 'struct(step_1.`field_1`.`subfield`)' due to data type mismatch: Only > foldable string expressions are allowed to appear at odd position, got: > NamePlaceholder > {quote} > If you modify the query in any of the following ways then it still works:: > * if you remove the field "name" and the "group by 1" part of the query > * if you remove the "struct()" from within the collect_list() > * if you use "named_struct()" instead of "struct()" within the collect_list() > Similarly collect_set() is broken and possibly more related functions, but I > haven't done thorough testing. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org