[ https://issues.apache.org/jira/browse/SPARK-29595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961783#comment-16961783 ]
Aman Omer edited comment on SPARK-29595 at 10/29/19 8:13 AM: ------------------------------------------------------------- {color:#172b4d}Parameters required for named_struct is Seq(name1, val1, name2, val2, ...). Validation step for named_struct only check for string at odd places. For example following query will add a row in _str_ table.{color}{color} {code:java} insert into str values named_struct( "ab", 1, "ba", 2);{code} According to the discussion in [https://github.com/apache/spark/pull/26275] , which was tackling similar issue, changing fields of struct type according to names will introduce complexity. So I think Spark should throw an exception when names does not match in named_struct. cc [~srowen] [~maropu] was (Author: aman_omer): Parameters required for named_struct is _{color:#172b4d}Seq(name1, val1, name2, val2, ...){color}_{color:#808080}{color:#172b4d}. Validation step for named_struct only check for string at odd places. For example following query will add a row in _str_ table.{color} {color} {code:java} insert into str values named_struct( "ab", 1, "ba", 2);{code} According to the discussion in [https://github.com/apache/spark/pull/26275] , which was tackling similar issue, changing fields of struct type according to names will introduce complexity. So I think Spark should throw an exception when names does not match in named_struct. cc [~srowen] [~maropu] > Insertion with named_struct should match by name > ------------------------------------------------ > > Key: SPARK-29595 > URL: https://issues.apache.org/jira/browse/SPARK-29595 > Project: Spark > Issue Type: Task > Components: SQL > Affects Versions: 3.0.0 > Reporter: Gengliang Wang > Priority: Major > > {code:java} > spark-sql> create table str using parquet as(select named_struct('a', 1, 'b', > 2) as data); > spark-sql> insert into str values named_struct("b", 3, "a", 1); > spark-sql> select * from str; > {"a":3,"b":1} > {"a":1,"b":2} > {code} > The result should be > {code:java} > {"a":1,"b":3} > {"a":1,"b":2} > {code} > Spark should match the field names of named_struct on insertion -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org