[ https://issues.apache.org/jira/browse/SPARK-20912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028265#comment-16028265 ]
Sean Owen commented on SPARK-20912: ----------------------------------- Can't you just write $"k" instead of "k" to get this behavior? > map function with columns as strings > ------------------------------------ > > Key: SPARK-20912 > URL: https://issues.apache.org/jira/browse/SPARK-20912 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.3.0 > Reporter: Jacek Laskowski > Priority: Trivial > > There's only {{map}} function that accepts {{Column}} values only. It'd be > very helpful to have a variant that accepted {{String}} for columns like > {{array}} or {{struct}}. > {code} > scala> val kvs = Seq(("key", "value")).toDF("k", "v") > kvs: org.apache.spark.sql.DataFrame = [k: string, v: string] > scala> kvs.printSchema > root > |-- k: string (nullable = true) > |-- v: string (nullable = true) > scala> kvs.withColumn("map", map("k", "v")).show > <console>:26: error: type mismatch; > found : String("k") > required: org.apache.spark.sql.Column > kvs.withColumn("map", map("k", "v")).show > ^ > <console>:26: error: type mismatch; > found : String("v") > required: org.apache.spark.sql.Column > kvs.withColumn("map", map("k", "v")).show > ^ > // note $ to create Columns per string > // not very dev-friendly > scala> kvs.withColumn("map", map($"k", $"v")).show > +---+-----+-----------------+ > | k| v| map| > +---+-----+-----------------+ > |key|value|Map(key -> value)| > +---+-----+-----------------+ > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org