Thanks Dongjoon, you're right. I referenced to a wrong caused-by jira link
in https://issues.apache.org/jira/browse/SPARK-32307.
Bests,
Yi
On Wed, Jul 15, 2020 at 3:03 AM Dongjoon Hyun <dongjoon.h...@gmail.com>
wrote:
> Hi, Yi.
>
> Could you explain why you think that is a blocker? For the given example
> from the JIRA description,
>
> spark.udf.register("key", udf((m: Map[String, String]) => m.keys.head.toInt))
>
> Seq(Map("1" -> "one", "2" -> "two")).toDF("a").createOrReplaceTempView("t")
>
> checkAnswer(sql("SELECT key(a) AS k FROM t GROUP BY key(a)"), Row(1) :: Nil)
>
>
> Apache Spark 3.0.0 seems to work like the following.
>
> scala> spark.version
> res0: String = 3.0.0
>
> scala> spark.udf.register("key", udf((m: Map[String, String]) =>
> m.keys.head.toInt))
> res1: org.apache.spark.sql.expressions.UserDefinedFunction =
> SparkUserDefinedFunction($Lambda$1958/948653928@5d6bed7b,IntegerType,List(Some(class[value[0]:
> map<string,string>])),None,false,true)
>
> scala> Seq(Map("1" -> "one", "2" ->
> "two")).toDF("a").createOrReplaceTempView("t")
>
> scala> sql("SELECT key(a) AS k FROM t GROUP BY key(a)").collect
> res3: Array[org.apache.spark.sql.Row] = Array([1])
>
>
> Could you provide a reproducible example?
>
> Bests,
> Dongjoon.
>
>
> On Tue, Jul 14, 2020 at 10:04 AM Yi Wu <yi...@databricks.com> wrote:
>
>> This probably be a blocker:
>> https://issues.apache.org/jira/browse/SPARK-32307
>>
>> On Tue, Jul 14, 2020 at 11:13 PM Sean Owen <sro...@gmail.com> wrote:
>>
>>> https://issues.apache.org/jira/browse/SPARK-32234 ?
>>>
>>> On Tue, Jul 14, 2020 at 9:57 AM Shivaram Venkataraman
>>> <shiva...@eecs.berkeley.edu> wrote:
>>> >
>>> > Hi all
>>> >
>>> > Just wanted to check if there are any blockers that we are still
>>> waiting for to start the new release process.
>>> >
>>> > Thanks
>>> > Shivaram
>>> >
>>>
>>
