Sumeet Sharma created SPARK-34804: ------------------------------------- Summary: registerFunction shouldnt be logging warning message for same function being re-registered Key: SPARK-34804 URL: https://issues.apache.org/jira/browse/SPARK-34804 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Reporter: Sumeet Sharma
{code:java} test("function registry warning") { implicit val ss = spark import ss.implicits._ val dd = Seq((1),(2)).toDF("a") val dd1 = udf((i: Int) => i*2) (1 to 4).foreach { _ => dd.sparkSession.udf.register("function", dd1) Thread.sleep(1000) } dd.withColumn("aa", expr("function(a)")).show(10) }{code} logs: 21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; some spark core configurations may not take effect. 21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; the static sql configurations will not take effect. 21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; some spark core configurations may not take effect. 21/03/19 22:39:43 WARN SimpleFunctionRegistry : The function function replaced a previously registered function. 21/03/19 22:39:44 WARN SimpleFunctionRegistry : The function function replaced a previously registered function. 21/03/19 22:39:45 WARN SimpleFunctionRegistry : The function function replaced a previously registered function. +---+---+ | a| aa| +---+---+ | 1| 2| | 2| 4| +---+---+ -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org