[
https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646760#comment-16646760
]
Apache Spark commented on SPARK-25690:
--------------------------------------
User 'maryannxue' has created a pull request for this issue:
https://github.com/apache/spark/pull/22701
> Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied
> infinitely
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-25690
> URL: https://issues.apache.org/jira/browse/SPARK-25690
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core, SQL
> Affects Versions: 2.4.0
> Reporter: Maryann Xue
> Priority: Major
>
> This was fixed in SPARK-24891 and was then broken by SPARK-25044.
> The unit test in {{AnalysisSuite}} added in SPARK-24891 should have failed
> but didn't because it wasn't properly updated after the {{ScalaUDF}}
> constructor signature change. In the meantime, the other two end-to-end tests
> added in SPARK-24891 were shadowed by SPARK-24865.
> So the unit test mentioned above, if updated properly, can reproduce this
> issue:
> {code:java}
> test("SPARK-24891 Fix HandleNullInputsForUDF rule") {
> val a = testRelation.output(0)
> val func = (x: Int, y: Int) => x + y
> val udf1 = ScalaUDF(func, IntegerType, a :: a :: Nil, nullableTypes = false
> :: false :: Nil)
> val udf2 = ScalaUDF(func, IntegerType, a :: udf1 :: Nil, nullableTypes =
> false :: false :: Nil)
> val plan = Project(Alias(udf2, "")() :: Nil, testRelation)
> comparePlans(plan.analyze, plan.analyze.analyze)
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
