voonhous commented on code in PR #19853:
URL: https://github.com/apache/hudi/pull/19853#discussion_r3961136757
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/HoodieProcedureFilterUtils.scala:
##########
@@ -367,12 +368,29 @@ object HoodieProcedureFilterUtils {
}
}
- private def evaluateExpressionOnRow(expression: Expression, row: Row,
schema: StructType): Boolean = {
+ // didn't match anything above, so ask Spark itself before we give up -
saves us from having
+ // to hand-list every builtin (concat, instr, if, ...) one by one
+ private def resolveViaFunctionRegistry(unresolvedFunc: UnresolvedFunction,
sparkSession: SparkSession): Expression = {
+ Try {
+ val nameParts = unresolvedFunc.nameParts
+ val functionIdentifier = nameParts match {
+ case Seq(funcName) => FunctionIdentifier(funcName)
+ case Seq(db, funcName) => FunctionIdentifier(funcName, Some(db))
+ case _ => FunctionIdentifier(nameParts.last)
+ }
+
sparkSession.sessionState.functionRegistry.lookupFunction(functionIdentifier,
unresolvedFunc.arguments)
Review Comment:
Addressed: `resolveViaFunctionRegistry` now unwraps `RuntimeReplaceable` via
`transformUp { case r => r.replacement }` and pushes `AggregateFunction`
results back to the rejection path. Pinned by the
`nvl`/`left`/`right`/`ifnull`/`nvl2`/`nullif` and `percentile`/`any_value`
cases in `TestHoodieProcedureFilterUtils`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
