voonhous commented on PR #19836: URL: https://github.com/apache/hudi/pull/19836#issuecomment-5569967128
Pushed `f24b3a35e84d`. Most of it walks back a suggestion of mine from the last round. I had asked for an all-decimal skip in `widenNumericOperands` to sidestep the `DecimalType.bounded` precision-38 clamp. That was wrong. Every filterable procedure calls `validateFilter` before `applyFilter` (`BaseProcedure.scala:123-139`, 17 of 17), and an uncoerced decimal-vs-decimal comparison is `resolved == false`, so `validateFilterExpression` rejects it and the filter throws. `dec > 1.00` worked in @w3lld1's original commit and stopped working once I added the skip. The skip also did not avoid the clamp it cited -- BIGINT against DECIMAL(31,30) still widens to DECIMAL(38,30). Removed. Also in this commit: - A NULL operand widens with the numeric ones, matching the plan Spark builds for `ts IN (1000, null)` and `ts <=> null`. - Arithmetic and `coalesce` route through the same helper. `ts + 1 > 1500` was rejected before. - Tests assert the `validate` half for every shape whose validation result this PR flips, since procedures validate before they filter. Previously all the new assertions went through `evaluateFilter` only, which skips that path. - `TestFsViewProcedure` now uses filter shapes master rejects, plus one that keeps no rows. The earlier `data_file_size > 0` passed on master unchanged and expected the same row count as the unfiltered call, so it could not fail. Spark 3.5.5 / 2.12 is green, 29 tests, scalastyle clean. The 4.1 / 2.13 suites need a re-run: they caught a `DECIMAL(38,0)` vs `DECIMAL(38,18)` assertion that 3.5 and 4.1 resolve differently (3.5 widens and overflows the cast, 4.1 declines to widen), which I dropped rather than pin to one version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
