jiangxt2 opened a new pull request, #57369: URL: https://github.com/apache/spark/pull/57369
### What changes were proposed in this pull request? Add a guard `xMk === 0.0 || yMk === 0.0` in `Corr.evaluateExpression`, branching to the existing `divideByZeroEvalResult` before evaluating `ck / sqrt(xMk * yMk)`. ### Why are the changes needed? When one of the input columns is constant, the corresponding Welford variance term (`xMk` or `yMk`) is exactly `0.0`, making the denominator zero. This produces `DIVIDE_BY_ZERO` in ANSI mode and `NaN` or `±Infinity` in non-ANSI mode. The sibling function `regr_r2` (same `PearsonCorrelation` base class) already guards against `yMk === 0.0` and `xMk === 0.0`. The Spark 3.1 migration guide lists `corr` among functions that should return `NULL` on divide-by-zero, but the guard was never added. ```sql SET spark.sql.ansi.enabled = true; SELECT corr(CAST(x AS DOUBLE), CAST(y AS DOUBLE)) FROM VALUES (1, 1), (1, 2), (1, 3) AS t(x, y); -- Before: DIVIDE_BY_ZERO -- After: NULL ``` ### Does this PR introduce _any_ user-facing change? Yes. `corr` on a constant column now returns `NULL` instead of throwing `DIVIDE_BY_ZERO` (ANSI mode) or returning `NaN`/`Infinity` (non-ANSI mode). ### How was this patch tested? - `linear-regression.sql`: x-constant and y-constant cases, both returning `NULL` - Corresponding analyzer golden file regenerated - `SQLQueryTestSuite` passes ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code with Claude Opus 4.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
