zhengruifeng opened a new pull request, #57366: URL: https://github.com/apache/spark/pull/57366
### What changes were proposed in this pull request? This PR replaces the correlation API's dedicated `VALUE_NOT_PEARSON` error condition with the existing generic `VALUE_NOT_ALLOWED` condition in both classic and Spark Connect DataFrames. It removes the unused dedicated condition and extends the Connect statistics test to assert the same structured error from both frontends. ### Why are the changes needed? Unsupported correlation methods have a single allowed value, so the generic error condition describes the validation consistently and avoids maintaining a method-specific condition. Checking both frontends guards their error-contract parity. ### Does this PR introduce _any_ user-facing change? Yes. Calling `DataFrame.stat.corr` with a method other than `pearson` now reports `VALUE_NOT_ALLOWED` instead of `VALUE_NOT_PEARSON`, with the standardized allowed-values message. ### How was this patch tested? Added a test that checks `VALUE_NOT_ALLOWED` and its message parameters for invalid correlation methods in classic and Spark Connect DataFrames. `git diff --check` passed. The PySpark test suite was not run. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
