marko-radoicic-db opened a new pull request, #58777: URL: https://github.com/apache/spark/pull/58777
### What changes were proposed in this pull request? This PR strengthens the shared PySpark column-resolution tests by asserting the exact error condition returned by Spark Classic and Spark Connect. A small helper keeps the shared test cases in `ColumnTestsMixin` while selecting the API-specific expected condition in the Connect parity suite. It also adds a regression test verifying that a generator cannot restore a tagged DataFrame column that an earlier projection removed. ### Why are the changes needed? The existing tests accepted any `AnalysisException`, which could hide regressions that surface the wrong public error condition. The generator case also lacked coverage for resolving a tagged column after it had been projected away. ### Does this PR introduce _any_ user-facing change? No. This is a test-only change. ### How was this patch tested? The following commands passed after rebasing onto the latest `master`: ``` build/sbt -Phive package .venv/bin/python python/run-tests.py --testnames "pyspark.sql.tests.test_column ColumnTests" .venv/bin/python python/run-tests.py --testnames "pyspark.sql.tests.connect.test_parity_column ColumnParityTests" .venv/bin/python python/run-tests.py --testnames "pyspark.sql.tests.connect.test_parity_column ColumnParityTestsWithNonStrictDFColResolution" ``` Both Connect suites had one expected skip: `test_validate_column_types`, which requires JVM access. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
