anblanco opened a new pull request, #55223: URL: https://github.com/apache/spark/pull/55223
## Summary Adds `SimpleWorkerTests` that runs the full `WorkerTests` suite through the non-daemon (simple-worker) path by setting `spark.python.use.daemon=false`. Windows always uses this path; Linux/macOS use it when `spark.python.use.daemon=false`. Includes explicit tests for: - DataFrame creation - UDF execution - Python Data Source reads These cover the worker paths affected by SPARK-53759, where the simple-worker socket was not explicitly closed before process exit. Split out from #55201 per reviewer feedback — the fix itself is in #55201, this PR adds the regression test coverage on master. ## Test plan - [x] `SimpleWorkerTests` passes locally on WSL (Ubuntu 24.04, Python 3.12.3) — 12/12 tests pass - [x] Red/green confirmed: without the fix (unfixed `pyspark.zip`), tests fail with `EOFException`; with the fix, all pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
