ericm-db opened a new pull request, #58247: URL: https://github.com/apache/spark/pull/58247
### What changes were proposed in this pull request? This is layer 5 of the nine-PR local Connect pool stack: #57684 -> #57685 -> #57907 -> #57686 -> RECOVERY_PR -> #57687 -> PURGE_PR -> #57102 -> #57688 Until #57686 merges, GitHub shows its cumulative diff. The review unit introduced here is commit `68833332d27`. This layer adds janitor and orphan recovery on top of normal retirement: - validated pending-launch records; - cleanup of dead, timed-out, malformed, and partially published launches; - retirement of dead, unreachable, idle, and orphaned claimed servers; - attendant PID-reuse checks and complete launch-process-group cleanup; - recovery of server PIDs from malformed records and daemon PID files; and - stale conf-file and unreferenced member-directory garbage collection. Acquisition, forceful purge, SparkSession integration, and warmup remain in later PRs. ### Why are the changes needed? Pool members outlive individual Python call frames and can be orphaned by crashes or SIGKILL. Separating recovery from normal release makes the adversarial state matrix and its idempotency contract reviewable without acquisition and user-facing orchestration. ### Does this PR introduce _any_ user-facing change? No. The pool is not wired into SparkSession in this layer. ### How was this patch tested? Added thirteen focused tests at this layer, bringing the suite to 46 tests. They cover pending launch failures, PID reuse, launch process groups, stale conf state, unusable and idle members, malformed records, dead clients, and member-directory garbage collection. ```bash python/run-tests --testnames pyspark.sql.tests.connect.test_connect_local_server_pool ``` The rebuilt commit passed Python AST parsing, `git diff --check`, and changed-file ASCII and 100-column checks. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Fable 5) and OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
