ericm-db opened a new pull request, #57685: URL: https://github.com/apache/spark/pull/57685
### What changes were proposed in this pull request? This is layer 2 of the local Connect pool stack, following #57684. It adds the filesystem-backed storage model for pool members: - stable member identities and state-file paths; - a per-pool cross-process file lock; - secure JSON state reads, writes, renames, and cleanup; - fingerprints for master, startup conf, working directory, and Python environment; and - FIFO claiming of reachable, version-compatible members. Process lifecycle, acquisition, SparkSession integration, and JIT warmup remain in later PRs. ### Why are the changes needed? The pool needs a small, independently reviewable state model before adding process supervision and background launching. Keeping this layer limited to storage and claiming makes the concurrency contract and on-disk state transitions reviewable without the attendant implementation. ### Does this PR introduce _any_ user-facing change? No. The storage model is internal and is not wired into SparkSession in this layer. ### How was this patch tested? Added seven focused tests in `pyspark.sql.tests.connect.test_connect_local_server_pool.LocalConnectServerPoolUnitTests` covering directory selection, pool-size parsing, zombie detection, fingerprint identity, FIFO claiming, fingerprint matching, and unreachable members. ```bash python -m unittest -v pyspark.sql.tests.connect.test_connect_local_server_pool.LocalConnectServerPoolUnitTests ``` All 7 tests passed. Ruff check, Ruff format check, and `git diff --check` also passed. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Fable 5) and OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
