mikamikasuki commented on issue #50482: URL: https://github.com/apache/arrow/issues/50482#issuecomment-5954825649
I have now independently reproduced this with a generated five-row Parquet file on macOS 15 (arm64), CPython 3.14.7, and the PyArrow 25.0.0 wheel. With 24 CPU threads, the file-object case timed out in 3 of 30 fresh processes; path reads exited cleanly in 12 of 12, and file-object reads with `use_threads=False` exited cleanly in 12 of 12. I captured native stacks from three hung processes. The main thread was waiting in Arrow's CPU `ThreadPool::Shutdown`; a worker was destroying a `ParquetFileFragment` / `PyReadableFile`, then blocked in `PyGILState_Ensure` → CPython thread-state attachment during finalization. This matches CPython's documented behavior that non-finalization threads attempting to attach during late interpreter finalization can be blocked permanently: https://docs.python.org/3.14/c-api/interp-lifecycle.html The source path leads to `OwnedRefNoGIL` in `python/pyarrow/src/arrow/python/common.h`, which reacquires the GIL to decref a retained Python object. I have a local candidate guard for the finalization path, but it still needs a source build and review of the check-to-GIL-acquire race, so I have not opened a PR. AI assistance was used to help trace the source and draft the candidate change; I ran the reproductions and examined the captured stacks. I would like to work on this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
