XiaoHongbo-Hope opened a new pull request, #10533: URL: https://github.com/apache/paimon/pull/10533
### Purpose Fix premature native Jindo filesystem destruction while Python streams are still open. Replacing a cached FileIO after a REST data-token change can drop the old handler while its streams are still in use, causing a native process crash. Keep a strong handler reference in input/output stream wrappers. This preserves the native filesystem for the stream's lifetime without changing token refresh, storage backend selection, or the Rust path. Resources remain collectible when the wrappers are released. ### Tests - Focused unittest suite: 32 tests run, 19 optional Jindo integration tests skipped; all 13 executed tests passed. - New tests cross the PyArrow PyFileSystem/PythonFile boundary and cover input/output lifetime, REST token-cache replacement, repeated close, and eventual client collection. All four new assertions/subcases fail against the pre-fix handler. - Controlled read-only OSS reproduction with PyArrow 19.0.1 and pyjindosdk 6.10.4.post1: both pre-fix cache-replacement runs crashed (SIGSEGV/SIGABRT); both patched runs read identical bytes and closed successfully. Token identity changes were forced; this is not a natural-expiry or long-running training test. - `git diff --check` passed. This narrowly fixes the PyArrow Jindo handler path. It does not address explicit filesystem close in the separate PVFS/fsspec path, and does not claim to establish the cause of any particular production incident. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
