linrrzqqq opened a new pull request, #62167: URL: https://github.com/apache/doris/pull/62167
### What problem does this PR solve? Issue Number: close #xxx Related PR: https://github.com/apache/doris/pull/61280 Problem Summary: When two imported modules share the same top-level package name, python_server imports them package by package, and clean up `sys.modules` after import, which causes incorrect lock key assignment and leads to thread contention or interference between imported states. For example, the symbols `pkg.mod.func1` and `pkg.mod.func2` may conflict when importing `pkg` or `pkg.mod` ```text 2026-04-07 16:35:30.108 ERROR [suite-thread-1] (ScriptContext.groovy:122) - Run test_pythonudtf_data_types_module in /mnt/disk7/linzhenqi/dv/doris/regression-test/suites/pythonudtf_p0/test_pythonudtf_data_types_module.groovy failed java.sql.SQLException: errCode = 2, detailMessage = (127.0.0.1)[RUNTIME_ERROR]Flight stream finish failed with message: 'pyudtf_module'. Detail: Python exception: Traceback (most recent call last): File "pyarrow/_flight.pyx", line 2315, in pyarrow._flight._do_exchange File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 2499, in do_exchange self._handle_exchange_udtf(python_udf_meta, reader, writer) File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 2316, in _handle_exchange_udtf adaptive_udtf = loader.load() ^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 877, in load func = self.load_udf_from_module(location, package_name, module_name, func_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 1135, in load_udf_from_module return self._load_package_udf( ^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 1088, in _load_package_udf udf_module = self._get_or_import_module(location, full_module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 978, in _get_or_import_module module = importlib.import_module(full_module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/miniconda3/envs/py312/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 946, in _load_unlocked KeyError: 'pyudtf_module' ... 2026-04-07 16:35:30.169 ERROR [suite-thread-6] (ScriptContext.groovy:122) - Run test_pythonudtf_exceptions_module in /mnt/disk7/linzhenqi/dv/doris/regression-test/suites/pythonudtf_p0/test_pythonudtf_exceptions_module.groovy failed java.sql.SQLException: errCode = 2, detailMessage = (127.0.0.1)[RUNTIME_ERROR]Flight stream finish failed with message: 'pyudtf_module'. Detail: Python exception: Traceback (most recent call last): File "pyarrow/_flight.pyx", line 2315, in pyarrow._flight._do_exchange File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 2499, in do_exchange self._handle_exchange_udtf(python_udf_meta, reader, writer) File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 2316, in _handle_exchange_udtf adaptive_udtf = loader.load() ^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 877, in load func = self.load_udf_from_module(location, package_name, module_name, func_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 1135, in load_udf_from_module return self._load_package_udf( ^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 1088, in _load_package_udf udf_module = self._get_or_import_module(location, full_module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/dv/doris/output/be/plugins/python_udf/python_server.py", line 978, in _get_or_import_module module = importlib.import_module(full_module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/disk7/linzhenqi/miniconda3/envs/py312/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1337, in _find_and_load_unlocked KeyError: 'pyudtf_module' ``` ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
