[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Vinay Sajip]

Change by Vinay Sajip :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Vinay Sajip]

Vinay Sajip  added the comment:


New changeset 3fd69991f47a6672c510fafd76bf183f17ac52ec by Irit Katriel in 
branch 'master':
bpo-38762: Extend logging.test_multiprocessing to cover missing cases. 
(GH-22142)
https://github.com/python/cpython/commit/3fd69991f47a6672c510fafd76bf183f17ac52ec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Irit Katriel]

Irit Katriel  added the comment:

We've decided for now to leave the code's behavior as it is (the alternatives 
are worse) and to keep only the test improvements. See discussion on PR 22142.

--
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Irit Katriel]

Change by Irit Katriel :


--
pull_requests: +21226
pull_request: https://github.com/python/cpython/pull/22142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Irit Katriel]

Irit Katriel  added the comment:

I just pushed a second PR with the alternative solution - it calculates the 
name on a best-effort basis, and doesn't import anything that's not already 
there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Irit Katriel]

Irit Katriel  added the comment:

As the PR currently is, it will, unless you turn this off by setting 
logging.logMultiprocessing=False.

The other alternative we are discussing is: revert back to not importing 
multiprocessing if it's not there.

If it's there we use it, but if it's not we set processName to something like 
f"pid={os.getpid()}" instead of "MainThread".

(1) os is already imported in logging.

(2) In most cases where multiprocessing is not imported it's because there is 
only one process, so processName will not likely be used as all.

(3) In the edge case of shutdown or someone clearing sys.modules the 
processName will not be pretty, but it will at least be correct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

So this will make logging depending on the multiprocess module? Even if you do 
not use multiprocessing it will be imported in any case when you use logging.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Vinay Sajip]

Vinay Sajip  added the comment:

See also bpo-8200, which relates to the way the code is currently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +patch
nosy: +iritkatriel
nosy_count: 2.0 -> 3.0
pull_requests: +21154
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22063

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Vinay Sajip]

Vinay Sajip  added the comment:

This will be a rare use case, and I would expect anyone clearing sys.modules in 
the child process to handle this consequence themselves (e.g. by reimporting 
multiprocessing after clearing out the other modules, since they apparently 
still want some of multiprocessing's functionality in the child process). I'll 
certainly look at PRs to address the issue using an import of multiprocessing 
if it's not already there. I'm not so keen on the other approaches you've 
suggested.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38762] Logging displays wrong "processName" if "sys.modules" is cleared in child process

[Delgan]

New submission from Delgan :

Hi.

In order to display the process name in logs, the "logging" module checks for 
the presence of "multiprocessing" in "sys.modules" before calling 
"current_process()". If "multiprocessing" is not found in "sys.modules", it 
assumes that the current process is the main one.

See : 
https://github.com/python/cpython/blob/af46450bb97ab9bd38748e75aa849c29fdd70028/Lib/logging/__init__.py#L340-L341

However, nothing prevents a child process to delete 
"sys.module['multiprocessing']", which causes the process name to be wrongly 
displayed as "MainProcess".

I attached a reproducible example, but this is straightforward to understand. 
Although it should not happen very often in practice, it is still theoretically 
possible. Obviously, one could say "just don't clear sys.modules", but I 
suppose there might exist tools doing such thing for good reasons (like 
resetting the test environment).

Issues which lead to the current implementation:
- issue4301
- issue7120
- issue8200

Possible fixes: 
- Force import "multiprocessing.current_process()" even if not already loaded
- Add function "os.main_pid()" and set "processName" to "MainProcess" only if 
"os.getpid() == os.main_pid()"

--
components: Library (Lib)
files: test.py
messages: 356338
nosy: Delgan
priority: normal
severity: normal
status: open
title: Logging displays wrong "processName" if "sys.modules" is cleared in 
child process
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file48705/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com