[issue38076] Make struct module PEP-384 compatible

Change by Petr Viktorin : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue38076] Make struct module PEP-384 compatible

miss-islington added the comment: New changeset 4590f72259ecbcea66e12a28af14d867255d2e66 by Eddie Elizondo in branch 'master': bpo-38076 Clear the interpreter state only after clearing module globals (GH-18039)

[issue38076] Make struct module PEP-384 compatible

Eric Snow added the comment: > there's still probably some underlying issue in multiprocessing. Whoa, I've never heard that before! -- ___ Python tracker ___

[issue38076] Make struct module PEP-384 compatible

Dino Viehland added the comment: One more data point: Backporting this change to Python 3.6 (I just happened to have it applied there already, so I haven't tried it on 3.7 or 3.8) has no crash and no hangs in multiprocessing on Linux. So something definitely changed in multiproessing which

[issue38076] Make struct module PEP-384 compatible

Dino Viehland added the comment: With either fix, or with both, on Linux I still see this hang at shutdown. Victor mentioned the fact that he had to hit Ctrl-C on Linux to see this, and I have to do the same thing. Then with the fixes in place the original test case still hangs on

[issue38076] Make struct module PEP-384 compatible

Eddie Elizondo added the comment: > I'm concerned by release blocker because 3.9.0a3 version is supposed to be > released soon, and usually release blocker do block a release :-) Ah! That makes sense! In any case, feel free to ping me if you need help on my side to get this PR through (or

[issue38076] Make struct module PEP-384 compatible

STINNER Victor added the comment: > That should close this issue, no need to work around the bug priority. I'm concerned by release blocker because 3.9.0a3 version is supposed to be released soon, and usually release blocker do block a release :-) --

[issue38076] Make struct module PEP-384 compatible

Eddie Elizondo added the comment: The PR that I sent out already fixes the issue. @vstinner, @pablogsal, please take a look again https://github.com/python/cpython/pull/18039 That should close this issue, no need to work around the bug priority. --

[issue38076] Make struct module PEP-384 compatible

STINNER Victor added the comment: > The error can still happened in other modules and under similar conditions, > no? The question was if the next 3.9 *alpha* release must be blocked by this issue. I don't think so. I reduce the priority to normal (not set). If someone disagrees, feel free

[issue38076] Make struct module PEP-384 compatible

Pablo Galindo Salgado added the comment: > Is it enough to reduce the issue priority from release blocker to normal? The error can still happened in other modules and under similar conditions, no? -- ___ Python tracker

[issue38076] Make struct module PEP-384 compatible

STINNER Victor added the comment: > https://github.com/python/cpython/pull/18038 is a partial fix for this Is it enough to reduce the issue priority from release blocker to normal? -- ___ Python tracker

[issue38076] Make struct module PEP-384 compatible

Eddie Elizondo added the comment: Hey all, I've got a fix for this bug and the CI is green: https://github.com/python/cpython/pull/18039 TL;DR: The module state have to be cleared at a later time. I explain in detail in the PR. Also, I didn't add a new test since there was a test that was

[issue38076] Make struct module PEP-384 compatible

Change by Eddie Elizondo : -- pull_requests: +17438 pull_request: https://github.com/python/cpython/pull/18039 ___ Python tracker ___

[issue38076] Make struct module PEP-384 compatible

Dino Viehland added the comment: https://github.com/python/cpython/pull/18038 is a partial fix for this. I think it fixes the crash at shutdown, although I'm still seeing a hang on master on Linux which is different then earlier versions of Python. I seem to have a really bogus stack

[issue38076] Make struct module PEP-384 compatible

Change by Dino Viehland : -- pull_requests: +17437 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/18038 ___ Python tracker ___

[issue38076] Make struct module PEP-384 compatible

Dino Viehland added the comment: And here's a variation which doesn't involve any instances from the module: import _struct class C: def __init__(self): self.pack = _struct.pack def __del__(self): self.pack('I', -42) _struct.x = C() --

[issue38076] Make struct module PEP-384 compatible

Dino Viehland added the comment: This is a relatively simple repro of the underlying problem: import _struct s = _struct.Struct('i') class C: def __del__(self): s.pack(42, 100) _struct.x = C() It's a little bit different in that it is actually causing the module to attempt to

[issue38076] Make struct module PEP-384 compatible

Dino Viehland added the comment: It seems problematic that_PyInterpreterState_ClearModules runs before all instances from a module have been cleared. If PyState_FindModule is no longer able to return module state then there's no way for a module to reliably work at shutdown other than

[issue38076] Make struct module PEP-384 compatible

Change by Dino Viehland : -- nosy: +eelizondo -dino.viehland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38076] Make struct module PEP-384 compatible

STINNER Victor added the comment: I can reproduce the crash on Linux. I interrupt the problem with CTRL+c (why is it blocked? I don't know). Then it does crash. First, _PyImport_Cleanup() does clear all modules including _struct. Then, _PyImport_Cleanup() calls gc.collect() which

[issue38076] Make struct module PEP-384 compatible

Pablo Galindo Salgado added the comment: Executing this simple code after this commit segfaults: from multiprocessing.pool import Pool class A(object): def __init__(self): self.pool = Pool() def __del__(self): self.pool.close() self.pool.join() a = A() [1]

[issue38076] Make struct module PEP-384 compatible

Change by Christian Heimes : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed type: -> enhancement ___ Python tracker ___

[issue38076] Make struct module PEP-384 compatible

Thomas Wouters added the comment: New changeset 4f384af067d05b16a554bfd976934fca9f87a1cf by T. Wouters (Dino Viehland) in branch 'master': bpo-38076: Make struct module PEP-384 compatible (#15805) https://github.com/python/cpython/commit/4f384af067d05b16a554bfd976934fca9f87a1cf --

[issue38076] Make struct module PEP-384 compatible

Change by Dino Viehland : -- keywords: +patch pull_requests: +15456 stage: -> patch review pull_request: https://github.com/python/cpython/pull/15805 ___ Python tracker ___

[issue38076] Make struct module PEP-384 compatible

New submission from Dino Viehland : Make struct module PEP-384 compatible -- assignee: dino.viehland components: Extension Modules messages: 351524 nosy: dino.viehland, eric.snow priority: normal severity: normal status: open title: Make struct module PEP-384 compatible versions: