[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Ma Lin  added the comment:

Thanks for review!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[miss-islington]

miss-islington  added the comment:


New changeset 57100c86baa8451a568348646834380cd425b858 by Miss Islington (bot) 
in branch '3.10':
[3.10] bpo-41735: Fix thread lock in zlib.Decompress.flush() may go wrong 
(GH-29587) (GH-29811)
https://github.com/python/cpython/commit/57100c86baa8451a568348646834380cd425b858


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Thanks malin!

--
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[miss-islington]

Change by miss-islington :


--
pull_requests: +28044
pull_request: https://github.com/python/cpython/pull/29812

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 86c1265cdc64030c8921e0da5fcae2ac64299c26 by Ma Lin in branch 
'3.9':
[3.9] bpo-41735: Fix thread lock in zlib.Decompress.flush() may go wrong 
(GH-29588)
https://github.com/python/cpython/commit/86c1265cdc64030c8921e0da5fcae2ac64299c26


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 7edb6270a78c695e4c2ae2432797dc18105374fc by Ma Lin in branch 
'main':
bpo-41735: Fix thread lock in zlib.Decompress.flush() may go wrong (GH-29587)
https://github.com/python/cpython/commit/7edb6270a78c695e4c2ae2432797dc18105374fc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +28043
pull_request: https://github.com/python/cpython/pull/29811

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Gregory P. Smith]

Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
resolution: later -> 
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Ma Lin  added the comment:

Sorry, I found an omission.

The previous PRs fixed the bug in these methods:

zlib.Compress.compress()
zlib.Decompress.decompress()

This method also has this bug, fix in PR29587 (main/3.10) and PR29588 (3.9-):

zlib.Decompress.flush()

Attached file `test_flush.py` can reliably reproduce the bug.

This time I carefully checked bz2/lzma/zlib modules, it should be no problem 
anymore.

Gregory P. Smith should understand these codes, add him to nosy list.

--
nosy: +gregory.p.smith
resolution: fixed -> later
status: closed -> open
Added file: https://bugs.python.org/file50445/test_flush.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Change by Ma Lin :


--
pull_requests: +27831
pull_request: https://github.com/python/cpython/pull/29588

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Change by Ma Lin :


--
pull_requests: +27830
pull_request: https://github.com/python/cpython/pull/29587

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Ma Lin  added the comment:

Thanks for review.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks! ✨ 🍰 ✨

--
nosy: +lukasz.langa
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Change by Ma Lin :


--
pull_requests: +21213
pull_request: https://github.com/python/cpython/pull/22132

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Change by Ma Lin :


--
pull_requests: +21211
pull_request: https://github.com/python/cpython/pull/22130

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

Change by Ma Lin :


--
keywords: +patch
pull_requests: +21208
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22126

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41735] Thread locks in zlib module may go wrong in rare case

[Ma Lin]

New submission from Ma Lin :

The code in zlib module:

self->zst.next_in = data->buf;  // set next_in
...
ENTER_ZLIB(self);   // acquire thread lock

`self->zst` is a `z_stream` struct defined in zlib, used to record states of a 
compress/decompress stream:

typedef struct z_stream_s {
Bytef*next_in;  /* next input byte */
uInt avail_in;  /* number of bytes available at next_in */
uLongtotal_in;  /* total number of input bytes read so far */

Bytef*next_out; /* next output byte will go here */
uInt avail_out; /* remaining free space at next_out */
uLongtotal_out; /* total number of bytes output so far */

... // Other states
} z_stream;

Setting `next_in` before acquiring the thread lock may mix up 
compress/decompress state in other threads.

Moreover, modify `ENTER_ZLIB` macro, don't release the GIL when the thread lock 
can be acquired immediately. This behavior is the same as the bz2/lzma modules.

--
components: Library (Lib)
messages: 376473
nosy: malin
priority: normal
severity: normal
status: open
title: Thread locks in zlib module may go wrong in rare case
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com