[issue4738] Patch to make zlib-objects better support threads

STINNER Victor added the comment: @pitrou: You added "Also, the GIL is now released when computing the CRC of a large buffer." in the NEWS. The GIL released for crc32 but also for adler32. ___ Python tracker _

[issue4738] Patch to make zlib-objects better support threads

Antoine Pitrou added the comment: Checked in r68165. Thanks! -- resolution: -> fixed status: open -> closed ___ Python tracker ___ __

[issue4738] Patch to make zlib-objects better support threads

ebfe added the comment: test-script Added file: http://bugs.python.org/file12532/zlibtest2.py ___ Python tracker ___ ___ Python-bugs-list mail

[issue4738] Patch to make zlib-objects better support threads

ebfe added the comment: Here is a small test-script with concurrent access to a single compressosbj. The original patch will immediately deadlock. The patch attached releases the GIL before trying to get the zlib-lock. This allows the other thread to release the zlib-lock but comes at the cost

[issue4738] Patch to make zlib-objects better support threads

Changes by ebfe : Removed file: http://bugs.python.org/file12466/zlib_threads-2.diff ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue4738] Patch to make zlib-objects better support threads

Antoine Pitrou added the comment: Same comment about potential deadlocks as in #4751. ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue4738] Patch to make zlib-objects better support threads

STINNER Victor added the comment: I opened a separate issue for the unicode problem: #4757. ___ Python tracker ___ ___ Python-bugs-list mailing

[issue4738] Patch to make zlib-objects better support threads

Antoine Pitrou added the comment: > May I also suggest to change the zlib module not to accept s* but y* You are probably right, but this would also break the API and can't be done lightheartedly. You can open a new bug entry about this though. ___ Python tr

[issue4738] Patch to make zlib-objects better support threads

Changes by ebfe : Removed file: http://bugs.python.org/file12463/zlib_threads-2.diff ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue4738] Patch to make zlib-objects better support threads

ebfe added the comment: new svn diff attached the indentation in this file is not my fault, it has tabs all over it... The 5kb limits protects from the overhead of releasing the GIL. With very small buffers the overall runtime in my benchmark tends to double. I set it based on my testing and i

[issue4738] Patch to make zlib-objects better support threads

STINNER Victor added the comment: Comments on zlib_threads-2.diff: - the indentation is strange: don't mix spaces and tabs! - I prefer ";" after a call to a macro: "ENTER_ZLIB(self);" instead of "ENTER_ZLIB(self)". It makes vim happy (auto indent code correctly) and it works for ENTER_ZLIB a

[issue4738] Patch to make zlib-objects better support threads

Changes by ebfe : Removed file: http://bugs.python.org/file12448/zlib_threads.diff ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue4738] Patch to make zlib-objects better support threads

ebfe added the comment: new svn diff attached - GIL is now released for adler32 and crc32 if the buffer is larger than 5kb (we don't want to risk burning cpu cycles by GIL-stuff) - adler32 got it's param by s# but now does s* - why s# anyway? - ENTER_ZLIB no longer gives away the GIL. It's dang

[issue4738] Patch to make zlib-objects better support threads

STINNER Victor added the comment: New comments about the last patch: - GIL is not released for adler() or crc32() whereas these functions may be slow for long strings: just add Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS before / after adler(...) and crc32(...) - (As ENTER_HASHLIB, issue #

[issue4738] Patch to make zlib-objects better support threads

Changes by Antoine Pitrou : -- nosy: +pitrou priority: -> normal stage: -> patch review versions: -Python 2.5, Python 2.6, Python 3.0 ___ Python tracker ___ ___

[issue4738] Patch to make zlib-objects better support threads

STINNER Victor added the comment: Ok, the patch looks fine and I like finer locks ;-) ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue4738] Patch to make zlib-objects better support threads

Changes by STINNER Victor : Removed file: http://bugs.python.org/file12440/zlib_threads.diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue4738] Patch to make zlib-objects better support threads

ebfe added the comment: new svn diff Added file: http://bugs.python.org/file12448/zlib_threads.diff ___ Python tracker ___ ___ Python-bugs-lis

[issue4738] Patch to make zlib-objects better support threads

STINNER Victor added the comment: A call to PyThread_free_lock(this->lock) in Comp_dealloc() and Decomp_dealloc(). Comp_dealloc() and Decomp_dealloc() code may also be factorized (write a common function to free unused_data, unconsumed_tail and self). -- nosy: +haypo ___

[issue4738] Patch to make zlib-objects better support threads

New submission from ebfe : My application needs to pack and unpack workunits all day long and does this using multiple threading.Threads. I've noticed that the zlib module seems to use only one thread at a time when using [de]compressobj(). As the comment in the sourcefile zlibmodule.c already sa