[issue19395] lzma hangs for a very long time when run in parallel using python's muptiprocessing module?
New submission from cantor: import lzma from functools import partial import multiprocessing def run_lzma(data,c): return c.compress(data) def split_len(seq, length): return [str.encode(seq[i:i+length]) for i in range(0, len(seq), length)] def lzma_mp(sequence,threads=3): lzc = lzma.LZMACompressor() blocksize = int(round(len(sequence)/threads)) strings = split_len(sequence, blocksize) lzc_partial = partial(run_lzma,c=lzc) pool=multiprocessing.Pool() lzc_pool = list(pool.map(lzc_partial,strings)) pool.close() pool.join() out_flush = lzc.flush() return b.join(lzc_pool + [out_flush]) sequence = 'AJKGJFKSHFKLHALWEHAIHWEOIAH IOAHIOWEHIOHEIOFEAFEASFEAFWEWEWFQWEWQWQGEWQFEWFDWEWEGEFGWEG' lzma_mp(sequence,threads=3) -- components: ctypes messages: 201278 nosy: cantor priority: normal severity: normal status: open title: lzma hangs for a very long time when run in parallel using python's muptiprocessing module? type: behavior versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19395 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19395] lzma hangs for a very long time when run in parallel using python's muptiprocessing module?
cantor added the comment: lzma -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19395 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19395] lzma hangs for a very long time when run in parallel using python's muptiprocessing module?
Changes by cantor cantorma...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19395 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19395] lzma hangs for a very long time when run in parallel using python's muptiprocessing module?
Changes by cantor cantorma...@gmail.com: -- components: -ctypes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19395 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com