[issue6692] asyncore kqueue support
Giampaolo Rodola' billiej...@users.sourceforge.net added the comment: I have a patch for adding epoll() support which is almost ready (asyncore supports poll(), not epoll()), it's just a matter of writing tests. kqueue() shouldn't be too difficult to implement considering that we can take Twisted as example. ...But aside from the implementation details I'd say there's a crucial problem with the current API: asyncore.loop([timeout[, use_poll[, map[, count) The use_poll argument should be deprecated in favor of something else, maybe a poller which accepts a callable, like this: asyncore.loop(poller=asyncore.select_poller) asyncore.loop(poller=asyncore.epoll_poller) asyncore.loop(poller=asyncore.kqueue_poller) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6692 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1706039] Added clearerr() to clear EOF state
Scott Dial sc...@scottdial.com added the comment: I've attached a patch that applies cleanly against py3k. I do not have an in-depth understanding of the new io module, but at a cursory glance, it seems to not use FILE streams, and would not be subject to this bug. However, Py_UniversalNewlineFgets() still remains, and I have patched it accordingly. Additionally, I have added the test to test_fileio. py3k doesn't fail any of these tests for me, but the original issue was only exposed on OS X. -- Added file: http://bugs.python.org/file15516/issue1706039-py3k.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1706039 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1706039] Added clearerr() to clear EOF state
Changes by Scott Dial sc...@scottdial.com: Added file: http://bugs.python.org/file15517/issue1706039-py3k.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1706039 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1706039] Added clearerr() to clear EOF state
Changes by Scott Dial sc...@scottdial.com: Removed file: http://bugs.python.org/file15516/issue1706039-py3k.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1706039 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3363] python version incorrectly reported in crash reports on Mac OS X 10.4.11 PPC
Changes by Ronald Oussoren ronaldousso...@mac.com: -- assignee: - ronaldoussoren nosy: +ronaldoussoren ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3363 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
Stefan Krah stefan-use...@bytereef.org added the comment: Segfault confirmed on 64 bit Ubuntu, Python 3.2a0: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f5074dea6e0 (LWP 11665)] 0x0042111b in _PyTuple_Resize (pv=0x7fff7ce03b10, newsize=25) at Objects/tupleobject.c:853 853 _PyObject_GC_UNTRACK(v); -- nosy: +skrah ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7468] PyErr_Format documentation doesn't mention all format codes
New submission from Ronald Oussoren ronaldousso...@mac.com: I haven't checked the 3.1, 2.7 and 2.6 documentation, but at least for 3.2 the documentation of PyErr_Format lists a very small number of supported format characters compared to PyUnicode_FromFormat even though PyErr_Format calls PyUnicode_FromFormatV to do the actual formatting. -- assignee: georg.brandl components: Documentation messages: 96198 nosy: georg.brandl, ronaldoussoren severity: normal status: open title: PyErr_Format documentation doesn't mention all format codes versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7468 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
flox la...@yahoo.fr added the comment: Yes, confirmed too. Both 3.1 and 3.2 are broken. Minimal sequence to trigger the bug: ~ $ ./python Python 3.2a0 (py3k:76743M, Dec 10 2009, 12:19:41) [GCC 4.3.2] on linux2 Type help, copyright, credits or license for more information. def m(i): if i == 10: return next(i for i in '0') MAX_LOOP = 110 t = range(11) for i in range(MAX_LOOP): d = map(m, t) tup = tuple(d) ## python: Objects/tupleobject.c:853: _PyTuple_Resize: ## Assertion `g-gc.gc_refs != (-2)' failed. -- components: +Interpreter Core nosy: +flox versions: +Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7469] Design and History FAQ entry on Floating Point does not mention short repr.
New submission from R. David Murray rdmur...@bitdance.com: See http://docs.python.org/dev/faq/design.html#why-are-floating-point-calculations-so-inaccurate. The 3.1 version is the same, so it also needs to be updated. The entry links to the tutorial, which has the correct information for 3.1/3.2, but not for trunk. -- keywords: easy messages: 96200 nosy: eric.smith, mark.dickinson, r.david.murray, rhettinger priority: low severity: normal stage: needs patch status: open title: Design and History FAQ entry on Floating Point does not mention short repr. type: behavior versions: Python 2.7, Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7469 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7470] logger.StreamHandler emit encoding fallback is wrong
New submission from Christian Boos cb...@neuf.fr: For a stream with a poor encoding, such as sys.stderr on Windows where encoding is cp437, the encoding of an unicode message will fail (expected) and then a fallback encoding to UTF-8 will be done: (in http://code.python.org/hg/trunk/file/bd98b2c097fe/Lib/logging/__init__.p y#l837): 837 except UnicodeError: 838 stream.write(fs % msg.encode(UTF-8)) However, that fallback won't work, as at this point, fs was already converted to unicode and is now u'%s\n', so the (msg.encode(UTF-8)) str will be decoded to unicode again using the default encoding, which will likely fail with a UnicodeDecodeError if msg contains non-ascii characters. The solution would be to keep using fs as %s\n in this line. This is similar to issue6991, but not exactly the same and it only happens for Python 2.7. Using logging_error.py, I've tested Python 2.3 to Python 2.6 (works) and Python 2.7a1 (fails), current trunk must have the same issue as 2.7a1. Patch follows. -- components: Library (Lib) files: logging_error.py messages: 96201 nosy: cboos, vinay.sajip severity: normal status: open title: logger.StreamHandler emit encoding fallback is wrong type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file15518/logging_error.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7470 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7470] logger.StreamHandler emit encoding fallback is wrong
Christian Boos cb...@neuf.fr added the comment: One way to solve the issue. PS: copy/pasting URLs to the Mercurial repo apparently doesn't work, for some reason (probably the line anchor); the URL I gave above was: http://code.python.org/hg/trunk/file/bd98b2c097fe/Lib/logging/__init__.py + #l837 -- keywords: +patch Added file: http://bugs.python.org/file15519/issue7470.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7470 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7470] logger.StreamHandler emit encoding fallback is wrong
Christian Boos cb...@neuf.fr added the comment: PPS: Ok, links to the Mercurial repo /really/ don't work, and it was not the #l837 part ;-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7470 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] gzip module too slow
New submission from asnakelover a3277...@uggsrock.com: It's not a big problem because we can just shell to zcat but compare these two: $ time python ./b.py /dev/null real0m10.977s user0m7.128s sys 0m0.888s $ time python ./a.py /dev/null real1m19.015s user1m18.185s sys 0m0.072s $ # Notice that the gzip module (a.py) had the benefit of the files being in a disk cache by too now... $ cat a.py import gzip import os apt_cache_dir = /var/cache/apt/apt-file for apt_cache_file in os.listdir(apt_cache_dir): if not apt_cache_file.endswith(.gz): continue f = gzip.open(os.path.join(apt_cache_dir, apt_cache_file)) for line in f: print line $ cat b.py import os import subprocess from cStringIO import StringIO apt_cache_dir = /var/cache/apt/apt-file for apt_cache_file in os.listdir(apt_cache_dir): if not apt_cache_file.endswith(.gz): continue p = subprocess.Popen([zcat, os.path.join(apt_cache_dir, apt_cache_file)], stdout = subprocess.PIPE) f = StringIO(p.communicate()[0]) assert p.returncode == 0 for line in f: print line Also tried this one just for completeness: $ cat c.py import gzip import os from cStringIO import StringIO apt_cache_dir = /var/cache/apt/apt-file for apt_cache_file in os.listdir(apt_cache_dir): if not apt_cache_file.endswith(.gz): continue f = gzip.open(os.path.join(apt_cache_dir, apt_cache_file)) f = StringIO(f.read()) for line in f: print line But after it had ran (with some thrashing) for 3 and a half minutes I killed it. -- components: Library (Lib) messages: 96204 nosy: asnakelover severity: normal status: open title: gzip module too slow type: performance versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] gzip module too slow
Antoine Pitrou pit...@free.fr added the comment: How does the following compare? f = gzip.open(...) s = f.read() for line in s.splitlines(): print line -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
Antoine Pitrou pit...@free.fr added the comment: (GZipFile.readline() is implemented in pure Python, which explains why it's rather slow) -- priority: - normal title: gzip module too slow - GZipFile.readline too slow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +pitrou priority: - high stage: - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
flox la...@yahoo.fr added the comment: Fix is coming soon... ## Sample code: import gc tuple(gc.collect() for i in range(11)) ## python: Objects/tupleobject.c:853: _PyTuple_Resize: ## Assertion `g-gc.gc_refs != (-2)' failed. And Trunk is affected -- versions: +Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
Antoine Pitrou pit...@free.fr added the comment: An even simpler way to reproduce: import gc def g(it): ... for x in it: ... gc.collect() ... yield x ... t = tuple(g(range(100))) Erreur de segmentation This is probably caused by the GC optimizations I introduced in trunk and 3.1. -- assignee: - pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
Antoine Pitrou pit...@free.fr added the comment: Ok, you've been faster than me, flox :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
Changes by flox la...@yahoo.fr: Removed file: http://bugs.python.org/file15520/issue7466.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7466] Segmentation fault after about 20 seconds on lenovo T500
Changes by flox la...@yahoo.fr: Added file: http://bugs.python.org/file15521/issue7466.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7472] email.encoders.encode_7or8bit(): typo iso-2202. iso-2022 is correct.
New submission from Yukihiro Nakadaira yukihiro.nakada...@gmail.com:
email.encoders.encode_7or8bit(): typo iso-2202. iso-2022 is correct.
Index: Lib/email/encoders.py
===
--- Lib/email/encoders.py (revision 76749)
+++ Lib/email/encoders.py (working copy)
@@ -62,7 +62,7 @@
# iso-2022-* is non-ASCII but still 7-bit
charset = msg.get_charset()
output_cset = charset and charset.output_charset
-if output_cset and output_cset.lower().startswith('iso-2202-'):
+if output_cset and output_cset.lower().startswith('iso-2022-'):
msg['Content-Transfer-Encoding'] = '7bit'
else:
msg['Content-Transfer-Encoding'] = '8bit'
--
components: Library (Lib)
messages: 96211
nosy: ynkdir
severity: normal
status: open
title: email.encoders.encode_7or8bit(): typo iso-2202. iso-2022 is correct.
type: behavior
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7472
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395552] add support for thread function result storage
Michal Božoň michal.bo...@gmail.com added the comment: why would we throw the thread function result out of the window? because there is a another way, to do it? there are always many ways, how to do it, so why would we want to do it very difficult way? i think that not having the result stored somewhere is a bug and i'm voting for fixing it -- nosy: +mykhal ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1395552 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7473] x84_64 arch Missing from 2.6.4 Mac universal binaries - Cripples building embedded 64-bit
New submission from Paul Nelson paul.nel...@jmp.com: By excluding the x86_64 architecture from the supplied MacOS install, you are crippling the ability to build an 64-bit application that embeds python. The Apple supplied universal binary includes all 3 architectures ppc, i386, and x86_64. If you are going to supply a Mac build you need to do the same. A 64-bit app has to link against 64-bit libraries, so the 64-bit build IS NECESSARY. You give no warning that you are excluding 1/3 of the architectures from your build. It also forces the unlucky downloader to circumvent the install, or rebuild from source. -- assignee: ronaldoussoren components: Macintosh messages: 96213 nosy: prniii, ronaldoussoren severity: normal status: open title: x84_64 arch Missing from 2.6.4 Mac universal binaries - Cripples building embedded 64-bit type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7473 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395552] add support for thread function result storage
R. David Murray rdmur...@bitdance.com added the comment: I agree with Stephen. You can implement this functionality easily through the current API (subclassing), so there's no need to complicate the existing API to handle this one communication method as a special case. Furthermore, this particular use case is the tip of a small iceberg usually called Futures, which provides a much cleaner API for the use case. There has been a suggestion to include such a package in core, though it will probably be some time before the library is mature enough for a decision to be made. -- nosy: +r.david.murray resolution: - rejected stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1395552 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7472] email.encoders.encode_7or8bit(): typo iso-2202. iso-2022 is correct.
Changes by Benjamin Peterson benja...@python.org: -- assignee: - barry nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7472 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7474] multiprocessing.managers.SyncManager managed object creation fails when started outside of invoked file
New submission from Eric etz...@hotmail.com: I am attempting to create a multiprocessing.SyncManager subclass instance, register a test class with it, start the manager, and instantiate the test class. The example I'm providing here is basically equivalent, and illustrates the same problem (example code is at the bottom of this message). This works without issue as long as the manager is started in the file you actually invoke from the command line. Handled in exactly the same way, however, with the call to the SyncManager instance's start method made in an included file, attempting to instantiate anything, including built-in list() and dict() objects, via the manager causes script to hang indefinitely. Output on ctrl+c breaking seems to indicate the call may be hanging in the challenge process. Invocation and output (starting/started are wrapped around the start() call; creating/created are wrapped around the manager's registered object instantiation): $ python test.py starting... ...started creating... ^CTraceback (most recent call last): File test.py, line 6, in module obj = test_manager.TC() File /usr/lib64/python2.6/multiprocessing/managers.py, line 634, in temp token, exp = self._create(typeid, *args, **kwds) File /usr/lib64/python2.6/multiprocessing/managers.py, line 532, in _create conn = self._Client(self._address, authkey=self._authkey) File /usr/lib64/python2.6/multiprocessing/connection.py, line 140, in Client answer_challenge(c, authkey) File /usr/lib64/python2.6/multiprocessing/connection.py, line 372, in answer_challenge message = connection.recv_bytes(256) # reject large message KeyboardInterrupt test.py: from test_inc import test_manager print creating... obj = test_manager.dict() print ...created test_inc.py: from multiprocessing.managers import SyncManager test_manager = SyncManager() print starting... test_manager.start() print ...started Environment: Linux 2.6.28-gentoo-r5 #4 SMP x86_64 Intel(R) Core(TM)2 Duo CPU T5470 @ 1.60GHz GenuineIntel GNU/Linux Version: Python 2.6.4 Apologies if this is my failure to understand how the library (or even Python) works, but it seems quite odd that this would be expected behavior. -- components: Extension Modules messages: 96215 nosy: etzool severity: normal status: open title: multiprocessing.managers.SyncManager managed object creation fails when started outside of invoked file versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7474 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7474] multiprocessing.managers.SyncManager managed object creation fails when started outside of invoked file
Changes by Eric etz...@hotmail.com: -- type: - crash ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7474 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
Changes by Brian Curtin cur...@acm.org: -- nosy: +brian.curtin ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1395552] add support for thread function result storage
Michal Božoň michal.bo...@gmail.com added the comment: i still do not agree, it will always feel somehow incomplete, being able to easily threadize the function: t = threading.Thread(target=func) and then not being able to get the target function result (easily) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1395552 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7474] multiprocessing.managers.SyncManager managed object creation fails when started outside of invoked file
Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +jnoller priority: - normal type: crash - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7474 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
asnakelover a3277...@uggsrock.com added the comment: Hope this reply works right, the python bug interface is a bit confusing for this newbie, it doesn't say Reply anywhere - sorry if it goes FUBAR. I tried the splitlines() version you suggested, it thrashed my machine so badly I pressed alt+sysrq+f (which invokes kernel oom_kill) after about 1 minute so I didn't lose anything important. About half a minute later the machine came back to life. In other words: the splitlines version used way, way too much memory - far worse even than making a cStringIO from a GzipFile instance.read(). It's not just a GzipFile.readline() issue either, c.py calls .read() and tries to turn the result into a cStringIO and that was the worst one of my three previous tests. I'm going to look at this purely from a consumer angle and not even look at gzip module source, from this angle (a consumer), zcat out performs it by a factor of 10 when gzip module is used with .readline() and by a good deal more when I try to read the whole gzip file as a string to turn into a cStringIO to emulate as closely as possible what happens with forking a zcat process. When I tried to splitlines() it was even worse. This is probably a RAM issue, but it just brings us back to - should gzip module eat so much ram when shelling out to zcat uses far less? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
New submission from flox la...@yahoo.fr: AFAIK these codecs were not ported to Python 3. 1. I found no hint in documentation on this matter. 2. Is it possible to contribute some of them, or there's a good reason to look elsewhere? -- components: Library (Lib) messages: 96218 nosy: flox severity: normal status: open title: codecs missing: base64 bz2 hex zlib ... versions: Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
Antoine Pitrou pit...@free.fr added the comment: I tried the splitlines() version you suggested, it thrashed my machine so badly I pressed alt+sysrq+f (which invokes kernel oom_kill) after about 1 minute so I didn't lose anything important. This sounds very weird. How much memory do you have, and how large are the .gz files you are reading? What is your system exactly? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
Changes by Skip Montanaro s...@pobox.com: -- nosy: +skip.montanaro ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
asnakelover a3277...@uggsrock.com added the comment: The gz in question is 17mb compressed and 247mb uncompressed. Calling zcat the python process uses between 250 and 260 mb with the whole string in memory using zcat as a fork. Numbers for the gzip module aren't obtainable except for readline(), which doesn't use much memory but is very slow. Other methods thrash the machine to death. The machine has 300mb free RAM from a total of 1024mb. It's not a new issue, I didn't find it when searching python bug archives for some reason but google: http://www.google.com/search?q=python+gzip+slow+zcat It's an issue which people have stumbled into before but nobody seems to have made headway any headway (I expect they just forked and called zcat or gzip, like I did...). It's a minor point since it's so easy to work around, in fact I have my IDS system completed already - thank you so much python for being so RAD friendly - yet the module is worthless if a user has zcat/gzip c programs and knows they exist as it stands. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
Antoine Pitrou pit...@free.fr added the comment: The gz in question is 17mb compressed and 247mb uncompressed. Calling zcat the python process uses between 250 and 260 mb with the whole string in memory using zcat as a fork. Numbers for the gzip module aren't obtainable except for readline(), which doesn't use much memory but is very slow. Other methods thrash the machine to death. The machine has 300mb free RAM from a total of 1024mb. That would be the explanation. Reading the whole file at once and then doing splitlines() on the result consumes twice the memory, since a list of lines must be constructed while the original data is still around. If you had more than 600 MB free RAM the splitlines() solution would probably be adequate :-) Doing repeated calls to splitlines() on chunks of limited size (say 1MB) would probably be fast enough without using too much memory. It would be a bit less trivial to implement though, and it seems you are ok with the subprocess solution. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
asnakelover a3277...@uggsrock.com added the comment: Yes, subprocess works fine and was the quickest to implement and probably the fastest to run too. How can I put this without being an ass? Hell, I'm no good at diplomacy - the gzip module blows chunks - if I can shell out to a standard unix util and it uses a tiny fraction of the memory to do the same job the module is inherently broken no matter how pretty it's code looks. You can of course tell me to STFU - but look around the web I won't be the first or last user to bemoan import gzip... :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
Martin v. Löwis mar...@v.loewis.de added the comment: These are not encodings, in that they don't convert characters to bytes. It was a mistake that they were integrated into the codecs interfaces in Python 2.x; this mistake is corrected in 3.x. -- nosy: +loewis resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7464] circular reference in HTTPResponse by urllib2
Kristján Valur Jónsson krist...@ccpgames.com added the comment: I have two solutions for this problem. The first is a mundane one, and what I employed in our production environment: class RecvAdapter(object): def __init__(self, wrapped): self.wrapped = wrapped def recv(self, amt): return self.wrapped.read(amt) def close(self): self.wrapped.close() ... fp = socket._fileobject(RecvAdapter(r), close=True) The second solution is a bit more interesting. It involves applying what I call a weakmethod: A bound method that holds a weak ref to the object instance: import weakref class WeakMethod(object): def __init__(self, bound): self.weakself = weakref.proxy(bound.im_self) self.methodname = bound.im_func.func_name def __call__(self, *args, **kw): return getattr(self.weakself, self.methodname)(*args, **kw) We then do: r.recv = WeakMethod(r.read) fp = socket._fileobject(r, close=True) I've had many uses for a WeakMethod through the years. I wonder if such a class might be considered useful enough to be put into the weakref module. -- keywords: +patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7464 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7476] pipes.quote does not handle zero-length args correctly
New submission from John Wiseman jjwise...@gmail.com:
In python 2.5.2, I believe pipes.quote gives the wrong result for zero-
length arguments.
from pipes import quote
args = ['arg1', '', 'arg3']
print 'mycommand %s' % (' '.join(quote(arg) for arg in args))
mycommand arg1 arg3
I think the result should be something like
mycommand arg1 '' arg3
--
components: Library (Lib)
messages: 96225
nosy: jjwiseman
severity: normal
status: open
title: pipes.quote does not handle zero-length args correctly
type: behavior
versions: Python 2.5
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7476
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
Marc-Andre Lemburg m...@egenix.com added the comment: Martin v. Löwis wrote: Martin v. Löwis mar...@v.loewis.de added the comment: These are not encodings, in that they don't convert characters to bytes. It was a mistake that they were integrated into the codecs interfaces in Python 2.x; this mistake is corrected in 3.x. Martin, I beg your pardon, but these codecs indeed implement valid encodings and the fact that these codecs were removed was a mistake. They should be readded to Python 3.x. Note that just because a codec doesn't convert between bytes and characters only, doesn't make it wrong in any way. The codec architecture in Python is designed to support same type encodings just as well as ones between bytes and characters. -- nosy: +lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
Marc-Andre Lemburg m...@egenix.com added the comment: Reopening the ticket. -- status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
Martin v. Löwis mar...@v.loewis.de added the comment: It's not possible to add these codecs back. Bytes objects (correctly) don't have an encode method, and string objects (correctly) don't have a decode method. The codec architecture of Python 3.x just doesn't support this kind of application; the codec architecture of 2.x was flawed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7464] circular reference in HTTPResponse by urllib2
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: The WeakMethod idea is not new: http://code.activestate.com/recipes/81253/ http://mindtrove.info/articles/python-weak-references/#toc-extending-weak- references -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7464 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7476] pipes.quote does not handle zero-length args correctly
Eric Smith e...@trueblade.com added the comment: In addition, quote is undocumented, although there is a test for it. It should also be documented and/or have a docstring. -- nosy: +eric.smith ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7476 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7470] logger.StreamHandler emit encoding fallback is wrong
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Sorry Christian, your patch appears just to rename 'fs' to 'ufs' so I'm not sure if your patch is exactly what you intended. -- assignee: - vinay.sajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7470 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7474] multiprocessing.managers.SyncManager managed object creation fails when started outside of invoked file
Changes by Benjamin Peterson benja...@python.org: -- assignee: - jnoller ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7474 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7475] codecs missing: base64 bz2 hex zlib ...
Benjamin Peterson benja...@python.org added the comment: I agree with Martin. gzip and bz2 convert bytes to bytes. Encodings deal strictly with unicode - bytes. -- nosy: +benjamin.peterson status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7449] A number tests crash if python is compiled --without-threads
Jerry Seutter jseut...@gmail.com added the comment:
The patch makes it so that tests that use threading skip rather than
generate errors when python is compiled --without-threads.
I added a skip_if_no('modulename') decorator to test_support.
Let me know if this patch is too big to review and I'll break it up. It
is 800 lines.
One thing to note: when threading is disabled, all sqlite3 tests are
skipped. There are a few test files that do not use threading, but I
couldn't think of an easy way to split them out.
Tested on OS X and Linux.
--
keywords: +patch
Added file: http://bugs.python.org/file15522/nothreads.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7449
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7449] A number tests crash if python is compiled --without-threads
Changes by Jerry Seutter jseut...@gmail.com: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7449 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7471] GZipFile.readline too slow
Antoine Pitrou pit...@free.fr added the comment: How can I put this without being an ass? Hell, I'm no good at diplomacy - the gzip module blows chunks - if I can shell out to a standard unix util and it uses a tiny fraction of the memory to do the same job the module is inherently broken no matter how pretty it's code looks. Well, let's say it is suboptimal. But it's certainly ok if you don't have any tight performance constraints. In any case, it won't improve until someone proposes a patch :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7471 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7464] circular reference in HTTPResponse by urllib2
Senthil Kumaran orsent...@gmail.com added the comment: weak method idea seems interesting. I have not used it anytime yet, and in this case,it seems okay. -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7464 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
