New submission from Daniel Svensson <[email protected]>:
I've noticed problems with the GC in two applications. In one case the
application did not respond to SOAP-requests for 9 seconds, every couple of
minutes, when it really shouldn't have taken more than 20ms. In another case we
had one operation executed by 20 threads that ought to have taken 0.5 seconds,
but instead took about 90 seconds.
Both cases were solved by disabling the garbage collector and hunting down
possible circular references. Once this was done, the first example went down
to its expected 20ms max latency, and the second one to its 0.5s processing
time.
Here is a short python script that demonstrates the issue, the JSON file in
this case is 1.2GB large:
> import cjson, time, gc
> def read_json_blob():
> t0 = time.time()
> fd = file("mytestfile")
> data = fd.read()
> fd.close()
> t1 = time.time()
> parsed = cjson.decode(data)
> t2 = time.time()
> print "read file in %.2fs, parsed json in %.2fs, total of %.2fs" % \
> (t1-t0, t2-t1, t2-t0)
> read_json_blob()
read file in 10.57s, parsed json in 531.10s, total of 541.67s
> gc.disable()
> read_json_blob()
read file in 0.59s, parsed json in 15.13s, total of 15.72s
> gc.collect()
0
I don't understand how Python can work like this default, at least not without
a warning, to me it looks like a joke gone too far. All documentation ought to
recommend people to disable the garbage collector at the first sign of
performance problems, or the garbage collector problem should be fixed, this
the "Documentation" or "Interpreter Core" Components in the ticket
classification.
----------
assignee: docs@python
components: Documentation, Interpreter Core
messages: 142305
nosy: docs@python, dsvensson
priority: normal
severity: normal
status: open
title: immense performance problems related to the garbage collector
type: performance
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue12775>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
