15 times more memory? That's a lot.
Interestingly Quora reported that their PyPy processes were only 50% larger
than CPython ones:
http://www.quora.com/Quora-Infrastructure/Did-Quoras-switch-to-PyPy-result-in-increased-memory-consumption
"our PyPy worker processes themselves take approximately 50% more memory than
our equivalent CPython worker processes, although we did not do a large amount
of tuning of the GC. Regardless, this wasn't the main cause of our memory
blowup.
"In our development, we found that certain functions were not worth being
ported from their C libraries to pure Python, things like crypto, lxml, PyML,
and a couple other random libraries. Our solution for those functions was to
run a parallel CPython process that would do nothing but take arguments via
an execnetchannel, and output return values via the same execnet channel.
"The overhead for some of these Python processes, especially for the ones that
required a lot of state (for example, PyML) is comparable to the amount of
memory taken by the master PyPy process, effectively causing a 2-3x blowup in
memory just to maintain the CPython processes; this is our main memory sink for
our PyPy branch."
----
I wonder what accounts for this large difference in PyPy memory consumption
(50% more vs. 1,400% more). What type of "large amount of tuning of the GC" did
Quora do?
________________________________
From: Igor Katson <igor.kat...@gmail.com>
To: Maciej Fijalkowski <fij...@gmail.com>
Cc: pypy-dev@python.org
Sent: Friday, October 7, 2011 2:07 PM
Subject: Re: [pypy-dev] Benchmarking PyPy performance on real-world Django app
Tried gunicorn, nothing special, the speed is roughly the same.
Unfortunately, I noticed that a single instance takes way to much memory
to bring that to production, where I pay for the actually used memory. 4
uWSGI workers eat 14 megabytes each, and pypy's memory usage increases,
and after a couple thousands or requests a single worker took 250mb,
more than 15 times more.
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
