So, I’ve been testing moving some of my projects to using PyPy as an
interpreter from CPython, but PyPy 6.0.0/ Python 3.5.3 seems to memory bloat
and trigger OOM protections in about 6 hours on a server with 2GB of RAM, while
CPython 3.6.2 ran for 24 days with a more or less constant memory footprint of
~670MB (+/- 40MB) on my discord bot, Birb.
Output of gc.get_stats() about 2-3 hours before the crash are as follows:
Total memory consumed:
GC used: 641.5MB (peak: 775.5MB)
in arenas: 536.5MB
rawmalloced: 102.9MB
nursery: 2.0MB
raw assembler used: 5.5MB
——————————————
Total: 647.0MB
Total memory allocated:
GC allocated: 1235.9MB (peak: 1303.8MB)
in arenas: 602.8MB
rawmalloced: 170.8MB
nursery: 2.0MB
raw assembler allocated: 6.0MB
——————————————
Total: 1241.9MB
This behaviour can be successfully replicated on brew’s macOS build of PyPy3
6.0.0 and the official PyPy3 6.0.0 release for 64-bit Ubuntu.
To replicate this behaviour, I can use the following code, after installing the
rewrite branch on discord.py from
git+https://github.com/Rapptz/discord.py@rewrite#egg=discord.py;
```
import gc
from discord.ext import commands
bot = commands.AutoShardedBot(“>>”) # Variable passed is prefix, changing will
not affect result
@bot.command()
async def get_gc_state(ctx):
await ctx.send(gc.get_stats())
bot.run(TOKEN) # Replace TOKEN with authentication token, number of
guilds/users visible will affect result
```
Unfortunately, I can only reproduce this issue on my production instance, with
~8.6k guilds, ~143.7k channels, and ~305k visible user accounts. On my test
instance with 1 guild, 7 channels and 12 users, there are no measurable memory
issues.
Of note is that under total memory allocated, the sum of the parts of the GC
allocation (602.8+170.8+2.0=775.6) is just a little over 62.75% of the total
memory reported as being allocated for the GC. Is this a PyPy bug?
Thanks for reading,
Oxylibrium.
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
