> > > Ah, another interesting point, this huge slowdown happens only when bm_pickle.py > > is executed through pyperformance. When run it directly, slowdown is > > not so large. > > pyperformance runs benchmarks in a virtual environment. I don't know > if it has any impact on bm_pickle. > > Most pyperformance can be run outside a virtual env if required > modules are installed on the system. (bm_pickle only requires the > stdlib and perf.) >
Bingo! Without venv: unpickle: Mean +- std dev: 26.9 us +- 0.0 us % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 28.78 0.000438 0 1440 read 27.33 0.000416 1 440 25 stat 9.72 0.000148 1 144 mmap ... 0.79 0.000012 1 11 munmap With venv: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 57.12 0.099023 2 61471 munmap 41.87 0.072580 1 61618 mmap 0.23 0.000395 1 465 27 stat unpickle and unpickle_list creates massive same-sized objects, then all objects are removed. If all pools in the arena is freed, munmap is called. I think we should save some arenas to reuse. On recent Linux, we may be able to use MADV_FREE instead of munmap.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com