New submission from Jason Fried: The default of 16k while good for memory usage it is not well suited for all cases. if we increased this to 4MB we saw a pretty large improvement to tar file creation and extraction on linux servers.
For a 1gb tar file containing 1024 random files each of 10MB in size. Time Delta for TarFile: 146.3240258693695 Time Delta for FastTarFile 4MB copybufsize: 102.76440262794495 Time Diff: 43.55962324142456 0.2976928975444698 ---------- components: Library (Lib) files: buftest.py messages: 267134 nosy: asvetlov, fried, lukasz.langa priority: normal severity: normal status: open title: TarFile expose copyfileobj bufsize to improve throughput type: performance versions: Python 3.5 Added file: http://bugs.python.org/file43158/buftest.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27199> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
