Re: Building Python 3.12 with no-omit-frame-pointer
On 23. 11. 22 9:12, Petr Viktorin wrote:
On 22. 11. 22 18:30, Miro Hrončok wrote:
On 21. 11. 22 11:01, Petr Viktorin wrote:
And since the Python slowdown comes from a single weird function, I think
that Fedora should ignore the Python benchmarks when evaluating the distro
default -- and if Fedora switches to no-omit-frame-pointer, Python 3.11
should be an exception (to be re-evaluated for 3.12).
Do I interpret correctly?
If (and only if) Fedora switches to -fno-omit-frame-pointer, python3.11 (and
possibly older Pythons) will explicitly use -fomit-frame-pointer instead (aka
opt-out).
Yes. If the change allows software to opt out (e.g. if it's slowed down too
much), then, based on the benchmarks, Python 3.11 should do that.
python3.11 has opted out.
I've also prepared pull requests for python3.{6..10} to do the same.
I don't intent to touch Python 2.7 unless I have to.
No idea how this affects PyPys, so keeping them untouched as well.
python3.12 will do the same for now.
Or not, if that would help testing. I don't think the speed of the preview
Python matters that much.
python3.12 has not opted out.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
Re: Building Python 3.12 with no-omit-frame-pointer
On 22. 11. 22 18:30, Miro Hrončok wrote: On 21. 11. 22 11:01, Petr Viktorin wrote: And since the Python slowdown comes from a single weird function, I think that Fedora should ignore the Python benchmarks when evaluating the distro default -- and if Fedora switches to no-omit-frame-pointer, Python 3.11 should be an exception (to be re-evaluated for 3.12). Do I interpret correctly? If (and only if) Fedora switches to -fno-omit-frame-pointer, python3.11 (and possibly older Pythons) will explicitly use -fomit-frame-pointer instead (aka opt-out). Yes. If the change allows software to opt out (e.g. if it's slowed down too much), then, based on the benchmarks, Python 3.11 should do that. python3.12 will do the same for now. Or not, if that would help testing. I don't think the speed of the preview Python matters that much. Once the massive changes to _PyEval_EvalFrameDefault land in Python 3.12, we will re-evaluate whether to use -fomit-frame-pointer or -fno-omit-frame-pointer (regardless of the Fedora global flags). That might not be a single point in time, so let's perhaps re-evaluate around Beta? ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Building Python 3.12 with no-omit-frame-pointer
On 21. 11. 22 11:01, Petr Viktorin wrote: And since the Python slowdown comes from a single weird function, I think that Fedora should ignore the Python benchmarks when evaluating the distro default -- and if Fedora switches to no-omit-frame-pointer, Python 3.11 should be an exception (to be re-evaluated for 3.12). Do I interpret correctly? If (and only if) Fedora switches to -fno-omit-frame-pointer, python3.11 (and possibly older Pythons) will explicitly use -fomit-frame-pointer instead (aka opt-out). python3.12 will do the same for now. Once the massive changes to _PyEval_EvalFrameDefault land in Python 3.12, we will re-evaluate whether to use -fomit-frame-pointer or -fno-omit-frame-pointer (regardless of the Fedora global flags). -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Building Python 3.12 with no-omit-frame-pointer
On 09. 11. 22 12:37, Petr Viktorin wrote: tl;dr: Python 3.12 should be built with no-omit-frame-pointer if upstream recommends it. Hello, You might be aware of a Fedora change proposal [0] (discussed on fedora-devel [1] and FESCo [2]) are discussing turning on C compiler flags that help with performance *measurement*, but might hurt performance itself: `-fno-omit-frame-pointer` and `-mno-omit-leaf-frame-pointer`. Apparently there are some benchmarks that make Python look extra slow when the flags are turned on -- which I don't quite understand. Update: Andrii Nakryiko looked at the disassembly: https://pagure.io/fesco/issue/2817#comment-826636 Looks like the slowdown comes from a single function, _PyEval_EvalFrameDefault, which essentially *is* the Python interpreter -- so it's very unlikely to be indicative of other code in the distro (including Python extension modules). That function is being overhauled for Python 3.12 (in the main branch it's now autogenerated [5], which should hopefully allow optimizations across any common code, and perhaps things like switching to a register-based VM [6]). Given these massive planned changes in the affected function, I think we should treat Python 3.11 and 3.12 as entirely separate when it comes to performance with no-omit-frame-pointer. And since the Python slowdown comes from a single weird function, I think that Fedora should ignore the Python benchmarks when evaluating the distro default -- and if Fedora switches to no-omit-frame-pointer, Python 3.11 should be an exception (to be re-evaluated for 3.12). (Most of the current benchmarks [7] are from Python, so more might be needed.) [5]: https://github.com/python/cpython/issues/98831 [6]: https://github.com/faster-cpython/ideas/issues/489 [7]: https://github.com/DaanDeMeyer/fpbench Meanwhile, on the upstream side, Python 3.12 (due next year, main Python for Fedora 39 [3]) has support for `perf`. Upstream plans to recommend compiling with these flags when measuring performance [4], and AFAIK, the plan is to recommend *always* compiling with them. The idea is that possible speedups from "allowing anyone to profile/optimize their workflow" are worth the initial slowdown. I'm not much of a performance expert myself, but I do get drawn into the relevant discussions on the CPython side. As far as I can see, performance geeks are enthusiastic for `perf` support, and I'd like to get them to (continue to) use Fedora builds. If CPython upstream does recommend these flags (or makes them default), I'm considering to turn the no-omit options on for Python 3.12 even if Fedora as a whole doesn't. Note that even a 2% slowdown will likely won back by general performance improvements – the Faster CPython team is targeting a 20% average speedup for pure-Python code in 3.12, on top of the ~25% for 3.11. And the people responsible for this speedup have a say in the upstream recommendations. Technically there are three separate places where the flags can be set, I think we should turn them on everywhere: - CPython itself & its standard library - The debug build (/usr/bin/python3-debug) - Default for libraries in Fedora (RPM macros) - Befault for libraries built by users (sysconfig settings) [0]: https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer [1]: https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/OOJDAKTJB5WGMOZRXTUX7FTPFBF3H7WE/ [2]: https://pagure.io/fesco/issue/2817 [3]: https://fedoraproject.org/wiki/Changes/Python3.12 [4]: https://docs.python.org/3.12/howto/perf_profiling.html#how-to-obtain-the-best-results ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Building Python 3.12 with no-omit-frame-pointer
On 15. 11. 22 20:17, Kevin Kofler wrote: tl;dr: Python 3.12 should be built with no-omit-frame-pointer if upstream recommends it. Absolutely not, because… Apparently there are some benchmarks that make Python look extra slow when the flags are turned on … considering those benchmarks, Python is one of the programs for which it would be the *least* appropriate to enable frame pointers! Oh, do you have more info on these benchmarks? The mentions I found were pretty vague. Is it DaanDeMeyer/fpbench? It seems pyperformance is the majority of those benchmarks, so there's not much to compare to. It's also the benchmark suite that upstream uses, so I'm certain any upstream recommendation will take it into account. Meanwhile, on the upstream side, Python 3.12 (due next year, main Python for Fedora 39 [3]) has support for `perf`. Upstream plans to recommend compiling with these flags when measuring performance [4], and AFAIK, the plan is to recommend *always* compiling with them. [3]: https://fedoraproject.org/wiki/Changes/Python3.12 [4]: https://docs.python.org/3.12/howto/perf_profiling.html#how-to-obtain-the-best-results Looking at the details in the link, would it not be sufficient to have the perf trampolines compiled with frame pointers enabled (which will only affect runs with perf support enabled ar runtime)? Or actually, since they are "compiled on the fly", do those trampolines not always have a frame pointer anyway, no matter how Python is compiled? That sounds like a question for upstream, do you want to bring it up there? (I won't, as I don't think I could follow up well on the discussion.) The idea is that possible speedups from "allowing anyone to profile/optimize their workflow" are worth the initial slowdown. I and others heavily dispute this claim. There is no evidence that anybody will be doing enough profiling and optimization to compensate for the up to 10% performance hit seen on performance-critical Python code in the benchmarks, nor that such big optimization is even possible to begin with. Also, profiling and optimizing individual Python programs (as opposed to the Python interpreter itself) will not help the vast majority of Python code out there at all. I know enough about the pyperformance benchmarks to suspect that talking about the highest slowdown is just a scare tactic. Not all benchmarks are indicative of real-world use. The one that got closest to 10% (scimark_sparse_mat_mult, with a 9.5% hit [0]) is pure-Python matrix multiplication -- perhaps the best example of code you would never put in production today. As far as I can see, performance geeks are enthusiastic for `perf` support, As you say yourself, this is a niche feature for "performance geeks". and I'd like to get them to (continue to) use Fedora builds. That is in theory a laudable goal, but not if it degrades the performance for everyone else. If CPython upstream does recommend these flags (or makes them default), I'm considering to turn the no-omit options on for Python 3.12 even if Fedora as a whole doesn't. Then I can only hope that FESCo will explicitly disallow that. Note that even a 2% slowdown will likely won back by general performance improvements – the Faster CPython team is targeting a 20% average speedup for pure-Python code in 3.12, on top of the ~25% for 3.11. And the people responsible for this speedup have a say in the upstream recommendations. Those 20% are a stated goal, not a guarantee. Even if attained, they will also not necessarily help all programs the same amount. And most importantly, Fedora will benefit from those upstream optimizations either way, even if it does not ship a build optimized for profiling rather than performance. So I do not see this as an argument for shipping Python with frame pointers. I am also very sceptical of that "grabbing for the stars" approach of accepting a 2-10% performance loss in the hopes of getting a 20% performance win that may or may not be actually achievable. In German, we have a proverb that literally translates to: "Better the sparrow in the hand than the dove on the roof." It means that it is better to take the small thing that you already have than to throw it away for a bigger thing that you have yet to catch. We don't “have” anything now, the Python 3.12 release is a year away. I'm pretty sure the upstream recommendation will take the then-current state into account. In fact, in your mail I don't really see anything Fedora-specific reason to override an upstream recommendation, when it comes. [0]: https://github.com/DaanDeMeyer/fpbench/blob/7233a6cfcd01467b429e1a22d961e3f77b6f4868/README.md ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
Re: Building Python 3.12 with no-omit-frame-pointer
> tl;dr: Python 3.12 should be built with no-omit-frame-pointer if > upstream recommends it. Absolutely not, because… > Apparently there are some benchmarks that make Python look extra slow > when the flags are turned on … considering those benchmarks, Python is one of the programs for which it would be the *least* appropriate to enable frame pointers! > which I don't quite understand. So then please try to figure it out. As long as the performance hit is as big as it is, enabling frame pointers is *not* acceptable. > Meanwhile, on the upstream side, Python 3.12 (due next year, main Python > for Fedora 39 [3]) has support for `perf`. Upstream plans to recommend > compiling with these flags when measuring performance [4], and AFAIK, > the plan is to recommend *always* compiling with them. > [3]: https://fedoraproject.org/wiki/Changes/Python3.12 > [4]: > https://docs.python.org/3.12/howto/perf_profiling.html#how-to-obtain-the-best-results Looking at the details in the link, would it not be sufficient to have the perf trampolines compiled with frame pointers enabled (which will only affect runs with perf support enabled ar runtime)? Or actually, since they are "compiled on the fly", do those trampolines not always have a frame pointer anyway, no matter how Python is compiled? > The idea is that possible speedups from "allowing anyone to > profile/optimize their workflow" are worth the initial slowdown. I and others heavily dispute this claim. There is no evidence that anybody will be doing enough profiling and optimization to compensate for the up to 10% performance hit seen on performance-critical Python code in the benchmarks, nor that such big optimization is even possible to begin with. Also, profiling and optimizing individual Python programs (as opposed to the Python interpreter itself) will not help the vast majority of Python code out there at all. > As far as I can see, performance geeks are enthusiastic for `perf` > support, As you say yourself, this is a niche feature for "performance geeks". > and I'd like to get them to (continue to) use Fedora builds. That is in theory a laudable goal, but not if it degrades the performance for everyone else. > If CPython upstream does recommend these flags (or makes them default), > I'm considering to turn the no-omit options on for Python 3.12 even if > Fedora as a whole doesn't. Then I can only hope that FESCo will explicitly disallow that. > Note that even a 2% slowdown will likely won back by general performance > improvements – the Faster CPython team is targeting a 20% average > speedup for pure-Python code in 3.12, on top of the ~25% for 3.11. And > the people responsible for this speedup have a say in the upstream > recommendations. Those 20% are a stated goal, not a guarantee. Even if attained, they will also not necessarily help all programs the same amount. And most importantly, Fedora will benefit from those upstream optimizations either way, even if it does not ship a build optimized for profiling rather than performance. So I do not see this as an argument for shipping Python with frame pointers. I am also very sceptical of that "grabbing for the stars" approach of accepting a 2-10% performance loss in the hopes of getting a 20% performance win that may or may not be actually achievable. In German, we have a proverb that literally translates to: "Better the sparrow in the hand than the dove on the roof." It means that it is better to take the small thing that you already have than to throw it away for a bigger thing that you have yet to catch. > Technically there are three separate places where the flags can be set, > I think we should turn them on everywhere: > - CPython itself & its standard library > - The debug build (/usr/bin/python3-debug) > - Default for libraries in Fedora (RPM macros) > - Befault for libraries built by users (sysconfig settings) IMHO, the debug build is the only one where it might possibly make sense to do so. Anything else would hurt the performance for real-world users who are not interested in profiling at all. Kevin Kofler ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
