[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Inada Naoki]

Inada Naoki  added the comment:

https://github.com/Homebrew/homebrew-core/blob/b2aff6271caa04508fb1529fdd5edbb22f4e7f21/Formula/python%402.rb#L75-L82

Homebrew checks compiler version too.

I don't think it's worth enough to add more code to avoid this trouble.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Inada Naoki]

Inada Naoki  added the comment:

I'm using Mojave.  I don't have regression.

$ /usr/local/bin/python2 x.py  # Homebrew python@2
1.681729

$ /usr/bin/python x.py  # System python2
1.891549

Could someone using High Sierra or Sierra test it?

x.py is test script in https://bugs.python.org/issue32616#msg310395

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Michael Romero]

Michael Romero  added the comment:

So is this now considered resolved for High Sierra users via 2.7.15rc1?

--
nosy: +Michael Romero

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Ned Deily]

Ned Deily  added the comment:

A followup - Ronald asked:
> w.r.t. detection of the clang/llvm version on Apple's system compiler: Is it 
> worthwhile to do so?

Now that Xcode 9.3 (for macOS 10.13+) is officially released, I ran a quick 
series of test on it and on the most recent Xcode versions for the last several 
macOS OS families: 10.12, 10.11, and 10.9 (I didn't have a 10.10 system 
available at the moment).  Only looking at the most recent supported 
Xcode/compiler version for each major release is reasonable since I think most 
people follow Apple's strong encouragement to keep software updated and only 
use the most recent releases.  And I think most people follow Apple's lead in 
using their build tools, via Xcode or the command line utilities, rather than a 
third-party compiler.

By that measure, it seems clear that (1) there is only one current version that 
exhibits the performance degradation, that is the Xcode 9.2 version labeled 
Apple LLVM 9.0.0 (clang-900.0.39.2) and (2) that is now only an issue for macOS 
10.12 (Sierra) where Xcode 9.2 is (and will likely remain) the most recent 
version.  For macOS 10.13 (High Sierra), the compiler in the newly released 
Xcode 9.3 does not exhibit the problem.  And the most recent versions of Xcode 
for the tested earlier macOS releases do not either.

BTW, the MacPorts project maintains a handy webpage listing Xcode releases and 
compiler versions by macOS release:
https://trac.macports.org/wiki/XcodeVersionInfo

Here are the results.  The methodology was to download and build the just 
released Python 2.7.15rc1 from source using the default configure options, i.e. 
just ./configure, and then run the test program 3 times with it and then three 
times with the Apple-provided system /usr/bin/python2.7 as a baseline.

ProductName:Mac OS X
ProductVersion: 10.13.4
BuildVersion:   17E199
2.7.15rc1 (default, Apr 15 2018, 00:22:29)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)]
1.181971
1.180467
1.173380
2.7.10 (default, Oct  6 2017, 22:29:07)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
1.567072
1.587779
1.570056

ProductName:Mac OS X
ProductVersion: 10.12.6
BuildVersion:   16G1314
2.7.15rc1 (default, Apr 15 2018, 00:31:39)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)]
6.290989
6.356802
6.295680
2.7.10 (default, Feb  7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
1.306507
1.312231
1.302826

ProductName:Mac OS X
ProductVersion: 10.11.6
BuildVersion:   15G20015
2.7.15rc1 (default, Apr 15 2018, 00:38:12)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
1.846332
1.855483
1.896600
2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)]
1.453702
1.426298
1.440348

ProductName:Mac OS X
ProductVersion: 10.9.5
BuildVersion:   13F1911
2.7.15rc1 (default, Apr 15 2018, 00:42:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
1.720303
1.712045
1.710216
2.7.5 (default, Mar  9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]
1.681696
1.685704
1.686414

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Antoine Pitrou]

Antoine Pitrou  added the comment:

This should remain everyone that backporting performance improvements is not a 
no-brainer.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I different question w.r.t. detection of the clang/llvm version on Apple's 
system compiler: Is it worthwhile to do so? 

If the compiler included in the Xcode 9.3 beta (and hence likely the one in 
Xcode 9.3 final) fixes the performance issue a very large subset of people 
building Python for themselves will get a fixed compiler fairly soon. It would 
then be enough to warn about this issue in a readme file for other users.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:

I don't know exactly.  But as far as I saw, Python 3's eval loop has less 
function-wide local variables.

For example, ROT_THREE uses only block local variable.
https://github.com/python/cpython/blob/a48e78a0b7761dd74f1d03fc69e0f6caa6f02fe6/Python/ceval.c#L1109-L

On the other hand, there are more function-wide local variables in Python 2.  
And some of them are used over `case`s actually.
https://github.com/python/cpython/blob/672fd7d8162f76aff8423fa5c7bfd2b1e91faf57/Python/ceval.c#L802-L807


I suspect that's why LLVM4 failed to optimize Python 2 but success to optimize 
Python 3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Ned Deily]

Ned Deily  added the comment:

Can anyone explain what the difference is between 2.7 and 3.6, i.e. why there 
is the performance regression for 2.7 but not for 3.6 using the same compiler 
instance?  It would be better to understand and solve that problem rather than 
trying to special case compiler versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:

How can we distinguish Apple LLVM with LLVM easily?
Or should we disable computed-gotos by default on LLVM?

It's only for Python 2.
5x slowdown is too large comparing to 10% speedup.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Zhiming Wang]

Zhiming Wang  added the comment:

Yeah, Apple LLVM versions are a major headache. I resorted to feature 
detection, using C++ coroutines support as the clang 5 distinguisher[1]:

$ cat /tmp/test/stub.cc
#include 

int main() {
return 0;
}

$ 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
 -v
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin17.4.0
Thread model: posix
InstalledDir: 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
 -o stub stub.cc -fcoroutines-ts -stdlib=libc++
stub.cc:1:10: fatal error: 'experimental/coroutine' file not found
#include 
 ^~~~
1 error generated.

$ 
/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
 -v
Apple LLVM version 9.1.0 (clang-902.0.31)
Target: x86_64-apple-darwin17.4.0
Thread model: posix
InstalledDir: 
/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ 
/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
 -o stub stub.cc -fcoroutines-ts -stdlib=libc++

Here Xcode.app is Xcode 9.2 and Xcode-beta.app is Xcode 9.3 beta 2.

The conclusion here seems to be that Apple LLVM 9.0.0 is based on LLVM 4, while 
Apple LLVM 9.1.0 is based on LLVM 5.

[1] 
http://releases.llvm.org/5.0.0/tools/clang/docs/ReleaseNotes.html#major-new-features

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:

I'm sorry, my patch doesn't work on Xcode (Apple LLVM).
computed-gotos is still enabled by default.

Apple doesn't expose LLVM version.  It's really annoying.

$ cat x.c
#include 

int main()
{
printf("__clang__ : %d\n", __clang__);
printf("__llvm__ : %d\n", __llvm__);
printf("__VERSION__ : %s\n", __VERSION__);
printf("__clang_version__ : %s\n", __clang_version__);
printf("__clang_major__   : %d\n", __clang_major__);
printf("__clang_minor__   : %d\n", __clang_minor__);
printf("__clang_patchlevel__ : %d\n", __clang_patchlevel__);
}

$ cc x.c && ./a.out
__clang__ : 1
__llvm__ : 1
__VERSION__ : 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)
__clang_version__ : 9.0.0 (clang-900.0.39.2)
__clang_major__   : 9
__clang_minor__   : 0
__clang_patchlevel__ : 0

--
resolution: fixed -> 
stage: resolved -> 
status: closed -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

Change by INADA Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:


New changeset 2942b909d9a428e6683d90b3436cfa4a81bd5d8a by INADA Naoki in branch 
'2.7':
bpo-32616: Disable computed gotos by default for clang < 5 (GH-5574)
https://github.com/python/cpython/commit/2942b909d9a428e6683d90b3436cfa4a81bd5d8a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

Change by INADA Naoki :


--
keywords: +patch
pull_requests: +5392
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Ned Deily]

Ned Deily  added the comment:

Quick test: there doesn't seem to be a similar regression when building 3.6 
with the current clang provided by Xcode 9.2, just with 2.7.  And both 2.7 and 
3.6 configure HAVE_COMPUTED_GOTOS on.  Benjamin?

(FWIW, the 2.7.x binaries provided by the python.org installers do not suffer 
from this performance regression as they are not built with clang.)

--
components:  -macOS
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Zhiming Wang]

Zhiming Wang  added the comment:

Turns out python 2.7.10 doesn't suffer from the performance issue even when 
compiled with stock clang 4.x, and upon further investigation, I tracked down 
the commit that introduced the regression:

commit 2c992a0788536087bfd78da8f2c62b30a461d7e2
Author: Benjamin Peterson 
Date:   Thu May 28 12:45:31 2015 -0500

backport computed gotos (#4753)

So Naoki was right that computed gotos is (solely) to blame here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:

Bad news: --enable-optimization doesn't solve it.
I hope same thing doesn't happen for Python 3.
Anyone tried Xcode 9.3?  What version of LLVM does Apple use?

Anyway, I think we need help of LLVM expert.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:

This is assembly code for FAST_DISPATCH()
https://paste.ubuntu.com/26523948/

It seems there are many redundant spills.  But I don't know how to remove them.
Are their clang expert?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

INADA Naoki  added the comment:

It seems clang4 failed to efficient register assigning.
FYI, --without-computed-gotos configure option make penalty smaller.

clang 5 (wihtout CGs): 2.653426
clang 5 (with CGs): 1.997584
clang 4 (without CGs): 3.330879
clang 4 (with CGs): 8.585673

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Is there anything we (the CPython developers) can do about this? 

If I read the issue correctly clang 5.x generates faster binaries than clang 
3.x and 4.x.  If that is indeed the issue there's probably not much we can do 
about this. 

BTW. I'm -1 on building the installer with anything but the compiler included 
in Xcode (and it would be nice to build with a recent version of Xcode to use 
an up-to-date compiler and SDK)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Terry J. Reedy]

Change by Terry J. Reedy :


--
components: +macOS
nosy: +ned.deily, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Tim Smith]

Change by Tim Smith :


--
nosy: +tdsmith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Zhiming Wang]

Zhiming Wang  added the comment:

My benchmarks above do contain py37 (master) stats.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Has anyone done the same analysis with Python 3.6 or 3.7?

--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
components: +Interpreter Core
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[INADA Naoki]

Change by INADA Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32616] Significant performance problems with Python 2.7 built with clang 3.x or 4.x

[Zhiming Wang]

New submission from Zhiming Wang :

Python 2.7 could be significantly slower (5x in some cases) when compiled with 
clang 3.x or 4.x, compared to clang 5.x. This is quite a problem on macOS, 
since the latest clang from Apple (which comes with Xcode 9.2) is based on LLVM 
4.x. This issue was first noticed by Bart Skowron and reported to the Homebrew 
project.[1]

I ran some preliminary benchmarks (here[2] are the exact setup scripts) with 
just a simple loop:

import time

def f(n):
while n > 0:
n -= 1

start = time.time()
f(5000)
stop = time.time()
print('%.6f' % (stop - start))

and here are my results:

- macOS 10.13.2 on a MacBook Pro:

2.082144/usr/bin/python2.7
7.964049/usr/local/bin/python2.7
8.750652dist/python27-apple-clang-900/bin/python2.7
8.476405dist/python27-clang-3.9/bin/python2.7
8.625660dist/python27-clang-4.0/bin/python2.7
1.760096dist/python27-clang-5.0/bin/python2.7
3.254814/usr/local/bin/python3.6
2.864716dist/python-master-apple-clang-900/bin/python3
3.071757dist/python-master-clang-3.9/bin/python3
2.925192dist/python-master-clang-4.0/bin/python3
2.908782dist/python-master-clang-5.0/bin/python3

- Ubuntu 17.10 in VirtualBox:

1.475095/usr/bin/python2.7
8.576817dist/python27-clang-3.9/bin/python2.7
8.165588dist/python27-clang-4.0/bin/python2.7
1.779193dist/python27-clang-5.0/bin/python2.7
1.728321dist/python27-gcc-5/bin/python2.7
1.570040dist/python27-gcc-6/bin/python2.7
1.604617dist/python27-gcc-7/bin/python2.7
2.323037/usr/bin/python3.6
2.964338dist/python-master-clang-3.9/bin/python3
3.054277dist/python-master-clang-4.0/bin/python3
2.734908dist/python-master-clang-5.0/bin/python3
2.490278dist/python-master-gcc-5/bin/python3
2.494691dist/python-master-gcc-6/bin/python3
2.642277dist/python-master-gcc-7/bin/python3

I haven't got time to run more rigorous benchmark suites (e.g., the 
performance[3] package). I did try the floating point benchmark from 
performance, and again saw a 2x difference in performance.

[1] https://github.com/Homebrew/homebrew-core/issues/22743
[2] https://gist.github.com/zmwangx/f8151ba8907ba8159a07fdd1528fc2b5
[3] https://pypi.python.org/pypi/performance

--
messages: 310395
nosy: zmwangx
priority: normal
severity: normal
status: open
title: Significant performance problems with Python 2.7 built with clang 3.x or 
4.x
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com