Re: [Pythonmac-SIG] Problem with pip
This would be a very good time to go over to Python-dev and see what’s being done for 3.7. And to help Ned if you want to see changes. -CHB On Thu, Jan 18, 2018 at 10:19 AM Joni Orponen wrote: > On Thu, Jan 18, 2018 at 5:05 PM, Karsten Wolf via Pythonmac-SIG < > pythonmac-sig@python.org> wrote: > >> >> > On 18.01.2018, at 14:58, Joni Orponen wrote: >> > >> > The offical CPython distributions seem to be compiled on 10.6 for >> compatibility reasons. It is easier to compile CPython forwards-compatible >> than to manually patch pyconfig.h after configure to exclude the syscalls >> not available on your older target platforms. >> >> With the following config line you compile a backwards compatible Python >> (here 64-bit only since I don't like fiddling around with hard to compile >> fat-libs). >> >> # for 2.7 >> env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch >> x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue >> >> and it yields a Python that can be py2apped and run on 10.6 as long as >> you don't use newer Frameworks or libs that need OSX > 1.6 >> > > I'm not sharing your experience. > > Adding -Wl,-no_weak_imports to LDFLAGS flags things up and those then > actually fail hard at runtime. The easiest one to spot and verify yourself > is getentropy() when using TLS. I'm having to patch out the detected > support for those syscalls manually out of the generated pyconfig.h when > compiling something backwards-compatibly for distribution. > > I'm also passing through -mmacosx-version-min in both LDFLAGS and CFLAGS > as not all egg specific build systems pass in the environment correctly, > but there is still some hope of them to use whatever it is that Python was > built with as Python exposes that at runtime. There are eggs one still has > to patch up manually for distribution. This is also useful to be in the > know of for some of the dependencies of any CPython modules you might have > to bundle as static libraries for distribution purposes. > > For the framework incompatibilities you can get to about the same > detection ability if you use -Wunguarded-availability in CFLAGS. There is > also -Wno-unguarded-availability-new which combines to provide the ability > for not yet caring of non-fatal issues for the newest one (as in the one > you are currently building on with the intention of being > backwards-compatible only), but this is rather new [1]. > > I do not recommend -Werror and -Wunguarded-availability together when > compiling backwards-compatible as you'd have to fix the use of all the > deprecated, but not yet broken, bits in CPython itself. > > So for targeting 10.6 I'd use something roughly like (also allowing you to > switch between Xcode versions via xcode-select, if needed): > > CC="xcrun cc" > CXX="$CC" > > MACOSX_DEPLOYMENT_TARGET="10.6" > > XCODE_ROOT="$(xcode-select --print-path)" > SYSROOT="$XCODE_ROOT/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" > > COMMON_FLAGS="--sysroot=$SYSROOT > -mmacosx-version-min=$MACOSX_DEPLOYMENT_TARGET" > > CFLAGS="-I$SYSROOT/usr/include $COMMON_FLAGS > -Wno-unguarded-availability-new -Wunguarded-availability" > CPPFLAGS="$CFLAGS" > > LDFLAGS="-L$SYSROOT/usr/lib $COMMON_FLAGS -Wl,-no_weak_imports" > > In practice I'd not target such an old OS version when building > backwards-compatibly as the associated workload would be rather large. > Building on the latest version and supporting a couple of versions back > down is still rather manageable and sane in my experience. > > Against this backdrop, I'm understanding of people, especially open source > volunteers, choosing to build both CPython and their distributable wheels > on 10.6 as the amount of effort to validate and support more permutations > is rather large for almost no benefit. > > The expectation levels on the Windows side have risen and thus since > Python 3.5 that stack has moved to build with a newer Visual Studio and is > also offering a nicer userspace install. I'm taking this thread as a signal > of the expectations of the macOS crowd having risen as well. > > A midway compromise point might be to coerce the community to start > building on 10.11 as 10.12 was again a great hardware support divider such > as 10.7 was and it's starting to have been long enough since 10.11. > > On the actual topic: If one explicitly needs some build-time details to be > specific, one can just build the egg oneself. Otherwise whatever pip pulls > in and is working is good enough for the job. > > -- > Joni Orponen > > [1] https://reviews.llvm.org/D34264 > ___ > Pythonmac-SIG maillist - Pythonmac-SIG@python.org > https://mail.python.org/mailman/listinfo/pythonmac-sig > unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython __
Re: [Pythonmac-SIG] Problem with pip
On Thu, Jan 18, 2018 at 5:05 PM, Karsten Wolf via Pythonmac-SIG < pythonmac-sig@python.org> wrote: > > > On 18.01.2018, at 14:58, Joni Orponen wrote: > > > > The offical CPython distributions seem to be compiled on 10.6 for > compatibility reasons. It is easier to compile CPython forwards-compatible > than to manually patch pyconfig.h after configure to exclude the syscalls > not available on your older target platforms. > > With the following config line you compile a backwards compatible Python > (here 64-bit only since I don't like fiddling around with hard to compile > fat-libs). > > # for 2.7 > env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch > x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue > > and it yields a Python that can be py2apped and run on 10.6 as long as you > don't use newer Frameworks or libs that need OSX > 1.6 > I'm not sharing your experience. Adding -Wl,-no_weak_imports to LDFLAGS flags things up and those then actually fail hard at runtime. The easiest one to spot and verify yourself is getentropy() when using TLS. I'm having to patch out the detected support for those syscalls manually out of the generated pyconfig.h when compiling something backwards-compatibly for distribution. I'm also passing through -mmacosx-version-min in both LDFLAGS and CFLAGS as not all egg specific build systems pass in the environment correctly, but there is still some hope of them to use whatever it is that Python was built with as Python exposes that at runtime. There are eggs one still has to patch up manually for distribution. This is also useful to be in the know of for some of the dependencies of any CPython modules you might have to bundle as static libraries for distribution purposes. For the framework incompatibilities you can get to about the same detection ability if you use -Wunguarded-availability in CFLAGS. There is also -Wno-unguarded-availability-new which combines to provide the ability for not yet caring of non-fatal issues for the newest one (as in the one you are currently building on with the intention of being backwards-compatible only), but this is rather new [1]. I do not recommend -Werror and -Wunguarded-availability together when compiling backwards-compatible as you'd have to fix the use of all the deprecated, but not yet broken, bits in CPython itself. So for targeting 10.6 I'd use something roughly like (also allowing you to switch between Xcode versions via xcode-select, if needed): CC="xcrun cc" CXX="$CC" MACOSX_DEPLOYMENT_TARGET="10.6" XCODE_ROOT="$(xcode-select --print-path)" SYSROOT="$XCODE_ROOT/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" COMMON_FLAGS="--sysroot=$SYSROOT -mmacosx-version-min=$MACOSX_DEPLOYMENT_TARGET" CFLAGS="-I$SYSROOT/usr/include $COMMON_FLAGS -Wno-unguarded-availability-new -Wunguarded-availability" CPPFLAGS="$CFLAGS" LDFLAGS="-L$SYSROOT/usr/lib $COMMON_FLAGS -Wl,-no_weak_imports" In practice I'd not target such an old OS version when building backwards-compatibly as the associated workload would be rather large. Building on the latest version and supporting a couple of versions back down is still rather manageable and sane in my experience. Against this backdrop, I'm understanding of people, especially open source volunteers, choosing to build both CPython and their distributable wheels on 10.6 as the amount of effort to validate and support more permutations is rather large for almost no benefit. The expectation levels on the Windows side have risen and thus since Python 3.5 that stack has moved to build with a newer Visual Studio and is also offering a nicer userspace install. I'm taking this thread as a signal of the expectations of the macOS crowd having risen as well. A midway compromise point might be to coerce the community to start building on 10.11 as 10.12 was again a great hardware support divider such as 10.7 was and it's starting to have been long enough since 10.11. On the actual topic: If one explicitly needs some build-time details to be specific, one can just build the egg oneself. Otherwise whatever pip pulls in and is working is good enough for the job. -- Joni Orponen [1] https://reviews.llvm.org/D34264 ___ Pythonmac-SIG maillist - Pythonmac-SIG@python.org https://mail.python.org/mailman/listinfo/pythonmac-sig unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG
Re: [Pythonmac-SIG] Problem with pip
> On 18.01.2018, at 14:58, Joni Orponen wrote: > > The offical CPython distributions seem to be compiled on 10.6 for > compatibility reasons. It is easier to compile CPython forwards-compatible > than to manually patch pyconfig.h after configure to exclude the syscalls not > available on your older target platforms. With the following config line you compile a backwards compatible Python (here 64-bit only since I don't like fiddling around with hard to compile fat-libs). # for 2.7 env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue and it yields a Python that can be py2apped and run on 10.6 as long as you don't use newer Frameworks or libs that need OSX > 1.6 -karsten ___ Pythonmac-SIG maillist - Pythonmac-SIG@python.org https://mail.python.org/mailman/listinfo/pythonmac-sig unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG
Re: [Pythonmac-SIG] Problem with pip
On Wed, Jan 17, 2018 at 6:42 PM, Shawn Tolidano wrote: > > So - to my knowledge, it has NO BUSINESS using the 10.6 as a compatible > wheel and should instead download the source. > The offical CPython distributions seem to be compiled on 10.6 for compatibility reasons. It is easier to compile CPython forwards-compatible than to manually patch pyconfig.h after configure to exclude the syscalls not available on your older target platforms. For the people compiling wheels for distribution it would make sense to mirror CPython for their distributions for similar reasons. Is there some newer syscall path you absolutely need to go through (for reasons of sandboxing requirements? a specification requirement for using getentropy()?) or what is the actual problem you are trying to fix here? -- Joni Orponen ___ Pythonmac-SIG maillist - Pythonmac-SIG@python.org https://mail.python.org/mailman/listinfo/pythonmac-sig unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG
