Attached is my manual verification steps of testing the proposed package for Focal.
Package version tested: 2.7.18-1~20.04.4 Part of the verification process is to ensure no regressions happen in no-change rebuilds of the current wheels compiled with python2.7 in our archives. It was determined that the best way to do this was to make this feature opt-in, so a test is performed to ensure that functionally no changes happen for no-change rebuilds. This was tested in this PPA - https://launchpad.net/~mitchdz/+archive/ubuntu/python-stdlib-extensions- py2-opt Observe the build logs and see The specific build log for Focal is https://launchpadlibrarian.net/714574260/buildlog_ubuntu-focal- amd64.python-stdlib-extensions_2.7.18-1ubuntu1~focal1_BUILDING.txt.gz Where you will see the following lines showing the workaround is not used: cd 2.7 && python2.7 setup.py build running build running build_ext There is a workaround to now inherit optimization CFLAGS when compiling wheels. To enable this, set APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND in your environment. See LP: https://launchpad.net/bugs/2002043 for further context. APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND not detected. building 'gdbm' extension creating build creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/Modules x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-CxOYiX/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I. -I/usr/include/python2.7 -c Modules/gdbmmodule.c -o build/temp.linux-x86_64-2.7/Modules/gdbmmodule.o creating build/lib.linux-x86_64-2.7 x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-CxOYiX/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-CxOYiX/python2.7-2.7.18=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC build/temp.linux-x86_64-2.7/Modules/gdbmmodule.o -lgdbm -o build/lib.linux-x86_64-2.7/gdbm.so building '_tkinter' extension Notice the lack of the OPT flags (-DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes) in the compile time flags for the module. ** Attachment added: "PYthon2.7_focal_opt_flags_verification.txt" https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+attachment/5746688/+files/PYthon2.7_focal_opt_flags_verification.txt ** Tags removed: verification-needed verification-needed-focal ** Tags added: verification-done verification-done-focal ** Description changed: [ Impact ] When compiling Python extensions using Python2, CFLAGS optimization flags are dropped. This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream. The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars(). [ Test Plan ] There will be 2 separate tests for this bug: * Ensuring no-change rebuilds are not changed * Ensuring local builds are not changed unless environment variable is set Test 1) No-change rebuilds To test that no-change rebuilds are not changed, the package python- stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs. Test 2) Functional test 1. Create test container $ lxc launch ubuntu:jammy jammy-2002043 - $ lxc shell ubuntu:jammy jammy-2002043 + $ lxc shell jammy-2002043 2. Install required packages For Jammy # apt update -y && apt install -y python2 python-pip For Focal - # apt update -y && apt install -y python2 python-setuptools + # apt update -y && apt install -y python2 python-setuptools gcc 3. Create test files # mkdir testprog # cd testprog # cat >setup.py <<EOL from setuptools import setup, Extension setup( name="test", ext_modules=[Extension("test", sources=["testmodule.c"])], zip_safe=False ) EOL # cat >testmodule.c <<EOL #include <stdio.h> int main(void) { printf("This is test program"); return 0; } EOL 4. Compile a test program # python2 setup.py build_ext --inplace 5. Check CFLAGS # python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))" -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security 6. Check the flags used to compile the test wheel # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans 7. Install fixed python Once updated, this will simply be an apt update && apt upgrade # add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y # apt install -y python2.7 # dpkg -s python2.7 | grep Version: Version: 2.7.18-13ubuntu1.2~jammy9 8. Clean build # rm -rf build/ test.so 9. Enable opt-in environment variable # export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND="" 9. Rebuild with new python2.7 installed # python2 setup.py build_ext --inplace 10. Check build flags # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection [ Where problems could occur ] * Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made * The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python2.7 in Ubuntu. https://bugs.launchpad.net/bugs/2002043 Title: Python extension modules get built using wrong compiler flags with python2 Status in python2.7 package in Ubuntu: Invalid Status in python2.7 source package in Bionic: Won't Fix Status in python2.7 source package in Focal: Fix Committed Status in python2.7 source package in Jammy: Fix Committed Status in python2.7 source package in Kinetic: Invalid Status in python2.7 source package in Lunar: Invalid Status in python2.7 source package in Mantic: Invalid Bug description: [ Impact ] When compiling Python extensions using Python2, CFLAGS optimization flags are dropped. This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream. The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars(). [ Test Plan ] There will be 2 separate tests for this bug: * Ensuring no-change rebuilds are not changed * Ensuring local builds are not changed unless environment variable is set Test 1) No-change rebuilds To test that no-change rebuilds are not changed, the package python- stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs. Test 2) Functional test 1. Create test container $ lxc launch ubuntu:jammy jammy-2002043 $ lxc shell jammy-2002043 2. Install required packages For Jammy # apt update -y && apt install -y python2 python-pip For Focal # apt update -y && apt install -y python2 python-setuptools gcc 3. Create test files # mkdir testprog # cd testprog # cat >setup.py <<EOL from setuptools import setup, Extension setup( name="test", ext_modules=[Extension("test", sources=["testmodule.c"])], zip_safe=False ) EOL # cat >testmodule.c <<EOL #include <stdio.h> int main(void) { printf("This is test program"); return 0; } EOL 4. Compile a test program # python2 setup.py build_ext --inplace 5. Check CFLAGS # python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))" -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security 6. Check the flags used to compile the test wheel # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans 7. Install fixed python Once updated, this will simply be an apt update && apt upgrade # add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y # apt install -y python2.7 # dpkg -s python2.7 | grep Version: Version: 2.7.18-13ubuntu1.2~jammy9 8. Clean build # rm -rf build/ test.so 9. Enable opt-in environment variable # export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND="" 9. Rebuild with new python2.7 installed # python2 setup.py build_ext --inplace 10. Check build flags # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection [ Where problems could occur ] * Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made * The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
