You can add something obvious, like -DMITCHELL_WAS_HERE :) On Thu, 8 Feb 2024, 18:35 Mitchell Dzurick, <2002...@bugs.launchpad.net> wrote:
> That's a great idea Andreas! It's a little difficult to pinpoint the > exact OPT flags as they are bunched up with the other compiler flags, so > I'll add a test in addition to the current ones which makes it easier to > see what's happening. > > -- > You received this bug notification because you are a member of Ubuntu > Pythoneers, which is subscribed to python2.7 in Ubuntu. > https://bugs.launchpad.net/bugs/2002043 > > Title: > Python extension modules get built using wrong compiler flags with > python2 > > Status in python2.7 package in Ubuntu: > Invalid > Status in python2.7 source package in Bionic: > Won't Fix > Status in python2.7 source package in Focal: > Fix Committed > Status in python2.7 source package in Jammy: > Fix Committed > Status in python2.7 source package in Kinetic: > Invalid > Status in python2.7 source package in Lunar: > Invalid > Status in python2.7 source package in Mantic: > Invalid > > Bug description: > [ Impact ] > > When compiling Python extensions using Python2, CFLAGS optimization > flags are dropped. > > This behavior has been caused by our update in this patch > > http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz > which differs from upstream. > > The fix modifies the portion of code in Lib/distutils/sysconfig.py > which gets the cflags from the environments, and includes the dropped > OPT flag from get_config_vars(). > > [ Test Plan ] > > There will be 2 separate tests for this bug: > * Ensuring no-change rebuilds are not changed > * Ensuring local builds are not changed unless environment variable is > set > > Test 1) No-change rebuilds > > To test that no-change rebuilds are not changed, the package python- > stdlib-extensions will be built against the new python2.7, and confirm > the compiler flags are not altered. This will be a manual test and > visual inspection of the build logs. > > Test 2) Functional test > > 1. Create test container > $ lxc launch ubuntu:jammy jammy-2002043 > $ lxc shell ubuntu:jammy jammy-2002043 > > 2. Install required packages > For Jammy > # apt update -y && apt install -y python2 python-pip > For Focal > # apt update -y && apt install -y python2 python-setuptools > > 3. Create test files > # mkdir testprog > # cd testprog > # cat >setup.py <<EOL > from setuptools import setup, Extension > > setup( > name="test", > ext_modules=[Extension("test", sources=["testmodule.c"])], > zip_safe=False > ) > EOL > # cat >testmodule.c <<EOL > #include <stdio.h> > > int main(void) > { > printf("This is test program"); > return 0; > } > EOL > > 4. Compile a test program > # python2 setup.py build_ext --inplace > > 5. Check CFLAGS > # python2 -c "import sysconfig; > print(sysconfig.get_config_var('CFLAGS'))" > -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes > -Wdate-time -D_FORTIFY_SOURCE=2 -g > -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto > -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong > -Wformat -Werror=format-security > > 6. Check the flags used to compile the test wheel > # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O > GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 > -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv > -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC > -fltrans > > 7. Install fixed python > Once updated, this will simply be an apt update && apt upgrade > # add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y > # apt install -y python2.7 > # dpkg -s python2.7 | grep Version: > Version: 2.7.18-13ubuntu1.2~jammy9 > > 8. Clean build > # rm -rf build/ test.so > > 9. Enable opt-in environment variable > # export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND="" > > 9. Rebuild with new python2.7 installed > # python2 setup.py build_ext --inplace > > 10. Check build flags > # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O > GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 > -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv > -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC > -fltrans > GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 > -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects > -fstack-protector-strong -fPIC -fasynchronous-unwind-tables > -fstack-protector-strong -fstack-clash-protection -fcf-protection > > [ Where problems could occur ] > > * Changing optimization flags can cause a myriad of unattended side > effects, but the change being opt-in means the users should be aware a > change is being made > * The change is opt-in, so an informative message is printed to the > console to spread awareness of the issue and how to use the workaround. A > questionably made CI/CD system might see the new console output and flag it > as a failure, so it is important to choose wording carefully. > > To manage notifications about this bug go to: > > https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions > > Launchpad-Notification-Type: bug > Launchpad-Bug: distribution=ubuntu; sourcepackage=python2.7; > component=universe; status=Invalid; importance=Undecided; assignee=None; > Launchpad-Bug: distribution=ubuntu; distroseries=bionic; > sourcepackage=python2.7; component=main; status=Won't Fix; importance=High; > assignee=None; > Launchpad-Bug: distribution=ubuntu; distroseries=focal; > sourcepackage=python2.7; component=universe; status=Fix Committed; > importance=High; assignee=mitchell.dzur...@canonical.com; > Launchpad-Bug: distribution=ubuntu; distroseries=jammy; > sourcepackage=python2.7; component=universe; status=Fix Committed; > importance=High; assignee=mitchell.dzur...@canonical.com; > Launchpad-Bug: distribution=ubuntu; distroseries=kinetic; > sourcepackage=python2.7; component=None; status=Invalid; importance=High; > assignee=None; > Launchpad-Bug: distribution=ubuntu; distroseries=lunar; > sourcepackage=python2.7; component=None; status=Invalid; > importance=Undecided; assignee=None; > Launchpad-Bug: distribution=ubuntu; distroseries=mantic; > sourcepackage=python2.7; component=None; status=Invalid; > importance=Undecided; assignee=None; > Launchpad-Bug-Tags: patch se-sponsor-halves verification-needed > verification-needed-focal verification-needed-jammy > Launchpad-Bug-Information-Type: Public > Launchpad-Bug-Private: no > Launchpad-Bug-Security-Vulnerability: no > Launchpad-Bug-Commenters: ahasenack crichton fabio.martins ghadi-rahme > halves mitchdz nafeabd rpocase tjaalton vorlon xnox > Launchpad-Bug-Reporter: Nafees (nafeabd) > Launchpad-Bug-Modifier: Mitchell Dzurick (mitchdz) > Launchpad-Message-Rationale: Subscriber (python2.7 in Ubuntu) @pythoneers > Launchpad-Message-For: pythoneers > > -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to python2.7 in Ubuntu. https://bugs.launchpad.net/bugs/2002043 Title: Python extension modules get built using wrong compiler flags with python2 Status in python2.7 package in Ubuntu: Invalid Status in python2.7 source package in Bionic: Won't Fix Status in python2.7 source package in Focal: Fix Committed Status in python2.7 source package in Jammy: Fix Committed Status in python2.7 source package in Kinetic: Invalid Status in python2.7 source package in Lunar: Invalid Status in python2.7 source package in Mantic: Invalid Bug description: [ Impact ] When compiling Python extensions using Python2, CFLAGS optimization flags are dropped. This behavior has been caused by our update in this patch http://archive.ubuntu.com/ubuntu/pool/universe/p/python2.7/python2.7_2.7.18-1~20.04.3.diff.gz which differs from upstream. The fix modifies the portion of code in Lib/distutils/sysconfig.py which gets the cflags from the environments, and includes the dropped OPT flag from get_config_vars(). [ Test Plan ] There will be 2 separate tests for this bug: * Ensuring no-change rebuilds are not changed * Ensuring local builds are not changed unless environment variable is set Test 1) No-change rebuilds To test that no-change rebuilds are not changed, the package python- stdlib-extensions will be built against the new python2.7, and confirm the compiler flags are not altered. This will be a manual test and visual inspection of the build logs. Test 2) Functional test 1. Create test container $ lxc launch ubuntu:jammy jammy-2002043 $ lxc shell ubuntu:jammy jammy-2002043 2. Install required packages For Jammy # apt update -y && apt install -y python2 python-pip For Focal # apt update -y && apt install -y python2 python-setuptools 3. Create test files # mkdir testprog # cd testprog # cat >setup.py <<EOL from setuptools import setup, Extension setup( name="test", ext_modules=[Extension("test", sources=["testmodule.c"])], zip_safe=False ) EOL # cat >testmodule.c <<EOL #include <stdio.h> int main(void) { printf("This is test program"); return 0; } EOL 4. Compile a test program # python2 setup.py build_ext --inplace 5. Check CFLAGS # python2 -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))" -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-W40Ff2/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security 6. Check the flags used to compile the test wheel # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans 7. Install fixed python Once updated, this will simply be an apt update && apt upgrade # add-apt-repository ppa:mitchdz/python2.7-optimization-flags -y # apt install -y python2.7 # dpkg -s python2.7 | grep Version: Version: 2.7.18-13ubuntu1.2~jammy9 8. Clean build # rm -rf build/ test.so 9. Enable opt-in environment variable # export APPLY_LP2002043_UBUNTU_CFLAGS_WORKAROUND="" 9. Rebuild with new python2.7 installed # python2 setup.py build_ext --inplace 10. Check build flags # strings build/lib.linux-x86_64-2.7/test.so | grep -- -O GNU GIMPLE 11.4.0 -mtune=generic -march=x86-64 -g -g -g -g -O2 -O2 -fno-openmp -fno-openacc -fcf-protection=full -fno-strict-aliasing -fwrapv -fstack-protector-strong -ffat-lto-objects -fstack-protector-strong -fPIC -fltrans GNU C17 11.4.0 -mtune=generic -march=x86-64 -g -g -O2 -fno-strict-aliasing -fwrapv -flto -flto -ffat-lto-objects -fstack-protector-strong -fPIC -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection [ Where problems could occur ] * Changing optimization flags can cause a myriad of unattended side effects, but the change being opt-in means the users should be aware a change is being made * The change is opt-in, so an informative message is printed to the console to spread awareness of the issue and how to use the workaround. A questionably made CI/CD system might see the new console output and flag it as a failure, so it is important to choose wording carefully. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2002043/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
