[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-40019: "test_gdb  should better detect when Python is optimized".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

STINNER Victor  added the comment:

There consensus seems to be towards the status quo: -Og is a good trade off 
between debug and performance for most core developers usage.

For the Fedora package, it may make sense to use -O0, but that should be 
discussed in Fedora ;-)

So I close the issue. Thanks for the interesting discussion :-)

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[Ned Deily]

Change by Ned Deily :


--
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The purpose of using -Og is that it significantly speeds up tests. You can 
always pass CFLAGS="-O0" if you debug particular debugger issues, but in normal 
cases -Og looks more beneficial. It saves hours of time of common developers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

STINNER Victor  added the comment:

> Do note though that if the -D_FORTIFY_SOURCE=2 hardening flag is used, the 
> compilation will fail with an optimization level less than -Og.

Does it make any sense to use a debug build in production? Does it make sense 
to enable hardening on a debug buil?d

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

It seems that it's still being worked on from gcc's side:

https://gcc.gnu.org/bugzilla//show_bug.cgi?id=78685

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[Benjamin Peterson]

Benjamin Peterson  added the comment:

If -Og is breaking debugging, isn't that a compiler bug? The GCC manpage claims 
" -Og enables optimizations that do not interfere with debugging."

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Correction, not fail, just a ton of warnings. The same is true for 
-D_FORTIFY_SOURCE=1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Do note though that if the -D_FORTIFY_SOURCE=2 hardening flag is used, the 
compilation will fail with an optimization level less than -Og.

Haven't tried yet with -D_FORTIFY_SOURCE=1 to see if it works with -O0.

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

STINNER Victor  added the comment:

My use case is to debug a crash using a Python compiled in debug module, like 
python3-debug program in Fedora. Since the debug ABI is now compatible with the 
release build, the idea is to attempt to reproduce a crash in gdb using 
python3-debug instead of python3, and then use gdb to see what's going on.

With -Og, the call stack is wrong sometimes, and some function arguments and 
local variables cannot be read (displayed as ).

On Travis CI, a few months ago, Python was built in debug mode using -O3. But 
it was a side effect of OpenSSL flags if I recall correctly.

With my PR 16544, Travis CI now uses -O0.

--
nosy: +pablogsal, pitrou, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +16133
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16544

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

STINNER Victor  added the comment:

test_gdb is failing in different architectures and different operating systems:

* Fedora, Python 3.8, x86-64:
  https://bugzilla.redhat.com/show_bug.cgi?id=1734327

* RHEL8, Python 3.6, s390x
  https://bugzilla.redhat.com/show_bug.cgi?id=1712977

* RHEL 8, Python 3.6, ppc64le
  https://bugzilla.redhat.com/show_bug.cgi?id=1714733

I'm quite sure that most issues are caused by the compiler optimization level 
which is not -O0.

See also bpo-37631: the debug build of the RHEL8 package is supposed to be 
compiled using -O0 using EXTRA_CFLAGS variable, but -O0 in EXTRA_CFLAGS 
variable is overriden by -02 in CFLAGS_NODIST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

STINNER Victor  added the comment:

I basically propose to revert bpo-23445 change:

commit 3d6c784371bccc2407048652bce50c5bccf9b1af
Author: Antoine Pitrou 
Date:   Wed Feb 11 19:39:16 2015 +0100

Issue #23445: pydebug builds now use "gcc -Og" where possible, to make the 
resulting executable faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38350] ./configure --with-pydebug should use -O0 rather than -Og

[STINNER Victor]

New submission from STINNER Victor :

In Python 2.7, when using ./configure --with-pydebug, Python is built using -O0 
compiler optimization level: disable all optimizations. The comment is quite 
explicit: "Optimization messes up debuggers, so turn it off for debug builds".

if test "$Py_DEBUG" = 'true' ; then
# Optimization messes up debuggers, so turn it off for
# debug builds.
OPT="-g -O0 -Wall $STRICT_PROTO"
else
OPT="-g $WRAP -O3 -Wall $STRICT_PROTO"
fi

In Python 3, -Og is preferred over -O0 for pydebug, if -Og is available:

if test "$Py_DEBUG" = 'true' ; then
# Optimization messes up debuggers, so turn it off for
# debug builds.
if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then
OPT="-g -Og -Wall"
else
OPT="-g -O0 -Wall"
fi
else
OPT="-g $WRAP -O3 -Wall"
fi

Problem: in my experience, gdb traceback doesn't make sense sometimes, and gdb 
fails to inspect some function arguments and some variables which are displayed 
as .

See a very concrete example with a test_gdb failure on x86-64 when Python is 
compiled using gcc -Og:
https://bugzilla.redhat.com/show_bug.cgi?id=1734327#c22

My colleague who is working on gdb suggests to use -O0:
https://bugzilla.redhat.com/show_bug.cgi?id=1734327#c27

Since I started contributing to Python, I always forced gcc -O0 because any 
other optimization level caused me many issues in gdb. I'm using -O0 for 10 
years with sucess.

The GCC documentation says "It is a better choice than -O0 for producing 
debuggable code because some compiler passes that collect debug information are 
disabled at -O0." But my experience says the opposite.

Note: instead of -g, we could use -g3 to include debug information for macros 
and defines.

--

I'm proposing to change the *default* compiler flags from -Og to -O0. 
Obviously, Linux distributions and developers are free to override the compiler 
optimization level. For example: ./configure --with-pydebug CFLAGS="-Og" 
ensures that Python is always built using -Og.

I propose to modify 3.7, 3.8 and master branches.

--
components: Build
messages: 353746
nosy: vstinner
priority: normal
severity: normal
status: open
title: ./configure --with-pydebug should use -O0 rather than -Og
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com