[issue24565] the f_lineno getter is broken

[daniel hahler]

daniel hahler  added the comment:

This is likely covered by existing/linked issues already, but wanted to leave 
it here nonetheless:

Given t-pdb.py:
```
import sys


def main():
sys.stdout.write("main...\n")
assert 0


if __name__ == "__main__":
main()
```

Without the fix from the PR:
```
% python3.8 -m pdb -c cont t-pdb.py
main...
Traceback (most recent call last):
  File "…/pyenv/3.8.0/lib/python3.8/pdb.py", line 1702, in main
pdb._runscript(mainpyfile)
  File "…/pyenv/3.8.0/lib/python3.8/pdb.py", line 1571, in _runscript
self.run(statement)
  File "…/pyenv/3.8.0/lib/python3.8/bdb.py", line 587, in run
exec(cmd, globals, locals)
  File "", line 1, in 
  File "…/Vcs/cpython/t-pdb.py", line 1, in 
import sys
  File "…/Vcs/cpython/t-pdb.py", line 6, in main
assert 0
AssertionError
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> …/Vcs/cpython/t-pdb.py(6)main()
-> assert 0
(Pdb)
```

With the fix:
```
% /tmp/cpython-bisect/bin/python3.8 -m pdb -c cont t-pdb.py
main...
Traceback (most recent call last):
  File "/tmp/cpython-bisect/lib/python3.8/pdb.py", line 1703, in main
pdb._runscript(mainpyfile)
  File "/tmp/cpython-bisect/lib/python3.8/pdb.py", line 1572, in _runscript
self.run(statement)
  File "/tmp/cpython-bisect/lib/python3.8/bdb.py", line 583, in run
exec(cmd, globals, locals)
  File "", line 1, in 
  File "…/Vcs/cpython/t-pdb.py", line 10, in 
main()
  File "…/Vcs/cpython/t-pdb.py", line 6, in main
assert 0
AssertionError
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> …/Vcs/cpython/t-pdb.py(6)main()
-> assert 0
(Pdb)
```

As you can see the traceback in the fixed case contains `main()` correctly,
while it has `import sys` (the scripts first line) otherwise.

I can only repeat myself to ask for reviewing/merging 
https://github.com/python/cpython/pull/12419.

--
nosy: +blueyed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Xavier de Gaye  added the comment:

Fixed a bug in the implementation of PR 12419 while running the coverage.py 
test suite: f_lineno must be valid upon 'call' trace events. The confusion 
stems from the reason why until now we have prevented line jumps from 'call' 
trace events, see below.

Summary:
  * Full coverage.py tests OK (including the C tracer).
  * Fixes issues #7238, #16482, #17277 and #17697.
  * We could now remove the restriction that prevents jumps from a call trace 
event. The restriction was based on the fact that f->f_trace is NULL until 
after the first return from call_trampoline() and setting f_lineno from within 
this first call to call_trampoline() would have had the f_lineno getter still 
return the value returned by PyCode_Addr2Line(), very confusing for the user !
  * A trace function may be set in an extension modules by PyEval_SetTrace() 
and the extension module may not use f->f_trace and still make jumps using 
f_lineno.
  * Fixes _PyTraceback_Add() at 
https://github.com/python/cpython/blob/58721a903074d28151d008d8990c98fc31d1e798/Python/traceback.c#L272

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[daniel hahler]

Change by daniel hahler :


--
versions: +Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Change by Xavier de Gaye :


--
pull_requests: +12373

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +5972
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Xavier de Gaye  added the comment:

Added PR 6233.
One of the GitHub bots failed to link this issue with PR 6233. Maybe this is 
related to the fact that connections to bpo are currently failing intermitently 
with the message:

An error occurred during a connection to bugs.python.org. Peer’s certificate 
has an invalid signature. Error code: SEC_ERROR_BAD_SIGNATURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Xavier de Gaye  added the comment:

A trace function may also be set in extension modules by PyEval_SetTrace() and 
it may not use f->f_trace. This is another reason why f->f_trace cannot be used 
in PyFrame_GetLineNumber() to know when f->f_lineno is valid.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Xavier de Gaye  added the comment:

I will work on it shortly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Could you please convert your patches to a PR Xavier?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

New submission from Xavier de Gaye:

The last paragraph of Objects/lnotab_notes.txt explains that the f_lineno 
member of the PyFrameObject structure is needed to store the line number of the 
last line tracing event so that this value may be used as the line number of 
the return event instead of the (sometimes confusing) value computed from 
f_lasti.  The f_lineno getter must then return the value of f-f_lineno 
(instead of the value computed from f-f_lasti) when tracing is set. The 
current implementation translates tracing is set as the local f_trace trace 
function is not NULL, this is wrong for the following reasons:

* AFAIK it is not documented anywhere that Python users implementing a trace 
function must delete the local f_trace functions of all the frames when setting 
the
  global trace function to None via sys.settrace(None) (issue 7238).

* Bdb.set_continue() in the bdb module of the std lib seems to know about this 
and attempts to do it, but fails to delete f_trace from the topmost frame, 
named botframe (sic) (issue 16482, issue 17697).

* It is not obvious how to delete the f_trace of all suspended generators when 
setting the global trace function to None (issue 17277).

* _PyTraceback_Add() in Python/traceback.c sets frame-f_lineno, obviously 
forgetting that it is useless since its f_trace is NULL.

This patch changes the semantics of f_lineno by stating that f_lineno is 
invalid when its value is -1. When tracing, the f_lineno of all the frames on 
the stack is valid. The f_lineno of a suspended generator is always invalid.

--
components: Interpreter Core
files: f_lineno.patch
keywords: patch
messages: 246275
nosy: belopolsky, xdegaye
priority: normal
severity: normal
status: open
title: the f_lineno getter is broken
versions: Python 3.6
Added file: http://bugs.python.org/file39865/f_lineno.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24565
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Xavier de Gaye added the comment:

Uploading the corresponding test cases.

--
Added file: http://bugs.python.org/file39866/f_lineno_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24565
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24565] the f_lineno getter is broken

[Xavier de Gaye]

Changes by Xavier de Gaye xdeg...@gmail.com:


--
type:  - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24565
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com