[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

It would be nice if it were enabled by default for fatal errors (and asserts 
perhaps?).

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Dave Malcolm]

Dave Malcolm  added the comment:

Various thoughts/nitpicking:
  - is it possible to indicate with a coding convention (e.g. comments) which 
parts of the code are intended to be called from a signal handler?  It seems 
worth making this explicit.  Or perhaps put it all in one file?
  - within tests.py, check_enabled and check_disabled seem to me to be 
misnamed; it's not at all clear what they do.
  I'd suggest renaming "get_output" to "run_code", perhaps (adding a 
docstring specifying the returned value)
  "check_enabled" seems to mean "assertCodeLeadsToOutput" or somesuch.

Within backtrace.py:
  - do all platforms supported by Python have a concept of numeric 
filedescriptors? I was wondering if FILE* might be a better abstraction here 
(with flushing), then read http://bugs.python.org/issue8863#msg124385 which 
gives the reason: fprintf etc are not signal-safe
  - all of the calls to "write" ignore the return code, leading to warnings 
from GCC.  I don't think there's any good way to handle errors from these 
calls, though.

Might be nice to also have SIGABRT (as per a c-level assertion failure), 
exposed 

NB: on Fedora/RHEL we also have a whole-system crash detection system (called 
"abrt": https://fedorahosted.org/abrt/ ), and in theory, that means that for 
me, crash reports get run using the gdb pretty-print hooks.

I'm wondering to what extent this would interract with whole-system 
crash-detection tools: would it e.g. mask a SIGSEGV, so that the crash is not 
seen by that system?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Le lundi 21 mars 2011 à 15:59 +, Dave Malcolm a écrit :
> Various thoughts/nitpicking:
>   - is it possible to indicate with a coding convention (e.g.
> comments) which parts of the code are intended to be called from a
> signal handler?  It seems worth making this explicit.  Or perhaps put
> it all in one file?

Ok, good idea, I will do that. I think that I will write it in the
function comment, something like: "This function is signal safe".

The following functions are signal safe:

faulthandler_fatal_error()
faulthandler_dump_backtrace()
faulthandler_dump_backtrace_threads()
faulthandler_user()

(.. and all helper functions, used by faulthandler_dump_backtrace():
reverse_string(), dump_decimal(), dump_hexadecimal(), dump_ascii(),
dump_frame().)

>   - within tests.py, check_enabled and check_disabled seem to me to be
> misnamed; it's not at all clear what they do.
>   I'd suggest renaming "get_output" to "run_code", perhaps (adding
> a docstring specifying the returned value)
>   "check_enabled" seems to mean "assertCodeLeadsToOutput" or
> somesuch.

Ok, I will try to find better names.

> Within backtrace.py:
>   - do all platforms supported by Python have a concept of numeric
> filedescriptors? I was wondering if FILE* might be a better
> abstraction here (with flushing), then read
> http://bugs.python.org/issue8863#msg124385 which gives the reason:
> fprintf etc are not signal-safe

Yes, I think that all platforms support numeric file descriptors. On
Windows, numeric file descriptors are not the native type for file, but
Windows provides a POSIX API.

And yes, FILE* API cannot be used because it uses buffers, and buffers
are not signal safe (at least, fwrite() is not signal safe).

I tested my module on Linux, FreeBSD and Windows. I don't have other
OS :-)

>   - all of the calls to "write" ignore the return code, leading to
> warnings from GCC.  I don't think there's any good way to handle
> errors from these calls, though.

Except exiting the signal handler, I don't see anything useful to do on
write() error. I think that it is safe to ignore write() errors, and I
prefer to keep the code simple.

I don't know how to make these warnings quiet.

> Might be nice to also have SIGABRT (as per a c-level assertion
> failure), exposed 

I think that it would do that while integrating faulthandler into Python
to check the interaction with Py_FatalError().

> NB: on Fedora/RHEL we also have a whole-system crash detection system
> (called "abrt": https://fedorahosted.org/abrt/ ), and in theory, that
> means that for me, crash reports get run using the gdb pretty-print
> hooks.
>
> I'm wondering to what extent this would interract with whole-system
> crash-detection tools: would it e.g. mask a SIGSEGV, so that the crash
> is not seen by that system?

faulthandler is compatible with gdb and abrt. For gdb, you get the
SIGSEGV signal before faulthandler: the "signal SIGSEGV" gdb command
will call faulthandler signal handler. For abrt, faulthandler does print
the backtrace, and then abrt is called. Execution order:

crash => gdb => faulthandler => abrt

(I think that gdb and abrt are exclusive)

I didn't try abrt, but I tried Ubuntu Apport which is smiliar (Apport
uses /proc/sys/kernel/core_pattern with a pipe).

You may test faulthandler on Fedora to tell me if it interacts correctly
with abrt :-)
https://github.com/haypo/faulthandler/wiki/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> Ok, good idea, I will do that. I think that I will write it in the
> function comment, something like: "This function is signal safe".

Done in 6685691dfcbd3644feffcb197491bce3168ff5de (git SHA-1)

While checking the usage of signal safe functions, I found a bug: 
dumpbacktrace_later() signal handler called Py_CLEAR() which is far from being 
signal safe. I tried to replace it be a pending call, but Py_AddPendingCall() 
doesn't look to be signal safe. So I just removed the call. 
cancel_dumpbacktrace_later() is already documented as being required to clear 
the reference to the file.

> >   - within tests.py, ...
> Ok, I will try to find better names.

Done in db99e17dea187bec4248aca9fc81f673d88dec93. I documented methods and 
renamed some of them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I renamed some functions to conform to the PEP 8 (and have more readable 
function names). I prefer to do it today instead having to keep ugly names for 
years :-)

Functions are now:

 * enable(file=sys.stderr, all_threads=False)
 * disable()
 * is_enabled()

 * dump_backtrace(file=sys.stderr, all_threads=False)
 * dump_backtrace_later(delay, repeat=False, file=sys.stderr, all_threads=False)
 * cancel_dump_backtrace_later()

 * register(signum, file=sys.stderr, all_threads=False)
 * unregister(signum)

 * sigbus()
 * sigfpe()
 * sigill()
 * sigsegv()

Refer to the README file for the details.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> It would be nice if it were enabled by default for fatal errors
> (and asserts perhaps?).

That would mean that the module should be a builtin module, or that it is 
always loaded in memory.

I am maybe ok to enable it by default for debug builds, but not for release 
builds. At least, not in a first time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Andreas Stührk]

Changes by Andreas Stührk :


--
nosy: +Trundle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
>> It would be nice if it were enabled by default for fatal errors
>> (and asserts perhaps?).
> 
> That would mean that the module should be a builtin module, or that it is 
> always loaded in memory.
> 
> I am maybe ok to enable it by default for debug builds, but not for release 
> builds. At least, not in a first time.

FWIW: We've been using a similar approach in mxTools.safecall()
(which is an undocumented function only and only available if
mxTools is compiled with a special define; see the mxTools.c source
code for details).

Our use case is different, though: We use this function to
call a Python callable wrapped with a longjump environment and
instead of dumping a traceback we issue an exception, so that
the calling code can deal with the problem in a more flexible way.

The main reasoning here is proper cleanup, signal the problem
to managing code and providing more information about the
interpreter state at the time of the segfault.

The function is mostly used when calling out to (possibly buggy)
3rd party extensions hooked into a long running web server
application, where it enables the server process to write
request information about the problem to a log file and then
shuts down the process very quickly.

Note that we haven't bothered dealing with threads, since we
normally use single threaded server processes and because signals
and threads don't interact reliably anyway.

Perhaps you could consider adding a similar approach (raising
an exception instead of writing a traceback) to the module.
We could then port our code to use your module, which is
more advanced.

BTW: Why do you call the traceback functions "*backtrace*" instead
of "*traceback*" ?

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com



::: Try our new mxODBC.Connect Python Database Interface for free ! 

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Le mardi 22 mars 2011 à 09:01 +, Marc-Andre Lemburg a écrit :
> Perhaps you could consider adding a similar approach (raising
> an exception instead of writing a traceback) to the module.
> We could then port our code to use your module, which is
> more advanced.

I already wrote a patch implementing this idea, 2 years ago, and the
idea was rejected. Re-read the history of the module, described in the
first message of this issue:

<< History of this module.

I first proposed a segfault handler using setjmp/longjmp to raise a
classic Python exception. So it was  possible to execute Python code
after a segfault (including stack overflow). But the idea was rejected
because the Python internal state may be corrupted if the segfault was
an invalid memory write (buffer overflow?), and anyway we cannot
warranty that the internal state is still consistent after a long jump.

=> http://bugs.python.org/issue3999 (sept. 2009)

... >>

You may dig python-dev archives to learn more about this proposition.

> BTW: Why do you call the traceback functions "*backtrace*" instead
> of "*traceback*" ?

Oh... I don't know. Since Python uses "traceback" word, I will rename my
functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
> Le mardi 22 mars 2011 à 09:01 +, Marc-Andre Lemburg a écrit :
>> Perhaps you could consider adding a similar approach (raising
>> an exception instead of writing a traceback) to the module.
>> We could then port our code to use your module, which is
>> more advanced.
> 
> I already wrote a patch implementing this idea, 2 years ago, and the
> idea was rejected. Re-read the history of the module, described in the
> first message of this issue:
> 
> << History of this module.
> 
> I first proposed a segfault handler using setjmp/longjmp to raise a
> classic Python exception. So it was  possible to execute Python code
> after a segfault (including stack overflow). But the idea was rejected
> because the Python internal state may be corrupted if the segfault was
> an invalid memory write (buffer overflow?), and anyway we cannot
> warranty that the internal state is still consistent after a long jump.
> 
> => http://bugs.python.org/issue3999 (sept. 2009)
> 
> ... >>
> 
> You may dig python-dev archives to learn more about this proposition.

I know that the state after a longjump in such a case cannot
be guaranteed, but from using the approach in practice over many
years, I know that it works quite well - most situations related
to programming errors in the 3rd party modules we plug into the
system (quant libs, ie. code that does pricing and risk analysis
of financial products) are segfaults related to NULL pointer
derefs and division by zero. Complete corruption of the Python
interpreter state is very rare.

Anyway, providing such an approach as option (with the default
being your implemented traceback printout), would leave the
decision to take this risk to the programmer and I think that's
a fair deal.

>> BTW: Why do you call the traceback functions "*backtrace*" instead
>> of "*traceback*" ?
> 
> Oh... I don't know. Since Python uses "traceback" word, I will rename my
> functions.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com



::: Try our new mxODBC.Connect Python Database Interface for free ! 

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Scott Dial]

Scott Dial  added the comment:

Antoine Pitrou wrote:
> It would be nice if it were enabled by default for fatal errors (and asserts 
> perhaps?).

I feel like a broken record. This code hardcodes fd=2 as a write target on 
crash, which is not safe thing to do at all. You can argue that adopters of 
Python 3.3 should have to deal with that fact, but it's obscure and there is no 
way to warn anyone about it except by putting a NEWS item, and if the PyCapsule 
discussion on python-dev have taught us anything: even well meaning programmers 
miss these things all the time.

I have stated this repeatedly on the other issues for this same discussion. I 
think creating a completely new issue for this same topic has segmented the 
discussion unfortunately. I wrote a much longer and more thoughtful explanation 
of why faulthandler writes to the wrong "thing" here:

http://bugs.python.org/msg124381

AFAICT, Victor has addressed my issue by giving programmers yet another 
interface to configure (that they may or may not be aware of). So, the only way 
this acceptable to me is if it's off by default and a programmer who wants this 
functionality opts-in and has taken care to make sure it does the right thing. 
My suggestion that faulthandler needs to find a way to be coupled to 
"sys.stderr" still stands.

--
nosy: +scott.dial

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Antoine Pitrou wrote:
> > It would be nice if it were enabled by default for fatal errors (and 
> > asserts perhaps?).
> 
> I feel like a broken record. This code hardcodes fd=2 as a write target on 
> crash,

For fatal errors, you needn't be async-safe, so the fatal error code
could read fileno(stderr) and give it to the traceback printing code.
What do you think, Victor?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> For fatal errors, you needn't be async-safe, so the fatal error code
> could read fileno(stderr) and give it to the traceback printing code.

My last patch for issue #8863 does exactly that:

##
 void
 Py_FatalError(const char *msg)
 {
-fprintf(stderr, "Fatal Python error: %s\n", msg);
-fflush(stderr); /* it helps in Windows debug build */
-if (PyErr_Occurred()) {
+const int fd = fileno(stderr);
+
+fputs("Fatal Python error: ", stderr);
+fputs(msg, stderr);
+fputc('\n', stderr);
+fflush(stderr);
+
+if (PyErr_Occurred())
 PyErr_PrintEx(0);
+else {
+fputc('\n', stderr);
+fflush(stderr);
+_Py_DumpBacktrace(fd);
 }
...
##

Yes, call fileno() here is safe.

--

The main problem was on the SIGSEGV handler which was first proposed as enabled 
by default. Extract of my old patch:

+static void
+fault_handler(int signum)
+{
+const int fd = 2; /* should be fileno(stderr) */
+unsigned int i;
+fault_handler_t *handler;
...

In the faulthandler module, the last call to faulthandler.enable() saves 
sys.stderr.fileno(). If this file descriptor is replaced by a critical file, we 
have a problem. It can occurs in two cases:
 - stderr was closed (after the call to enable) and a new file gets its file 
descriptor number
 - dup2() was used

Both cases may occur on a server application.

But I think that everybody agrees to disable the SIGSEGV handler by default.

--

I'm preparing the integration of faulthandler in the following Mercurial repo:
http://hg.python.org/features/faulthandler/

I didn't write the fatal error hook yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Let's try the "Remote hg repo" feature.

--
hgrepos: +8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


--
keywords: +patch
Added file: http://bugs.python.org/file21393/ec274420e9e2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Ok, here is a full patch tested on Linux and Windows:
 - Add faulthandler module (code, doc, tests) as a builtin module
 - Add "-X faulthandler=1" command line option and PYTHONFAULTHANDLER=1 en var 
to enable the fault handler at startup
 - Add _Py_DumpTraceback() and _Py_DumpTracebackThreads() functions to 
Python/traceback.c
 - Py_FatalError() calls _Py_DumpTraceback() if there is not exception
 - Initialize/Unload faulthandler explicitly in Py_InitializeEx()/Py_Finalize() 
to do it in the right order

I added a section in the doc to explain the file descriptor issue.

I merged different .c and .h files into faulthandler.c, the code is easier to 
read and maintain like that.

I don't know if the following test is the best test to check if SIGARLM 
constant and alarm() function are available:

+#ifdef SIGALRM
+#  define FAULTHANDLER_LATER
+#endif

(The test works at least on Linux and Windows)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

(Reminder for me: add something to Doc/whatsnew/3.3.rst and Misc/NEWS)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I've posted a review at http://bugs.python.org/review/11393/show.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21403/174bc8c03e9d.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file21393/ec274420e9e2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I updated the (Hg repo and the) patch to fix all Antoine's remarks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I don't know if it is important to document that some functions keep an 
internal reference to the file argument. For example, I wrote "It keeps a 
reference to file: use disable() to clear this reference." in 
faulthandler.enable() documentation. Is it necessary?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file21403/174bc8c03e9d.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21404/605422430234.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

New patch 605422430234.diff:
 - Remove test_refcount(), it changed faulthandler internal state, and so it 
was no more possible to use faulthandler in regrtest.py
 - Use "#ifdef HAVE_ALARM" to decide if dump_traceback_later() is available, 
instead of "#ifdef SIGALRM"
 - Don't document _sig*() functions or _fatal_error() anymore

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> I updated the (Hg repo and the) patch to fix all Antoine's remarks.

Can you make the suggested changes to the tests? Thank you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I have pushed a new branch "faulthandler-thread" in 
http://hg.python.org/features/faulthandler/. It contains an implementation of 
dump_tracebacks_later() using a watchdog thread, instead of alarm().

It has two advantages:
- it works under Windows
- it won't disrupt use of alarm() or SIGALRM by user code (including the test 
suite)

It has one drawback: you can only display all threads, since the watchdog 
thread is not a Python thread.

I haven't fixed the tests for it, I'm waiting for Victor's changes first ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

regrtest_timeout.patch: add --timeout option to regrtest.py to display the 
traceback and then exit if a test takes more than TIMEOUT seconds. It adds also 
a check on the subprocess exit code (regrtest.py -j): raise a exception if it 
is different than zero.

--
Added file: http://bugs.python.org/file21463/regrtest_timeout.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21467/c684b1e59aaa.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file21404/605422430234.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I regenerated the patch.

Last changes:

 - I merged Antoine's new implementation of 
faulthandler.dump_backtraces_later() using a thread
 - dump_backtraces_later() has a new exit option: if exit=True, call _exit()
 - disable (remove) register() and unregister() on Windows just because these 
functions are useless. No signal can be used. Only SIGSEGV and SIGBUS can be 
handled by the process, and these signals cannot be used with register(), only 
with enable()
 - fix register() and unregister() to be "signal safe": don't move the signal 
list in memory (because of realloc) or free memory (on unregister) because the 
signal handler may be running at the same time. Use a static list instead, with 
a "enabled" flag per signal. The code becomes simpler.
 - fix register(): don't reinstall the signal if it was already installed
 - speed up the test: because dump_backtraces_later() has now a subsecond 
resolution, we can use sleep of 50 ms instead of 1 sec

It tested the patch on Linux and Windows: it works. Cool, 
dump_backtraces_later() is now also supported on Windows, thanks to Antoine!

My TODO list is empty (the last item was "fix register() to be signal safe") so 
I think that the patch is ready to be commited.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Remainder for me: the patch contains some whitespace changes in Makefile.pre.in 
and Setup.dist.

dump_traceback(), enable(), dump_tracebacks_later(), register() flush the input 
file. It is not said in the doc. Should it be documented? For enable(), 
dump_tracebacks_later(), register(), it may be useless because the traceback is 
dumped later, and there may be new buffered data in the file. But I don't think 
that it hurts to flush the file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

>  - speed up the test: because dump_backtraces_later() has now a
> subsecond resolution, we can use sleep of 50 ms instead of 1 sec

This is too short, there may be random failures on some slow buildbots.
IMO, 0.5s is the minimum you can use.

+def _check_dump_tracebacks_later(self, repeat, cancel, filename):
+"""
+Call dump_tracebacks_later() two times, or three times if
repeat is True.
+Check the output: the traceback may be written 1, 2 or 3 times
+depending on repeat and cancel options.
+
+Raise an error if the output doesn't match the expect format.
+"""

The docstring is outdated. It also seems the "cancel" option isn't
useful anymore, you could remove it and simplify the test.

+process = script_helper.spawn_python('-c', code)
+stdout, stderr = process.communicate()

Shouldn't you check the return code as well?

+code = "\n".join(code)

Again, I think it would make the code simpler and more maintainable if
you used triple-quoted strings instead of lists/tuples.

When you launch a waiting thread in a subprocess, I think it's better to
set it in daemon mode so as to avoid blocking if the main thread raises
an exception.

You have a "#ifdef MS_WINDOWS" in check_signum() but that function is
not compiled under Windows (it is inside "#ifdef FAULTHANDLER_USER").

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> >  - speed up the test: because dump_backtraces_later() has now a
> > subsecond resolution, we can use sleep of 50 ms instead of 1 sec
> 
> This is too short, there may be random failures on some slow buildbots.
> IMO, 0.5s is the minimum you can use.

Ok, changed.

> The docstring is outdated.

fixed

>  It also seems the "cancel" option isn't
> useful anymore, you could remove it and simplify the test.

cancel is used by test_dump_tracebacks_later_repeat_cancel() to check
that faulthandler.cancel_dump_tracebacks_later() did cancel the last
dump_backtraces_later() request.

> +process = script_helper.spawn_python('-c', code)
> +stdout, stderr = process.communicate()
> 
> Shouldn't you check the return code as well?

Yes, except for fatal errors, because the C library exits with a
non-zero exit code (e.g. 139 for a SIGSEGV on Linux). I added a check on
the exit code.

> +code = "\n".join(code)
> 
> Again, I think it would make the code simpler and more maintainable if
> you used triple-quoted strings instead of lists/tuples.

Ok ok, done

> When you launch a waiting thread in a subprocess, I think it's better to
> set it in daemon mode so as to avoid blocking if the main thread raises
> an exception.

I doesn't know that. fixed

> You have a "#ifdef MS_WINDOWS" in check_signum() but that function is
> not compiled under Windows (it is inside "#ifdef FAULTHANDLER_USER").

Ah yes, the code became useless: removed

--

Other changes:

 - faulthandler_fatal_error() calls the previous signal handler using
raise(signum). Before, the signal was raised again because the same
instruction was executed again, and it raised the same fault. But if the
user sends manually a fatal signal to the process, the signal was
ignored because of the fault handler (but the traceback was displayed).
 - use SA_NODEFER and SA_RESTART flags in enable() and register()
 - faulthandler_get_fileno() accepts fileno()==0
 - cleanup the doc

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21470/a979bb83a94b.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file21467/c684b1e59aaa.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> faulthandler_fatal_error() calls the previous signal handler
> using raise(signum)

It doesn't work as expected on Windows: Windows doesn't call its own signal 
handler anymore. Use the previous code (only on Windows).

I also added a test for stack overflow: it fails on FreeBSD because I called 
sigaltstack() with the wrong arguments. It is now fixed.

test_faulthandler pass on Linux, Windows and FreeBSD.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21474/4adbea7c832e.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file21470/a979bb83a94b.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21476/f5a11df83d98.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file21474/4adbea7c832e.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> My TODO list is empty (the last item was "fix register() to be signal
> safe") so I think that the patch is ready to be commited.

As I wrote, I did some tests on FreeBSD, found bugs and fixed them. I also 
fixed the weird behaviour if the user sends manually a fatal signal (like 
SIGSEGV): it is no more ignored. And I patched test_faulthandler to not create 
core files. On Windows, there is still a popup warning the user a fatal error 
occured. I don't know how to disable (temporary) this popup.

Here is the updated patch and I hope the final patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

>  - speed up the test: because dump_backtraces_later() has now a
> subsecond resolution, we can use sleep of 50 ms instead of 1 sec

This is too short, there may be random failures on some slow buildbots.
IMO, 0.5s is the minimum you can use.

+def _check_dump_tracebacks_later(self, repeat, cancel, filename):
+"""
+Call dump_tracebacks_later() two times, or three times if
repeat is True.
+Check the output: the traceback may be written 1, 2 or 3 times
+depending on repeat and cancel options.
+
+Raise an error if the output doesn't match the expect format.
+"""

The docstring is outdated. It also seems the "cancel" option isn't
useful anymore, you could remove it and simplify the test.

+process = script_helper.spawn_python('-c', code)
+stdout, stderr = process.communicate()

Shouldn't you check the return code as well?

+code = "\n".join(code)

Again, I think it would make the code simpler and more maintainable if
you used triple-quoted strings instead of lists/tuples.

When you launch a waiting thread in a subprocess, I think it's better to
set it in daemon mode so as to avoid blocking if the main thread raises
an exception.

You have a "#ifdef MS_WINDOWS" in check_signum() but that function is
not compiled under Windows (it is inside "#ifdef FAULTHANDLER_USER").

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I merged the faulthandler branch into the default branch. I removed __version__ 
field: the Python version should be enough. I also fixed an infinite loop 
raised by test_capi.

test_faulthandler pass on Solaris and OpenIndiana, but it fails on PPC:



==
FAIL: test_enable_file (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 166, in test_enable_file
filename=filename)
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 84, in check_fatal_error
self.assertEqual(lines, expected)
AssertionError: Lists differ: ['Fatal Python error: Bus erro... != ['Fatal 
Python error: Segmenta...

First differing element 0:
Fatal Python error: Bus error
Fatal Python error: Segmentation fault

- ['Fatal Python error: Bus error',
+ ['Fatal Python error: Segmentation fault',
   '',
   'Traceback (most recent call first):',
   '  File "", line 4 in ']

==
FAIL: test_enable_threads (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 176, in test_enable_threads
all_threads=True)
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 84, in check_fatal_error
self.assertEqual(lines, expected)
AssertionError: Lists differ: ['Fatal Python error: Bus erro... != ['Fatal 
Python error: Segmenta...

First differing element 0:
Fatal Python error: Bus error
Fatal Python error: Segmentation fault

- ['Fatal Python error: Bus error',
+ ['Fatal Python error: Segmentation fault',
   '',
   'Current thread XXX:',
   '  File "", line 3 in ']

==
FAIL: test_gil_released (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 154, in test_gil_released
'Segmentation fault')
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 84, in check_fatal_error
self.assertEqual(lines, expected)
AssertionError: Lists differ: ['Fatal Python error: Bus erro... != ['Fatal 
Python error: Segmenta...

First differing element 0:
Fatal Python error: Bus error
Fatal Python error: Segmentation fault

- ['Fatal Python error: Bus error',
+ ['Fatal Python error: Segmentation fault',
   '',
   'Traceback (most recent call first):',
   '  File "", line 3 in ']

==
FAIL: test_sigfpe (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 104, in test_sigfpe
'Floating point exception')
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 84, in check_fatal_error
self.assertEqual(lines, expected)
AssertionError: Lists differ: [] != ['Fatal Python error: Floating...

Second list contains 4 additional elements.
First extra element 0:
Fatal Python error: Floating point exception

- []
+ ['Fatal Python error: Floating point exception',
+  '',
+  'Traceback (most recent call first):',
+  '  File "", line 3 in ']

==
FAIL: test_sigsegv (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 93, in test_sigsegv
'Segmentation fault')
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 84, in check_fatal_error
self.assertEqual(lines, expected)
AssertionError: Lists differ: ['Fatal Python error: Bus erro... != ['Fatal 
Python error: Segmenta...

First differing element 0:
Fatal Python error: Bus error
Fatal Python error: Segmentation fault

- ['Fatal Python error: Bus error',
+ ['Fatal Python error: Segmentation fault',
   '',
   'Traceback (most recent call first):',
   '  File "", line 3 in ']

--

--

___
Python tracker 

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

http://hg.python.org/features/faulthandler/rev/0943ef33495a should fix 
test_faulthandler to support SIGBUS on reading from NULL. But it doesn't fix 
the failure on SIGFPE yet. I'm running the buildbot to check if it's better or 
not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file21483/d4902114f720.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset b0680b5a5215 by Victor Stinner in branch 'default':
Issue #11393: Add the new faulthandler module
http://hg.python.org/cpython/rev/b0680b5a5215

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset df240014e72f by Victor Stinner in branch 'default':
Issue #11393: reenable all tests in regrtest.py (wooops, sorry Antoine)
http://hg.python.org/cpython/rev/df240014e72f

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

First failure: AMD64 Gentoo Wide 3.x.
http://www.python.org/dev/buildbot/all/builders/AMD64%20Gentoo%20Wide%203.x/builds/1363/

[1/1] test_faulthandler
test test_faulthandler failed -- Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.x.ochtman-gentoo-amd64/build/Lib/test/test_faulthandler.py",
 line 165, in test_stack_overflow
'(?:Segmentation fault|Bus error)')
  File 
"/home/buildbot/buildarea/3.x.ochtman-gentoo-amd64/build/Lib/test/test_faulthandler.py",
 line 95, in check_fatal_error
self.assertRegex(output, regex)
AssertionError: Regex didn't match: '^Fatal Python error: (?:Segmentation 
fault|Bus error)\n\nTraceback\\ \\(most\\ recent\\ call\\ first\\):\n  File 
"", line 3 in $' not found in ''

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

test_faulthandler failed also on AMD64 OpenIndiana 3.x: regrtest.py (only 
executing test_faulthandler) was killed after 11 min 20 sec.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 25a2aeecb34b by Victor Stinner in branch 'default':
Issue #11393: Disable test_stack_overflow of test_faulthandler
http://hg.python.org/cpython/rev/25a2aeecb34b

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I disabled test_stack_overflow because the test fails on AMD64 Gentoo Wide 3.x, 
and maybe other buildbots (AMD64 OpenIndiana 3.x?). I fear that the test uses a 
lot of CPU, memory and/or swap.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

stack_overflow.patch should fix AMD64 Gentoo Wide 3.x and AMD64 OpenIndiana 3.x 
failures. I schedule a test on their custom slaves, but these buildbots are 
still running test_faulthandler on their 3.x slaves. I interrupted the test, 
but it doesn't work (click on Cancel doesn't stop the tests). Schedule tests:

http://www.python.org/dev/buildbot/all/builders/AMD64%20OpenIndiana%20custom
http://www.python.org/dev/buildbot/all/builders/AMD64%20Gentoo%20Wide%20custom

I'm impatient to test regrtest_timeout.patch on x86 XP-4 3.x to try to learn 
more on test_multiprocessing timeout (1200 seconds)!

--
Added file: http://bugs.python.org/file21485/stack_overflow.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset a6d2c703586a by Victor Stinner in branch 'default':
Issue #11393: Fix the documentation (cancel_dump_traceback_later)
http://hg.python.org/cpython/rev/a6d2c703586a

New changeset e289b64f355d by Victor Stinner in branch 'default':
Issue #11393: limit stack overflow test to 100 MB
http://hg.python.org/cpython/rev/e289b64f355d

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

test_faulthandler blocked AMD64 Gentoo Wide 3.x and AMD64 OpenIndiana 3.x 
buildbots because of the stack overflow test.

> New changeset e289b64f355d by Victor Stinner in branch 'default':
> Issue #11393: limit stack overflow test to 100 MB
> http://hg.python.org/cpython/rev/e289b64f355d

Ok, it fixed test_faulthandler on AMD64 OpenIndiana 3.x and AMD64 Gentoo Wide 
3.x.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I opened #11727 for the patch on regrtest.py. All buildbots look happy (no more 
failure on test_faulthandler), so let's close this issue.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

There is a failure on FreeBSD:

==
FAIL: test_dump_tracebacks_later_repeat 
(test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_faulthandler.py",
 line 406, in test_dump_tracebacks_later_repeat
self.check_dump_tracebacks_later(repeat=True)
  File 
"/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_faulthandler.py",
 line 400, in check_dump_tracebacks_later
self._check_dump_tracebacks_later(repeat, cancel, None)
  File 
"/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_faulthandler.py",
 line 381, in _check_dump_tracebacks_later
trace = self.get_output(code, True, filename)
  File 
"/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_faulthandler.py",
 line 57, in get_output
self.assertEqual(exitcode, 0)
AssertionError: -11 != 0

http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%203.x/builds/1333/steps/test/logs/stdio

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset aa2ac1581d23 by Victor Stinner in branch 'default':
Issue #11393: test_faulthandler checks the exitcode after the output
http://hg.python.org/cpython/rev/aa2ac1581d23

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Same issue on PPC Leopard custom:

[224/354] test_faulthandler
test test_faulthandler failed -- Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 412, in test_dump_tracebacks_later_file
self.check_dump_tracebacks_later(file=True)
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 398, in check_dump_tracebacks_later
self._check_dump_tracebacks_later(repeat, cancel, filename)
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 381, in _check_dump_tracebacks_later
trace = self.get_output(code, True, filename)
  File 
"/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_faulthandler.py",
 line 57, in get_output
self.assertEqual(exitcode, 0)
AssertionError: 1 != 0

Reopen the issue until this test is fixed.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset e91de993964c by Victor Stinner in branch 'default':
Issue #11393: check that stdout is empty if we use a file
http://hg.python.org/cpython/rev/e91de993964c

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 61626c3f3a54 by Victor Stinner in branch 'default':
Issue #11393: get more information on assertion error (test_faulthandler)
http://hg.python.org/cpython/rev/61626c3f3a54

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Other failures on "x86 XP-4 3.x":
http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/4308

==
FAIL: test_dump_tracebacks_later (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 404, in test_dump_tracebacks_later
self.check_dump_tracebacks_later()
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 401, in check_dump_tracebacks_later
self._check_dump_tracebacks_later(repeat, cancel, None)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 390, in _check_dump_tracebacks_later
self.assertRegex(trace, '^%s$' % regex)
AssertionError: Regex didn't match: '^^Thread 0x[0-9a-f]+:\n  File 
"\\", line 7 in func\\n  File "\\", line 30 in 
\\$$' not found in 'Thread 0x07cc:\n  File "", line 7 in 
func\n  File "", line 30 in \nTraceback (most recent call 
last):\n  File "", line 30, in \n  File "", line 11, in 
func\nAssertionError'

==
FAIL: test_dump_tracebacks_later_file (test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 413, in test_dump_tracebacks_later_file
self.check_dump_tracebacks_later(file=True)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 399, in check_dump_tracebacks_later
self._check_dump_tracebacks_later(repeat, cancel, filename)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 391, in _check_dump_tracebacks_later
self.assertEqual(exitcode, 0)
AssertionError: 1 != 0

==
FAIL: test_dump_tracebacks_later_repeat 
(test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 407, in test_dump_tracebacks_later_repeat
self.check_dump_tracebacks_later(repeat=True)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 401, in check_dump_tracebacks_later
self._check_dump_tracebacks_later(repeat, cancel, None)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 390, in _check_dump_tracebacks_later
self.assertRegex(trace, '^%s$' % regex)
AssertionError: Regex didn't match: '^^Thread 0x[0-9a-f]+:\n  File 
"\\", line 7 in func\\n  File "\\", line 30 in 
\\\nThread 0x[0-9a-f]+:\n  File "\\", line 7 in func\\n  
File "\\", line 30 in \\$$' not found in 'Thread 
0x0fd8:\n  File "", line 7 in func\n  File "", line 30 in 
\nThread 0x0fd8:\n  File "", line 7 in func\n  File 
"", line 30 in \nTraceback (most recent call last):\n  File 
"", line 30, in \n  File "", line 11, in 
func\nAssertionError'

==
FAIL: test_dump_tracebacks_later_repeat_cancel 
(test.test_faulthandler.FaultHandlerTests)
--
Traceback (most recent call last):
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 410, in test_dump_tracebacks_later_repeat_cancel
self.check_dump_tracebacks_later(repeat=True, cancel=True)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 401, in check_dump_tracebacks_later
self._check_dump_tracebacks_later(repeat, cancel, None)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_faulthandler.py",
 line 390, in _check_dump_tracebacks_later
self.assertRegex(trace, '^%s$' % regex)
AssertionError: Regex didn't match: '^^Thread 0x[0-9a-f]+:\n  File 
"\\", line 7 in func\\n  File "\\", line 30 in 
\\\nThread 0x[0-9a-f]+:\n  File "\\", line 7 in func\\n  
File "\\", line 30 in \\$$' not found in 'Thread 
0x082c:\n  File "", line 7 in func\n  File "", line 30 in 
\nThread 0x082c:\n  File "", line 7 in func\n  File 
"", line 30 in \nTraceback (most recent call last):\n  File 
"", line 30, in \n  File "", line 11, in 
func\nAssertionError'


The 3 assertion errors mean that time.sleep(pause) takes less than pause 
seconds. 61626c3f3a54 adds more information about these failure.

test_dump_tracebacks_later_file() failure is the same than the PPC Leopard 
failure.

--

__

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 69f58be4688a by Victor Stinner in branch 'default':
Issue #11393: test_faulthandler is more tolerant on inaccurate time
http://hg.python.org/cpython/rev/69f58be4688a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Cool, 69f58be4688a fixed all failures (all related to dump_backtraces_later()) 
on "x86 XP-4 3.x". So "PPC Leopard" should also be fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 8b1341d51fe6 by Victor Stinner in branch 'default':
Issue #11393: Fix faulthandler_thread(): release cancel lock before join lock
http://hg.python.org/cpython/rev/8b1341d51fe6

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> New changeset 8b1341d51fe6 by Victor Stinner in branch 'default':
> Issue #11393: Fix faulthandler_thread(): release cancel lock before join lock
> http://hg.python.org/cpython/rev/8b1341d51fe6

This is wrong, it should always be released, not only when explicitly
cancelled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 0fb0fbd442b4 by Victor Stinner in branch 'default':
Issue #11393: New try to fix faulthandler_thread()
http://hg.python.org/cpython/rev/0fb0fbd442b4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 3558eecd84f0 by Victor Stinner in branch 'default':
Issue #11393: fix usage of locks in faulthandler
http://hg.python.org/cpython/rev/3558eecd84f0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

The last commits fixed test_faulthandler on FreeBSD, there are no more 
faulthandler bugs. Close this issue gain.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset e51d8a160a8a by Victor Stinner in branch 'default':
Issue #11393: fault handler uses raise(signum) for SIGILL on Windows
http://hg.python.org/cpython/rev/e51d8a160a8a

New changeset a4fa79b0d478 by Victor Stinner in branch 'default':
Issue #11393: The fault handler handles also SIGABRT
http://hg.python.org/cpython/rev/a4fa79b0d478

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset a27755b10448 by Victor Stinner in branch 'default':
Issue #11393: Fix faulthandler.disable() and add a test
http://hg.python.org/cpython/rev/a27755b10448

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 7e3ed426962f by Victor Stinner in branch 'default':
Issue #11393: _Py_DumpTraceback() writes the header even if there is no frame
http://hg.python.org/cpython/rev/7e3ed426962f

New changeset e609105dff64 by Victor Stinner in branch 'default':
Issue #11393: signal of user signal displays tracebacks even if tstate==NULL
http://hg.python.org/cpython/rev/e609105dff64

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

New submission from STINNER Victor :

<< Fault handler for SIGSEGV, SIGFPE, SIGBUS and SIGILL signals: display the
Python backtrace and restore the previous handler. Allocate an alternate stack
for this handler, if sigaltstack() is available, to be able to allocate memory
on the stack, even on stack overflow (not available on Windows).

Import the module and call faulthandler.enable() to enable the fault handler.

The fault handler is called on catastrophic cases and so it can only use
signal-safe functions (eg. it doesn't allocate memory on the heap). That's why
the backtrace is limited: it only supports ASCII encoding (use the
backslashreplace error handler for non-ASCII characters), doesn't print
the source code in the backtrace (only the filename, the function name and the
line number), is limited to 100 frames and 100 threads.

By default, the Python backtrace is written to the standard error stream. Start
your graphical applications in a terminal and run your server in foreground to
see the backtrace, or pass a file to faulthandler.enable().

faulthandler is implemented in C using signal handlers to be able to dump a
backtrace on a crash or when Python is blocked (eg. deadlock). >>

https://github.com/haypo/faulthandler

faulthandler works on Python 2.5 .. 3.3.

--

faulthandler is an alternative to gdb7 + python-gdb.py to dump the backtrace. 
faulthandler already helped me to debug a PyQt crash on Windows: I don't know 
how to install gdb on Windows, especially with faulthandler!

But I didn't get a lot of feedbacks.

--

I don't know if the whole module should be included or not.

TODO list:

 * add something to enable faulthandler on the command line: an option on the 
command line (eg. python -x faulthandler script.py) or an environment variable 
(PYTHONFAULTHANDLER=y python script.py)?
 * decide what to do with child processes: disable the faulthandler after a 
fork?
 * use something else than Py_AtExit() to unload the module?
 * dump the Python backtrace on a fatal error (and/or maybe catch SIGABRT)
 * use maybe something else than alarm()+SIGARLM for dumpbacktrace_later() 
because it interrupts the current system call

--

History of this module.

I first proposed a segfault handler using setjmp/longjmp to raise a classic 
Python exception. So it was  possible to execute Python code after a segfault 
(including stack overflow). But the idea was rejected because the Python 
internal state may be corrupted if the segfault was an invalid memory write 
(buffer overflow?), and anyway we cannot warranty that the internal state is 
still consistent after a long jump.

=> http://bugs.python.org/issue3999 (sept. 2009)

After that, I proposed to just display the Python backtrace on segfault. The 
signal handler was enabled by default. Because some people was concerned about 
possible bugs or security vulnerabilities introduced by the signal handler, I 
proposed options to disable it. Finally, I proposed to disable it by default 
and add options to enable it, but it was too late for an inclusion into Python 
3.2. Anyways, the project (a patch) was young and the API not correctly defined.

=> http://bugs.python.org/issue8863 (may 2010)

I converted my patch into a module. During this conversion, I realized that a 
module API is more natural than options using environment variables or 
functions in the sys module (I proposed sys.setfaulthandlerenabled(), an 
horrible name :-)). And it becomes more natural to enable the faulthandler by 
importing a module and calling a function (faulthandler.enable()).

Later I added functions to display the Python backtrace not only on a crash, 
but after a certain amount of time, or when an user signal is raised. You can 
also call the function directly to dump the backtrace. And finally, I added 
many option to configure how the backtrace is displayed: current thread of all 
threads, sys.stderr or another file, ...

--

I hope that this module will help to understand multiprocessing issues on the 
buildbots!

--
components: Library (Lib)
messages: 130013
nosy: haypo
priority: normal
severity: normal
status: open
title: Integrate faulthandler module into Python 3.3
versions: Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +dmalcolm

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I tested faulthandler on Linux, FreeBSD and Windows XP: it works well. I 
suppose that it works on any operating systems.

You can also try it on know crashers: Lib/test/crashers/ (you have to modify 
the files to add: import faulthandler; faulthandler.enable()). Example:


$ ./python Lib/test/crashers/bogus_code_obj.py 
Fatal Python error: Segmentation fault

Traceback (most recent call first):
  File "", line 1 in 
  File "Lib/test/crashers/bogus_code_obj.py", line 20 in 
Erreur de segmentation


The backtrace is not revelant here because the file is very short, but 
faulthandler is more useful on an huge application.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11393] Integrate faulthandler module into Python 3.3

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com