Re: Track down SIGABRT
Israel Brewster isr...@ravnalaska.net writes: ... Running it through GDB didn't seem to give me much more information than that crash report - I can see that it is in thread 0, and the call stack is identical to what the crash report shows, but I can't see where in the python code it is. ... Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib0x98176be6 __select + 10 1 time.so 0x0052fdda 0x52f000 + 3546 2 org.python.python 0x0007e90a PyCFunction_Call + 98 3 org.python.python 0x000ccef4 PyEval_EvalFrameEx + 8182 4 org.python.python 0x000caeb2 PyEval_EvalCodeEx + 1777 This tells you: the crash happens somewhere in a call to a function defined in time.so. I suppose that all such functions are harmless in themselves. I see two potential causes: * a stack overflow some Python builds are made with quite a small thread runtime stack. However, you would usually get a SIGSEGV rather than a SIGABRT. * a bad pointer passed in to the function. Again, I would expect rather a SIGSEGV than a SIGABRT -- but maybe __select has a special sanity check for its arguments and raises SIGABRT when the check fails. In order to determine where in your Python code the problem happens, you likely need a Python with debugging symbols (system installed Python interpreters usually are stripped, e.g. they no longer have debugging symbols). Python comes with GDB macros able to show you information at the Python (rather than the C) level. Here (Ubuntu 12.4), it is at /usr/share/doc/python2.7/gdbinit.gz. Those macros need debugging symbols. Read the top of this file, for how to get those macros defined in your GDB session. I doubt that being able to relate the problem to code at the Python level will help you much. Your SIGABRT problem is likely not caused on the Python level (but at C level). If it is not caused by a runtime stack overflow, the cause is likely a memory corruption, made non locally (far away) in some C extension. Using a tool to track down memory corruption might help you (however, in my experience, it is usually a huge task). -- https://mail.python.org/mailman/listinfo/python-list
Re: Track down SIGABRT
On Jan 13, 2015, at 6:27 AM, William Ray Wing w...@mac.com wrote: On Jan 9, 2015, at 12:40 PM, Israel Brewster isr...@ravnalaska.net wrote: I have a long-running python/CherryPy Web App server process that I am running on Mac OS X 10.8.5. Python 2.7.2 running in 32-bit mode (for now, I have the code in place to change over to 64 bit, but need to schedule the downtime to do it). On the 6th of this month, during normal operation from what I can tell, and after around 33 days of trouble-free uptime, the python process crashed with a SIGABRT. I restarted the process, and everything looked good again until yesterday, when it again crashed with a SIGABRT. The crash dump the system gave me doesn't tell me much, other than that it looks like python is calling some C function when it crashes. I've attached the crash report, in case it can mean something more to someone else. Can anyone give me some hints as to how to track down the cause of this crash? It's especially problematic since I can't mess with the live server for testing, and it is quite a while between crashes, making it difficult, if not impossible, to reproduce in testing. Thanks. --- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 --- Can you run the application in an IDE? Yes - I run it through Wing during development. I don't think that would be such a good option for my production machine, however. If it gets really bad I'll consider it though - that should at least tell me where it is crashing. -Bill --- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 --- -- https://mail.python.org/mailman/listinfo/python-list
Re: Track down SIGABRT
Assuming you have gdb available, you should be able to attach to the running process, then set a breakpoint in relevant functions (like exit() or abort()). Once there, you can pick through the C stack manually (kind of tedious) or use the gdbinit file which comes with Python to get a Python stack trace (much less tedious, once you've made sure any version dependencies have been eliminated). Or, with the latest versions of gdb (7.x I think), you get more stuff built into gdb itself. More details here: https://wiki.python.org/moin/DebuggingWithGdb Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Track down SIGABRT
On Jan 13, 2015, at 8:26 AM, Skip Montanaro skip.montan...@gmail.com wrote: Assuming you have gdb available, you should be able to attach to the running process, then set a breakpoint in relevant functions (like exit() or abort()). Once there, you can pick through the C stack manually (kind of tedious) or use the gdbinit file which comes with Python to get a Python stack trace (much less tedious, once you've made sure any version dependencies have been eliminated). Or, with the latest versions of gdb (7.x I think), you get more stuff built into gdb itself. More details here: https://wiki.python.org/moin/DebuggingWithGdb Thanks, I'll look into that. Hopefully running with the debugger attached won't slow things down to much. The main thing I think will be getting the python extensions installed - the instructions only talk about doing this for linux packages. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Track down SIGABRT
On Jan 12, 2015, at 5:51 PM, Jason Friedman jsf80...@gmail.com wrote: I have a long-running python/CherryPy Web App server process that I am running on Mac OS X 10.8.5. Python 2.7.2 running in 32-bit mode (for now, I have the code in place to change over to 64 bit, but need to schedule the downtime to do it). On the 6th of this month, during normal operation from what I can tell, and after around 33 days of trouble-free uptime, the python process crashed with a SIGABRT. I restarted the process, and everything looked good again until yesterday, when it again crashed with a SIGABRT. Can you monitor disk and memory on the host? Perhaps it is climbing towards an unacceptable value right before crashing. Good thought. I'm pretty sure that the system monitor still showed a couple of gigs free memory before the last crash, but the process could still be using unacceptable amounts of resources Do you have the option of stopping and starting your process every night or every week? Yes, that's an option, and as a work-around I'll consider it. Of course, I'd much rather not have the thing crash in the first place :-) -- https://mail.python.org/mailman/listinfo/python-list --- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 --- -- https://mail.python.org/mailman/listinfo/python-list
Re: Track down SIGABRT
Israel Brewster isr...@ravnalaska.net writes: when it again crashed with a SIGABRT. The crash dump the system gave me doesn't tell me much, other than that it looks like python is calling some C function when it crashes. I've attached the crash report, in case it can mean something more to someone else. Somehow I missed the original post: did you get a core dump? The next thing to do is examine it under gdb. Can anyone give me some hints as to how to track down the cause of this crash? If it's recurring, recompile python with -g to generate debugging symbols, so you'll have an easier time examining the dump. You can even run the whole interpreter under gdb, though doing that for 33 days might be uncharted territory. -- https://mail.python.org/mailman/listinfo/python-list
Track down SIGABRT
I have a long-running python/CherryPy Web App server process that I am running on Mac OS X 10.8.5. Python 2.7.2 running in 32-bit mode (for now, I have the code in place to change over to 64 bit, but need to schedule the downtime to do it). On the 6th of this month, during normal operation from what I can tell, and after around 33 days of trouble-free uptime, the python process crashed with a SIGABRT. I restarted the process, and everything looked good again until yesterday, when it again crashed with a SIGABRT. The crash dump the system gave me doesn't tell me much, other than that it looks like python is calling some C function when it crashes. I've attached the crash report, in case it can mean something more to someone else.Can anyone give me some hints as to how to track down the cause of this crash? It's especially problematic since I can't mess with the live server for testing, and it is quite a while between crashes, making it difficult, if not impossible, to reproduce in testing. Thanks. ---Israel BrewsterSystems Analyst IIRavn Alaska5245 Airport Industrial RdFairbanks, AK 99709(907) 450-7293---BEGIN:VCARD VERSION:3.0 N:Brewster;Israel;;; FN:Israel Brewster ORG:Frontier Flying Service;MIS TITLE:PC Support Tech II EMAIL;type=INTERNET;type=WORK;type=pref:isr...@frontierflying.com TEL;type=WORK;type=pref:907-450-7293 item1.ADR;type=WORK;type=pref:;;5245 Airport Industrial Wy;Fairbanks;AK;99701; item1.X-ABADR:us CATEGORIES:General X-ABUID:36305438-95EA-4410-91AB-45D16CABCDDC\:ABPerson END:VCARD Python_2015-01-08-152219_minilogger.crash Description: Binary data -- https://mail.python.org/mailman/listinfo/python-list
Re: Track down SIGABRT
I have a long-running python/CherryPy Web App server process that I am running on Mac OS X 10.8.5. Python 2.7.2 running in 32-bit mode (for now, I have the code in place to change over to 64 bit, but need to schedule the downtime to do it). On the 6th of this month, during normal operation from what I can tell, and after around 33 days of trouble-free uptime, the python process crashed with a SIGABRT. I restarted the process, and everything looked good again until yesterday, when it again crashed with a SIGABRT. Can you monitor disk and memory on the host? Perhaps it is climbing towards an unacceptable value right before crashing. Do you have the option of stopping and starting your process every night or every week? -- https://mail.python.org/mailman/listinfo/python-list
