[Python-Dev] segfault - potential double free when using iterparse

2012-06-12 Thread Alon Horev
Hi All, First of all, I'm not opening a bug yet as I'm not certain whether this is a CPython bug or lxml bug. I'm getting a segfault within python's GC (garbage collector) module. here's the stack trace: #0 0x7fc7e9f6b76e in gc_list_remove (op=0x7fc79cef3d98) at Modules/gcmodule.c:211 #1

Re: [Python-Dev] segfault - potential double free when using iterparse

2012-06-12 Thread Terry Reedy
On 6/12/2012 9:40 AM, Alon Horev wrote: Hi All, First of all, I'm not opening a bug yet as I'm not certain whether this is a CPython bug or lxml bug. lxml is more likely, making this a topic for python-list and whatever lxml list. ... from lxml.etree import iterparse ... anyone

Re: [Python-Dev] segfault in struct module

2008-06-11 Thread Gregory P. Smith
On Tue, Jun 10, 2008 at 9:04 AM, Jean-Paul Calderone [EMAIL PROTECTED] wrote: [EMAIL PROTECTED]:~$ ~/Projects/python/trunk/python Python 2.6a3+ (trunk:63964, Jun 5 2008, 16:49:12) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type help, copyright, credits or license for more

Re: [Python-Dev] segfault in struct module

2008-06-11 Thread Martin v. Löwis
Martin, since you committed 60793 in Feb, any others like this that need merging from release25-maint to trunk off the top of your head? That's the entire chunk of Google bug fixes, and yes, all of it needs to be ported to 2.6 still. I'll look into it, unless you volunteer :-) (it doesn't need

Re: [Python-Dev] segfault in struct module

2008-06-11 Thread Gregory P. Smith
On Wed, Jun 11, 2008 at 1:19 AM, Martin v. Löwis [EMAIL PROTECTED] wrote: Martin, since you committed 60793 in Feb, any others like this that need merging from release25-maint to trunk off the top of your head? That's the entire chunk of Google bug fixes, and yes, all of it needs to be

[Python-Dev] segfault in struct module

2008-06-10 Thread Jean-Paul Calderone
[EMAIL PROTECTED]:~$ ~/Projects/python/trunk/python Python 2.6a3+ (trunk:63964, Jun 5 2008, 16:49:12) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type help, copyright, credits or license for more information. import struct struct.pack(357913941c, 'a') Segmentation fault

[Python-Dev] segfault in 2.5.1

2008-04-25 Thread Neal Becker
Attempt to write to a mmap which was opened mmap.PROT_READ causes python to segfault. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] segfault in 2.5.1

2008-04-25 Thread Ralf Schmitt
On Fri, Apr 25, 2008 at 3:13 PM, Neal Becker [EMAIL PROTECTED] wrote: Attempt to write to a mmap which was opened mmap.PROT_READ causes python to segfault. http://bugs.python.org/issue2111 ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Segfault

2007-08-27 Thread Hrvoje Nikšić
On Fri, 2007-08-24 at 09:01 -0700, Neal Norwitz wrote: Right. I looked at this with Jeffrey Yasskin and agree that a lock is needed for f_fp. The fix is ugly because it's needed around all accesses to f_fp and there are a ton of them. Some are with the GIL held and others when it isn't.

Re: [Python-Dev] Segfault

2007-08-27 Thread Maciej Fijalkowski
Honestly, the argument that this code is already gone in 3.0 is not very valid. 2.x version would be probably used for many years. Cheers, fijal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

Re: [Python-Dev] Segfault

2007-08-24 Thread Hrvoje Nikšić
On Thu, 2007-08-23 at 11:57 -0700, Neal Norwitz wrote: allow threads section that runs with the GIL released, file_close might acquire the GIL and be running in parallel to this code. If file_close sets f_fp to NULL after the if condition evaluates, but before the call to _portable_fseek

Re: [Python-Dev] Segfault

2007-08-24 Thread Neal Norwitz
On 8/24/07, Hrvoje Nikšić [EMAIL PROTECTED] wrote: On Thu, 2007-08-23 at 11:57 -0700, Neal Norwitz wrote: allow threads section that runs with the GIL released, file_close might acquire the GIL and be running in parallel to this code. If file_close sets f_fp to NULL after the if

Re: [Python-Dev] Segfault

2007-08-23 Thread Neal Norwitz
On 8/23/07, Hrvoje Nikšić [EMAIL PROTECTED] wrote: On Wed, 2007-08-22 at 21:32 -0700, Neal Norwitz wrote: Py_BEGIN_ALLOW_THREADS errno = 0; - ret = _portable_fseek(f-f_fp, offset, whence); + if (f-f_fp != NULL) + ret = _portable_fseek(f-f_fp,

Re: [Python-Dev] Segfault

2007-08-22 Thread Hrvoje Nikšić
On Tue, 2007-08-21 at 09:14 -0700, Neal Norwitz wrote: The patch is insufficient to prevent all types of crashes that occur when accessing a file from 2 threads (closing in one and doing whatever in another). You are right. I wouldn't go so far to say the file object thread-unsafe, but it

Re: [Python-Dev] Segfault

2007-08-22 Thread Hrvoje Nikšić
On Wed, 2007-08-22 at 10:35 +0200, Hrvoje Nikšić wrote: I think we need a reliable mechanism to prevent file_close messing with f_fp while other operations are being performed. Since each FILE has an internal lock associated with it, flockfile could be used to lock out the sections that

Re: [Python-Dev] Segfault

2007-08-22 Thread Neal Norwitz
On 8/22/07, Hrvoje Nikšić [EMAIL PROTECTED] wrote: On Tue, 2007-08-21 at 09:14 -0700, Neal Norwitz wrote: The patch is insufficient to prevent all types of crashes that occur when accessing a file from 2 threads (closing in one and doing whatever in another). You are right. I wouldn't go

Re: [Python-Dev] Segfault

2007-08-21 Thread Hrvoje Nikšić
On Mon, 2007-08-20 at 20:27 +0200, Maciej Fijalkowski wrote: import thread while 1: f = open(/tmp/dupa, w) thread.start_new_thread(f.close, ()) f.close() file_close inadvertently allows fclose to be called twice on the same stdio file. This patch should fix the problem:

Re: [Python-Dev] Segfault

2007-08-21 Thread Ronald Oussoren
Please create a patch for this in the tracker to ensure that this patch doesn't get lost. Ronald On Tuesday, August 21, 2007, at 10:47AM, Hrvoje Nik?i? [EMAIL PROTECTED] wrote: On Mon, 2007-08-20 at 20:27 +0200, Maciej Fijalkowski wrote: import thread while 1: f = open(/tmp/dupa, w)

Re: [Python-Dev] Segfault

2007-08-21 Thread Neal Norwitz
On 8/21/07, Hrvoje Nikšić [EMAIL PROTECTED] wrote: On Mon, 2007-08-20 at 20:27 +0200, Maciej Fijalkowski wrote: import thread while 1: f = open(/tmp/dupa, w) thread.start_new_thread(f.close, ()) f.close() file_close inadvertently allows fclose to be called twice on the

Re: [Python-Dev] Segfault

2007-08-21 Thread Martin v. Löwis
I think there might already be a bug report about file not being thread-safe. (It could have also been socket, but I think Martin fixed a problem in socket a while ago.) This was a different problem, though: the socket object contained an address structure, which it didn't really need and

[Python-Dev] Segfault

2007-08-20 Thread Maciej Fijalkowski
IMHO this shouldn't segfault: import thread while 1: f = open(/tmp/dupa, w) thread.start_new_thread(f.close, ()) f.close() while it does on cpython 2.5.1, linux box. May I consider this a bug? ___ Python-Dev mailing list

Re: [Python-Dev] Segfault

2007-08-20 Thread Andrew Bennetts
Maciej Fijalkowski wrote: IMHO this shouldn't segfault: import thread while 1: f = open(/tmp/dupa, w) thread.start_new_thread(f.close, ()) f.close() while it does on cpython 2.5.1 , linux box. May I consider this a bug? Yes, that's a bug. Please file it at

Re: [Python-Dev] Segfault in python 2.5

2006-10-19 Thread Steve Holden
Mike Klaas wrote: On 10/18/06, Tim Peters [EMAIL PROTECTED] wrote: [...] Shouldn't the thread state generally be the same anyway? (I seem to recall some gloomy warning against resuming generators in separate threads). Is this an indication that generators aren't thread-safe? regards Steve

[Python-Dev] Segfault in python 2.5

2006-10-18 Thread Mike Klaas
[http://sourceforge.net/tracker/index.php?func=detailaid=1579370group_id=5470atid=105470] Hello, I'm managed to provoke a segfault in python2.5 (occasionally it just a invalid argument to internal function error). I've posted a traceback and a general idea of what the code consists of in the

Re: [Python-Dev] Segfault in python 2.5

2006-10-18 Thread Michael Hudson
Mike Klaas [EMAIL PROTECTED] writes: [http://sourceforge.net/tracker/index.php?func=detailaid=1579370group_id=5470atid=105470] Hello, I'm managed to provoke a segfault in python2.5 (occasionally it just a invalid argument to internal function error). I've posted a traceback and a general

Re: [Python-Dev] Segfault in python 2.5

2006-10-18 Thread Jack Jansen
On 18-Oct-2006, at 22:08 , Michael Hudson wrote: Unfortunately, I've been attempting for hours to reduce the problem to a completely self-contained script, but it is resisting my efforts due to timing problems. Has anyone ever tried to use helgrind (the valgrind module, not the heavy metal

Re: [Python-Dev] Segfault in python 2.5

2006-10-18 Thread Mike Klaas
On 10/18/06, Michael Hudson [EMAIL PROTECTED] wrote: Mike Klaas [EMAIL PROTECTED] writes: I've been reading the bug report with interest, but unless I can reproduce it it's mighty hard for me to debug, as I'm sure you know. Indeed. Unfortunately, I've been attempting for hours to reduce

Re: [Python-Dev] Segfault in python 2.5

2006-10-18 Thread Tim Peters
[Michael Hudson] I've been reading the bug report with interest, but unless I can reproduce it it's mighty hard for me to debug, as I'm sure you know. [Mike Klaas] Indeed. Note that I just attached a much simpler pure-Python script that fails very quickly, on Windows, using a debug build.

Re: [Python-Dev] Segfault in python 2.5

2006-10-18 Thread Mike Klaas
On 10/18/06, Tim Peters [EMAIL PROTECTED] wrote: [Mike Klaas] Indeed. Note that I just attached a much simpler pure-Python script that fails very quickly, on Windows, using a debug build. Read the new comment to learn why both Windows and debug build are essential to it failing reliably

[Python-Dev] segfault when using PyGILState_Ensure/Release in Python2.3.4

2006-07-21 Thread Travis E. Oliphant
I'm hoping somebody here can help me with an error I'm getting in Python 2.3.4 but not in Python 2.4.2 when I use PyGILState_Ensure in NumPy on Linux. Perhaps somebody can point out what I'm doing wrong because while I've tried to understand the threading API it can be a bit confusing and

Re: [Python-Dev] segfault (double free?) when '''-string crosses line

2006-04-10 Thread Martin v. Löwis
Tim Peters wrote: #ifdef __VMS extern char* vms__StdioReadline(FILE *sys_stdin, FILE *sys_stdout, char *prompt); #endif may be (it's apparently not checked in, and Google couldn't find it either). Does anyone still work on the VMS port? If not, screw it ;-) I would have no concerns

Re: [Python-Dev] segfault (double free?) when '''-string crosses line

2006-04-10 Thread Martin v. Löwis
Jean-François Piéronne wrote: I (and a few others guys) work on the VMS port,also, HP has publish in the VMS technical journal an article about Python on VMS. What do you need? In the specific case: an answer to Tim Peter's question (where is vms__StdioReadline, and how can its memory

[Python-Dev] segfault (double free?) when '''-string crosses line

2006-04-09 Thread Guido van Rossum
On Linux, In HEAD 2.5, but only with the non-debug version, I get a segfault when I do this: ''' ... ''' It seems to occur for any triple-quoted string crossing a line boundary. A bit of the stack trace: #0 0x40030087 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #1 0x4207ad18 in

Re: [Python-Dev] segfault (double free?) when '''-string crosses line

2006-04-09 Thread Tim Peters
[Guido] On Linux, In HEAD 2.5, but only with the non-debug version, I get a segfault when I do this: ''' ... ''' It seems to occur for any triple-quoted string crossing a line boundary. A bit of the stack trace: #0 0x40030087 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #1

Re: [Python-Dev] segfault (double free?) when '''-string crosses line

2006-04-09 Thread Neal Norwitz
On 4/9/06, Tim Peters [EMAIL PROTECTED] wrote: [Guido] On Linux, In HEAD 2.5, but only with the non-debug version, I get a segfault when I do this: ''' ... ''' It rings a bell here only in that the front end had lots of allocate-versus-free mismatches between the PyObject_ and

Re: [Python-Dev] segfault (double free?) when '''-string crosses line

2006-04-09 Thread Tim Peters
[Neal Norwitz] http://python.org/sf/1467512 fixes the problem for me on linux. It converts all the PyMem_* APIs to PyObject_* APIs. Assigned to Guido until he changes that. :-) Thanks! I didn't take that route, instead not changing anything I didn't take the time to understand first. For