Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi,
On Tue, Mar 15, 2011 at 5:55 PM, Matthew Brett wrote:
> Hi,
>
> On Tue, Mar 15, 2011 at 5:30 PM, Christoph Gohlke wrote:
>>
>>
>> On 3/15/2011 5:13 PM, Matthew Brett wrote:
>>> Hi,
>>>
>>> On Tue, Mar 15, 2011 at 10:23 AM, Matthew Brett
>>> wrote:
Hi,
On Tue, Mar 15, 2011 at 10:12 AM, Pauli Virtanen wrote:
> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
>> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
>> but is this the right fix to this problem? I was confused by the
>> presence of the old PyString_AsString function.
>
> It's not a correct fix. The original code seems also wrong ("index" can
> either be Unicode or Bytes/String object), and will probably bomb when
> indexing with Unicode strings on Python 2. The right thing to do is to
> make it show the repr of the "index" object.
OK - I realize I'm being very lazy here but, do you mean:
PyErr_Format(PyExc_ValueError,
>> "field named %s not found.",
>> PyString_AsString(PyObject_Repr(index)));
>>>
>>> Being less lazy, and having read the cpython source, and read
>>> Christoph's mail more carefully, I believe Christoph's patch is
>>> correct...
>>>
>>> Unicode indexing of structured array fields doesn't raise an error on
>>> python 2.x; I assume because PyString_AsString is returning a char*
>>> using the Unicode default encoding, as per the docs.
>>>
>>
>> I think the patch is correct for Python 3 but, as Pauli pointed out, the
>> original code can crash also under Python 2.x when indexing with an
>> unicode string that contains non-ascii7 characters, which seems much
>> less likely and apparently has been undetected for quite a while. For
>> example, this crashes for me on Python 2.7:
>>
>> import numpy as np
>> a = np.zeros((1,), dtype=[('f1', 'f')])
>> a[u's'] = 1 # works
>> a[u'µ'] = 1 # crash
>>
>> So, the proposed patch works for Python 3, but there could be a better
>> patch fixing also the corner case on Python 2.
>
> Thanks. How about something like the check further up the file:
>
> if (PyUnicode_Check(op)) {
> temp = PyUnicode_AsUnicodeEscapeString(op);
> }
> PyErr_Format(PyExc_ValueError,
> "field named %s not found.",
> PyBytes_AsString(temp));
>
> ? I'm happy to submit a pull request with tests if that kind of thing
> looks sensible to y'all,
That fix works for 2.6 but crashes 3.2 when it is rejecting (by design
I think) field indexing with byte strings.
I've put a test function in this branch which exercises the routes I
could think of. It will crash current 2.x because of the unicode
error that Pauli pointed out, but I think it corresponds to the
expected behavior:
https://github.com/matthew-brett/numpy/compare/numpy:master...matthew-brett:struct-arr-fields
Do these tests look correct? I'd be happy to hear why the code above
does not work, it wasn't obvious to me.
See you,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi,
On Tue, Mar 15, 2011 at 5:30 PM, Christoph Gohlke wrote:
>
>
> On 3/15/2011 5:13 PM, Matthew Brett wrote:
>> Hi,
>>
>> On Tue, Mar 15, 2011 at 10:23 AM, Matthew Brett
>> wrote:
>>> Hi,
>>>
>>> On Tue, Mar 15, 2011 at 10:12 AM, Pauli Virtanen wrote:
Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
> but is this the right fix to this problem? I was confused by the
> presence of the old PyString_AsString function.
It's not a correct fix. The original code seems also wrong ("index" can
either be Unicode or Bytes/String object), and will probably bomb when
indexing with Unicode strings on Python 2. The right thing to do is to
make it show the repr of the "index" object.
>>>
>>> OK - I realize I'm being very lazy here but, do you mean:
>>>
>>> PyErr_Format(PyExc_ValueError,
> "field named %s not found.",
> PyString_AsString(PyObject_Repr(index)));
>>
>> Being less lazy, and having read the cpython source, and read
>> Christoph's mail more carefully, I believe Christoph's patch is
>> correct...
>>
>> Unicode indexing of structured array fields doesn't raise an error on
>> python 2.x; I assume because PyString_AsString is returning a char*
>> using the Unicode default encoding, as per the docs.
>>
>
> I think the patch is correct for Python 3 but, as Pauli pointed out, the
> original code can crash also under Python 2.x when indexing with an
> unicode string that contains non-ascii7 characters, which seems much
> less likely and apparently has been undetected for quite a while. For
> example, this crashes for me on Python 2.7:
>
> import numpy as np
> a = np.zeros((1,), dtype=[('f1', 'f')])
> a[u's'] = 1 # works
> a[u'µ'] = 1 # crash
>
> So, the proposed patch works for Python 3, but there could be a better
> patch fixing also the corner case on Python 2.
Thanks. How about something like the check further up the file:
if (PyUnicode_Check(op)) {
temp = PyUnicode_AsUnicodeEscapeString(op);
}
PyErr_Format(PyExc_ValueError,
"field named %s not found.",
PyBytes_AsString(temp));
? I'm happy to submit a pull request with tests if that kind of thing
looks sensible to y'all,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On 3/15/2011 5:13 PM, Matthew Brett wrote:
> Hi,
>
> On Tue, Mar 15, 2011 at 10:23 AM, Matthew Brett
> wrote:
>> Hi,
>>
>> On Tue, Mar 15, 2011 at 10:12 AM, Pauli Virtanen wrote:
>>> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
but is this the right fix to this problem? I was confused by the
presence of the old PyString_AsString function.
>>>
>>> It's not a correct fix. The original code seems also wrong ("index" can
>>> either be Unicode or Bytes/String object), and will probably bomb when
>>> indexing with Unicode strings on Python 2. The right thing to do is to
>>> make it show the repr of the "index" object.
>>
>> OK - I realize I'm being very lazy here but, do you mean:
>>
>> PyErr_Format(PyExc_ValueError,
"field named %s not found.",
PyString_AsString(PyObject_Repr(index)));
>
> Being less lazy, and having read the cpython source, and read
> Christoph's mail more carefully, I believe Christoph's patch is
> correct...
>
> Unicode indexing of structured array fields doesn't raise an error on
> python 2.x; I assume because PyString_AsString is returning a char*
> using the Unicode default encoding, as per the docs.
>
I think the patch is correct for Python 3 but, as Pauli pointed out, the
original code can crash also under Python 2.x when indexing with an
unicode string that contains non-ascii7 characters, which seems much
less likely and apparently has been undetected for quite a while. For
example, this crashes for me on Python 2.7:
import numpy as np
a = np.zeros((1,), dtype=[('f1', 'f')])
a[u's'] = 1 # works
a[u'µ'] = 1 # crash
So, the proposed patch works for Python 3, but there could be a better
patch fixing also the corner case on Python 2.
Christoph
> Thanks,
>
> Matthew
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi,
On Tue, Mar 15, 2011 at 10:23 AM, Matthew Brett wrote:
> Hi,
>
> On Tue, Mar 15, 2011 at 10:12 AM, Pauli Virtanen wrote:
>> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
>>> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
>>> but is this the right fix to this problem? I was confused by the
>>> presence of the old PyString_AsString function.
>>
>> It's not a correct fix. The original code seems also wrong ("index" can
>> either be Unicode or Bytes/String object), and will probably bomb when
>> indexing with Unicode strings on Python 2. The right thing to do is to
>> make it show the repr of the "index" object.
>
> OK - I realize I'm being very lazy here but, do you mean:
>
> PyErr_Format(PyExc_ValueError,
>>> "field named %s not found.",
>>> PyString_AsString(PyObject_Repr(index)));
Being less lazy, and having read the cpython source, and read
Christoph's mail more carefully, I believe Christoph's patch is
correct...
Unicode indexing of structured array fields doesn't raise an error on
python 2.x; I assume because PyString_AsString is returning a char*
using the Unicode default encoding, as per the docs.
Thanks,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On 3/15/2011 11:34 AM, Christoph Gohlke wrote:
>
>
> On 3/15/2011 10:12 AM, Pauli Virtanen wrote:
>> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
>>> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
>>> but is this the right fix to this problem? I was confused by the
>>> presence of the old PyString_AsString function.
>>
>> It's not a correct fix. The original code seems also wrong ("index" can
>> either be Unicode or Bytes/String object), and will probably bomb when
>> indexing with Unicode strings on Python 2. The right thing to do is to
>> make it show the repr of the "index" object.
>>
>> The PyString_AsString is present, as it's mapped on Py3 to
>> PyBytes_AsString by "npy_3kcompat.h".
>>
>> Pauli
>
> I think the proposed patch does correctly fix ticket #1770 and the OP's
> problem under Python 3.
>
> On Python 3.x str() and repr() are the same for byte strings:
>
str(b's')
> "b's'"
repr(b's')
> "b's'"
>
> I agree that the code is broken for Python 2.x when indexing with a byte
> object.
sorry, this should read: I agree that the original code is broken for
Python 2.x when indexing with a unicode object.
Christoph
>
> Christoph
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On 3/15/2011 10:12 AM, Pauli Virtanen wrote:
> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
>> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
>> but is this the right fix to this problem? I was confused by the
>> presence of the old PyString_AsString function.
>
> It's not a correct fix. The original code seems also wrong ("index" can
> either be Unicode or Bytes/String object), and will probably bomb when
> indexing with Unicode strings on Python 2. The right thing to do is to
> make it show the repr of the "index" object.
>
> The PyString_AsString is present, as it's mapped on Py3 to
> PyBytes_AsString by "npy_3kcompat.h".
>
> Pauli
I think the proposed patch does correctly fix ticket #1770 and the OP's
problem under Python 3.
On Python 3.x str() and repr() are the same for byte strings:
>>> str(b's')
"b's'"
>>> repr(b's')
"b's'"
I agree that the code is broken for Python 2.x when indexing with a byte
object.
Christoph
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi, On Tue, Mar 15, 2011 at 11:07 AM, Pauli Virtanen wrote: > Tue, 15 Mar 2011 10:23:35 -0700, Matthew Brett wrote: > [clip] >> OK - I realize I'm being very lazy here but, do you mean: >> >> PyErr_Format(PyExc_ValueError, "field named %s not found.", PyString_AsString(PyObject_Repr(index))); >> >>> The PyString_AsString is present, as it's mapped on Py3 to >>> PyBytes_AsString by "npy_3kcompat.h". >> >> Oh - dear - I think I felt a blood vessel pop somewhere in my brain :) > > This was an "answer" to your question as I understood it: > "PyString_AsString is no longer a part of the API on Python 3.x. So how > come this code can work on Python 3 if it appears here?" Oh - dear - again. Yes it was a helpful answer and direct to my question - implication otherwise entirely accidental. The blood vessel was only because my brain was already twisted from trying to unlearn the correspondence between string and byte in python 3... I fear that my suggestion as to what you meant by repr doesn't make sense though. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Tue, 15 Mar 2011 10:23:35 -0700, Matthew Brett wrote: [clip] > OK - I realize I'm being very lazy here but, do you mean: > > PyErr_Format(PyExc_ValueError, >>> "field named %s not found.", >>> PyString_AsString(PyObject_Repr(index))); > >> The PyString_AsString is present, as it's mapped on Py3 to >> PyBytes_AsString by "npy_3kcompat.h". > > Oh - dear - I think I felt a blood vessel pop somewhere in my brain :) This was an "answer" to your question as I understood it: "PyString_AsString is no longer a part of the API on Python 3.x. So how come this code can work on Python 3 if it appears here?" Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On Tue, Mar 15, 2011 at 11:12 AM, Pauli Virtanen wrote:
> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
> > Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
> > but is this the right fix to this problem? I was confused by the
> > presence of the old PyString_AsString function.
>
> It's not a correct fix. The original code seems also wrong ("index" can
> either be Unicode or Bytes/String object), and will probably bomb when
> indexing with Unicode strings on Python 2. The right thing to do is to
> make it show the repr of the "index" object.
>
> The PyString_AsString is present, as it's mapped on Py3 to
> PyBytes_AsString by "npy_3kcompat.h".
>
>
Well, I applied the fix, so that needs to be fixed ;) What does repr return
in python 3k, utf-8?
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi,
On Tue, Mar 15, 2011 at 10:12 AM, Pauli Virtanen wrote:
> Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
>> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
>> but is this the right fix to this problem? I was confused by the
>> presence of the old PyString_AsString function.
>
> It's not a correct fix. The original code seems also wrong ("index" can
> either be Unicode or Bytes/String object), and will probably bomb when
> indexing with Unicode strings on Python 2. The right thing to do is to
> make it show the repr of the "index" object.
OK - I realize I'm being very lazy here but, do you mean:
PyErr_Format(PyExc_ValueError,
>> "field named %s not found.",
>> PyString_AsString(PyObject_Repr(index)));
> The PyString_AsString is present, as it's mapped on Py3 to
> PyBytes_AsString by "npy_3kcompat.h".
Oh - dear - I think I felt a blood vessel pop somewhere in my brain :)
See you,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Tue, 15 Mar 2011 10:06:09 -0700, Matthew Brett wrote:
> Sorry to ask, and I ask partly because I'm in the middle of a py3k port,
> but is this the right fix to this problem? I was confused by the
> presence of the old PyString_AsString function.
It's not a correct fix. The original code seems also wrong ("index" can
either be Unicode or Bytes/String object), and will probably bomb when
indexing with Unicode strings on Python 2. The right thing to do is to
make it show the repr of the "index" object.
The PyString_AsString is present, as it's mapped on Py3 to
PyBytes_AsString by "npy_3kcompat.h".
Pauli
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi,
On Sun, Mar 13, 2011 at 12:07 PM, Matthew Brett wrote:
> Hi,
>
> On Sun, Mar 13, 2011 at 11:51 AM, Christoph Gohlke wrote:
>>
>>
>> On 3/13/2011 11:29 AM, Matthew Brett wrote:
>>>
>>> Hi
>>>
>>> On Sun, Mar 13, 2011 at 9:54 AM, Christoph Gohlke wrote:
On 3/13/2011 1:57 AM, Matthew Brett wrote:
>
> Hi,
> I have this on my OSX 10.6 system and numpy 1.5.1 and current numpy
> head (30ee1d352):
>
> $ python3.2
> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
import numpy as np
a = np.zeros((1,), dtype=[('f1', 'f')])
a['f1'] = 1
a['f2'] = 1
>
> Segmentation fault
>
> All tests pass with np.test()
>
> Expected behavior with same code on python2.6:
>
a['f2'] = 1
>
> Traceback (most recent call last):
> File "", line 1, in
> ValueError: field named f2 not found.
>
> Cheers,
>
> Matthew
Confirmed on Windows. The crash is in line 816 of mapping.c:
PyErr_Format(PyExc_ValueError,
"field named %s not found.",
PyString_AsString(index));
This works with Python 3.x:
PyErr_Format(PyExc_ValueError,
"field named %S not found.",
index);
>>>
>>> Sure enough, in python3.2:
>>>
>> a[b'f2'] = 1
>>>
>>> Traceback (most recent call last):
>>> File "", line 1, in
>>> ValueError: field named f2 not found.
>>>
>>> That is - it is specifically passing a python 3 string that causes the
>>> segmentation fault.
>>>
>>> Is this a release blocker? I'm afraid I don't know the code well
>>> enough to be confident of the right fix. Is there something I can do
>>> to help?
>>
>> Please open a ticket at and refer to this
>> discussion and proposed fix.
>>
>> diff --git a/numpy/core/src/multiarray/mapping.c
>> b/numpy/core/src/multiarray/mapping.c
>> index 8db85bf..3a72811 100644
>> --- a/numpy/core/src/multiarray/mapping.c
>> +++ b/numpy/core/src/multiarray/mapping.c
>> @@ -812,10 +812,16 @@ array_ass_sub(PyArrayObject *self, PyObject *index,
>> PyObject *op)
>> }
>> }
>> }
>> -
>> +#if defined(NPY_PY3K)
>> + PyErr_Format(PyExc_ValueError,
>> + "field named %S not found.",
>> + index);
>> +#else
>> PyErr_Format(PyExc_ValueError,
>> "field named %s not found.",
>> PyString_AsString(index));
>> +#endif
>> +
>> return -1;
>> }
>
> http://projects.scipy.org/numpy/ticket/1770
Sorry to ask, and I ask partly because I'm in the middle of a py3k
port, but is this the right fix to this problem? I was confused by
the presence of the old PyString_AsString function.
Best,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On Sun, Mar 13, 2011 at 12:51 PM, Christoph Gohlke wrote:
>
>
> On 3/13/2011 11:29 AM, Matthew Brett wrote:
> > Hi
> >
> > On Sun, Mar 13, 2011 at 9:54 AM, Christoph Gohlke
> wrote:
> >> On 3/13/2011 1:57 AM, Matthew Brett wrote:
> >>> Hi,
> >>> I have this on my OSX 10.6 system and numpy 1.5.1 and current numpy
> >>> head (30ee1d352):
> >>>
> >>> $ python3.2
> >>> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
> >>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >> import numpy as np
> >> a = np.zeros((1,), dtype=[('f1', 'f')])
> >> a['f1'] = 1
> >> a['f2'] = 1
> >>> Segmentation fault
> >>>
> >>> All tests pass with np.test()
> >>>
> >>> Expected behavior with same code on python2.6:
> >>>
> >> a['f2'] = 1
> >>> Traceback (most recent call last):
> >>> File "", line 1, in
> >>> ValueError: field named f2 not found.
> >>>
> >>> Cheers,
> >>>
> >>> Matthew
> >>
> >> Confirmed on Windows. The crash is in line 816 of mapping.c:
> >>
> >> PyErr_Format(PyExc_ValueError,
> >> "field named %s not found.",
> >> PyString_AsString(index));
> >>
> >> This works with Python 3.x:
> >>
> >> PyErr_Format(PyExc_ValueError,
> >> "field named %S not found.",
> >> index);
> >
> > Sure enough, in python3.2:
> >
> a[b'f2'] = 1
> > Traceback (most recent call last):
> >File "", line 1, in
> > ValueError: field named f2 not found.
> >
> > That is - it is specifically passing a python 3 string that causes the
> > segmentation fault.
> >
> > Is this a release blocker? I'm afraid I don't know the code well
> > enough to be confident of the right fix. Is there something I can do
> > to help?
>
> Please open a ticket at and refer to
> this discussion and proposed fix.
>
> diff --git a/numpy/core/src/multiarray/mapping.c
> b/numpy/core/src/multiarray/mapping.c
> index 8db85bf..3a72811 100644
> --- a/numpy/core/src/multiarray/mapping.c
> +++ b/numpy/core/src/multiarray/mapping.c
> @@ -812,10 +812,16 @@ array_ass_sub(PyArrayObject *self, PyObject
> *index, PyObject *op)
> }
> }
> }
> -
> +#if defined(NPY_PY3K)
> +PyErr_Format(PyExc_ValueError,
> + "field named %S not found.",
> + index);
> +#else
> PyErr_Format(PyExc_ValueError,
> "field named %s not found.",
> PyString_AsString(index));
> +#endif
> +
> return -1;
> }
>
>
> Christoph
>
>
There are two more occurrences of this construction in descriptor.c.
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi,
On Sun, Mar 13, 2011 at 11:51 AM, Christoph Gohlke wrote:
>
>
> On 3/13/2011 11:29 AM, Matthew Brett wrote:
>>
>> Hi
>>
>> On Sun, Mar 13, 2011 at 9:54 AM, Christoph Gohlke wrote:
>>>
>>> On 3/13/2011 1:57 AM, Matthew Brett wrote:
Hi,
I have this on my OSX 10.6 system and numpy 1.5.1 and current numpy
head (30ee1d352):
$ python3.2
Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import numpy as np
>>> a = np.zeros((1,), dtype=[('f1', 'f')])
>>> a['f1'] = 1
>>> a['f2'] = 1
Segmentation fault
All tests pass with np.test()
Expected behavior with same code on python2.6:
>>> a['f2'] = 1
Traceback (most recent call last):
File "", line 1, in
ValueError: field named f2 not found.
Cheers,
Matthew
>>>
>>> Confirmed on Windows. The crash is in line 816 of mapping.c:
>>>
>>> PyErr_Format(PyExc_ValueError,
>>> "field named %s not found.",
>>> PyString_AsString(index));
>>>
>>> This works with Python 3.x:
>>>
>>> PyErr_Format(PyExc_ValueError,
>>> "field named %S not found.",
>>> index);
>>
>> Sure enough, in python3.2:
>>
> a[b'f2'] = 1
>>
>> Traceback (most recent call last):
>> File "", line 1, in
>> ValueError: field named f2 not found.
>>
>> That is - it is specifically passing a python 3 string that causes the
>> segmentation fault.
>>
>> Is this a release blocker? I'm afraid I don't know the code well
>> enough to be confident of the right fix. Is there something I can do
>> to help?
>
> Please open a ticket at and refer to this
> discussion and proposed fix.
>
> diff --git a/numpy/core/src/multiarray/mapping.c
> b/numpy/core/src/multiarray/mapping.c
> index 8db85bf..3a72811 100644
> --- a/numpy/core/src/multiarray/mapping.c
> +++ b/numpy/core/src/multiarray/mapping.c
> @@ -812,10 +812,16 @@ array_ass_sub(PyArrayObject *self, PyObject *index,
> PyObject *op)
> }
> }
> }
> -
> +#if defined(NPY_PY3K)
> + PyErr_Format(PyExc_ValueError,
> + "field named %S not found.",
> + index);
> +#else
> PyErr_Format(PyExc_ValueError,
> "field named %s not found.",
> PyString_AsString(index));
> +#endif
> +
> return -1;
> }
http://projects.scipy.org/numpy/ticket/1770
Thanks,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On 3/13/2011 11:29 AM, Matthew Brett wrote:
> Hi
>
> On Sun, Mar 13, 2011 at 9:54 AM, Christoph Gohlke wrote:
>> On 3/13/2011 1:57 AM, Matthew Brett wrote:
>>> Hi,
>>> I have this on my OSX 10.6 system and numpy 1.5.1 and current numpy
>>> head (30ee1d352):
>>>
>>> $ python3.2
>>> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
>>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>> import numpy as np
>> a = np.zeros((1,), dtype=[('f1', 'f')])
>> a['f1'] = 1
>> a['f2'] = 1
>>> Segmentation fault
>>>
>>> All tests pass with np.test()
>>>
>>> Expected behavior with same code on python2.6:
>>>
>> a['f2'] = 1
>>> Traceback (most recent call last):
>>> File "", line 1, in
>>> ValueError: field named f2 not found.
>>>
>>> Cheers,
>>>
>>> Matthew
>>
>> Confirmed on Windows. The crash is in line 816 of mapping.c:
>>
>> PyErr_Format(PyExc_ValueError,
>> "field named %s not found.",
>> PyString_AsString(index));
>>
>> This works with Python 3.x:
>>
>> PyErr_Format(PyExc_ValueError,
>> "field named %S not found.",
>> index);
>
> Sure enough, in python3.2:
>
a[b'f2'] = 1
> Traceback (most recent call last):
>File "", line 1, in
> ValueError: field named f2 not found.
>
> That is - it is specifically passing a python 3 string that causes the
> segmentation fault.
>
> Is this a release blocker? I'm afraid I don't know the code well
> enough to be confident of the right fix. Is there something I can do
> to help?
Please open a ticket at and refer to
this discussion and proposed fix.
diff --git a/numpy/core/src/multiarray/mapping.c
b/numpy/core/src/multiarray/mapping.c
index 8db85bf..3a72811 100644
--- a/numpy/core/src/multiarray/mapping.c
+++ b/numpy/core/src/multiarray/mapping.c
@@ -812,10 +812,16 @@ array_ass_sub(PyArrayObject *self, PyObject
*index, PyObject *op)
}
}
}
-
+#if defined(NPY_PY3K)
+PyErr_Format(PyExc_ValueError,
+ "field named %S not found.",
+ index);
+#else
PyErr_Format(PyExc_ValueError,
"field named %s not found.",
PyString_AsString(index));
+#endif
+
return -1;
}
Christoph
>
> Cheers,
>
> Matthew
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
Hi
On Sun, Mar 13, 2011 at 9:54 AM, Christoph Gohlke wrote:
> On 3/13/2011 1:57 AM, Matthew Brett wrote:
>> Hi,
>> I have this on my OSX 10.6 system and numpy 1.5.1 and current numpy
>> head (30ee1d352):
>>
>> $ python3.2
>> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>> Type "help", "copyright", "credits" or "license" for more information.
> import numpy as np
> a = np.zeros((1,), dtype=[('f1', 'f')])
> a['f1'] = 1
> a['f2'] = 1
>> Segmentation fault
>>
>> All tests pass with np.test()
>>
>> Expected behavior with same code on python2.6:
>>
> a['f2'] = 1
>> Traceback (most recent call last):
>> File "", line 1, in
>> ValueError: field named f2 not found.
>>
>> Cheers,
>>
>> Matthew
>
> Confirmed on Windows. The crash is in line 816 of mapping.c:
>
> PyErr_Format(PyExc_ValueError,
> "field named %s not found.",
> PyString_AsString(index));
>
> This works with Python 3.x:
>
> PyErr_Format(PyExc_ValueError,
> "field named %S not found.",
> index);
Sure enough, in python3.2:
>>> a[b'f2'] = 1
Traceback (most recent call last):
File "", line 1, in
ValueError: field named f2 not found.
That is - it is specifically passing a python 3 string that causes the
segmentation fault.
Is this a release blocker? I'm afraid I don't know the code well
enough to be confident of the right fix. Is there something I can do
to help?
Cheers,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Segfault with python 3.2 structured array non-existent field
On 3/13/2011 1:57 AM, Matthew Brett wrote:
> Hi,
>
> I have this on my OSX 10.6 system and numpy 1.5.1 and current numpy
> head (30ee1d352):
>
> $ python3.2
> Python 3.2 (r32:88452, Feb 20 2011, 11:12:31)
> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
import numpy as np
a = np.zeros((1,), dtype=[('f1', 'f')])
a['f1'] = 1
a['f2'] = 1
> Segmentation fault
>
> All tests pass with np.test()
>
> Expected behavior with same code on python2.6:
>
a['f2'] = 1
> Traceback (most recent call last):
>File "", line 1, in
> ValueError: field named f2 not found.
>
> Cheers,
>
> Matthew
Confirmed on Windows. The crash is in line 816 of mapping.c:
PyErr_Format(PyExc_ValueError,
"field named %s not found.",
PyString_AsString(index));
This works with Python 3.x:
PyErr_Format(PyExc_ValueError,
"field named %S not found.",
index);
Christoph
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
