Re: [Numpy-discussion] better error message possible?
On Jun 7, 2012, at 4:32 AM, Paul Anton Letnes wrote: > > On 7. juni 2012, at 10:30, Thouis (Ray) Jones wrote: > >> I've opened a PR at https://github.com/numpy/numpy/pull/296 for discussion. >> >> A typical result >> > np.zeros((3,3))[[1,2,3]] >> Traceback (most recent call last): >> File "", line 1, in >> IndexError: index 3 is out of bounds for axis 0: [-3,3) >> >> Ray Jones >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > I would prefer: > IndexError: index 3 is out of bounds for axis 0: [-3,2] > as I find the 3) notation a bit weird - after all, indices are not floats, so > 2.999 or 2.3 doesn't make sense as an index. Actually, with slicing, you can attempt indexing using floats (as referred to here: http://stackoverflow.com/questions/8514547/why-ndarray-allow-floating-point-index), so I disagree with the integer notation [-3,2] I actually prefer the subsequent suggestion of reporting the shape, since that's what I always check when debugging index errors anyway -- having it in the error message would save me a ton of time. > > An alternative is to not refer to negative indices and say > IndexError: index 3 is out of bounds for axis 0, shape was: (3,) > (print more axes when the number of axes is higher.) > > BTW, I'm really glad someone is taking an interest in these error messages, > it's a great idea! > … an enthusiastic +1! > Paul > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Thu, Jun 7, 2012 at 11:44 AM, Dave Hirschfeld wrote: > Paul Anton Letnes gmail.com> writes: > >> I would prefer: >> IndexError: index 3 is out of bounds for axis 0: [-3,2] >> as I find the 3) notation a bit weird - after all, indices are not floats, so > 2.999 or 2.3 doesn't make sense as >> an index. >> >> An alternative is to not refer to negative indices and say >> IndexError: index 3 is out of bounds for axis 0, shape was: (3,) >> (print more axes when the number of axes is higher.) >> > > +1 for the latter suggestion - if the array shape is available it's a great > help in debugging the error. I agree that reporting the shape would be good, but it's usually not available at the point that the indices are found to be out-of-bounds, due to (often implicit) flattening. I think it might be possible to track and report that the array was flattened, which might help avoid some confusion when the maximum index reported in the Exception doesn't match any of the dimensions of the array being indexed due to flattening. Another possibility I entertained was to split the too-high vs. too-low cases: IndexError: index 3 is out of bounds for axis 0: must be < 3 IndexError: index -4 is out of bounds for axis 0: must be >= -3 Ray Jones ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
Paul Anton Letnes gmail.com> writes: > I would prefer: > IndexError: index 3 is out of bounds for axis 0: [-3,2] > as I find the 3) notation a bit weird - after all, indices are not floats, so 2.999 or 2.3 doesn't make sense as > an index. > > An alternative is to not refer to negative indices and say > IndexError: index 3 is out of bounds for axis 0, shape was: (3,) > (print more axes when the number of axes is higher.) > +1 for the latter suggestion - if the array shape is available it's a great help in debugging the error. An alternative wording could be: IndexError: Index 3 is out of bounds for axis 0 of an array of shape (3,2,4) -Dave ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On 7. juni 2012, at 10:30, Thouis (Ray) Jones wrote: > I've opened a PR at https://github.com/numpy/numpy/pull/296 for discussion. > > A typical result > np.zeros((3,3))[[1,2,3]] > Traceback (most recent call last): > File "", line 1, in > IndexError: index 3 is out of bounds for axis 0: [-3,3) > > Ray Jones > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion I would prefer: IndexError: index 3 is out of bounds for axis 0: [-3,2] as I find the 3) notation a bit weird - after all, indices are not floats, so 2.999 or 2.3 doesn't make sense as an index. An alternative is to not refer to negative indices and say IndexError: index 3 is out of bounds for axis 0, shape was: (3,) (print more axes when the number of axes is higher.) BTW, I'm really glad someone is taking an interest in these error messages, it's a great idea! Paul ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
I've opened a PR at https://github.com/numpy/numpy/pull/296 for discussion. A typical result >>> np.zeros((3,3))[[1,2,3]] Traceback (most recent call last): File "", line 1, in IndexError: index 3 is out of bounds for axis 0: [-3,3) Ray Jones ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Tue, Jun 5, 2012 at 12:15 PM, Thouis Jones wrote: > On Mon, Jun 4, 2012 at 11:49 PM, Nathaniel Smith wrote: >> On Mon, Jun 4, 2012 at 10:00 PM, Thouis (Ray) Jones wrote: >>> On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones wrote: I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change. >>> >>> My initial estimate was low, but not overly so. An initial pass at >>> adding index/dimension information to IndexErrors is here: >>> https://github.com/thouis/numpy/tree/index_error_info >> >> Fabulous! I made a few comments there, but also: >> >>> A typical result: >>> >> numpy.zeros(3)[5] >>> Traceback (most recent call last): >>> File "", line 1, in >>> IndexError: index 5 out of bounds in dimension 0 >> >> I would say "for", not "in". >> >> "index 5" is a bit ambiguous too... people might mis-read it as the >> dimension, like, "the 5th index value I gave"? Not sure how to make it >> unambiguous. Maybe: >> >> "IndexError: dimension 0 index out of bounds: got 5, size is 3" >> >> ? > > How about: > IndexError: 5 is out of bounds for dimension 0: must be in [-3, 3). > > to be maximally explicit about what values are allowed, and avoid the > "index" confusion. Or perhaps "axis" instead of "dimension", since this is how they are referred to in most numpy argument lists. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Mon, Jun 4, 2012 at 11:49 PM, Nathaniel Smith wrote: > On Mon, Jun 4, 2012 at 10:00 PM, Thouis (Ray) Jones wrote: >> On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones wrote: >>> I could look into this. There are only ~10 places the code generates >>> this error, so it should be a pretty minor change. >> >> My initial estimate was low, but not overly so. An initial pass at >> adding index/dimension information to IndexErrors is here: >> https://github.com/thouis/numpy/tree/index_error_info > > Fabulous! I made a few comments there, but also: > >> A typical result: >> > numpy.zeros(3)[5] >> Traceback (most recent call last): >> File "", line 1, in >> IndexError: index 5 out of bounds in dimension 0 > > I would say "for", not "in". > > "index 5" is a bit ambiguous too... people might mis-read it as the > dimension, like, "the 5th index value I gave"? Not sure how to make it > unambiguous. Maybe: > > "IndexError: dimension 0 index out of bounds: got 5, size is 3" > > ? How about: IndexError: 5 is out of bounds for dimension 0: must be in [-3, 3). to be maximally explicit about what values are allowed, and avoid the "index" confusion. Ray Jones ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Mon, Jun 4, 2012 at 10:00 PM, Thouis (Ray) Jones wrote: > On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones wrote: >> I could look into this. There are only ~10 places the code generates >> this error, so it should be a pretty minor change. > > My initial estimate was low, but not overly so. An initial pass at > adding index/dimension information to IndexErrors is here: > https://github.com/thouis/numpy/tree/index_error_info Fabulous! I made a few comments there, but also: > A typical result: > numpy.zeros(3)[5] > Traceback (most recent call last): > File "", line 1, in > IndexError: index 5 out of bounds in dimension 0 I would say "for", not "in". "index 5" is a bit ambiguous too... people might mis-read it as the dimension, like, "the 5th index value I gave"? Not sure how to make it unambiguous. Maybe: "IndexError: dimension 0 index out of bounds: got 5, size is 3" ? > I thought it best to have erroring indices report their initial value: > numpy.zeros(3)[-15] > Traceback (most recent call last): > File "", line 1, in > IndexError: index -15 out of bounds in dimension 0 > > This is different from some places in the code where IndexErrors > already had bad index and dimension information (including the maximum > value possible for an index in that dimension). I left these alone, > though most of them would report that the bad index was -12 instead of > -15. For instance: > https://github.com/thouis/numpy/blob/index_error_info/numpy/core/src/multiarray/mapping.c#L1640 I think this code you link to is actually correct, but yeah, it should definitely report whatever the user passed in, or it will be a debugging hindrance rather than a debugging help! > Also there were a few indexing errors that were throwing ValueErrors. > I changed these to IndexErrors. > > If someone could give this a cursory review before I issue a PR, I'd > appreciate it. I don't expect that most of these code paths are > heavily exercised in the tests (but I could be wrong). Perhaps the easiest thing would be to just add a test? It should be about 1 line each per code path... or 2 if you check both the negative and positive versions. def test_index_bound_checking(): assert_raises(IndexError, my_array.__getitem__, (0, 100)) assert_raises(IndexError, my_array.__getitem__, (0, -101)) # etc. - N ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Mon, Jun 4, 2012 at 4:27 PM, Thouis (Ray) Jones wrote: > On Fri, Jun 1, 2012 at 6:56 PM, Chris Withers wrote: >> On 01/06/2012 16:39, Benjamin Root wrote: >>> >>> >>> > >>> import numpy >>> > >>> numpy.zeros(10)[-123] >>> > Traceback (most recent call last): >>> > File "", line 1, in >>> > IndexError: index out of bounds >>> > >>> > ...could say this: >>> > >>> > >>> numpy.zeros(10)[-123] >>> > Traceback (most recent call last): >>> > File "", line 1, in >>> > IndexError: -123 is out of bounds >>> >>> Only that no-one has implemented it, I guess. If you want to then >>> that'd be cool :-). >>> >>> To be generally useful for debugging, it would probably be good for >>> the error message to also mention which dimension is involved, and/or >>> the actual size of the array in that dimension. You can also get such >>> error messages from expressions like 'arr[i, j, k]', after all, where >>> it's even less obvious what went wrong. >>> >>> -- Nathaniel >>> >>> >>> +1, please! >> >> Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that >> Python's built-in exceptions often tell you what went wrong but not what >> data caused the error, even when it's easily to hand when raising the >> exception. > > I could look into this. There are only ~10 places the code generates > this error, so it should be a pretty minor change. My initial estimate was low, but not overly so. An initial pass at adding index/dimension information to IndexErrors is here: https://github.com/thouis/numpy/tree/index_error_info A typical result: >>> numpy.zeros(3)[5] Traceback (most recent call last): File "", line 1, in IndexError: index 5 out of bounds in dimension 0 I thought it best to have erroring indices report their initial value: >>> numpy.zeros(3)[-15] Traceback (most recent call last): File "", line 1, in IndexError: index -15 out of bounds in dimension 0 This is different from some places in the code where IndexErrors already had bad index and dimension information (including the maximum value possible for an index in that dimension). I left these alone, though most of them would report that the bad index was -12 instead of -15. For instance: https://github.com/thouis/numpy/blob/index_error_info/numpy/core/src/multiarray/mapping.c#L1640 Also there were a few indexing errors that were throwing ValueErrors. I changed these to IndexErrors. If someone could give this a cursory review before I issue a PR, I'd appreciate it. I don't expect that most of these code paths are heavily exercised in the tests (but I could be wrong). Ray Jones ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On 4. juni 2012, at 16:27, Thouis (Ray) Jones wrote: > On Fri, Jun 1, 2012 at 6:56 PM, Chris Withers wrote: >> On 01/06/2012 16:39, Benjamin Root wrote: >>> >>> >>> > >>> import numpy >>> > >>> numpy.zeros(10)[-123] >>> > Traceback (most recent call last): >>> > File "", line 1, in >>> > IndexError: index out of bounds >>> > >>> > ...could say this: >>> > >>> > >>> numpy.zeros(10)[-123] >>> > Traceback (most recent call last): >>> > File "", line 1, in >>> > IndexError: -123 is out of bounds >>> >>> Only that no-one has implemented it, I guess. If you want to then >>> that'd be cool :-). >>> >>> To be generally useful for debugging, it would probably be good for >>> the error message to also mention which dimension is involved, and/or >>> the actual size of the array in that dimension. You can also get such >>> error messages from expressions like 'arr[i, j, k]', after all, where >>> it's even less obvious what went wrong. >>> >>> -- Nathaniel >>> >>> >>> +1, please! >> >> Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that >> Python's built-in exceptions often tell you what went wrong but not what >> data caused the error, even when it's easily to hand when raising the >> exception. > > I could look into this. There are only ~10 places the code generates > this error, so it should be a pretty minor change. > > Ray Jones Isn't it useful even if you change it in just one of those locations? Better to have the information available when you can, than to never have it. Paul ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Fri, Jun 1, 2012 at 6:56 PM, Chris Withers wrote: > On 01/06/2012 16:39, Benjamin Root wrote: >> >> >> > >>> import numpy >> > >>> numpy.zeros(10)[-123] >> > Traceback (most recent call last): >> > File "", line 1, in >> > IndexError: index out of bounds >> > >> > ...could say this: >> > >> > >>> numpy.zeros(10)[-123] >> > Traceback (most recent call last): >> > File "", line 1, in >> > IndexError: -123 is out of bounds >> >> Only that no-one has implemented it, I guess. If you want to then >> that'd be cool :-). >> >> To be generally useful for debugging, it would probably be good for >> the error message to also mention which dimension is involved, and/or >> the actual size of the array in that dimension. You can also get such >> error messages from expressions like 'arr[i, j, k]', after all, where >> it's even less obvious what went wrong. >> >> -- Nathaniel >> >> >> +1, please! > > Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that > Python's built-in exceptions often tell you what went wrong but not what > data caused the error, even when it's easily to hand when raising the > exception. I could look into this. There are only ~10 places the code generates this error, so it should be a pretty minor change. Ray Jones ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Fri, Jun 1, 2012 at 10:56 AM, Chris Withers wrote: > On 01/06/2012 16:39, Benjamin Root wrote: > > > > > > > >>> import numpy > > > >>> numpy.zeros(10)[-123] > > > Traceback (most recent call last): > > > File "", line 1, in > > > IndexError: index out of bounds > > > > > > ...could say this: > > > > > > >>> numpy.zeros(10)[-123] > > > Traceback (most recent call last): > > > File "", line 1, in > > > IndexError: -123 is out of bounds > > > > Only that no-one has implemented it, I guess. If you want to then > > that'd be cool :-). > > > > To be generally useful for debugging, it would probably be good for > > the error message to also mention which dimension is involved, and/or > > the actual size of the array in that dimension. You can also get such > > error messages from expressions like 'arr[i, j, k]', after all, where > > it's even less obvious what went wrong. > > > > -- Nathaniel > > > > > > +1, please! > > Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that > Python's built-in exceptions often tell you what went wrong but not what > data caused the error, even when it's easily to hand when raising the > exception. > > Where's the right place to raise an issue that a numpy developer can > hopefully make the (I suspect) simple change to get this behaviour? > > Hmm, how about I enable github issues for the numpy repository? It looks like we are headed that way and maybe now is a good time to get started. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On 01/06/2012 16:39, Benjamin Root wrote: > > > > >>> import numpy > > >>> numpy.zeros(10)[-123] > > Traceback (most recent call last): > > File "", line 1, in > > IndexError: index out of bounds > > > > ...could say this: > > > > >>> numpy.zeros(10)[-123] > > Traceback (most recent call last): > > File "", line 1, in > > IndexError: -123 is out of bounds > > Only that no-one has implemented it, I guess. If you want to then > that'd be cool :-). > > To be generally useful for debugging, it would probably be good for > the error message to also mention which dimension is involved, and/or > the actual size of the array in that dimension. You can also get such > error messages from expressions like 'arr[i, j, k]', after all, where > it's even less obvious what went wrong. > > -- Nathaniel > > > +1, please! Indeed, sadly I'm not a C developer. It's a pet bugbear of mine that Python's built-in exceptions often tell you what went wrong but not what data caused the error, even when it's easily to hand when raising the exception. Where's the right place to raise an issue that a numpy developer can hopefully make the (I suspect) simple change to get this behaviour? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
>> On Fri, Jun 1, 2012 at 10:46 AM, Chris Withers >> > Any reason why this: >> > >> > >>> import numpy >> > >>> numpy.zeros(10)[-123] >> > Traceback (most recent call last): >> > File "", line 1, in >> > IndexError: index out of bounds >> > >> > ...could say this: >> > >> > >>> numpy.zeros(10)[-123] >> > Traceback (most recent call last): >> > File "", line 1, in >> > IndexError: -123 is out of bounds >> >> Only that no-one has implemented it, I guess. If you want to then >> that'd be cool :-). That would be nice, but to be fair, python itself doesn't do it either: >>> l = range(10) >>> l[12] Traceback (most recent call last): File "", line 1, in IndexError: list index out of range Though Python's standard error messages are lacking in a lot of places... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Fri, Jun 1, 2012 at 9:14 AM, Nathaniel Smith wrote: > On Fri, Jun 1, 2012 at 10:46 AM, Chris Withers > wrote: > > Hi All, > > > > Any reason why this: > > > > >>> import numpy > > >>> numpy.zeros(10)[-123] > > Traceback (most recent call last): > > File "", line 1, in > > IndexError: index out of bounds > > > > ...could say this: > > > > >>> numpy.zeros(10)[-123] > > Traceback (most recent call last): > > File "", line 1, in > > IndexError: -123 is out of bounds > > Only that no-one has implemented it, I guess. If you want to then > that'd be cool :-). > > To be generally useful for debugging, it would probably be good for > the error message to also mention which dimension is involved, and/or > the actual size of the array in that dimension. You can also get such > error messages from expressions like 'arr[i, j, k]', after all, where > it's even less obvious what went wrong. > > -- Nathaniel > +1, please! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] better error message possible?
On Fri, Jun 1, 2012 at 10:46 AM, Chris Withers wrote: > Hi All, > > Any reason why this: > > >>> import numpy > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "", line 1, in > IndexError: index out of bounds > > ...could say this: > > >>> numpy.zeros(10)[-123] > Traceback (most recent call last): > File "", line 1, in > IndexError: -123 is out of bounds Only that no-one has implemented it, I guess. If you want to then that'd be cool :-). To be generally useful for debugging, it would probably be good for the error message to also mention which dimension is involved, and/or the actual size of the array in that dimension. You can also get such error messages from expressions like 'arr[i, j, k]', after all, where it's even less obvious what went wrong. -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] better error message possible?
Hi All, Any reason why this: >>> import numpy >>> numpy.zeros(10)[-123] Traceback (most recent call last): File "", line 1, in IndexError: index out of bounds ...could say this: >>> numpy.zeros(10)[-123] Traceback (most recent call last): File "", line 1, in IndexError: -123 is out of bounds cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
