Roundup Robot added the comment:
New changeset 369300948f3f by Stefan Krah in branch 'default':
Issue #22445: PyBuffer_IsContiguous() now implements precise contiguity
https://hg.python.org/cpython/rev/369300948f3f
--
nosy: +python-dev
___
Python
Changes by Stefan Krah ste...@bytereef.org:
--
resolution: - fixed
stage: patch review - resolved
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
___
Sebastian Berg added the comment:
Antoine, sounds good to me, I don't mind this being in python rather sooner
then later, for NumPy itself it does not matter I think. I just wanted to warn
that there were problems when we first tried to switch in NumPy, which, if I
remember correctly, is now
Antoine Pitrou added the comment:
Is this related to the NPY_RELAXED_STRIDES_CHECKING compilation flag?
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
___
Sebastian Berg added the comment:
@pitrou, yes of course. This would make python do the same thing as numpy does
(currently only with that compile flag given).
About the time schedule, I think I will try to see if some other numpy dev has
an opinion. Plus, should look into documenting it for
Antoine Pitrou added the comment:
Like Stefan I think this would be good to go in 3.5. The PyBuffer APIs are
relatively new so there shouldn't be a lot of breakage.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
Stefan Krah added the comment:
Okay, the whole thing isn't that urgent either.
Sorry for the confusion w.r.t slicing: I misremembered what the
latest numpy version did:
a)
x = np.array([[1,2,3,]])
x.strides
(9223372036854775807, 8)
b)
x = np.array([[1,2,3], [4,5,6]])[:1]
Stefan Krah added the comment:
FWIW, I think it would be good to make this change early in the
3.5 release cycle, so issues can be found. Sebastian, do you
have an idea when the change will be decided in numpy?
Regarding the discussion here ...
https://github.com/numpy/numpy/issues/5085
Sebastian Berg added the comment:
Numpy 1.9. was only released recently, so 1.10. might be a while. If no
problems show up during release or until then, we will likely switch it
by then. But that could end up being a year from now, so I am not sure
if 3.6 might not fit better. The problems
Stefan Krah added the comment:
Ok, here's my take on the situation:
1) As far as Python is concerned, shape[0] == 1 was already special-cased, so
people could not rely on canonical Fortran or C strides anyway.
2) Accessing an element via strides should be done using PyBuffer_GetPointer(),
Stefan Krah added the comment:
Since the functions in abstract.c have been committed by Travis Oliphant:
Could there have been a reason why the {shape=[1], strides=[-5]}
case was considered but the general case was not?
Or is it generally accepted among the numpy devs that not considering
the
Sebastian Berg added the comment:
Yeah, the code does much the same as the old numpy code (at least most of the
same funny little things, though I seem to remember the old numpy code had
something yet a bit weirder, would have to check).
To be honest, I do not know. It isn't implausible that
Sebastian Berg added the comment:
I am very sorry. The attached patch fixes this (not sure if quite right, but if
anything should be more general then necessary). One test fails, but it looks
like exactly the intended change.
--
Added file:
Changes by Sebastian Berg sebast...@sipsolutions.net:
Added file: http://bugs.python.org/file36677/relaxed-strides-checking.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
___
Changes by Sebastian Berg sebast...@sipsolutions.net:
Added file: http://bugs.python.org/file36678/contiguous.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
___
Changes by Sebastian Berg sebast...@sipsolutions.net:
Added file: http://bugs.python.org/file36680/contiguous.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
___
Stefan Krah added the comment:
Thanks! I still have to review the patch in depth, but generally
I'm +1 now for relaxing the contiguity check.
Curiously enough the existing code already considered e.g. shape=[1],
strides=[-5] as contiguous.
--
___
Stefan Krah added the comment:
Ok, so it is a debug thing in the current NumPy sources.
IMO ultimately the getbufferproc needs to return valid strides, even
if the first value isn't used.
For that matter, the getbufferproc is free to translate the multi-
dimensional corner case array to a
Sebastian Berg added the comment:
An extra dimension is certainly not irrelevant! The strides *are* valid
and numpy currently actually commonly creates such arrays when slicing.
The question is whether or not we want to ignore them for contiguity
checks even if they have no effect on the memory
Stefan Krah added the comment:
I think it would help discussing your options if the patch passes test_buffer
first. Currently it segfaults because shape can be NULL. Also, code in
memoryobject.c relies on the fact that ndim==0 means contiguous.
Then, it would help enormously if you give
New submission from Sebastian Berg:
In NumPy we decided some time ago that if you have a multi dimensional buffer,
shaped for example 1x10, then this buffer should be considered both C- and
F-contiguous. Currently, some buffers which can be used validly in a contiguous
fashion are rejected.
Stefan Krah added the comment:
There is another oddity: #12845. Does NumPy have a formal definition of
array contiguity somewhere?
--
nosy: +skrah
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
Stefan Krah added the comment:
BTW, if you have NumPy installed and run test_buffer in Python3.3+,
numpy.ndarray has many tests against memoryview and _testbuffer.ndarray
(the latter is our exegesis of PEP-3118).
--
___
Python tracker
Sebastian Berg added the comment:
#12845 should be closed, seems like a bug in some old version. The definition
now is simply that the array is contiguous if you can legally access it in a
contiguous fashion. Which means first stride is itemsize, second is
itemsize*shape[0] for Fortran,
Sebastian Berg added the comment:
To be clear, the important part here, is that to me all elements *can* be
accessed using that scheme. It is not correct to assume that `stride[-1]` or
`stride[0]` is actually equal to `itemsize`.
In other words, you have to be able to pass the pointer to the
Stefan Krah added the comment:
Thanks, #12845 is indeed fixed in NumPy.
Why does NumPy consider an array with a stride that will almost
certainly lead to undefined behavior (unless you compile with
-fwrapv) as valid?
In CPython we try to eliminate these kinds of issues (though
they may still
Sebastian Berg added the comment:
Well, the 9223372036854775807 is certainly no good for production code and we
would never have it in a release version, it is just there currently to expose
if there are more problems. However I don't care what happens on overflow (as
long as it is not an
Changes by Alexander Belopolsky alexander.belopol...@gmail.com:
--
nosy: +belopolsky
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22445
___
___
28 matches
Mail list logo
