Re: [Numpy-discussion] When to stop supporting Python 2.6?

[Christoph Deil]

> On 11 Dec 2015, at 07:52, Paul Hobson  wrote:
> 
> 
> 
> On Thu, Dec 10, 2015 at 4:20 AM, Julian Taylor  > wrote:
> On 12/09/2015 12:10 AM, Ralf Gommers wrote:
> 
> 
> On Wed, Dec 9, 2015 at 12:01 AM, Chris Barker  
> >> wrote:
> 
> drop 2.6
> 
> I still don't understand why folks insist that they need to run a
> (very)) old python on an old OS, but need the latest and greatest numpy.
> 
> Chuck's list was pretty long and compelling.
> 
> -CHB
> 
> 
> 
> On Mon, Dec 7, 2015 at 1:38 AM, Sturla Molden
> mailto:sturla.mol...@gmail.com> 
> >> wrote:
> 
> Charles R Harris  
>  >> wrote:
> 
> > As a strawman proposal, how about dropping moving to 2.7 and 3.4 
> minimum
> > supported version next fall, say around numpy 1.12 or 1.13 
> depending on how
> > the releases go.
> >
> > I would like to here from the scipy folks first.
> 
> 
> +1 for dropping Python 2.6, 3.2 and 3.3 after branching 1.11.x. We're
> already behind other projects like ipython, pandas and matplotlib as
> usual, so there really isn't much point in being the only project
> (together with scipy) of the core stack to keep on supporting more or
> less obsolete Python versions.
> 
> Ralf
> 
> 
> I don't see how that is a relevant point. NumPy is the lowest component of 
> the stack, we have to be the last to drop support for Python 2.6. And we 
> aren't yet the last even when only looking at the high profile components. 
> Astropy still supports 2.6 for another release.
> Though by the time 1.11 comes out we might be so I'm ok with dropping it 
> after that even when I'm not convinced we gain anything significant from 
> doing so.
> 
> Purely from a user-perspective, I don't understand why the numpy team would 
> want to continue support Python <= 2.6 and <= 3.3. The old versions of numpy 
> aren't going anywhere, so they can still be used if, for example, you're 
> stuck on a 6-yr old license of ArcGIS, and therefore stuck on Python 2.6
> 
> I started using Python with version 2.4 or 2.5 and there was zero discussion 
> about supporting old Python 1.X versions then. I know those situations are 
> aren't directly comparable, but when can we let the past go?
> -paul
>   
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> https://mail.scipy.org/mailman/listinfo/numpy-discussion 
> 

Another numpy user here.

At work we have the old Scientific Linux 6, which has Python 2.6 and an old 
Numpy version.

For most of my work I want and often need a newer Python and Numpy, which I can 
install in $HOME with conda.

For the old system Python 2.6 the sysadmin would never install Numpy 1.12, even 
if it was supported.
The whole idea is to leave it alone to make sure it’s stable.

I don’t understand the use case.
Is there anyone that really needs to install the future Numpy 1.12 into very 
old Python 2.6 installs?

Christoph

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] When to stop supporting Python 2.6?

[Paul Hobson]

On Thu, Dec 10, 2015 at 4:20 AM, Julian Taylor <
jtaylor.deb...@googlemail.com> wrote:

> On 12/09/2015 12:10 AM, Ralf Gommers wrote:
>
>>
>>
>> On Wed, Dec 9, 2015 at 12:01 AM, Chris Barker > > wrote:
>>
>> drop 2.6
>>
>> I still don't understand why folks insist that they need to run a
>> (very)) old python on an old OS, but need the latest and greatest
>> numpy.
>>
>> Chuck's list was pretty long and compelling.
>>
>> -CHB
>>
>>
>>
>> On Mon, Dec 7, 2015 at 1:38 AM, Sturla Molden
>> mailto:sturla.mol...@gmail.com>> wrote:
>>
>> Charles R Harris > > wrote:
>>
>> > As a strawman proposal, how about dropping moving to 2.7 and
>> 3.4 minimum
>> > supported version next fall, say around numpy 1.12 or 1.13
>> depending on how
>> > the releases go.
>> >
>> > I would like to here from the scipy folks first.
>>
>>
>> +1 for dropping Python 2.6, 3.2 and 3.3 after branching 1.11.x. We're
>> already behind other projects like ipython, pandas and matplotlib as
>> usual, so there really isn't much point in being the only project
>> (together with scipy) of the core stack to keep on supporting more or
>> less obsolete Python versions.
>>
>> Ralf
>>
>
>
> I don't see how that is a relevant point. NumPy is the lowest component of
> the stack, we have to be the last to drop support for Python 2.6. And we
> aren't yet the last even when only looking at the high profile components.
> Astropy still supports 2.6 for another release.
> Though by the time 1.11 comes out we might be so I'm ok with dropping it
> after that even when I'm not convinced we gain anything significant from
> doing so.


Purely from a user-perspective, I don't understand why the numpy team would
want to continue support Python <= 2.6 and <= 3.3. The old versions of
numpy aren't going anywhere, so they can still be used if, for example,
you're stuck on a 6-yr old license of ArcGIS, and therefore stuck on Python
2.6

I started using Python with version 2.4 or 2.5 and there was zero
discussion about supporting old Python 1.X versions then. I know those
situations are aren't directly comparable, but when can we let the past go?
-paul
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] volunteers for 1.11.0 release manager?

[Charles R Harris]

Hi All,

Thought I'd bring this up. I think the 1.11.0 release should be fairly easy
as releases go, so if someone wants to get some practice with making
releases this is probably a good one to start with. We should branch 1.11.x
sometime before the end of January. It is almost in branchable condition as
is, IMHO, but there are some things to take care of: the pile of PRs,
`__numpy_ufunc__`, and maybe a few more deprecations that should be changed
to errors.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy intermittent seg fault

[Nathaniel Smith]

On Thu, Dec 10, 2015 at 4:05 PM, Jacopo Sabbatini
 wrote:
> Hi,
>
> I'm experiencing random segmentation faults from numpy. I have generated a
> core dumped and extracted a stack trace, the following:
>
> #0  0x7f3a8d921d5d in getenv () from /lib64/libc.so.6
> #1  0x7f3a843bde21 in blas_set_parameter () from
> /opt/apps/sidescananalysis-9.7.1-42-gdd3e068+dev/lib/python2.7/site-packages/numpy/core/../../../../libopenblas.so.0
> #2  0x7f3a843bcd91 in blas_memory_alloc () from
> /opt/apps/sidescananalysis-9.7.1-42-gdd3e068+dev/lib/python2.7/site-packages/numpy/core/../../../../libopenblas.so.0
> #3  0x7f3a843bd4e5 in blas_thread_server () from
> /opt/apps/sidescananalysis-9.7.1-42-gdd3e068+dev/lib/python2.7/site-packages/numpy/core/../../../../libopenblas.so.0
> #4  0x7f3a8e09ff18 in start_thread () from /lib64/libpthread.so.0
> #5  0x7f3a8d9ceb2d in clone () from /lib64/libc.so.6

Given the backtrace this is almost certainly some sort of bug in
openblas, and I'd suggest filing a bug with them.

It's possible that we might have accidentally added a workaround in
1.10.2 (release candidate currently available, final should be out
soon). There was some environment variable handling code in numpy 1.9
through 1.10.1 that triggered problems in some buggy libraries (see
numpy issues #6460 and 6622); possibly the workaround for those issues
will also workaround this issue. But if not then I'm not sure what
else we can do, and it's probably a good idea to file a bug with
openblas regardless.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Numpy intermittent seg fault

[Jacopo Sabbatini]

Hi,

I'm experiencing random segmentation faults from numpy. I have generated a
core dumped and extracted a stack trace, the following:

#0  0x7f3a8d921d5d in getenv () from /lib64/libc.so.6
#1  0x7f3a843bde21 in blas_set_parameter () from
/opt/apps/sidescananalysis-9.7.1-42-gdd3e068+dev/lib/python2.7/site-packages/numpy/core/../../../../libopenblas.so.0
#2  0x7f3a843bcd91 in blas_memory_alloc () from
/opt/apps/sidescananalysis-9.7.1-42-gdd3e068+dev/lib/python2.7/site-packages/numpy/core/../../../../libopenblas.so.0
#3  0x7f3a843bd4e5 in blas_thread_server () from
/opt/apps/sidescananalysis-9.7.1-42-gdd3e068+dev/lib/python2.7/site-packages/numpy/core/../../../../libopenblas.so.0
#4  0x7f3a8e09ff18 in start_thread () from /lib64/libpthread.so.0
#5  0x7f3a8d9ceb2d in clone () from /lib64/libc.so.6

I have experience the segfault from several code paths but they all have
the same stack trace.

I use conda to run python and numpy. The dump of the packages version is:

conda 3.18.9   py27_0
geos  3.3.3 0
matplotlib1.4.3np19py27_2
networkx  1.10 py27_0
numpy 1.9.2py27_2
openblas  0.2.143
pandas0.16.2   np19py27_0
python2.7.102
scikit-image  0.11.3   np19py27_0
scipy 0.15.1   np19py27_0
shapely   1.5.7py27_0
system5.8   1

Cheers,
Jacopo
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Should dtypes have an ndim attribute?

[Nathaniel Smith]

On Dec 10, 2015 4:11 AM, "Andy Ray Terrel"  wrote:
>
> That's essentially what datashape did over in the blaze ecosystem. It
gets a bit fancier to support ragged arrays and optional types.
>
> http://datashape.readthedocs.org/en/latest/

IIUC this is a much more modest proposal, for numpy's existing dtypes that
represent subarrays.

It seems pretty harmless I guess. It's a little uncomfortable to be adding
yet another attribute to all dtypes that will only be used by one obscure
subset of dtypes, but it's no worse than a bunch of other already existing
attributes in that respect.

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Memory mapping and NPZ files

[Mathieu Dubois]

On 10/12/2015 13:55, Sturla Molden wrote:

Mathieu Dubois  wrote:


Does it make sense?

No. Memory mapping should just memory map, not do all sorts of crap.
The point is precisely that, you can't do memory mapping with Npz files 
(while it works with Npy files).


Mathieu
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Memory mapping and NPZ files

[Mathieu Dubois]

On 10/12/2015 15:35, Sebastian Berg wrote:

On Mi, 2015-12-09 at 15:51 +0100, Mathieu Dubois wrote:

Dear all,

If I am correct, using mmap_mode with Npz files has no effect i.e.:
f = np.load("data.npz", mmap_mode="r")
X = f['X']
will load all the data in memory.


My take on it is, that no, I do not want implicit extraction/copy of the
file.

I agree it's controversial.

However, npz files are not necessarily compressed, and I expect that in
the non-compressed version, memory-mapping is possible on the
uncompressed version.
If that is possible, it would ideally work for uncompressed npz files
and could raise an error which suggests to manually uncompress the file
when mmap_mode is given.

I got the same idea this afternoon. I will test that soon.

Thanks for your constructive answer!
Mathieu


- Sebastian


Can somebody confirm that?

If I'm correct, the mmap_mode argument could be passed to the NpzFile
class which could in turn perform the correct operation. One way to
handle that would be to use the ZipFile.extract method to write the
Npy file on disk and then load it with numpy.load with the mmap_mode
argument. Note that the user will have to remove the file to reclaim
disk space (I guess that's OK).

One problem that could arise is that the extracted Npy file can be
large (it's the purpose of using memory mapping) and therefore it may
be useful to offer some control on where this file is extracted (for
instance /tmp can be too small to extract the file here). numpy.load
could offer a new option for that (passed to ZipFile.extract).

Does it make sense?

Thanks in advance,
Mathieu
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Memory mapping and NPZ files

[Sebastian Berg]

On Mi, 2015-12-09 at 15:51 +0100, Mathieu Dubois wrote:
> Dear all,
> 
> If I am correct, using mmap_mode with Npz files has no effect i.e.:
> f = np.load("data.npz", mmap_mode="r")
> X = f['X']
> will load all the data in memory.
> 

My take on it is, that no, I do not want implicit extraction/copy of the
file.
However, npz files are not necessarily compressed, and I expect that in
the non-compressed version, memory-mapping is possible on the
uncompressed version.
If that is possible, it would ideally work for uncompressed npz files
and could raise an error which suggests to manually uncompress the file
when mmap_mode is given.

- Sebastian

> Can somebody confirm that?
> 
> If I'm correct, the mmap_mode argument could be passed to the NpzFile
> class which could in turn perform the correct operation. One way to
> handle that would be to use the ZipFile.extract method to write the
> Npy file on disk and then load it with numpy.load with the mmap_mode
> argument. Note that the user will have to remove the file to reclaim
> disk space (I guess that's OK).
> 
> One problem that could arise is that the extracted Npy file can be
> large (it's the purpose of using memory mapping) and therefore it may
> be useful to offer some control on where this file is extracted (for
> instance /tmp can be too small to extract the file here). numpy.load
> could offer a new option for that (passed to ZipFile.extract).
> 
> Does it make sense?
> 
> Thanks in advance,
> Mathieu
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Memory mapping and NPZ files

[Sturla Molden]

Mathieu Dubois  wrote:

> Does it make sense?

No. Memory mapping should just memory map, not do all sorts of crap.

Sturla

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] When to stop supporting Python 2.6?

[Julian Taylor]

On 12/09/2015 12:10 AM, Ralf Gommers wrote:



On Wed, Dec 9, 2015 at 12:01 AM, Chris Barker mailto:chris.bar...@noaa.gov>> wrote:

drop 2.6

I still don't understand why folks insist that they need to run a
(very)) old python on an old OS, but need the latest and greatest numpy.

Chuck's list was pretty long and compelling.

-CHB



On Mon, Dec 7, 2015 at 1:38 AM, Sturla Molden
mailto:sturla.mol...@gmail.com>> wrote:

Charles R Harris mailto:charlesr.har...@gmail.com>> wrote:

> As a strawman proposal, how about dropping moving to 2.7 and 3.4 
minimum
> supported version next fall, say around numpy 1.12 or 1.13 depending 
on how
> the releases go.
>
> I would like to here from the scipy folks first.


+1 for dropping Python 2.6, 3.2 and 3.3 after branching 1.11.x. We're
already behind other projects like ipython, pandas and matplotlib as
usual, so there really isn't much point in being the only project
(together with scipy) of the core stack to keep on supporting more or
less obsolete Python versions.

Ralf



I don't see how that is a relevant point. NumPy is the lowest component 
of the stack, we have to be the last to drop support for Python 2.6. And 
we aren't yet the last even when only looking at the high profile 
components. Astropy still supports 2.6 for another release.
Though by the time 1.11 comes out we might be so I'm ok with dropping it 
after that even when I'm not convinced we gain anything significant from 
doing so.

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Should dtypes have an ndim attribute?

[Andy Ray Terrel]

That's essentially what datashape did over in the blaze ecosystem. It gets
a bit fancier to support ragged arrays and optional types.

http://datashape.readthedocs.org/en/latest/
On Dec 10, 2015 5:14 AM, "Gerrit Holl"  wrote:

> Hi,
>
> I have made a modest proposal in issue #6752
> .  Basically, the proposal
> is to add an `ndim` attribute to dtypes.  Currently, arrays have a
> shape and an ndim attribute, where ndim equals len(shape).  dtype
> objects have a shape attribute, but no corresponding ndim.
>
> An ndim attribute would help in immediately determining whether a
> field in a structured dtype is multidimensional or not.
>
> Thoughts?
>
> Gerrit.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Should dtypes have an ndim attribute?

[Gerrit Holl]

Hi,

I have made a modest proposal in issue #6752
.  Basically, the proposal
is to add an `ndim` attribute to dtypes.  Currently, arrays have a
shape and an ndim attribute, where ndim equals len(shape).  dtype
objects have a shape attribute, but no corresponding ndim.

An ndim attribute would help in immediately determining whether a
field in a structured dtype is multidimensional or not.

Thoughts?

Gerrit.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion