Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Christoph Gohlke]

On 6/13/2011 5:58 AM, Ralf Gommers wrote:
> Hi,
>
> I am pleased to announce the availability of the first release candidate
> of NumPy 1.6.1. This is a bugfix release, list of fixed bugs:
> #1834   einsum fails for specific shapes
> #1837   einsum throws nan or freezes python for specific array shapes
> #1838   object <-> structured type arrays regression
> #1851   regression for SWIG based code in 1.6.0
> #1863   Buggy results when operating on array copied with astype()
>
> If no problems are reported, the final release will be in one week.
> Sources and binaries can be found
> at
> https://sourceforge.net/projects/numpy/files/NumPy/1.6.1rc1/
>
> Enjoy,
> Ralf
>

Hi,

in case there is going to be another release candidate: could a fix for 
ticket #1843 (numpy.rec.array fails with open file on Py3) be included 
in numpy 1.6.1? It's not crucial though.

http://projects.scipy.org/numpy/ticket/1843

Christoph
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Bruce Southey]

On 06/19/2011 05:21 AM, Ralf Gommers wrote:



On Tue, Jun 14, 2011 at 5:28 AM, Bruce Southey > wrote:


On Mon, Jun 13, 2011 at 8:31 PM, Pauli Virtanen mailto:p...@iki.fi>> wrote:
> On Mon, 13 Jun 2011 11:08:18 -0500, Bruce Southey wrote:
> [clip]
>> OSError:
>>
/usr/local/lib/python3.2/site-packages/numpy/core/multiarray.pyd:
cannot
>> open shared object file: No such file or directory
>
> I think that's a result of Python 3.2 changing the extension module
> file naming scheme (IIRC to a versioned one).
>
> The '.pyd' ending and its Unix counterpart are IIRC hardcoded to the
> failing test, or some support code in Numpy. Clearly, they
should instead
> ask Python how it names the extensions modules. This information
may be
> available somewhere in the `sys` module.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

Pauli,
I am impressed yet again!
It really saved a lot of time to understand the reason.

A quick comparison between Python versions:
Python2.7: multiarray.so
Python3.2: multiarray.cpython-32m.so


Search for the extension leads to PEP 3149
http://www.python.org/dev/peps/pep-3149/


That looked familiar:
http://projects.scipy.org/numpy/ticket/1749
https://github.com/numpy/numpy/commit/ee0831a8


This is POSIX only which excludes windows. I tracked it down to the
list 'libname_ext' defined in file "numpy/ctypeslib.py"  line 100:
libname_ext = ['%s.so' % libname, '%s.pyd' % libname]

On my 64-bit Linux system, I get the following results:
$ python2.4 -c "from distutils import sysconfig; print
sysconfig.get_config_var('SO')"
.so
$ python2.5 -c "from distutils import sysconfig; print
sysconfig.get_config_var('SO')"
.so
$ python2.7 -c "from distutils import sysconfig; print
sysconfig.get_config_var('SO')"
.so
$ python3.1 -c "from distutils import sysconfig;
print(sysconfig.get_config_var('SO'))"
.so
$ python3.2 -c "from distutils import sysconfig;
print(sysconfig.get_config_var('SO'))"
.cpython-32m.so

My Windows 32-bit Python2.6  install:
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('SO')
'.pyd'

Making the bug assumption that other OSes including 32-bit and 64-bit
versions also provide the correct suffix, this suggest adding the
import and modification as:

import from distutils import sysconfig
libname_ext = ['%s%s' % (libname, sysconfig.get_config_var('SO'))]

Actually if that works for Mac, Sun etc. then 'load_library' function
could be smaller.


The issue is that libraries built as Python extensions have the extra 
".cpython-mu32", but other libraries on your system don't. And 
sysconfig.get_config_var('SO') is used for both. I don't see another 
config var that allows to distinguish between the two. Unless on 
platforms where it matters there are separate return values in 
sysconfig.get_config_vars('SO'). What do you get for that on Linux?


The logic of figuring out the right extension string should not be 
duplicated, distutils.misc_util seems like a good place for it. Can 
you test if this works for you: 
https://github.com/rgommers/numpy/tree/sharedlib-ext



Finally the test_basic2 in "tests/test_ctypeslib.py" needs to be
changed as well because it is no longer correct. Just commenting out
the 'fix' appears to work.

It works in the case of trying to load multiarray, but the function 
under test would still be broken.


Ralf


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Okay,
So how do I get these changes and apply them to the release candidate?
(This part of the development/test workflow needs some attention.)

Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] replacing the mechanism for dispatching ufuncs

[Mark Wiebe]

NumPy has a mechanism built in to allow subclasses to adjust or override
aspects of the ufunc behavior. While this goal is important, this mechanism
only allows for very limited customization, making for instance the masked
arrays unable to work with the native ufuncs in a full and proper way. I
would like to deprecate the current mechanism, in particular
__array_prepare__ and __array_wrap__, and introduce a new method I will
describe below. If you've ever used these mechanisms, please review this
design to see if it meets your needs.


Any class type which would like to override its behavior in ufuncs would
define a method called _numpy_ufunc_, and optionally an attribute
__array_priority__ as can already be done. The class which wins the priority
battle gets its _numpy_ufunc_ function called as follows:

return arr._numpy_ufunc_(current_ufunc, *args, **kwargs)


To support this overloading, the ufunc would get a new support method,
result_type, and there would be a new global function, broadcast_empty_like.

The function ufunc.empty_like behaves like the global np.result_type, but
produces the output type or a tuple of output types specific to the ufunc,
which may follow a different convention than regular arithmetic type
promotion. This allows for a class to create an output array of the correct
type to pass to the ufunc if it needs to be different than the default.

The function broadcast_empty_like is just like empty_like, but takes a list
or tuple of arrays which are to be broadcast together for producing the
output, instead of just one.

Thanks,
Mark


A simple class which overrides the ufuncs might look as follows:

def sin(ufunc, *args, **kwargs):
# Convert degrees to radians
args[0] = np.deg2rad(args[0])
# Return a regular array, since the result is not in degrees
return ufunc(*args, **kwargs)

class MyDegreesClass:
"""Array-like object with a degrees unit"""

def __init__(arr):
self.arr = arr

def _numpy_ufunc_(ufunc, *args, **kwargs):
override = globals().get(ufunc.name)
if override:
return override(ufunc, *args, **kwargs)
else:
raise TypeError, 'ufunc %s incompatible with MyDegreesClass' %
ufunc.name



A more complex example will be something like this:

def my_general_ufunc(ufunc, *args, **kwargs):
# Extract the 'out' argument. This only supports ufuncs with
# one output, currently.
out = kwargs.get('out')
if len(args) > ufunc.nin:
if out is None:
out = args[ufunc.nin]
else:
raise ValueError, "'out' given as both a position and keyword
argument"

# Just want the inputs from here on
args = args[:ufunc.nin]

# Strip out MyArrayClass, but allow operations with regular ndarrays
raw_in = []
for a in args:
if isinstance(a, MyArrayClass):
raw_in.append(a.arr)
else:
raw_in.append(a)

# Allocate the output array
if not out is None:
if isinstance(out, MyArrayClass):
raise TypeError, "'out' must have type MyArrayClass"
else:
# Create the output array, obeying the 'order' parameter,
# but disallowing subclasses
out = np.broadcast_empty_like([args,
order=kwargs.get('order'),
dtype=ufunc.result_type(args),
subok=False)

# Override the output argument
kwargs['out'] = out.arr

# Call the ufunc
ufunc(*args, **kwargs)

# Return the output
return out

class MyArrayClass:
def __init__(arr):
self.arr = arr

def _numpy_ufunc_(ufunc, *args, **kwargs):
override = globals().get(ufunc.name)
if override:
return override(ufunc, *args, **kwargs)
else:
return my_general_ufunc(ufunc, *args, **kwargs)
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Bruce Southey]

On 06/19/2011 05:21 AM, Ralf Gommers wrote:



 On Tue, Jun 14, 2011 at 5:28 AM, Bruce Southey mailto:bsout...@gmail.com>> wrote:

 On Mon, Jun 13, 2011 at 8:31 PM, Pauli Virtanen mailto:p...@iki.fi>> wrote:
> On Mon, 13 Jun 2011 11:08:18 -0500, Bruce Southey wrote: [clip]
>> OSError:
>> /usr/local/lib/python3.2/site-packages/numpy/core/multiarray.pyd:
>> cannot open shared object file: No such file or directory
>
> I think that's a result of Python 3.2 changing the extension
> module file naming scheme (IIRC to a versioned one).
>
> The '.pyd' ending and its Unix counterpart are IIRC hardcoded to
> the failing test, or some support code in Numpy. Clearly, they
> should instead ask Python how it names the extensions modules. This
> information may be available somewhere in the `sys` module.
>
> ___ NumPy-Discussion
> mailing list NumPy-Discussion@scipy.org
> 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

 Pauli, I am impressed yet again! It really saved a lot of time to
 understand the reason.

 A quick comparison between Python versions: Python2.7: multiarray.so
 Python3.2: multiarray.cpython-32m.so
 

 Search for the extension leads to PEP 3149
 http://www.python.org/dev/peps/pep-3149/


 That looked familiar: http://projects.scipy.org/numpy/ticket/1749
 https://github.com/numpy/numpy/commit/ee0831a8


 This is POSIX only which excludes windows. I tracked it down to the
 list 'libname_ext' defined in file "numpy/ctypeslib.py" line 100:
 libname_ext = ['%s.so' % libname, '%s.pyd' % libname]

 On my 64-bit Linux system, I get the following results: $ python2.4
 -c "from distutils import sysconfig; print
 sysconfig.get_config_var('SO')" .so $ python2.5 -c "from distutils
 import sysconfig; print sysconfig.get_config_var('SO')" .so $
 python2.7 -c "from distutils import sysconfig; print
 sysconfig.get_config_var('SO')" .so $ python3.1 -c "from distutils
 import sysconfig; print(sysconfig.get_config_var('SO'))" .so $
 python3.2 -c "from distutils import sysconfig;
 print(sysconfig.get_config_var('SO'))" .cpython-32m.so

 My Windows 32-bit Python2.6 install:
>>> from distutils import sysconfig sysconfig.get_config_var('SO')
 '.pyd'

 Making the bug assumption that other OSes including 32-bit and
 64-bit versions also provide the correct suffix, this suggest adding
 the import and modification as:

 import from distutils import sysconfig libname_ext = ['%s%s' %
 (libname, sysconfig.get_config_var('SO'))]

 Actually if that works for Mac, Sun etc. then 'load_library'
 function could be smaller.


 The issue is that libraries built as Python extensions have the extra
 ".cpython-mu32", but other libraries on your system don't. And
 sysconfig.get_config_var('SO') is used for both. I don't see another
 config var that allows to distinguish between the two. Unless on
 platforms where it matters there are separate return values in
 sysconfig.get_config_vars('SO'). What do you get for that on Linux?

 The logic of figuring out the right extension string should not be
 duplicated, distutils.misc_util seems like a good place for it. Can
 you test if this works for you:
 https://github.com/rgommers/numpy/tree/sharedlib-ext


 Finally the test_basic2 in "tests/test_ctypeslib.py" needs to be
 changed as well because it is no longer correct. Just commenting out
 the 'fix' appears to work.

 It works in the case of trying to load multiarray, but the function
 under test would still be broken.

 Ralf


 ___ NumPy-Discussion
 mailing list NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion
I copied the files but that just moves the problem. So that patch is 
incorrect.


I get the same errors on Fedora 15 supplied Python3.2 for numpy 1.6.0 
and using git from 'https://github.com/rgommers/numpy.git'.  Numpy is 
getting Fedora supplied Atlas (1.5.1 does not).


It appears that there is a misunderstanding of the PEP because 'SO' and 
'SOABI' do exactly what the PEP says on my systems:

>> from distutils import sysconfig sysconfig.get_config_var('SO')

'.cpython-32m.so'

>> sysconfig.get_config_var('SOABI')

'cpython-32m'

Consequently, the name, 'multiarray.pyd', created within numpy is invalid.

Looking the code, I see this line which makes no sense given that the 
second part is true under Linux:


if (not is_python_ext) and 'SOABI' in distutils.sysconfig.get_config_vars():

So I think the 'get_shared_lib_extension' function is wrong and probably 
unneeded.



Bruce


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Ralf Gommers]

On Mon, Jun 20, 2011 at 4:20 PM, Bruce Southey  wrote:

> **
> On 06/19/2011 05:21 AM, Ralf Gommers wrote:
>
>
>
> On Tue, Jun 14, 2011 at 5:28 AM, Bruce Southey  wrote:
>
>>  On Mon, Jun 13, 2011 at 8:31 PM, Pauli Virtanen  wrote:
>> > On Mon, 13 Jun 2011 11:08:18 -0500, Bruce Southey wrote:
>> > [clip]
>> >> OSError:
>> >> /usr/local/lib/python3.2/site-packages/numpy/core/multiarray.pyd:
>> cannot
>> >> open shared object file: No such file or directory
>> >
>> > I think that's a result of Python 3.2 changing the extension module
>> > file naming scheme (IIRC to a versioned one).
>> >
>> > The '.pyd' ending and its Unix counterpart are IIRC hardcoded to the
>> > failing test, or some support code in Numpy. Clearly, they should
>> instead
>> > ask Python how it names the extensions modules. This information may be
>> > available somewhere in the `sys` module.
>> >
>> > ___
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion@scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>>
>>  Pauli,
>> I am impressed yet again!
>> It really saved a lot of time to understand the reason.
>>
>> A quick comparison between Python versions:
>> Python2.7: multiarray.so
>> Python3.2:  multiarray.cpython-32m.so
>>
>> Search for the extension leads to PEP 3149
>> http://www.python.org/dev/peps/pep-3149/
>>
>
> That looked familiar:
> http://projects.scipy.org/numpy/ticket/1749
> https://github.com/numpy/numpy/commit/ee0831a8
>
>>
>> This is POSIX only which excludes windows. I tracked it down to the
>> list 'libname_ext' defined in file "numpy/ctypeslib.py"  line 100:
>> libname_ext = ['%s.so' % libname, '%s.pyd' % libname]
>>
>> On my 64-bit Linux system, I get the following results:
>> $ python2.4 -c "from distutils import sysconfig; print
>> sysconfig.get_config_var('SO')"
>> .so
>> $ python2.5 -c "from distutils import sysconfig; print
>> sysconfig.get_config_var('SO')"
>> .so
>> $ python2.7 -c "from distutils import sysconfig; print
>> sysconfig.get_config_var('SO')"
>> .so
>> $ python3.1 -c "from distutils import sysconfig;
>> print(sysconfig.get_config_var('SO'))"
>> .so
>> $ python3.2 -c "from distutils import sysconfig;
>> print(sysconfig.get_config_var('SO'))"
>> .cpython-32m.so
>>
>> My Windows 32-bit Python2.6  install:
>> >>> from distutils import sysconfig
>> >>> sysconfig.get_config_var('SO')
>> '.pyd'
>>
>> Making the bug assumption that other OSes including 32-bit and 64-bit
>> versions also provide the correct suffix, this suggest adding the
>> import and modification as:
>>
>> import from distutils import sysconfig
>> libname_ext = ['%s%s' % (libname, sysconfig.get_config_var('SO'))]
>>
>> Actually if that works for Mac, Sun etc. then 'load_library' function
>> could be smaller.
>>
>
> The issue is that libraries built as Python extensions have the extra
> ".cpython-mu32", but other libraries on your system don't. And
> sysconfig.get_config_var('SO') is used for both. I don't see another config
> var that allows to distinguish between the two. Unless on platforms where it
> matters there are separate return values in sysconfig.get_config_vars('SO').
> What do you get for that on Linux?
>
> The logic of figuring out the right extension string should not be
> duplicated, distutils.misc_util seems like a good place for it. Can you test
> if this works for you:
> https://github.com/rgommers/numpy/tree/sharedlib-ext
>
>
>> Finally the test_basic2 in "tests/test_ctypeslib.py" needs to be
>> changed as well because it is no longer correct. Just commenting out
>> the 'fix' appears to work.
>>
>>  It works in the case of trying to load multiarray, but the function
> under test would still be broken.
>
> Ralf
>
>
> ___
> NumPy-Discussion mailing 
> listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>  Okay,
> So how do I get these changes and apply them to the release candidate?
> (This part of the development/test workflow needs some attention.)
>
> You could just have tested that branch, it should have the same issue. To
test on 1.6.x, you can just checkout 1.6.x and cherry-pick the relevant
commit. I've done that for you at
https://github.com/rgommers/numpy/tree/sharedlib-ext-1.6.x

For completeness, this is how you grab it:
$ git remote add rgommers https://github.com/rgommers/numpy.git
$ git fetch rgommers
$ git co rgommers/sharedlib-ext-1.6.x

Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Ralf Gommers]

On Mon, Jun 20, 2011 at 12:48 AM, Mark Wiebe  wrote:

> On Mon, Jun 13, 2011 at 7:58 AM, Ralf Gommers  > wrote:
>
>> Hi,
>>
>> I am pleased to announce the availability of the first release candidate
>> of NumPy 1.6.1. This is a bugfix release, list of fixed bugs:
>> #1834   einsum fails for specific shapes
>> #1837   einsum throws nan or freezes python for specific array shapes
>> #1838   object <-> structured type arrays regression
>> #1851   regression for SWIG based code in 1.6.0
>> #1863   Buggy results when operating on array copied with astype()
>>
>> If no problems are reported, the final release will be in one week.
>> Sources and binaries can be found 
>> at
>> https://sourceforge.net/projects/numpy/files/NumPy/1.6.1rc1/
>
>
> I've committed a fix to #1870, a regression in 1.6.0, in master. I'll leave
> you with the judgement call on what to do with it.
>
> It's a regression, so it would be good to include it. This together with
the ctypes.load_library thing would require a second RC, but that's fine.

I don't really understand that code though and don't have much time to test
it either, so I would appreciate if someone else can review it.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Ralf Gommers]

On Mon, Jun 20, 2011 at 9:19 AM, Christoph Gohlke  wrote:

>
>
> On 6/13/2011 5:58 AM, Ralf Gommers wrote:
> > Hi,
> >
> > I am pleased to announce the availability of the first release candidate
> > of NumPy 1.6.1. This is a bugfix release, list of fixed bugs:
> > #1834   einsum fails for specific shapes
> > #1837   einsum throws nan or freezes python for specific array shapes
> > #1838   object <-> structured type arrays regression
> > #1851   regression for SWIG based code in 1.6.0
> > #1863   Buggy results when operating on array copied with astype()
> >
> > If no problems are reported, the final release will be in one week.
> > Sources and binaries can be found
> > at
> > https://sourceforge.net/projects/numpy/files/NumPy/1.6.1rc1/
> >
> > Enjoy,
> > Ralf
> >
>
> Hi,
>
> in case there is going to be another release candidate: could a fix for
> ticket #1843 (numpy.rec.array fails with open file on Py3) be included
> in numpy 1.6.1? It's not crucial though.
>
> http://projects.scipy.org/numpy/ticket/1843
>
> This should be reviewed and committed to master first. If someone can do
that in the next couple of days I'm fine with backporting it.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Ralf Gommers]

On Mon, Jun 20, 2011 at 8:50 PM, Bruce Southey  wrote:

>
> I copied the files but that just moves the problem. So that patch is
> incorrect.
>
> I get the same errors on Fedora 15 supplied Python3.2 for numpy 1.6.0 and
> using git from 'https://github.com/rgommers/numpy.git'.  Numpy is getting
> Fedora supplied Atlas (1.5.1 does not).
>
> It appears that there is a misunderstanding of the PEP because 'SO' and
> 'SOABI' do exactly what the PEP says on my systems:
>

It doesn't on OS X. But that's not even the issue. As I explained before,
the issue is that get_config_var('SO') is used to determine the extension of
system libraries (such as liblapack.so) and python-related ones (such as
multiarray.cpython-32m.so).  And the current functions don't do mindreading.

>
> >>> from distutils import sysconfig sysconfig.get_config_var('SO')
> '.cpython-32m.so'
> >>> sysconfig.get_config_var('SOABI')
> 'cpython-32m'
>
> Consequently, the name, 'multiarray.pyd', created within numpy is invalid.
>

I removed the line in ctypeslib that was trying this, so I think you are not
testing my patch.

Ralf


> Looking the code, I see this line which makes no sense given that the
> second part is true under Linux:
>
> if (not is_python_ext) and 'SOABI' in
> distutils.sysconfig.get_config_vars():
>
> So I think the 'get_shared_lib_extension' function is wrong and probably
> unneeded.
>
>
> Bruce
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] fast grayscale conversion

[Alex Flint]

At the moment I'm using numpy.dot to convert a WxHx3 RGB image to a
grayscale image:

src_mono = np.dot(src_rgb.astype(np.float), np.ones(3)/3.);

This seems quite slow though (several seconds for a 3 megapixel image) - is
there a more specialized routine better suited to this?

Cheers,
Alex
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Mark Wiebe]

On Mon, Jun 20, 2011 at 2:33 PM, Ralf Gommers
wrote:

> On Mon, Jun 20, 2011 at 12:48 AM, Mark Wiebe  wrote:
>
>> On Mon, Jun 13, 2011 at 7:58 AM, Ralf Gommers <
>> ralf.gomm...@googlemail.com> wrote:
>>
>>> Hi,
>>>
>>> I am pleased to announce the availability of the first release candidate
>>> of NumPy 1.6.1. This is a bugfix release, list of fixed bugs:
>>> #1834   einsum fails for specific shapes
>>> #1837   einsum throws nan or freezes python for specific array shapes
>>> #1838   object <-> structured type arrays regression
>>> #1851   regression for SWIG based code in 1.6.0
>>> #1863   Buggy results when operating on array copied with astype()
>>>
>>> If no problems are reported, the final release will be in one week.
>>> Sources and binaries can be found 
>>> at
>>> https://sourceforge.net/projects/numpy/files/NumPy/1.6.1rc1/
>>
>>
>> I've committed a fix to #1870, a regression in 1.6.0, in master. I'll
>> leave you with the judgement call on what to do with it.
>>
>> It's a regression, so it would be good to include it. This together with
> the ctypes.load_library thing would require a second RC, but that's fine.
>
> I don't really understand that code though and don't have much time to test
> it either, so I would appreciate if someone else can review it.
>

I've created a pull request to try and make this as easy as possible:

https://github.com/numpy/numpy/pull/92

-Mark


>
> Ralf
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fast grayscale conversion

[Zachary Pincus]

You could try:
src_mono = src_rgb.astype(float).sum(axis=-1) / 3.

But that speed does seem slow. Here are the relevant timings on my machine (a 
recent MacBook Pro) for a 3.1-megapixel-size array:
In [16]: a = numpy.empty((2048, 1536, 3), dtype=numpy.uint8)

In [17]: timeit numpy.dot(a.astype(float), numpy.ones(3)/3.)
10 loops, best of 3: 116 ms per loop

In [18]: timeit a.astype(float).sum(axis=-1)/3.
10 loops, best of 3: 85.3 ms per loop

In [19]: timeit a.astype(float)
10 loops, best of 3: 23.3 ms per loop




On Jun 20, 2011, at 4:15 PM, Alex Flint wrote:

> At the moment I'm using numpy.dot to convert a WxHx3 RGB image to a grayscale 
> image:
> 
> src_mono = np.dot(src_rgb.astype(np.float), np.ones(3)/3.);
> 
> This seems quite slow though (several seconds for a 3 megapixel image) - is 
> there a more specialized routine better suited to this?
> 
> Cheers,
> Alex
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fast grayscale conversion

[Eric Firing]

On 06/20/2011 10:41 AM, Zachary Pincus wrote:
> You could try:
> src_mono = src_rgb.astype(float).sum(axis=-1) / 3.
>
> But that speed does seem slow. Here are the relevant timings on my machine (a 
> recent MacBook Pro) for a 3.1-megapixel-size array:
> In [16]: a = numpy.empty((2048, 1536, 3), dtype=numpy.uint8)
>
> In [17]: timeit numpy.dot(a.astype(float), numpy.ones(3)/3.)
> 10 loops, best of 3: 116 ms per loop
>
> In [18]: timeit a.astype(float).sum(axis=-1)/3.
> 10 loops, best of 3: 85.3 ms per loop
>
> In [19]: timeit a.astype(float)
> 10 loops, best of 3: 23.3 ms per loop
>
>

On my slower machine (older laptop, core2 duo), you can speed it up more:

In [3]: timeit a.astype(float).sum(axis=-1)/3.0
1 loops, best of 3: 235 ms per loop

In [5]: timeit b = a.astype(float).sum(axis=-1); b /= 3.0
1 loops, best of 3: 181 ms per loop

In [7]: timeit b = a.astype(np.float32).sum(axis=-1); b /= 3.0
10 loops, best of 3: 148 ms per loop

If you really want float64, it is still faster to do the first operation 
with single precision:

In [8]: timeit b = a.astype(np.float32).sum(axis=-1).astype(np.float64); 
b /= 3.0
10 loops, best of 3: 163 ms per loop

Eric


>
>
> On Jun 20, 2011, at 4:15 PM, Alex Flint wrote:
>
>> At the moment I'm using numpy.dot to convert a WxHx3 RGB image to a 
>> grayscale image:
>>
>> src_mono = np.dot(src_rgb.astype(np.float), np.ones(3)/3.);
>>
>> This seems quite slow though (several seconds for a 3 megapixel image) - is 
>> there a more specialized routine better suited to this?
>>
>> Cheers,
>> Alex
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Numpy/Scipy Testing Guidelines URL?

[Skipper Seabold]

Are the testing guidelines included in the HTML docs anywhere? If I
recall, they used to be, and I couldn't find them with a brief
look/google. I'd like to link to them. Maybe the rendered rst page is
considered their new home?

https://github.com/numpy/numpy/blob/master/doc/TESTS.rst.txt

Skipper
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy/Scipy Testing Guidelines URL?

[Ralf Gommers]

On Mon, Jun 20, 2011 at 11:36 PM, Skipper Seabold wrote:

> Are the testing guidelines included in the HTML docs anywhere? If I
> recall, they used to be, and I couldn't find them with a brief
> look/google. I'd like to link to them. Maybe the rendered rst page is
> considered their new home?
>

That is the new home. Should be linked from the old home. A patch to add
those to the numpy devguide is welcome I think.

Ralf



> https://github.com/numpy/numpy/blob/master/doc/TESTS.rst.txt
>
> Skipper
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy/Scipy Testing Guidelines URL?

[Skipper Seabold]

On Mon, Jun 20, 2011 at 5:41 PM, Ralf Gommers
 wrote:
>
>
> On Mon, Jun 20, 2011 at 11:36 PM, Skipper Seabold 
> wrote:
>>
>> Are the testing guidelines included in the HTML docs anywhere? If I
>> recall, they used to be, and I couldn't find them with a brief
>> look/google. I'd like to link to them. Maybe the rendered rst page is
>> considered their new home?
>
> That is the new home. Should be linked from the old home. A patch to add
> those to the numpy devguide is welcome I think.
>

numpy/doc/source/dev? This is what I was thinking. But I didn't know
if there was some other grand design to having a lot of the developer
specific information in numpy/doc.

Skipper
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy/Scipy Testing Guidelines URL?

[Ralf Gommers]

On Mon, Jun 20, 2011 at 11:44 PM, Skipper Seabold wrote:

> On Mon, Jun 20, 2011 at 5:41 PM, Ralf Gommers
>  wrote:
> >
> >
> > On Mon, Jun 20, 2011 at 11:36 PM, Skipper Seabold 
> > wrote:
> >>
> >> Are the testing guidelines included in the HTML docs anywhere? If I
> >> recall, they used to be, and I couldn't find them with a brief
> >> look/google. I'd like to link to them. Maybe the rendered rst page is
> >> considered their new home?
> >
> > That is the new home. Should be linked from the old home. A patch to add
> > those to the numpy devguide is welcome I think.
> >
>
> numpy/doc/source/dev?


Yep, ends up at http://docs.scipy.org/doc/numpy/dev/


> This is what I was thinking. But I didn't know
> if there was some other grand design to having a lot of the developer
> specific information in numpy/doc.
>
> If there is, I haven't seen the blueprint.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fast grayscale conversion

[Alex Flint]

Thanks, that's helpful. I'm now getting comparable times on a different
machine, it must be something else slowing down my machine more generally,
not just numpy.

On Mon, Jun 20, 2011 at 5:11 PM, Eric Firing  wrote:

> On 06/20/2011 10:41 AM, Zachary Pincus wrote:
> > You could try:
> > src_mono = src_rgb.astype(float).sum(axis=-1) / 3.
> >
> > But that speed does seem slow. Here are the relevant timings on my
> machine (a recent MacBook Pro) for a 3.1-megapixel-size array:
> > In [16]: a = numpy.empty((2048, 1536, 3), dtype=numpy.uint8)
> >
> > In [17]: timeit numpy.dot(a.astype(float), numpy.ones(3)/3.)
> > 10 loops, best of 3: 116 ms per loop
> >
> > In [18]: timeit a.astype(float).sum(axis=-1)/3.
> > 10 loops, best of 3: 85.3 ms per loop
> >
> > In [19]: timeit a.astype(float)
> > 10 loops, best of 3: 23.3 ms per loop
> >
> >
>
> On my slower machine (older laptop, core2 duo), you can speed it up more:
>
> In [3]: timeit a.astype(float).sum(axis=-1)/3.0
> 1 loops, best of 3: 235 ms per loop
>
> In [5]: timeit b = a.astype(float).sum(axis=-1); b /= 3.0
> 1 loops, best of 3: 181 ms per loop
>
> In [7]: timeit b = a.astype(np.float32).sum(axis=-1); b /= 3.0
> 10 loops, best of 3: 148 ms per loop
>
> If you really want float64, it is still faster to do the first operation
> with single precision:
>
> In [8]: timeit b = a.astype(np.float32).sum(axis=-1).astype(np.float64);
> b /= 3.0
> 10 loops, best of 3: 163 ms per loop
>
> Eric
>
>
> >
> >
> > On Jun 20, 2011, at 4:15 PM, Alex Flint wrote:
> >
> >> At the moment I'm using numpy.dot to convert a WxHx3 RGB image to a
> grayscale image:
> >>
> >> src_mono = np.dot(src_rgb.astype(np.float), np.ones(3)/3.);
> >>
> >> This seems quite slow though (several seconds for a 3 megapixel image) -
> is there a more specialized routine better suited to this?
> >>
> >> Cheers,
> >> Alex
> >>
> >> ___
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion@scipy.org
> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Object array from list in 1.6.0 (vs. 1.5.1)

[Mark Wiebe]

This pull request which needs some testing should fix the issue:

https://github.com/numpy/numpy/pull/92

-Mark

On Fri, Jun 10, 2011 at 11:40 AM, Ken Basye  wrote:

> Dear folks,
>   I have some code that stopped working with 1.6.0 and I'm wondering if
> there's a better way to replace it than what I came up with.  Here's a
> condensed version:
>
> x = [()]  # list containing an empty tuple; this isn't the only case,
> but it's one that must be handled correctly
> y = np.empty(len(x), dtype=object)
> y[:] = x[:]
>
> In 1.5.1 this works, giving y as "array([()], dtype=object)"
>
> In 1.6.0, it raises a ValueError:
> ValueError: output operand requires a reduction, but reduction is not
> enabled
>
> I didn't see anything in the release notes about this; admittedly it's a
> very small corner case.   I also don't understand what the error message
> is trying to tell me here.
>
> Most of the more straightforward ways to construct the desired array
> don't work because the interpretation of a nested structure is as a
> multi-dimensional array, which is reasonable.
>
> At this point my workaround is to replace the assignment with a loop:
>
> for i, v in enumerate(x):
>  y[i] = v
>
> but this seems non-Numpyish.  Any light on what the error means or a
> better way to do this would be most welcome.
>
> Thanks,
> Ken
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] fast grayscale conversion

[Christopher Barker]

Alex Flint wrote:
> Thanks, that's helpful. I'm now getting comparable times on a different 
> machine, it must be something else slowing down my machine more 
> generally, not just numpy.

you also might want to get a bit fancier than simply scaling linearly 
R,G, and B don't necessarily all contribute equally to our sense of 
"whiteness"

For instance, PIL uses:

"""
When from a colour image to black and white, the library uses the ITU-R 
601-2 luma transform:

 L = R * 299/1000 + G * 587/1000 + B * 114/1000
"""

which would be easy enough to do with numpy.

-Chris


> 
> On Mon, Jun 20, 2011 at 5:11 PM, Eric Firing  > wrote:
> 
> On 06/20/2011 10:41 AM, Zachary Pincus wrote:
>  > You could try:
>  > src_mono = src_rgb.astype(float).sum(axis=-1) / 3.
>  >
>  > But that speed does seem slow. Here are the relevant timings on
> my machine (a recent MacBook Pro) for a 3.1-megapixel-size array:
>  > In [16]: a = numpy.empty((2048, 1536, 3), dtype=numpy.uint8)
>  >
>  > In [17]: timeit numpy.dot(a.astype(float), numpy.ones(3)/3.)
>  > 10 loops, best of 3: 116 ms per loop
>  >
>  > In [18]: timeit a.astype(float).sum(axis=-1)/3.
>  > 10 loops, best of 3: 85.3 ms per loop
>  >
>  > In [19]: timeit a.astype(float)
>  > 10 loops, best of 3: 23.3 ms per loop
>  >
>  >
> 
> On my slower machine (older laptop, core2 duo), you can speed it up
> more:
> 
> In [3]: timeit a.astype(float).sum(axis=-1)/3.0
> 1 loops, best of 3: 235 ms per loop
> 
> In [5]: timeit b = a.astype(float).sum(axis=-1); b /= 3.0
> 1 loops, best of 3: 181 ms per loop
> 
> In [7]: timeit b = a.astype(np.float32).sum(axis=-1); b /= 3.0
> 10 loops, best of 3: 148 ms per loop
> 
> If you really want float64, it is still faster to do the first operation
> with single precision:
> 
> In [8]: timeit b = a.astype(np.float32).sum(axis=-1).astype(np.float64);
> b /= 3.0
> 10 loops, best of 3: 163 ms per loop
> 
> Eric
> 
> 
>  >
>  >
>  > On Jun 20, 2011, at 4:15 PM, Alex Flint wrote:
>  >
>  >> At the moment I'm using numpy.dot to convert a WxHx3 RGB image
> to a grayscale image:
>  >>
>  >> src_mono = np.dot(src_rgb.astype(np.float), np.ones(3)/3.);
>  >>
>  >> This seems quite slow though (several seconds for a 3 megapixel
> image) - is there a more specialized routine better suited to this?
>  >>
>  >> Cheers,
>  >> Alex
>  >>
>  >> ___
>  >> NumPy-Discussion mailing list
>  >> NumPy-Discussion@scipy.org 
>  >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>  >
>  > ___
>  > NumPy-Discussion mailing list
>  > NumPy-Discussion@scipy.org 
>  > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> 
> 
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] datetime pull request for review and build/test

[Mark Wiebe]

https://github.com/numpy/numpy/pull/93

The summary:

* Tighten up date unit vs time unit casting rules, and integrate the
NPY_CASTING enum deeper into the datetime conversions
* Determine a unit when converting from a string array, similar to when
converting from lists of strings
* Switch local/utc handling to a timezone= parameter, which also accepts a
datetime.tzinfo object. This, for example, enables the use of the pytz
library with numpy.datetime64
* Remove the events metadata, make the old API functions raise exceptions,
and rename the "frequency" metadata name to "timeunit"
* Abstract the flexible dtype mechanism into a function, so that it can be
more easily changed without having to add code to many places

-Mark
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] npyio -> gzip 271 Error -3 while decompressing ?

[Derek Homeier]

Moin Denis,

On 20 Jun 2011, at 19:04, denis wrote:
>  a separate question, have you run genfromtxt( "xx.csv.gz" ) lately ?

I haven't, and I was not particularly involved with it before this  
patch, so this would possibly be better addressed to the list.

> On on .../scikits.learn-0.8/scikits/learn/datasets/data.digits.csv.gz
> numpy 1.6.0, py 2.6 mac I get
>
>X = np.genfromtxt( filename, delimiter="," )
>  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
> python2.6/site-packages/numpy/lib/npyio.py", line 1271, in genfromtxt
>first_line = fhd.next()
>  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
> python2.6/gzip.py", line 438, in next
>line = self.readline()
>  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
> python2.6/gzip.py", line 393, in readline
>c = self.read(readsize)
>  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
> python2.6/gzip.py", line 219, in read
>self._read(readsize)
>  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
> python2.6/gzip.py", line 271, in _read
>uncompress = self.decompress.decompress(buf)
> zlib.error: Error -3 while decompressing: invalid distance too far
> back
>
> It would be nice to fix this too, if it hasn't been already.
> Btw the file gunzips fine.

I could reproduce that error for the gzip'ed csv files in that  
directory; it can be isolated to the underlying gzip call above -
fhd = gzip.open('digits.csv.gz', 'rbU'); fhd.next()
produces the same error for these files with all python2.x versions on  
my Mac, but not with python3.x. Also only with the 'U' mode specified,  
yet the same mode is parsing other .gz files just fine. I could not  
really track down what the 'U' flag is doing in gzip.py, but I assume  
it is specifying some kind of unbuffered read. Also it's a mystery to  
me what is different in those files that would trigger the error. I  
even read them in with loadtxt() and wrote them back using constant  
line width and/or spaces as separators, still producing the same  
exception.
The obvious place to fix this (or work around a bug in python2's  
gzip.py, whatever), would be changing the open command in genfromtxt
fhd = iter(np.lib._datasource.open(fname, 'rbU'))
to omit the 'U' at least with python2. Alternatively one could do a  
test read and catch the exception, to then re-open the file with mode  
'rb'...
Pierre, if you are reading this, can you comment how important the 'U'  
is for performance considerations or such?

HTH,
Derek


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Bruce Southey]

On Mon, Jun 20, 2011 at 2:43 PM, Ralf Gommers
 wrote:
>
>
> On Mon, Jun 20, 2011 at 8:50 PM, Bruce Southey  wrote:
>>
>> I copied the files but that just moves the problem. So that patch is
>> incorrect.
>>
>> I get the same errors on Fedora 15 supplied Python3.2 for numpy 1.6.0 and
>> using git from 'https://github.com/rgommers/numpy.git'.  Numpy is getting
>> Fedora supplied Atlas (1.5.1 does not).
>>
>> It appears that there is a misunderstanding of the PEP because 'SO' and
>> 'SOABI' do exactly what the PEP says on my systems:
>
> It doesn't on OS X. But that's not even the issue. As I explained before,
> the issue is that get_config_var('SO') is used to determine the extension of
> system libraries (such as liblapack.so) and python-related ones (such as
> multiarray.cpython-32m.so).  And the current functions don't do mindreading.
>>
>> >>> from distutils import sysconfig sysconfig.get_config_var('SO')
>> '.cpython-32m.so'
>> >>> sysconfig.get_config_var('SOABI')
>> 'cpython-32m'
>>
>> Consequently, the name, 'multiarray.pyd', created within numpy is invalid.
>
> I removed the line in ctypeslib that was trying this, so I think you are not
> testing my patch.
>
> Ralf
>
>>
>> Looking the code, I see this line which makes no sense given that the
>> second part is true under Linux:
>>
>> if (not is_python_ext) and 'SOABI' in
>> distutils.sysconfig.get_config_vars():
>>
>> So I think the 'get_shared_lib_extension' function is wrong and probably
>> unneeded.
>>
>>
>> Bruce
>>

Just to show that this is the new version, I added two print
statements in the 'get_shared_lib_extension' function:
>>> from numpy.distutils.misc_util import get_shared_lib_extension
>>> get_shared_lib_extension(True)
first so_ext .cpython-32mu.so
returned so_ext .cpython-32mu.so
'.cpython-32mu.so'
>>> get_shared_lib_extension(False)
first so_ext .cpython-32mu.so
returned so_ext .so
'.so'

The reason for the same location is obvious because all the patch does
is move  the code to get the extension into that function. So the
'get_shared_lib_extension' function returns the extension '.so' to the
load_library function. However that name is wrong under Linux as it
has to be 'multiarray.cpython-32mu.so' and hence the error in the same
location. I did come across this thread
'http://bugs.python.org/issue10262' which indicates why Linux is
different by default.

So what is the actual name of the multiarray shared library with the Mac?
If it is ' 'multiarray.so' then the correct name is "libname +
sysconfig.get_config_var('SO')" as I previously indicated.


Bruce




$ python3
Python 3.2 (r32:88445, Feb 21 2011, 21:11:06)
[GCC 4.6.0 20110212 (Red Hat 4.6.0-0.7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.test()
Running unit tests for numpy
NumPy version 2.0.0.dev-Unknown
NumPy is installed in /usr/lib64/python3.2/site-packages/numpy
Python version 3.2 (r32:88445, Feb 21 2011, 21:11:06) [GCC 4.6.0
20110212 (Red Hat 4.6.0-0.7)]
nose version 1.0.0
first so_ext .cpython-32mu.so
returned so_ext .so
...F...S..F.E..KK..K..K...first
so_ext .cpython-32mu.so
returned so_ext .so
...S/usr/lib64/python3.2/site-

[Numpy-discussion] numpy submodules missing in intersphinx objects.inv?

[Skipper Seabold]

I was just trying to link to the numpy.testing module but can't using
intersphinx. Does anyone know why certain submodules aren't included
in objects.inv? It looks as though it has something to do either with
having a reference at the top of the rst file (so you can :ref: link
to it) or having that module autodoc'd (then you can :mod: link to
it), but I can't find any actual documentation about this actually
being the case. This is the relevant ticket
http://projects.scipy.org/numpy/ticket/1432

Skipper
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Ralf Gommers]

On Tue, Jun 21, 2011 at 3:55 AM, Bruce Southey  wrote:

> On Mon, Jun 20, 2011 at 2:43 PM, Ralf Gommers
>  wrote:
> >
> >
> > On Mon, Jun 20, 2011 at 8:50 PM, Bruce Southey 
> wrote:
> >>
> >> I copied the files but that just moves the problem. So that patch is
> >> incorrect.
> >>
> >> I get the same errors on Fedora 15 supplied Python3.2 for numpy 1.6.0
> and
> >> using git from 'https://github.com/rgommers/numpy.git'.  Numpy is
> getting
> >> Fedora supplied Atlas (1.5.1 does not).
> >>
> >> It appears that there is a misunderstanding of the PEP because 'SO' and
> >> 'SOABI' do exactly what the PEP says on my systems:
> >
> > It doesn't on OS X. But that's not even the issue. As I explained before,
> > the issue is that get_config_var('SO') is used to determine the extension
> of
> > system libraries (such as liblapack.so) and python-related ones (such as
> > multiarray.cpython-32m.so).  And the current functions don't do
> mindreading.
> >>
> >> >>> from distutils import sysconfig sysconfig.get_config_var('SO')
> >> '.cpython-32m.so'
> >> >>> sysconfig.get_config_var('SOABI')
> >> 'cpython-32m'
> >>
> >> Consequently, the name, 'multiarray.pyd', created within numpy is
> invalid.
> >
> > I removed the line in ctypeslib that was trying this, so I think you are
> not
> > testing my patch.
> >
> > Ralf
> >
> >>
> >> Looking the code, I see this line which makes no sense given that the
> >> second part is true under Linux:
> >>
> >> if (not is_python_ext) and 'SOABI' in
> >> distutils.sysconfig.get_config_vars():
> >>
> >> So I think the 'get_shared_lib_extension' function is wrong and probably
> >> unneeded.
> >>
> >>
> >> Bruce
> >>
>
> Just to show that this is the new version, I added two print
> statements in the 'get_shared_lib_extension' function:
> >>> from numpy.distutils.misc_util import get_shared_lib_extension
> >>> get_shared_lib_extension(True)
> first so_ext .cpython-32mu.so
> returned so_ext .cpython-32mu.so
> '.cpython-32mu.so'
> >>> get_shared_lib_extension(False)
> first so_ext .cpython-32mu.so
> returned so_ext .so
> '.so'
>

This all looks correct. Before you were saying you were still getting
'multiarray.pyd', now your error says 'multiarray.so'. So now you are
testing the right thing. Test test_basic2() in test_ctypeslib was fixed, but
I forgot to fix it in two other places. I updated both my branches on
github, please try again.

>
> The reason for the same location is obvious because all the patch does
> is move  the code to get the extension into that function. So the
> 'get_shared_lib_extension' function returns the extension '.so' to the
> load_library function. However that name is wrong under Linux as it
> has to be 'multiarray.cpython-32mu.so' and hence the error in the same
> location. I did come across this thread
> 'http://bugs.python.org/issue10262' which indicates why Linux is
> different by default.
>
> So what is the actual name of the multiarray shared library with the Mac?
> If it is ' 'multiarray.so' then the correct name is "libname +
> sysconfig.get_config_var('SO')" as I previously indicated.
>
> It is, and yes that's correct. Orthogonal to the actual issue though.

Ralf



>
> Bruce
>
>
>
>
> $ python3
> Python 3.2 (r32:88445, Feb 21 2011, 21:11:06)
> [GCC 4.6.0 20110212 (Red Hat 4.6.0-0.7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy as np
> >>> np.test()
> Running unit tests for numpy
> NumPy version 2.0.0.dev-Unknown
> NumPy is installed in /usr/lib64/python3.2/site-packages/numpy
> Python version 3.2 (r32:88445, Feb 21 2011, 21:11:06) [GCC 4.6.0
> 20110212 (Red Hat 4.6.0-0.7)]
> nose version 1.0.0
> first so_ext .cpython-32mu.so
> returned so_ext .so
>
> ...F...S..F.E..KK..K..K...first
> so_ext .cpython-32mu.so
> returned so_ext .so
>
>