Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-29 Thread Sebastian Berg 
On Di, 2014-10-28 at 20:37 -0400, Alexander Belopolsky wrote:
> 
> On Tue, Oct 28, 2014 at 1:42 PM, Stephan Hoyer 
> wrote:
> .flat lets you iterate over all elements of a N-dimensional
> array as if it was 1D, without ever needing to make a copy of
> the array. In contrast, ravel() and reshape(-1) cannot always
> avoid a copy, because they need to return another ndarray.
> 
> 
> In some cases ravel() returns a copy where a view can be easily
> constructed.  For example,
> 

Yeah, but we just changed that for 1.10, not ravel can even get further
if you use order='K'.

- Sebastian

> 
> >>> x = np.arange(10)
> >>> y = x[::2]
> >>> y.ravel().flags['OWNDATA']
> True
> 
> 
> Interestingly, in the same case reshape(-1) returns a view:
> 
> 
> >>> y.reshape(-1).flags['OWNDATA']
> False
> 
> 
> (This suggests at least a documentation bug - numpy.ravel
> documentation says that it is equivalent to reshape(-1).)
> 
> 
> It is only in situations like this
> 
> 
> >>> a = np.arange(16).reshape((4,4))
> >>> a[1::2,1::2].ravel()
> array([ 5,  7, 13, 15])
> 
> 
> where flat view cannot be an ndarray, but .flat can still return
> something that is at least duck-typing compatible with ndarray (if not
> an ndarray subclass) and behaves as a view into original data.
> 
> 
> My preferred design would be for x.flat to return a flat view into x.
> This would be consistent with the way .T and .real attributes are
> defined and close enough to .imag.  An obvious way to obtain a flat
> copy would be x.flat.copy().  Once we have this, ravel() and flatten()
> can be deprecated and reshape(-1) discouraged.
> 
> 
> I think this would be backward compatible except for rather
> questionable situations like this:
> 
> 
> >>> i = x.flat
> >>> list(i)
> [0, 1, 2, 3, 4, 0, 6, 7, 8, 9]
> >>> list(i)
> []
> >>> np.array(i)
> array([0, 1, 2, 3, 4, 0, 6, 7, 8, 9])  
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-29 Thread Sebastian Berg 
On Di, 2014-10-28 at 14:03 -0400, Alan G Isaac wrote:
> On 10/28/2014 1:42 PM, Stephan Hoyer wrote:
> > np.nditer is a reasonable alternative to .flat (and it's documented as 
> > such), but it's a rather inelegant, kitchen-sink type function.
> 
> 
> I'm not sure what "reasonable" means here,
> other than "in principle, possible to use".
> 
> In particular, `flat` is much more elegant,
> and includes an automatic guarantee that the
> iterations will be in C-contiguous style.
> 

I don't really like flat (it is a pretty old part of numpy), but I
agree, while you can force nditer to be C-contiguous, nditer has its own
problems and is also pretty complex. I would argue that nditer it is a
"you know what you are doing" type of function... Or of course if you
want to understand the C-Api.
Unless you keep in mind how buffers are copied around inside it, using
half its features is dangerous.

- Sebastian

> Alan
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Andrew Nelson 
Dear list,
I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I wish to perform
matrix multiplication of the 'NY' 2x2 matrices, resulting in the matrix B.
B would have shape (NX, 2, 2).  I believe that np.einsum would be up to the
task, but I'm not quite sure of the subscripts I would need to achieve this.

Can anyone help, please?

cheers,
Andrew.

-- 
_
Dr. Andrew Nelson


_
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Eelco Hoogendoorn 
My point isn't about speed; its about the scope of numpy. typing np.fft.fft
isn't more or less convenient than using some other symbol from the
scientific python stack.

Numerical algorithms should be part of the stack, for sure; but should they
be part of numpy? I think its cleaner to have them in a separate package.
Id rather have us discuss how to facilitate the integration of as many
possible fft libraries with numpy behind a maximally uniform interface,
rather than having us debate which fft library is 'best'.

On Tue, Oct 28, 2014 at 6:21 PM, Sturla Molden 
wrote:

> Eelco Hoogendoorn  wrote:
>
> > Perhaps the 'batteries included' philosophy made sense in the early days
> of
> > numpy; but given that there are several fft libraries with their own pros
> > and cons, and that most numpy projects will use none of them at all, why
> > should numpy bundle any of them?
>
> Because sometimes we just need to compute a DFT, just like we sometimes
> need to compute a sine or an exponential. It does that job perfectly well.
> It is not always about speed. Just typing np.fft.fft(x) is convinient.
>
> Sturla
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Dave Hirschfeld 
Andrew Nelson writes:

> 
> Dear list,I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I 
wish to perform matrix multiplication of the 'NY' 2x2 matrices, resulting 
in the matrix B.  B would have shape (NX, 2, 2).  I believe that np.einsum 
would be up to the task, but I'm not quite sure of the subscripts I would 
need to achieve this.
> 
> Can anyone help, please?
> 
> cheers,
> Andrew.

Sorry, I'm a little unclear on what is supposed to be multiplied with 
what. You've got (NX x NY) 2x2 matricies, how do you end up with NX 2x2 
matricies?

Perhaps if you show the code using loops with `np.dot` it will be clearer 
how to translate to using `np.einsum`

-Dave




___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Eelco Hoogendoorn 
You need to specify your input format. Also, if your output matrix misses
the NY dimension, that implies you wish to contract (sum) over it, which
contradicts your statement that the 2x2 subblocks form the matrices to
multiply with. In general, I think it would help if you give a little more
background on your problem.

On Wed, Oct 29, 2014 at 10:39 AM, Andrew Nelson  wrote:

> Dear list,
> I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I wish to
> perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the
> matrix B.  B would have shape (NX, 2, 2).  I believe that np.einsum would
> be up to the task, but I'm not quite sure of the subscripts I would need to
> achieve this.
>
> Can anyone help, please?
>
> cheers,
> Andrew.
>
> --
> _
> Dr. Andrew Nelson
>
>
> _
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Daπid 
On 29 October 2014 10:48, Eelco Hoogendoorn  wrote:
> My point isn't about speed; its about the scope of numpy. typing np.fft.fft
> isn't more or less convenient than using some other symbol from the
> scientific python stack.

The problem is in distribution. For many users, installing a new
library is not easy (computing cluster, company regulations...). And
this assuming the alternative library is held to the same quality
standards as Numpy.

Another argument is that this should only be living in Scipy, that is,
after all, quite standard; but it requires a FORTRAN compiler, that is
quite a big dependency.

/David.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread David Cournapeau 
On Wed, Oct 29, 2014 at 9:48 AM, Eelco Hoogendoorn <
hoogendoorn.ee...@gmail.com> wrote:

> My point isn't about speed; its about the scope of numpy. typing
> np.fft.fft isn't more or less convenient than using some other symbol from
> the scientific python stack.
>
> Numerical algorithms should be part of the stack, for sure; but should
> they be part of numpy? I think its cleaner to have them in a separate
> package. Id rather have us discuss how to facilitate the integration of as
> many possible fft libraries with numpy behind a maximally uniform
> interface, rather than having us debate which fft library is 'best'.
>

I would agree if it were not already there, but removing it (like
Blas/Lapack) is out of the question for backward compatibility reason. Too
much code depends on it.

David


>
> On Tue, Oct 28, 2014 at 6:21 PM, Sturla Molden 
> wrote:
>
>> Eelco Hoogendoorn  wrote:
>>
>> > Perhaps the 'batteries included' philosophy made sense in the early
>> days of
>> > numpy; but given that there are several fft libraries with their own
>> pros
>> > and cons, and that most numpy projects will use none of them at all, why
>> > should numpy bundle any of them?
>>
>> Because sometimes we just need to compute a DFT, just like we sometimes
>> need to compute a sine or an exponential. It does that job perfectly well.
>> It is not always about speed. Just typing np.fft.fft(x) is convinient.
>>
>> Sturla
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecate pkgload, PackageLoader

2014-10-29 Thread Robert Kern 
On Wed, Oct 29, 2014 at 4:30 AM, Charles R Harris
 wrote:
> Hi All,
>
> It is proposed to deprecate, then remove, pkgload and PackageLoader.
>
> Complaints? Cries of Anguish?

Tears of joy.

-- 
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Sebastian Berg 
On Mi, 2014-10-29 at 20:39 +1100, Andrew Nelson wrote:
> Dear list,
> I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I wish to
> perform matrix multiplication of the 'NY' 2x2 matrices, resulting in
> the matrix B.  B would have shape (NX, 2, 2).  I believe that
> np.einsum would be up to the task, but I'm not quite sure of the
> subscripts I would need to achieve this.
> 

Just remember that you sum over the columns of the first matrix and the
rows of the second so those share the index:

np.einsum('...ix, ...xj->...', a, b)

in the future, the np.dot predecessor (whatever it is) or the @ operator
should be better at it though.

- Sebastian

> Can anyone help, please?
> 
> 
> cheers,
> Andrew.
> 
> 
> -- 
> _
> Dr. Andrew Nelson
> 
> 
> _
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Sebastian Berg 
On Mi, 2014-10-29 at 13:05 +0100, Sebastian Berg wrote:
> On Mi, 2014-10-29 at 20:39 +1100, Andrew Nelson wrote:
> > Dear list,
> > I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I wish to
> > perform matrix multiplication of the 'NY' 2x2 matrices, resulting in
> > the matrix B.  B would have shape (NX, 2, 2).  I believe that
> > np.einsum would be up to the task, but I'm not quite sure of the
> > subscripts I would need to achieve this.
> > 
> 
> Just remember that you sum over the columns of the first matrix and the
> rows of the second so those share the index:
> 
> np.einsum('...ix, ...xj->...', a, b)
> 

Nevermind, didn't read carefully. This is not possible since the
reduction operation is a sum, you will have to do at least two
operations.

> in the future, the np.dot predecessor (whatever it is) or the @ operator
> should be better at it though.
> 
> - Sebastian
> 
> > Can anyone help, please?
> > 
> > 
> > cheers,
> > Andrew.
> > 
> > 
> > -- 
> > _
> > Dr. Andrew Nelson
> > 
> > 
> > _
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecate pkgload, PackageLoader

2014-10-29 Thread Benjamin Root 
/me looks at pydoc numpy.pkgload
What in the world?!

On Wed, Oct 29, 2014 at 7:12 AM, Robert Kern  wrote:

> On Wed, Oct 29, 2014 at 4:30 AM, Charles R Harris
>  wrote:
> > Hi All,
> >
> > It is proposed to deprecate, then remove, pkgload and PackageLoader.
> >
> > Complaints? Cries of Anguish?
>
> Tears of joy.
>
> --
> Robert Kern
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Generalize hstack/vstack --> stack; Block matrices like in matlab

2014-10-29 Thread Stefan Otte 
Hey,

there are several ways how to proceed.

- My proposed solution covers the 80% case quite well (at least I use
it all the time). I'd convert the doctests into unittests and we're
done.

- We could slightly change the interface to leave out the surrounding
square brackets, i.e. turning `stack([[a, b], [c, d]])` into
`stack([a, b], [c, d])`

- We could extend it even further allowing a "filler value" for non
set values and a "shape" argument. This could be done later as well.

- `bmat` is not really matrix specific. We could refactor `bmat` a bit
to use the same logic in `stack`. Except the `matrix` calls `bmat` and
`_from_string` are pretty agnostic to the input.

I'm in favor of the first or last approach. The first: because it
already works and is quite simple. The last: because the logic and
tests of both `bmat` and `stack` would be the same and the feature to
specify a string representation of the block matrix is nice.


Best,
 Stefan



On Tue, Oct 28, 2014 at 7:46 PM, Nathaniel Smith  wrote:
> On 28 Oct 2014 18:34, "Stefan Otte"  wrote:
>>
>> Hey,
>>
>> In the last weeks I tested `np.asarray(np.bmat())` as `stack`
>> function and it works quite well. So the question persits:  If `bmat`
>> already offers something like `stack` should we even bother
>> implementing `stack`? More code leads to more
>> bugs and maintenance work. (However, the current implementation is
>> only 5 lines and by using `bmat` which would reduce that even more.)
>
> In the long run we're trying to reduce usage of np.matrix and ideally
> deprecate it entirely. So yes, providing ndarray equivalents of matrix
> functionality (like bmat) is valuable.
>
> -n
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Pierre-Andre Noel 
>> Id rather have us discuss how to facilitate the integration of as 
many possible fft libraries with numpy behind a maximally uniform 
interface, rather than having us debate which fft library is 'best'.


I agree with the above.

> I would agree if it were not already there, but removing it (like 
Blas/Lapack) is out of the question for backward compatibility reason. 
Too much code depends on it.


And I definitely agree with that too.

I think that numpy.fft should be left there in its current state 
(although perhaps as deprecated). Now scipy.fft should have a good 
generic algorithm as default, and easily allow for different 
implementations to be accessed through the same interface.


Pierre-André

On 10/29/2014 03:33 AM, David Cournapeau wrote:



On Wed, Oct 29, 2014 at 9:48 AM, Eelco Hoogendoorn 
mailto:hoogendoorn.ee...@gmail.com>> wrote:


My point isn't about speed; its about the scope of numpy. typing
np.fft.fft isn't more or less convenient than using some other
symbol from the scientific python stack.

Numerical algorithms should be part of the stack, for sure; but
should they be part of numpy? I think its cleaner to have them in
a separate package. Id rather have us discuss how to facilitate
the integration of as many possible fft libraries with numpy
behind a maximally uniform interface, rather than having us debate
which fft library is 'best'.


I would agree if it were not already there, but removing it (like 
Blas/Lapack) is out of the question for backward compatibility reason. 
Too much code depends on it.


David


On Tue, Oct 28, 2014 at 6:21 PM, Sturla Molden
mailto:sturla.mol...@gmail.com>> wrote:

Eelco Hoogendoorn mailto:hoogendoorn.ee...@gmail.com>> wrote:

> Perhaps the 'batteries included' philosophy made sense in
the early days of
> numpy; but given that there are several fft libraries with
their own pros
> and cons, and that most numpy projects will use none of them
at all, why
> should numpy bundle any of them?

Because sometimes we just need to compute a DFT, just like we
sometimes
need to compute a sine or an exponential. It does that job
perfectly well.
It is not always about speed. Just typing np.fft.fft(x) is
convinient.

Sturla

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org 
http://mail.scipy.org/mailman/listinfo/numpy-discussion



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org 
http://mail.scipy.org/mailman/listinfo/numpy-discussion




___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-29 Thread Alexander Belopolsky 
On Tue, Oct 28, 2014 at 10:11 PM, Nathaniel Smith  wrote:

> > I don't think so - I think all the heavy lifting is already done in
> flatiter.  The missing parts are mostly trivial things like .size or .shape
> or can be fudged by coercing to true ndarray using existing
> flatiter.__array__ method.
>
> Now try .resize()...
>

Simple:

def resize(self, shape):
if self.shape == shape:
return
else:
raise ValueError


>From ndarray.resize documentation:

Raises
--
ValueError
If `a` does not own its own data or references or views to it exist,
and the data memory must be changed.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-29 Thread Alexander Belopolsky 
On Tue, Oct 28, 2014 at 10:11 PM, Nathaniel Smith  wrote:

> .diagonal has no magic, it just turns out that the diagonal of any strided
> array is also expressible as a strided array. (Specifically, new_strides =
> (sum(old_strides),).)



This is genius!  Once you mentioned this, it is obvious how the new
diagonal() works and one can only wonder why it took over 20 years to get
this feature in NumPy.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?

2014-10-29 Thread Stephan Hoyer 
On Wed, Oct 29, 2014 at 2:16 AM, Sebastian Berg 
wrote:

> On Di, 2014-10-28 at 14:03 -0400, Alan G Isaac wrote:
> I don't really like flat (it is a pretty old part of numpy), but I
> agree, while you can force nditer to be C-contiguous, nditer has its own
> problems and is also pretty complex. I would argue that nditer it is a
> "you know what you are doing" type of function... Or of course if you
> want to understand the C-Api.
> Unless you keep in mind how buffers are copied around inside it, using
> half its features is dangerous.
>

One other subtle way in which nditer and falt differ is that nditer returns
0-dimensional arrays, not scalars.

It turns out there is actually no way in numpy to iterate in the fastest
possible order ('K') over an N-dimensional array in Python without the
overhead of creating 0-dimensional arrays. This came up recently an as
issue for numba:
https://github.com/numba/numba/issues/841

So although this discussion has mostly been about consolidating options,
this is at least one feature that we might like to add. Honestly, this is
likely only to be an issue for projects like Numba that take valid
python/numpy code and compile it. If you're writing C/Cython, you can just
use the NumPy C API.

Cheers,
Stephan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecate pkgload, PackageLoader

2014-10-29 Thread Julian Taylor 
On 29.10.2014 05:30, Charles R Harris wrote:
> Hi All,
> 
> It is proposed to deprecate, then remove, pkgload and PackageLoader.
> 
> Complaints? Cries of Anguish?
> 

I don't mind the deprecation, but I have to ask why? is it causing issues?
it does look like something some people use in their workflows.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Alexander Eberspächer 
On 29.10.2014 18:03, Pierre-Andre Noel wrote:
>>> Id rather have us discuss how to facilitate the integration of as
> many possible fft libraries with numpy behind a maximally uniform
> interface, rather than having us debate which fft library is 'best'.
> 
> I agree with the above.

Absolutely. I think the NumPy/Scipy interfaces are easy and convenient
enough to serve as a front-end to different FFT codes.

>> I would agree if it were not already there, but removing it (like
> Blas/Lapack) is out of the question for backward compatibility reason.
> Too much code depends on it.
> 
> And I definitely agree with that too.
> 
> I think that numpy.fft should be left there in its current state
> (although perhaps as deprecated). Now scipy.fft should have a good
> generic algorithm as default, and easily allow for different
> implementations to be accessed through the same interface.

Definitely. My attempt at streamlining the use of pyfftw even further
can be found here:

https://github.com/aeberspaecher/transparent_pyfftw

This small package does nothing more than to automatically load fftw
wisdom on export and add a keyword that gives the number of threads to
use to NumPy/Scipy style FFT calls. I think similar attempts could be
made with other FFT libraries. The mission statement would be to map
each library's interface to the simple and convenient SciPy/NumPy-style
interface, and also to wisely choose default parameters (such as e.g.
pyfftw's planner_effort).

Also, I think it's obvious that a generic and easy-to-use implementation
cannot deliver exactly the same performance as hand-tuned code, but
anyway I see plenty room for improvement for SciPy/NumPy's FFT modules.

Alex
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecate pkgload, PackageLoader

2014-10-29 Thread Robert Kern 
On Wed, Oct 29, 2014 at 6:05 PM, Julian Taylor
 wrote:
> On 29.10.2014 05:30, Charles R Harris wrote:
>> Hi All,
>>
>> It is proposed to deprecate, then remove, pkgload and PackageLoader.
>>
>> Complaints? Cries of Anguish?
>
> I don't mind the deprecation, but I have to ask why? is it causing issues?
> it does look like something some people use in their workflows.

Why do you think so? It's old code that was used in scipy/__init__.py,
but we removed it from there years ago. It has been unofficially
deprecated (and widely considered a bad idea) for years. The only
modifications to it in the past 6 years or so have been repo-wide
cleanups, Python 3 compatibility and the like, so it *is* adding to
the maintenance burden for those tasks. Do you know of anyone that is
actually currently using it?

-- 
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Henry Gomersall 
On 29/10/14 18:23, Alexander Eberspächer wrote:
> Definitely. My attempt at streamlining the use of pyfftw even further
> can be found here:
>
> https://github.com/aeberspaecher/transparent_pyfftw

There could be an argument that this sort of capability should be added 
to the pyfftw package, as a package level config.

Something like:

import pyfftw
pyfftw.default_threads = 4

import pyfftw.interfaces.numpy_fft as fft

The wisdom code can be added at the package import level too, but that 
doesn't need anything more.

Cheers,

Henry
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Alexander Eberspächer 
On 29.10.2014 19:40, Henry Gomersall wrote:

> There could be an argument that this sort of capability should be added 
> to the pyfftw package, as a package level config.
> 
> Something like:
> 
> import pyfftw
> pyfftw.default_threads = 4

I think that would be great, though probably slightly off-topic here.

> import pyfftw.interfaces.numpy_fft as fft
> 
> The wisdom code can be added at the package import level too, but that 
> doesn't need anything more.

If NumPy/SciPy provided interfaces to different FFT implementations,
implementation specific routines (e.g. wisdom load/save or creation of
byte-aligned empty arrays in the pyfftw case) could be made available
through a subpackage, e.g. np.fft.implementation_specific. That
subpackage then exposed routines specific to the implementation that
lives below the simple interfaces.

For implementation-specific configuration, perhaps a user-level
configuration file or set of environment variables could be read on
import of the specific implementation.

At the very heart of allowing NumPy to use different FFT implementations
could be a definition of an intermediate layer, much like LAPACK is for
linear algebra. This probably would have to happen at the C-level. I'm
only wildly speculating here as I don't have enough experience with
interfaces to different FFT libraries, so I don't know whether the
individual interfaces are close enough to be able to define a suitable
"common interface".

Alex
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Andrew Nelson 
On Wed, Oct 29, 2014 at 10:39 AM, Andrew Nelson  wrote:

> Dear list,
> I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I wish to
> perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the
> matrix B.  B would have shape (NX, 2, 2).  I believe that np.einsum would
> be up to the task, but I'm not quite sure of the subscripts I would need
to
> achieve this.

Ok, I'll try to explain in more detail of what I'm trying to do (I'm not
skilled in matrix algebra).

Say I have a series of matrices, M, which are all 2x2: M_0, M_1, ..., M_{NY
- 1}. These all need to be multiplied by each other.
i.e. N = M_0 x M_1 x  ... x M_{NY - 1}.
Note that I want to multiply M_0 by M_1, the result of that by M_2, the
result of that by M_3 and so on.
I can hold the (NY) matrices in a single array that has shape (NY, 2, 2).
The first row in that array would be M_0, the last would be M_{NY-1}.  The
output of all that matrix multiplication would be a single 2x2 matrix.  So
I would've thought an operation would do something like this:

 #there are NY-1 matrix multiplications involved here.
M[NY, 2, 2] ->  N[2, 2]

Now let's make the next level of complication, I have NX of those M[NY, 2,
2] matrices. So I need to do the above matrix multiplication series NX
times.  I could hold all this in an array, P, with shape (NX, NY, 2, 2).
Each of the NX rows are independent.
Currently I am doing this in a nested loop, pseudocode follows:


output = np.zeros((NX, 2, 2))

for i in range(NX):
temp = np.identity(2)
for j in range(NY):
temp = np.dot(temp, P[i, j])

output[i] = temp


My original question was posted as I would like to remove that doubly
nested loop, with something more elegant, as well as a whole load faster.
Is there an np.einsum that can furnish that?
(Please forgive me if this still isn't clear or precise enough).

-- 
_
Dr. Andrew Nelson


_
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication

2014-10-29 Thread Alexander Belopolsky 
On Wed, Oct 29, 2014 at 5:39 AM, Andrew Nelson  wrote:

> I have a 4D array, A, that has the shape (NX, NY, 2, 2).  I wish to
> perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the
> matrix B.  B would have shape (NX, 2, 2).


What you are looking for is dot.reduce and NumPy does not implement that.
You can save an explicit loop by doing reduce(np.dot, matrices).  For
example


In [6] A
Out[6]
array([[[ 1.,  0.],
[ 0.,  1.]],

   [[ 2.,  0.],
[ 0.,  2.]],

   [[ 3.,  0.],
[ 0.,  3.]]])

In [7] reduce(np.dot, A)
Out[7]
array([[ 6.,  0.],
   [ 0.,  6.]])
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] [ANN - JOB] Assistant Researcher - Berkeley Institute for Data Science

2014-10-29 Thread Fernando Perez 
Hi all,

the newly founded Berkeley Institute for Data Science is hiring researchers
with a focus on open source tools for scientific computing, please see here
for details:

https://aprecruit.berkeley.edu/apply/JPF00590

Cheers,

f
-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Sturla Molden 
On 29/10/14 10:48, Eelco Hoogendoorn wrote:

 > Id rather have us discuss how to facilitate the integration of
 > as many possible fft libraries with numpy behind a maximally uniform
 > interface, rather than having us debate which fft library is 'best'.

I am happy with the NumPy interface. There are minor differences between 
the SciPy and NumPy FFT interfaces (e.g. for rfft, see below). 
Personally I prefer the NumPy interface because it makes it easier to 
map Fourier coeffs to frequencies.

One thing we could do, without too much hassle, is to use FFTs from MKL 
or Accelerate (vDSP) if we link with these libararies for BLAS/LAPACK.

MKL has an API compatible with FFTW, so FFTW and MKL can be supported 
with the same C code. FFTW and MKL also have a Fortran 77 API which we 
could wrap with f2py (no Fortran compiler are needed). It is actually 
possible to use the FFTs in FFTW and MKL from Python without any C 
coding at all. We just need to add a Python interface on top of the f2py 
wrappers, which is similar to what we do for scipy.linalg.

The FFTs in Accelerate have a very simple C interface, but only support 
power-of-two array sizes, so we would need to use them with Bluestein's 
algorithm. Again, because of their simplicity, it is possible to wrap 
these FFT functions with f2py.

We cannot bundle NumPy or SciPy binaries with FFTW due to GPL [*], but 
as I understand it we already have permission from Intel to bundle 
binary wheels linked with MKL. Accelerate is a system library, so that 
does not pose a license problem.

[*] Actually, we could, but the binaries would be tainted with a viral 
license.



 >>> a = np.random.rand(8)

 >>> scipy.fftpack.rfft(a)[:,None]

array([[ 3.47756851],
[-0.45869926],
[-0.21730867],
[-0.43763425],
[-0.67338213],
[-0.28799   ],
[ 0.17321793],
[-0.31514119]])

 >>> np.fft.rfft(a)[:,None]

array([[ 3.47756851+0.j],
[-0.45869926-0.21730867j],
[-0.43763425-0.67338213j],
[-0.28799000+0.17321793j],
[-0.31514119+0.j]])





Sturla

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FFTS for numpy's FFTs (was: Re: Choosing between NumPy and SciPy functions)

2014-10-29 Thread Nathaniel Smith 
On 30 Oct 2014 03:58, "Sturla Molden"  wrote:
[...]
> We cannot bundle NumPy or SciPy binaries with FFTW due to GPL [*], but
> as I understand it we already have permission from Intel to bundle
> binary wheels linked with MKL. Accelerate is a system library, so that
> does not pose a license problem.
>
> [*] Actually, we could, but the binaries would be tainted with a viral
> license.

And binaries linked with MKL are tainted by a proprietary license... They
have very similar effects, in both cases you can use the binary just fine
for whatever you want, but if you redistribute it as part of a larger work,
then you must (follow the terms of the GPL/follow the terms of Intel's
license).

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion