[Numpy-discussion] Has IPython been useful to you? Please let me know...

[Fernando Perez]

Hi all,

[ apologies for the semi-spam, I'll keep this brief and expect all
replies off-list ]

IPython is a project that many of you on this list are likely to use
in your daily work, either directly or indirectly (if you've embedded
it or used it as a component of some other system).  I would simply
like to ask you, if IPython has been significantly useful for a
project you use, lead, develop, etc., to let me know.

For legal/professional reasons, I need to gather information about who
has found IPython to be of value.  I started IPython as a toy
'afternoon hack' in late 2001, and today it continues to grow, as the
nicely summarized Ohloh stats show:  https://www.ohloh.net/p/ipython
(obviously, this is now the result of the work of many, not just
myself, as is true of any healthy open source project as it grows).
But I have never systematically tracked its impact, and now I need to
do so.

So, if you have used IPython and it has made a significant
contribution to your project, work, research, company, whatever, I'd
be very grateful if you let me know.  A short paragraph on what this
benefit has been is all I ask.  Once I gather any information I get, I
would contact directly some of the responders to ask for your
authorization before quoting you.

I should stress that any information you give me will only go in a
documentation packet in support of my legal/residency process here in
the USA (think of it as an oversized, obnoxiously detailed CV that
goes beyond just publications and regular academic information).

To keep traffic off this list, please send your replies directly to
me, either at this address or my regular work one:

fernando.pe...@berkeley.edu

In advance, many thanks to anyone willing to reply.  I've never asked
for anything in return for working on IPython and the ecosystem of
scientific Python tools, but this is actually very important, so any
information you can provide me will be very useful.

Best regards,

Fernando Perez.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Superfluous array transpose (cf. ticket #1054)

[Pauli Virtanen]

Sun, 15 Mar 2009 19:57:10 +0100, Sturla Molden wrote:

> Regarding ticket #1054. What is the reason for this strange behaviour?
> 
 a = np.zeros((10,10),order='F')
 a.flags
>   C_CONTIGUOUS : False
>   F_CONTIGUOUS : True
>   OWNDATA : True
>   WRITEABLE : True
>   ALIGNED : True
>   UPDATEIFCOPY : False
 (a+1).flags
>   C_CONTIGUOUS : True
>   F_CONTIGUOUS : False
>   OWNDATA : True
>   WRITEABLE : True
>   ALIGNED : True
>   UPDATEIFCOPY : False

New numpy arrays are by default in C-order, I believe.

-- 
Pauli Virtanen

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Py_ssize_t

[David Cournapeau]

On Mon, Mar 16, 2009 at 4:10 AM, Charles R Harris
 wrote:
> Some of the calls to the python c-api have been changed to use Py_ssize_t.
> As Py_ssize_t was not available in Python 2.4 I wonder if we check if it is
> defined and set it to int if not.

Yes, we do, in ndarrayobject.h

> Also, are we running tests on Python 2.4
> for the release?

At least I do - I have access to a RHEL machine, where the default
python is 2.4. Since it is 64 bits as well, it is pretty good to
detect those kind of problems.

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Py_ssize_t

[Charles R Harris]

Some of the calls to the python c-api have been changed to use Py_ssize_t.
As Py_ssize_t was not available in Python 2.4 I wonder if we check if it is
defined and set it to int if not. Also, are we running tests on Python 2.4
for the release?

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Superfluous array transpose (cf. ticket #1054)

[Sturla Molden]

Regarding ticket #1054. What is the reason for this strange behaviour?

>>> a = np.zeros((10,10),order='F')
>>> a.flags
  C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  UPDATEIFCOPY : False
>>> (a+1).flags
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  UPDATEIFCOPY : False


Sturla Molden

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[Sturla Molden]

> Well, that's nearly as good. (Though submitting a single svn diff
> containing all changes could have been a bit more easy to handle than
> separate patches for each file. But a small nitpick only.)

The problem is I am really bad at using these tools. I have TortoiseSVN
installed, but no idea how to use it. :(

I cannot delete any file attachment in trac, but I can overwrite the files
I've posted.

S.M.



___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[Pauli Virtanen]

Sun, 15 Mar 2009 17:48:51 +0100, Sturla Molden wrote:

>> Mon, 16 Mar 2009 00:33:28 +0900, David Cournapeau wrote:
>>
>> Also, you could post the patch on the http://codereview.appspot.com
>> site. Then it would be easier to both review and to keep track of its
>> revisions
> 
> I have posted the files here:
> 
> http://projects.scipy.org/numpy/ticket/1055

Well, that's nearly as good. (Though submitting a single svn diff 
containing all changes could have been a bit more easy to handle than 
separate patches for each file. But a small nitpick only.)

But I wonder if there is a way to improve the behavior of Trac with 
attachments/patches, there seem to be currently some warts:

- Can a non-admin user delete or mark some attachments obsolete?

- Trac doesn't want to show all patches in HTML, and doesn't recognize
  the .diff suffix. Maybe this can be fixed.

- Inline comments in patches would be nice, as in codereview.

-- 
Pauli Virtanen

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[Sturla Molden]

> Mon, 16 Mar 2009 00:33:28 +0900, David Cournapeau wrote:
>
> Also, you could post the patch on the http://codereview.appspot.com site.
> Then it would be easier to both review and to keep track of its
> revisions

I have posted the files here:

http://projects.scipy.org/numpy/ticket/1055


Sturla

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[Sturla Molden]

> Would it be possible to make the changes as a patch (svn diff) - this
> makes things easier to review.

I've added diff files to ticket #1055.


> Yes, I would be more comfortable without them (for 1.3). This is
> typically the kind of small changes which can be a PITA to deal with
> just before a release because it breaks some platforms in non obvious
> ways.

Ok, they are commented out.


> For the restrict keyword support, I will add a distutils check to avoid
> the compiler-specifics (again after 1.3).

I've added a header file npy_restrict.h that defines a NPY_RESTRICT symbol.

Best regards,

Sturla







___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[Pauli Virtanen]

Mon, 16 Mar 2009 00:33:28 +0900, David Cournapeau wrote:

> Sturla Molden wrote:
>>
>> There is a version of fftpack_litemodule.c, fftpack.c and fftpack.h
>> that does this attached to ticket #1055. The two important changes are
>> releasing the GIL and using npy_intp for 64 bit support.
>>   
>>   
> Would it be possible to make the changes as a patch (svn diff) - this
> makes things easier to review.

Also, you could post the patch on the http://codereview.appspot.com site. 
Then it would be easier to both review and to keep track of its 
revisions. (Attachments, especially whole-file ones sent on the mailing 
list are IMHO significantly more cumbersome for all parties concerned.)

In practice the code review tool is also easier to use than sending SVN 
diffs to the mailing list. If you are working on a SVN checkout, I 
recommend using the upload tool

http://codereview.appspot.com/static/upload.py

to upload the patch to the code review site. Just do

python upload.py

on the SVN checkout containing your changes. (You'll need a Google 
account for this, though.) When revising the patch after the initial 
upload, specify the Codereview site issue number to the upload:

python upload.py -i 12345

so that the new version of the patch is marked as an improved version of 
the old one.

-- 
Pauli Virtanen

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[David Cournapeau]

Sturla Molden wrote:
>
> There is a version of fftpack_litemodule.c, fftpack.c and fftpack.h that
> does this attached to ticket #1055. The two important changes are
> releasing the GIL and using npy_intp for 64 bit support.
>   

Would it be possible to make the changes as a patch (svn diff) - this
makes things easier to review.

> Minor changes:
>
> There is a restrict qualifier in fftpack.c. If it is not compiled with
> C99, it tries to use similar GNU or MS extensions.
>
> There is some OpenMP pragmas in ftpack_litemodule.c. If you don't compile
> with OpenMP support, they do nothing. If you do compile with OpenMP, they
> will make certain FFTs run in parallel.  I can comment them out if you
> prefer.
>   

Yes, I would be more comfortable without them (for 1.3). This is
typically the kind of small changes which can be a PITA to deal with
just before a release because it breaks some platforms in non obvious ways.

For the restrict keyword support, I will add a distutils check to avoid
the compiler-specifics (again after 1.3).

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[Sturla Molden]

>> 1) I have noticed that fftpack_litemodule.c does not release the GIL
>> around calls to functions in fftpack.c. I cannot se any obvious reason
>> for
>> this. As far as I can tell, the functions in fftpack.c are re-entrant.
>>
>> 2) If fftpack_lite did release the GIL, it would allow functions in
>> numpy.fft to use multithreading for multiple FFTs in parallel
>> (threading.Thread are ok, not special compilation needed).
>
> Both are fines to modify for 1.3.

There is a version of fftpack_litemodule.c, fftpack.c and fftpack.h that
does this attached to ticket #1055. The two important changes are
releasing the GIL and using npy_intp for 64 bit support.

Minor changes:

There is a restrict qualifier in fftpack.c. If it is not compiled with
C99, it tries to use similar GNU or MS extensions.

There is some OpenMP pragmas in ftpack_litemodule.c. If you don't compile
with OpenMP support, they do nothing. If you do compile with OpenMP, they
will make certain FFTs run in parallel.  I can comment them out if you
prefer.

Sturla Molden




___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] suggestion for generalizing numpy functions

[Darren Dale]

Hi Travis,

On Mon, Mar 9, 2009 at 6:08 PM, Travis E. Oliphant
wrote:

> Darren Dale wrote:
> > On Mon, Mar 9, 2009 at 9:50 AM, Darren Dale  > > wrote:
> >
> > I spent some time over the weekend fixing a few bugs in numpy that
> > were exposed when attempting to use ufuncs with ndarray
> > subclasses. It got me thinking that, with relatively little work,
> > numpy's functions could be made to be more general. For example,
> > the numpy.ma  module redefines many of the
> > standard ufuncs in order to do some preprocessing before the
> > builtin ufunc is called. Likewise, in the units/quantities package
> > I have been working on, I would like to perform a dimensional
> > analysis to make sure an operation is allowed before I call a
> > ufunc that might change data in place.
> >
>
> The suggestions behind this idea are interesting.   It seems related to
> me, to the concept of "contexts" that Eric presented at SciPy a couple
> of years ago that keeps coming up at Enthought.It may be of benefit
> to solve the problem from that perspective rather than the "sub-class"
> perspective.
>
> Unfortunately, I don't have time to engage this discussion as it
> deserves, but I wanted to encourage you because I think there are good
> ideas in what you are doing.The sub-class route may be a decent
> solution, but it also might be worthwhile to think from the perspective
> of contexts as well.
>
> Basically, the context idea is that rather than "sub-class" the ndarray,
> you create a more powerful name-space for code that uses arrays to live
> in.   Because python code can execute using a namespace that is any
> dictionary-like thing, you can create a "namespace" object with more
> powerful getters and setters that intercepts the getting and setting of
> names as the Python code is executing.
>
> This allows every variable to be "adapted" in a manner analagous to
> "type-maps" in SWIG --- but in a more powerful way.We have been
> taking advantage of this basic but powerful idea quite a bit.
> Unit-handling is a case where "contexts" and generic functions rather
> than sub-classes appears to be an approach  to solving the problem.
>
> The other important idea about contexts is that you can layer-on
> adapters on getting and setting variables into the namespace which
> provide more hooks for doing some powerful things in easy-to-remember
> ways.
>
> I apologize if it sounds like I'm hi-jacking your question to promote an
> agenda.   I really like the generality you are trying to reach with your
> suggestions and just wanted to voice the opinion that it might be better
> to look for a solution using the two dimensions of "objects" and
> "namespaces" (o.k.  generic functions are probably another dimension in
> my metaphor) rather than just sub-classes of objects.
>

Contexts may be an alternative approach, but I do not understand the vision
or how they would be applied to the problem of unit handling.

The Quantities package is already in a useful and working state, based on an
ndarray subclass. My goal at this point is to make quantities more useful
with numpy/scipy. There is already a mechanism for doing so, it just needs
to be tweaked in order for it to more generally applicable.

Hopefully I can interest some of the current numpy developers to engage in
the discussion after 1.3 is released.

Darren
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[David Cournapeau]

On Sun, Mar 15, 2009 at 12:46 PM, Charles R Harris
 wrote:
>
>
> As it is easier to read and understand without going to the macro
> definition. Note that David has a quite involved check for the inline
> keyword implementation and I expect he would want to do the same for the
> restrict keyword. I think using lots of #if defined(xxx) might be easier

It is easier but less maintainable and less portable. With checks, we
can deal with new platforms more easily. And it is more robust. Things
depending on the platform should be the exception, not the norm.

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Enhancements for NumPy's FFTs

[David Cournapeau]

On Sun, Mar 15, 2009 at 5:16 AM, Sturla Molden  wrote:
>
>
> 1) I have noticed that fftpack_litemodule.c does not release the GIL
> around calls to functions in fftpack.c. I cannot se any obvious reason for
> this. As far as I can tell, the functions in fftpack.c are re-entrant.
>
> 2) If fftpack_lite did release the GIL, it would allow functions in
> numpy.fft to use multithreading for multiple FFTs in parallel
> (threading.Thread are ok, not special compilation needed).

Both are fines to modify for 1.3.

>
> 3) Is there any reason numpy.fft does not have dct? If not, I'd suggest
> addition of numpy.fft.dct and numpy.fft.idct.

numpy.fft is only here for compatibility with older array packages
AFAIK. So we should not add new features. I would much prefer to see
those kind of things in scipy.fftpack (which already have DCT I, II
and III). Adding dct to numpy.fft will only add to the confusion, I
think. There is already enough duplication between numpy and scipy,
let's not add more.

>
> 4) Regarding ticket #400: Cython now makes this easy. NumPy's FFTs should
> be exposed to C extesions without calling back to Python.

Agreed - that's one area where we could much more, but this has to be
done carefully. I am a bit worried of doing this kind of things at the
last minute. 1, 2 are OK for the trunk, but not 4 (because it impacts
public API), IMO.

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion