Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?

[Eli Stevens (Gmail)]

On Thu, Mar 31, 2011 at 11:45 AM, Ralf Gommers
 wrote:
> On Thu, Mar 31, 2011 at 8:37 PM, Eli Stevens (Gmail)
>  wrote:
>> Next up for me is to get a patch onto the CPython issue tracker, but
>> as soon as I have that done, I'll start working on adding unit tests
>> to my numpy fork.  I anticipate being able to get that done today.
>
> Sounds good.

CPython issue here:
http://bugs.python.org/issue11734

My numpy changes so far:
https://github.com/wickedgrey/numpy/compare/23965b6b665985390d7a...maintenance%2F1.6.x

>From my last commit message:
"Note that my changes to _internal were mostly of the "make it work"
variety; I'm not really understanding the wider scope.  Very much
needs review by wiser eyes."

I'd also appreciate some feedback on the unit test coverage; the
test_roundtrip_half function uncovered a problem that the other two
didn't (which surprised me), and I'd like to make sure that there
aren't more gaps in the tests that I've written.

Thanks for all the help!
Eli
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Charles R Harris]

On Thu, Mar 31, 2011 at 8:42 AM, Ralf Gommers
wrote:

> On Thu, Mar 31, 2011 at 4:53 AM, Charles R Harris
>  wrote:
> >
> >
> > On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
> >  wrote:
> >>
>



If you look in Trac under "All Tickets by Milestone" you'll find all
> nine tickets together under 1.6.0. Five are bug fixes, four are
> enhancements. There are some missing tests, but all tickets have
> proposed patches.
>
>
OK. I changed 1562 to enhancement because it adds a keyword. With that
change the current status looks like this.

Bug Fixes:

1163 -- convert int64 correctly
1458 -- make loadtxt unpack structured arrays
1071 -- loadtxt fails if the last column contains empty value, under
discussion
1565 -- duplicate of 1163

Enhancements:

1107 -- support for blocks of data, adds two keywords.
1562 -- add ndmin keyword to aid in getting correct dimensions, doesn't
apply on top of previous.
1616 -- remove use of readline so input isn't restricted to files.
1731 -- allow loadtxt to read given number of rows, adds keyword.
1752 -- return empty array when empty file encountered, conflicts with 1616.


Some of this might should go into genfromtxt. None of the patches have
tests.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.0 beta 1

[Paul Ivanov]

Ralf Gommers, on 2011-03-23 17:07,  wrote:
> Please test this beta and report any problems on the Numpy
> mailing list.

Hi,

I reopened http://projects.scipy.org/numpy/ticket/1768, as
'test_chisquare' still fails for me on a 32 bit machine. Lowering
the precision from 14 to 13 decimals makes the test pass,
however.

Other than that:

NumPy version 1.6.0b1
NumPy is installed in
/home/pi/.local/lib/python2.6/site-packages/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC
4.4.3]
nose version 0.11.1
--snip--
Ran 3399 tests in 23.052s

FAILED (KNOWNFAIL=3, SKIP=4, failures=1)

best,
-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 


signature.asc
Description: Digital signature
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1

[David]

On 03/31/2011 06:37 PM, Pearu Peterson wrote:
>
>
> On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau  > wrote:
>
> On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen  > wrote:
>  > In article
>  >  >,
>  >  Ralf Gommers  > wrote:
>  >
>  >> Hi,
>  >>
>  >> I am pleased to announce the availability of the first beta of NumPy
>  >> 1.6.0. Due to the extensive changes in the Numpy core for this
>  >> release, the beta testing phase will last at least one month. Please
>  >> test this beta and report any problems on the Numpy mailing list.
>  >>
>  >> Sources and binaries can be found at:
>  >> http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/
>  >> For (preliminary) release notes see below.
>
> I see a segfault on Ubuntu 64 bits for the test
> TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py.
> Am I the only one seeing it ?
>
> The test work here ok on Ubuntu 64 with numpy master. Could you try the
> maintenance/1.6.x branch where the related bugs are fixed.

I did test that as well, and got the same issue, but could not reproduce 
it on another machine. I do get the error every time on my main work 
machine, though. I will look more into it, but it is most likely 
something due to my machine,

cheers,

David
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Willing to contribute to SciPy & NumPy ...

[Benjamin Root]

On Thu, Mar 31, 2011 at 3:29 PM, Christopher Barker
wrote:

> On 3/31/11 12:04 PM, Anthony Scopatz wrote:
>
> > (with which I am inclined to agree), documentation is probably not the
> > best way to get started.
>
> I agree as well. However, I think newbies can be a great help in
> identifying holes in the documentation -- so when documentation is
> missing or unclear, ask here, and then you'll have something to contribute.
>
>
+1 on this.  The biggest problem with developers writing documentation for
tools like NumPy is that we have already become accustom to the quirks and
assumptions of the tool.  Fresh minds are perfect for identifying problem
areas in the docs.

The problem I have seen, though, is that it becomes so easy to just answer
the user's question and then just move on.  We really need to better remind
ourselves that each time someone has to ask an "easy" question, it is a
failing with our documentation.

"What if it was already in the docs", you ask?  Then the question is, why
couldn't they find it (or recognize it)?

"What if they didn't read the documentation", you say?  Then the question
becomes how did our documentation fail to be read?

Just food for thought...

Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Willing to contribute to SciPy & NumPy ...

[Christopher Barker]

On 3/31/11 12:04 PM, Anthony Scopatz wrote:
> A good place to get started is helping out with documentation (see
> http://docs.scipy.org/numpy/Front%20Page/).
>
>
> Actually, according to the Jacob Kaplan-Moss's talk at PyCon this year
> (http://pycon.blip.tv/file/4881071/)

Nice talk -- thanks for the link.

> (with which I am inclined to agree), documentation is probably not the
> best way to get started.

I agree as well. However, I think newbies can be a great help in 
identifying holes in the documentation -- so when documentation is 
missing or unclear, ask here, and then you'll have something to contribute.

-Chris

> Do check out the developer zone, maybe search the the open tickets and
> see if there are any bugs you want to tackle.
> Maybe write some tests for untested code you find.

Good ideas as well, of course.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Orion Poplawski]

On 03/31/2011 01:21 PM, Robert Kern wrote:
> On Thu, Mar 31, 2011 at 14:16, Orion Poplawski  wrote:
>> On 03/31/2011 01:12 PM, Robert Kern wrote:
>>> Well, they're meant to be copied into your own code, which is why they
>>> end up under a doc/ directory. Lots of things like this tend to end up
>>> in doc/ directories.
>>
>> perhaps /usr/share/doc/numpy then.
>
> This is up to the various Linux packagers of numpy. They have their
> own tools to do this according to their own policies. Doing *anything*
> in our setup.py to install these files will just make their jobs
> harder. That's why I think that this needs to be handled in the binary
> installers, not the setup.py.
>

Fedora already copies everything in doc/* to /usr/share/doc/numpy-%{version}, 
so yeah, no need to put it anywhere.

-- 
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Robert Kern]

On Thu, Mar 31, 2011 at 14:16, Orion Poplawski  wrote:
> On 03/31/2011 01:12 PM, Robert Kern wrote:
>> Well, they're meant to be copied into your own code, which is why they
>> end up under a doc/ directory. Lots of things like this tend to end up
>> in doc/ directories.
>
> perhaps /usr/share/doc/numpy then.

This is up to the various Linux packagers of numpy. They have their
own tools to do this according to their own policies. Doing *anything*
in our setup.py to install these files will just make their jobs
harder. That's why I think that this needs to be handled in the binary
installers, not the setup.py.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Orion Poplawski]

On 03/31/2011 01:12 PM, Robert Kern wrote:
> Well, they're meant to be copied into your own code, which is why they
> end up under a doc/ directory. Lots of things like this tend to end up
> in doc/ directories.
>

perhaps /usr/share/doc/numpy then.


-- 
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Orion Poplawski]

On 03/31/2011 01:08 PM, Ralf Gommers wrote:
> But they're not documentation files. The line is a bit blurry, but
> something like the numpy.i SWIG interface face is meant for using
> numpy, not documenting usage of SWIG and numpy (maybe most similar to
> what's in core/include/ ?). The actual documentation of how to use
> numpy.i is included in the reference guide, and is not present in that
> dir.
>
> Perhaps to make clear this is not documentation, put it in something
> like numpy/interfacing/swig/, etc.?

How are these files used?  Are they used by the python code directly?  Just 
used by people wanting to use swig/cython/pyrex to interface with numpy?  If 
the latter, they don't belong in python/site-packages but something lib 
/usr/share/numpy.


-- 
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Robert Kern]

On Thu, Mar 31, 2011 at 14:08, Ralf Gommers  wrote:
> On Thu, Mar 31, 2011 at 8:52 PM, Robert Kern  wrote:

>> Linux distributions start to complain when data files (especially
>> documentation data files not used at runtime) are placed into the
>> Python packages. I would argue that it's the wrong place for them on
>> *any* platform.
>
> But they're not documentation files. The line is a bit blurry, but
> something like the numpy.i SWIG interface face is meant for using
> numpy, not documenting usage of SWIG and numpy (maybe most similar to
> what's in core/include/ ?).

Yeah, and they hate that, too. As the originator of that idea, I'm
loathe to push it any further. :-)

> The actual documentation of how to use
> numpy.i is included in the reference guide, and is not present in that
> dir.
>
> Perhaps to make clear this is not documentation, put it in something
> like numpy/interfacing/swig/, etc.?

Well, they're meant to be copied into your own code, which is why they
end up under a doc/ directory. Lots of things like this tend to end up
in doc/ directories.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Ralf Gommers]

On Thu, Mar 31, 2011 at 8:52 PM, Robert Kern  wrote:
> On Thu, Mar 31, 2011 at 13:39, Ralf Gommers  
> wrote:
>> On Thu, Mar 31, 2011 at 8:32 PM, Robert Kern  wrote:
>>> On Thu, Mar 31, 2011 at 12:00, Ralf Gommers  
>>> wrote:
 On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski  
 wrote:
> I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.  
> It
> appears to install a bunch of suff in 
> /usr/lib64/python2.7/site-packages/doc
> (see below for examples).  This seems wrong.  Thoughts?

 That is indeed wrong. It's caused by the following in setup.py:

    # we want these files also in binaries/installed files, so it belongs 
 here
    # instead of in Manifest.in
    config.add_data_files(('doc/cython/'),
                          ('doc/pyrex/'),
                          ('doc/swig/'))

 Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
 source tree?
>>>
>>> It's not clear to me how they were included in the Windows .exe
>>> installer originally (the original ticket that led to this
>>> change[1088]). I don't think it was through the generic setup.py. The
>>> proper fix would probably be something specific to each binary
>>> installer.
>>>
>>> [1088] http://projects.scipy.org/numpy/ticket/1088
>>
>> I'm not sure how this was done at first (maybe someone remembers?), it
>> kind of hard to search for that in the history. But something specific
>> for each way of installing sounds fragile. The first question to be
>> answered is where these files should be installed to. The current
>> location is obviously wrong.
>
> Well, the answer is different for each platform. That's why it ought
> to be up to each binary installer.
>
>> If it is under numpy/doc/swig/ etc. then moving them there in the
>> source tree itself seems logical. Those dirs are not used to build the
>> user and reference guides, so there's no need for them to stay where
>> they are.
>
> Linux distributions start to complain when data files (especially
> documentation data files not used at runtime) are placed into the
> Python packages. I would argue that it's the wrong place for them on
> *any* platform.

But they're not documentation files. The line is a bit blurry, but
something like the numpy.i SWIG interface face is meant for using
numpy, not documenting usage of SWIG and numpy (maybe most similar to
what's in core/include/ ?). The actual documentation of how to use
numpy.i is included in the reference guide, and is not present in that
dir.

Perhaps to make clear this is not documentation, put it in something
like numpy/interfacing/swig/, etc.?

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Willing to contribute to SciPy & NumPy ...

[Anthony Scopatz]

On Thu, Mar 31, 2011 at 3:04 AM, Scott Sinclair  wrote:

> On 31 March 2011 07:27, Sylvain Bellemare  wrote:
> >  I would like to seriously start contributing to NumPy and/or SciPy, as
> much as I possibly can.
>
> I'm sure that your help will be welcomed!
>
> A good place to get started is helping out with documentation (see
> http://docs.scipy.org/numpy/Front%20Page/).


Actually, according to the Jacob Kaplan-Moss's talk at PyCon this year (
http://pycon.blip.tv/file/4881071/)
(with which I am inclined to agree), documentation is probably not the best
way to get started.

Do check out the developer zone, maybe search the the open tickets and see
if there are any bugs you want to tackle.
Maybe write some tests for untested code you find.  If you have questions,
don't hesitate to ask.  And don't forget
to test and document your own code!

Be Well
Anthony


> SciPy has plenty of work
> required - you'll probably learn your way into the code that way.
> Another place to look is http://www.scipy.org/Developer_Zone. It's
> worthwhile learning how to work with Git and Github if you want to get
> patches accepted more easily.
>
> Cheers,
> Scott
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Robert Kern]

On Thu, Mar 31, 2011 at 13:39, Ralf Gommers  wrote:
> On Thu, Mar 31, 2011 at 8:32 PM, Robert Kern  wrote:
>> On Thu, Mar 31, 2011 at 12:00, Ralf Gommers  
>> wrote:
>>> On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski  
>>> wrote:
 I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.  It
 appears to install a bunch of suff in 
 /usr/lib64/python2.7/site-packages/doc
 (see below for examples).  This seems wrong.  Thoughts?
>>>
>>> That is indeed wrong. It's caused by the following in setup.py:
>>>
>>>    # we want these files also in binaries/installed files, so it belongs 
>>> here
>>>    # instead of in Manifest.in
>>>    config.add_data_files(('doc/cython/'),
>>>                          ('doc/pyrex/'),
>>>                          ('doc/swig/'))
>>>
>>> Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
>>> source tree?
>>
>> It's not clear to me how they were included in the Windows .exe
>> installer originally (the original ticket that led to this
>> change[1088]). I don't think it was through the generic setup.py. The
>> proper fix would probably be something specific to each binary
>> installer.
>>
>> [1088] http://projects.scipy.org/numpy/ticket/1088
>
> I'm not sure how this was done at first (maybe someone remembers?), it
> kind of hard to search for that in the history. But something specific
> for each way of installing sounds fragile. The first question to be
> answered is where these files should be installed to. The current
> location is obviously wrong.

Well, the answer is different for each platform. That's why it ought
to be up to each binary installer.

> If it is under numpy/doc/swig/ etc. then moving them there in the
> source tree itself seems logical. Those dirs are not used to build the
> user and reference guides, so there's no need for them to stay where
> they are.

Linux distributions start to complain when data files (especially
documentation data files not used at runtime) are placed into the
Python packages. I would argue that it's the wrong place for them on
*any* platform.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?

[Ralf Gommers]

On Thu, Mar 31, 2011 at 8:37 PM, Eli Stevens (Gmail)
 wrote:
> On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers
>  wrote:
>> It would definitely need to get into beta 2, and even then I'm a
>> little hesitant to push in such a change at the last moment.
>
> If I miss the 1.6.0b2 cutoff, would a change like this be appropriate
> for 1.6.1, or will it need to wait until 1.7.0?  I wish I'd known that
> half floats were going to be in 1.6.0 (the bug tracker says 2.0); I'd
> have started working on this a few weeks earlier.  :)

There likely will not be a 1.7.0, if things with 2.0 go according to
plan. It probably wouldn't be a very appropriate change for 1.6.1.

>
>> It would
>> need some testing with other libraries that use the numpy buffer
>> protocol.
>
> I am currently working on a patch for cython to accept half-floats.
> ATM, it is able to accept the data from numpy (though it does not yet
> handle the bit-fiddling to expand the values into a float32 on read,
> nor collapse it from a float32 back into a float16 on write).  The
> code change to numpy was literally a one-line change to add NPY_HALF
> to a switch statement (though I haven't tried to import half floats
> yet).  Is one other library enough, or will I need to find another
> library to patch as well?  I think that OpenCL would probably be my
> next choice, since I think that we'll eventually want float16 support
> there too.

If it works with Cython I think that's enough. With other libraries I
did not mean that you should write patches for those, just some
testing to make sure the change is 100% backwards compatible.

>> Also, reading back the thread, Pauli seemed to disagree with
>> this.
>
> On Fri, Mar 25, 2011 at 10:21 AM, Pauli Virtanen  wrote:
>> The buffer interface cannot be used to export the half-float types, since
>> the type is not specified in PEP 3118. Numpy cannot unilaterally add
>> nonstandard format codes to the spec.
>
> I'm currently in the process of submitting a patch to CPython to
> update the struct module to handle half-floats.  So far, the response
> on python-ideas has been generally positive (a couple +1s from people
> with commit bits, GVR has chimed in on the thread with an aside (but
> not an objection), etc.).  Unless I'm misunderstanding the objection,
> making half-floats an official part of the spec (the discussion on
> python-ideas explicitly notes that numpy driving force behind the
> change) would satisfy Pauli's concern.
>
> Next up for me is to get a patch onto the CPython issue tracker, but
> as soon as I have that done, I'll start working on adding unit tests
> to my numpy fork.  I anticipate being able to get that done today.

Sounds good.

Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Bruce Southey]

On 03/31/2011 12:02 PM, Derek Homeier wrote:
> On 31 Mar 2011, at 17:03, Bruce Southey wrote:
>
>> This is an invalid ticket because the docstring clearly states that in
>> 3 different, yet critical places, that missing values are not handled
>> here:
>>
>> "Each row in the text file must have the same number of values."
>> "genfromtxt : Load data with missing values handled as specified."
>> "   This function aims to be a fast reader for simply formatted
>> files.  The
>> `genfromtxt` function provides more sophisticated handling of,
>> e.g.,
>> lines with missing values."
>>
>> Really I am trying to separate the usage of loadtxt and genfromtxt to
>> avoid unnecessary duplication and confusion. Part of this is
>> historical because loadtxt was added in 2007 and genfromtxt was added
>> in 2009. So really certain features of loadtxt have been  'kept' for
>> backwards compatibility purposes yet these features can be 'abused' to
>> handle missing data. But I really consider that any missing values
>> should cause loadtxt to fail.
>>
> OK, I was not aware of the design issues of loadtxt vs. genfromtxt -
> you could probably say also for historical reasons since I have not
> used genfromtxt much so far.
> Anyway the docstring statement "Converters can also be used to
>   provide a default value for missing data:"
> then appears quite misleading, or an invitation to abuse, if you will.
> This should better be removed from the documentation then, or users
> explicitly discouraged from using converters instead of genfromtxt
> (I don't see how you could completely prevent using converters in
> this way).
>
>> The patch is incorrect because it should not include a space in the
>> split() as indicated in the comment by the original reporter. Of
> The split('\r\n') alone caused test_dtype_with_object(self) to fail,
> probably
> because it relies on stripping the blanks. But maybe the test is ill-
> formed?
>
>> course a corrected patch alone still is not sufficient to address the
>> problem without the user providing the correct converter. Also you
>> start to run into problems with multiple delimiters (such as one space
>> versus two spaces) so you start down the path to add all the features
>> that duplicate genfromtxt.
> Given that genfromtxt provides that functionality more conveniently,
> I agree again users should be encouraged to use this instead of
> converters.
> But the actual tab-problem causes in fact an issue not related to
> missing
> values at all (well, depending on what you call a missing value).
> I am describing an example on the ticket.
>
> Cheers,
>   Derek
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
I am really not disagreeing that much with you. Rather that, as you have 
shown, it is very easy to increase the complexity of examples that 
loadtxt does not handle.  By missing value I mean "when no data value is 
stored for the variable in the current observation" (via Wikipedia) 
since encoded missing values (such as '.', 'NA' and 'NaN') can be 
recovered.


Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Ralf Gommers]

On Thu, Mar 31, 2011 at 8:32 PM, Robert Kern  wrote:
> On Thu, Mar 31, 2011 at 12:00, Ralf Gommers  
> wrote:
>> On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski  wrote:
>>> I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.  It
>>> appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc
>>> (see below for examples).  This seems wrong.  Thoughts?
>>
>> That is indeed wrong. It's caused by the following in setup.py:
>>
>>    # we want these files also in binaries/installed files, so it belongs here
>>    # instead of in Manifest.in
>>    config.add_data_files(('doc/cython/'),
>>                          ('doc/pyrex/'),
>>                          ('doc/swig/'))
>>
>> Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
>> source tree?
>
> It's not clear to me how they were included in the Windows .exe
> installer originally (the original ticket that led to this
> change[1088]). I don't think it was through the generic setup.py. The
> proper fix would probably be something specific to each binary
> installer.
>
> [1088] http://projects.scipy.org/numpy/ticket/1088

I'm not sure how this was done at first (maybe someone remembers?), it
kind of hard to search for that in the history. But something specific
for each way of installing sounds fragile. The first question to be
answered is where these files should be installed to. The current
location is obviously wrong.

If it is under numpy/doc/swig/ etc. then moving them there in the
source tree itself seems logical. Those dirs are not used to build the
user and reference guides, so there's no need for them to stay where
they are.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?

[Eli Stevens (Gmail)]

On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers
 wrote:
> It would definitely need to get into beta 2, and even then I'm a
> little hesitant to push in such a change at the last moment.

If I miss the 1.6.0b2 cutoff, would a change like this be appropriate
for 1.6.1, or will it need to wait until 1.7.0?  I wish I'd known that
half floats were going to be in 1.6.0 (the bug tracker says 2.0); I'd
have started working on this a few weeks earlier.  :)

> It would
> need some testing with other libraries that use the numpy buffer
> protocol.

I am currently working on a patch for cython to accept half-floats.
ATM, it is able to accept the data from numpy (though it does not yet
handle the bit-fiddling to expand the values into a float32 on read,
nor collapse it from a float32 back into a float16 on write).  The
code change to numpy was literally a one-line change to add NPY_HALF
to a switch statement (though I haven't tried to import half floats
yet).  Is one other library enough, or will I need to find another
library to patch as well?  I think that OpenCL would probably be my
next choice, since I think that we'll eventually want float16 support
there too.

> Also, reading back the thread, Pauli seemed to disagree with
> this.

On Fri, Mar 25, 2011 at 10:21 AM, Pauli Virtanen  wrote:
> The buffer interface cannot be used to export the half-float types, since
> the type is not specified in PEP 3118. Numpy cannot unilaterally add
> nonstandard format codes to the spec.

I'm currently in the process of submitting a patch to CPython to
update the struct module to handle half-floats.  So far, the response
on python-ideas has been generally positive (a couple +1s from people
with commit bits, GVR has chimed in on the thread with an aside (but
not an objection), etc.).  Unless I'm misunderstanding the objection,
making half-floats an official part of the spec (the discussion on
python-ideas explicitly notes that numpy driving force behind the
change) would satisfy Pauli's concern.

Next up for me is to get a patch onto the CPython issue tracker, but
as soon as I have that done, I'll start working on adding unit tests
to my numpy fork.  I anticipate being able to get that done today.

Thanks!
Eli
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Robert Kern]

On Thu, Mar 31, 2011 at 12:00, Ralf Gommers  wrote:
> On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski  wrote:
>> I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.  It
>> appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc
>> (see below for examples).  This seems wrong.  Thoughts?
>
> That is indeed wrong. It's caused by the following in setup.py:
>
>    # we want these files also in binaries/installed files, so it belongs here
>    # instead of in Manifest.in
>    config.add_data_files(('doc/cython/'),
>                          ('doc/pyrex/'),
>                          ('doc/swig/'))
>
> Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
> source tree?

It's not clear to me how they were included in the Windows .exe
installer originally (the original ticket that led to this
change[1088]). I don't think it was through the generic setup.py. The
proper fix would probably be something specific to each binary
installer.

[1088] http://projects.scipy.org/numpy/ticket/1088

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?

[Mark Wiebe]

On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers
wrote:

> On Thu, Mar 31, 2011 at 7:06 PM, Mark Wiebe  wrote:
> > On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) <
> wickedg...@gmail.com>
> > wrote:
> >>
> >> On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail)
> >>  wrote:
> >> > Can anyone please give me some suggestions on how to go about writing
> >> > a unit test for this?  Or should I just submit a pull request?
> >>
> >> I've gotten a bit of positive feedback to adding the 'e' type to the
> >> struct module on the python-ideas list (per my understanding, not
> >> before python 3.3, but I don't think that should hinder adoption in
> >> other libraries), so I'd like to ask again about unit testing a change
> >> like this.  Can anyone offer some advice for where to start?
> >
> > The tests for the buffer protocol are in
> > numpy/core/tests/test_multiarray.py, starting at line 1847. It does some
> > round-trip testing of all the types through the memoryview object.
> > You'll also need to change the _dtype_from_pep3118 in
> > numpy/core/_internal.py, called from the _descriptor_from_pep3118_format
> > function, so that NumPy can accept the half type from a buffer as well.
> >>
> >> Also, what kind of timeframe / cutoff am I looking at to get this into
> >> 1.6.0 or 1.6.x?
> >
> > Since the changes to NumPy are pretty small, I think it should be ok to
> get
> > into 1.6.0. How to handle it is up to Ralf, however, and it would
> probably
> > be best to get it into beta 2 which I believe he would like to release
> over
> > the weekend (please chime in to correct me).
>
> It would definitely need to get into beta 2, and even then I'm a
> little hesitant to push in such a change at the last moment. It would
> need some testing with other libraries that use the numpy buffer
> protocol. Also, reading back the thread, Pauli seemed to disagree with
> this.


My reading of Pauli's thoughts was that putting it in unilaterally is
undesirable, something I definitely agree with. I think with Eli doing the
legwork of getting input and acceptance from the relevant parties, we should
help him out as much as possible. Not getting this change into 1.6 makes the
Cython support much more difficult because of their design based around the
buffer protocol.

Looking at the thread on python-users, I see mostly concern that the type be
standard or have precedent elsewhere, which as an IEEE standard it
definitely satisfies. The other question is on the chosen letter - we picked
'e' as 'h' was taken and it is close to 'f' and 'd', the other float types.
If a letter change is deemed necessary, it would be good to get that into
1.6 as well, since this kind of change is easy now, but much more difficult
later.

For testing, Eli implementing the Cython support seems like a good start to
me. Another nice test would be to get PIL's OpenEXR support to export a half
buffer to NumPy.

-Mark
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.histogramdd of empty data

[Benjamin Root]

On Thu, Mar 31, 2011 at 12:32 PM, Ralf Gommers
wrote:

> -- Forwarded message --
> From: Ralf Gommers 
> Date: Thu, Mar 31, 2011 at 7:31 PM
> Subject: Re: [Numpy-discussion] np.histogramdd of empty data
> To: Nils Becker 
>
>
> On Thu, Mar 31, 2011 at 12:33 PM, Nils Becker  wrote:
> > Hi Ralf,
> >
> > I cloned numpy/master and played around a little.
> >
> > when giving the bins explicitely, now histogram2d and histogramdd work
> > as expected in all tests i tried.
> >
> >
> > However, some of the cases with missing bin specification appear
> > somewhat inconsistent.
> >
> > The first question is if creating arbitrary bins for empty data and
> > empty bin specification is better than raising an Exception:
> >
> > Specifically:
>
> Bins of size 0 should give a meaningful error, I was just fixing that
> as part of #1788 in
> https://github.com/rgommers/numpy/tree/ticket-1788-histogramdd
>
> > numpy.histogram2d([],[],bins=[0,0])
> >> (array([ 0.,  0.]), array([ 0.]), array([ 0.]))
>
> Now gives:
>ValueError: Element at index 0 in `bins` should be a positive integer.
>
> > numpy.histogram([],bins=0)
> >> ValueError: zero-size array to minimum.reduce without identity
>
> Now gives:
>ValueError: `bins` should be a positive integer.
>
>
> > so 1-d and 2-d behave not quite the same.
> >
> > also, these work (although with arbitrary bin edges):
> >
> > numpy.histogram2d([],[],bins=[1,1])
> >> (array([ 0.,  0.]), array([ 0.,  1.]), array([ 0.,  1.]))
> >
> > numpy.histogram2d([],[],bins=[0,1])
> >> (array([ 0.,  0.]), array([ 0.]), array([ 0.,  1.]))
> >
> > while this raises an error:
> >
> > numpy.histogram([],bins=1)
> >> ValueError: zero-size array to minimum.reduce without identity
>
> Now gives:
>(array([0]), array([ 0.,  1.]))
>
>
Just for consistency's sake, maybe the same should be done for np.bincount()
and np.digitize() for empty data (but known bins)?  I don't know if your fix
to histogram does this or not, but the latest pull from numpy master doesn't
do this.

Thanks,
Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?

[Ralf Gommers]

On Thu, Mar 31, 2011 at 7:06 PM, Mark Wiebe  wrote:
> On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) 
> wrote:
>>
>> On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail)
>>  wrote:
>> > Can anyone please give me some suggestions on how to go about writing
>> > a unit test for this?  Or should I just submit a pull request?
>>
>> I've gotten a bit of positive feedback to adding the 'e' type to the
>> struct module on the python-ideas list (per my understanding, not
>> before python 3.3, but I don't think that should hinder adoption in
>> other libraries), so I'd like to ask again about unit testing a change
>> like this.  Can anyone offer some advice for where to start?
>
> The tests for the buffer protocol are in
> numpy/core/tests/test_multiarray.py, starting at line 1847. It does some
> round-trip testing of all the types through the memoryview object.
> You'll also need to change the _dtype_from_pep3118 in
> numpy/core/_internal.py, called from the _descriptor_from_pep3118_format
> function, so that NumPy can accept the half type from a buffer as well.
>>
>> Also, what kind of timeframe / cutoff am I looking at to get this into
>> 1.6.0 or 1.6.x?
>
> Since the changes to NumPy are pretty small, I think it should be ok to get
> into 1.6.0. How to handle it is up to Ralf, however, and it would probably
> be best to get it into beta 2 which I believe he would like to release over
> the weekend (please chime in to correct me).

It would definitely need to get into beta 2, and even then I'm a
little hesitant to push in such a change at the last moment. It would
need some testing with other libraries that use the numpy buffer
protocol. Also, reading back the thread, Pauli seemed to disagree with
this.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.histogramdd of empty data

[Ralf Gommers]

-- Forwarded message --
From: Ralf Gommers 
Date: Thu, Mar 31, 2011 at 7:31 PM
Subject: Re: [Numpy-discussion] np.histogramdd of empty data
To: Nils Becker 


On Thu, Mar 31, 2011 at 12:33 PM, Nils Becker  wrote:
> Hi Ralf,
>
> I cloned numpy/master and played around a little.
>
> when giving the bins explicitely, now histogram2d and histogramdd work
> as expected in all tests i tried.
>
>
> However, some of the cases with missing bin specification appear
> somewhat inconsistent.
>
> The first question is if creating arbitrary bins for empty data and
> empty bin specification is better than raising an Exception:
>
> Specifically:

Bins of size 0 should give a meaningful error, I was just fixing that
as part of #1788 in
https://github.com/rgommers/numpy/tree/ticket-1788-histogramdd

> numpy.histogram2d([],[],bins=[0,0])
>> (array([ 0.,  0.]), array([ 0.]), array([ 0.]))

Now gives:
   ValueError: Element at index 0 in `bins` should be a positive integer.

> numpy.histogram([],bins=0)
>> ValueError: zero-size array to minimum.reduce without identity

Now gives:
   ValueError: `bins` should be a positive integer.


> so 1-d and 2-d behave not quite the same.
>
> also, these work (although with arbitrary bin edges):
>
> numpy.histogram2d([],[],bins=[1,1])
>> (array([ 0.,  0.]), array([ 0.,  1.]), array([ 0.,  1.]))
>
> numpy.histogram2d([],[],bins=[0,1])
>> (array([ 0.,  0.]), array([ 0.]), array([ 0.,  1.]))
>
> while this raises an error:
>
> numpy.histogram([],bins=1)
>> ValueError: zero-size array to minimum.reduce without identity

Now gives:
   (array([0]), array([ 0.,  1.]))


> another thing with non-empty data:
>
> numpy.histogram([1],bins=1)
>> (array([1]), array([ 0.5,  1.5]))

That is the correct answer.

> numpy.histogram([1],bins=0)
>> (array([], dtype=int64), array([ 0.5]))

Now gives:
   ValueError: `bins` should be a positive integer.


> while
>
> numpy.histogram2d([1],[1],bins=A)
>> ValueError: zero-size array to minimum.reduce without identity
>
> (here A==[0,0] or A==[0,1] but not A==[1,1] which gives a result)

Same sensible errors now, telling you bins elements shouldn't be 0.

Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?

[Mark Wiebe]

On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail)
wrote:

> On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail)
>  wrote:
> > Can anyone please give me some suggestions on how to go about writing
> > a unit test for this?  Or should I just submit a pull request?
>
> I've gotten a bit of positive feedback to adding the 'e' type to the
> struct module on the python-ideas list (per my understanding, not
> before python 3.3, but I don't think that should hinder adoption in
> other libraries), so I'd like to ask again about unit testing a change
> like this.  Can anyone offer some advice for where to start?
>

The tests for the buffer protocol are in
numpy/core/tests/test_multiarray.py, starting at line 1847. It does some
round-trip testing of all the types through the memoryview object.

You'll also need to change the _dtype_from_pep3118 in
numpy/core/_internal.py, called from the _descriptor_from_pep3118_format
function, so that NumPy can accept the half type from a buffer as well.

Also, what kind of timeframe / cutoff am I looking at to get this into
> 1.6.0 or 1.6.x?


Since the changes to NumPy are pretty small, I think it should be ok to get
into 1.6.0. How to handle it is up to Ralf, however, and it would probably
be best to get it into beta 2 which I believe he would like to release over
the weekend (please chime in to correct me).

-Mark
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Derek Homeier]

On 31 Mar 2011, at 17:03, Bruce Southey wrote:

> This is an invalid ticket because the docstring clearly states that in
> 3 different, yet critical places, that missing values are not handled
> here:
>
> "Each row in the text file must have the same number of values."
> "genfromtxt : Load data with missing values handled as specified."
> "   This function aims to be a fast reader for simply formatted  
> files.  The
>`genfromtxt` function provides more sophisticated handling of,  
> e.g.,
>lines with missing values."
>
> Really I am trying to separate the usage of loadtxt and genfromtxt to
> avoid unnecessary duplication and confusion. Part of this is
> historical because loadtxt was added in 2007 and genfromtxt was added
> in 2009. So really certain features of loadtxt have been  'kept' for
> backwards compatibility purposes yet these features can be 'abused' to
> handle missing data. But I really consider that any missing values
> should cause loadtxt to fail.
>
OK, I was not aware of the design issues of loadtxt vs. genfromtxt -
you could probably say also for historical reasons since I have not
used genfromtxt much so far.
Anyway the docstring statement "Converters can also be used to
 provide a default value for missing data:"
then appears quite misleading, or an invitation to abuse, if you will.
This should better be removed from the documentation then, or users
explicitly discouraged from using converters instead of genfromtxt
(I don't see how you could completely prevent using converters in
this way).

> The patch is incorrect because it should not include a space in the
> split() as indicated in the comment by the original reporter. Of

The split('\r\n') alone caused test_dtype_with_object(self) to fail,  
probably
because it relies on stripping the blanks. But maybe the test is ill- 
formed?

> course a corrected patch alone still is not sufficient to address the
> problem without the user providing the correct converter. Also you
> start to run into problems with multiple delimiters (such as one space
> versus two spaces) so you start down the path to add all the features
> that duplicate genfromtxt.

Given that genfromtxt provides that functionality more conveniently,
I agree again users should be encouraged to use this instead of  
converters.
But the actual tab-problem causes in fact an issue not related to  
missing
values at all (well, depending on what you call a missing value).
I am describing an example on the ticket.

Cheers,
Derek

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Ralf Gommers]

On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski  wrote:
> I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.  It
> appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc
> (see below for examples).  This seems wrong.  Thoughts?

That is indeed wrong. It's caused by the following in setup.py:

# we want these files also in binaries/installed files, so it belongs here
# instead of in Manifest.in
config.add_data_files(('doc/cython/'),
  ('doc/pyrex/'),
  ('doc/swig/'))

Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
source tree?

Ralf


>
>    /usr/lib64/python2.7/site-packages/doc/cython/MANIFEST
>    /usr/lib64/python2.7/site-packages/doc/cython/Makefile
>    /usr/lib64/python2.7/site-packages/doc/cython/README.txt
>    /usr/lib64/python2.7/site-packages/doc/cython/c_numpy.pxd
>    /usr/lib64/python2.7/site-packages/doc/cython/c_python.pxd
>    /usr/lib64/python2.7/site-packages/doc/cython/numpyx.pyx
>    /usr/lib64/python2.7/site-packages/doc/cython/run_test.py
>    /usr/lib64/python2.7/site-packages/doc/cython/run_test.pyc
>    /usr/lib64/python2.7/site-packages/doc/cython/run_test.pyo
>    /usr/lib64/python2.7/site-packages/doc/cython/setup.py
>    /usr/lib64/python2.7/site-packages/doc/cython/setup.pyc
>    /usr/lib64/python2.7/site-packages/doc/cython/setup.pyo
>    /usr/lib64/python2.7/site-packages/doc/pyrex/MANIFEST
>    /usr/lib64/python2.7/site-packages/doc/pyrex/Makefile
>    /usr/lib64/python2.7/site-packages/doc/pyrex/README.txt
>    /usr/lib64/python2.7/site-packages/doc/pyrex/c_numpy.pxd
>    /usr/lib64/python2.7/site-packages/doc/pyrex/c_python.pxd
>    /usr/lib64/python2.7/site-packages/doc/pyrex/notes
>    /usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.c
>    /usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.pyx
>    /usr/lib64/python2.7/site-packages/doc/pyrex/run_test.py
>    /usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyc
>    /usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyo
>    /usr/lib64/python2.7/site-packages/doc/pyrex/setup.py
>    /usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyc
>    /usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyo
>    /usr/lib64/python2.7/site-packages/doc/swig/Array.i
>    /usr/lib64/python2.7/site-packages/doc/swig/Array1.cxx
>    /usr/lib64/python2.7/site-packages/doc/swig/Array1.h
>
>
> --
> Orion Poplawski
> Technical Manager                     303-415-9701 x222
> NWRA/CoRA Division                    FAX: 303-415-9702
> 3380 Mitchell Lane                  or...@cora.nwra.com
> Boulder, CO 80301              http://www.cora.nwra.com
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc

[Orion Poplawski]

I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.  It 
appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc 
(see below for examples).  This seems wrong.  Thoughts?

/usr/lib64/python2.7/site-packages/doc/cython/MANIFEST
/usr/lib64/python2.7/site-packages/doc/cython/Makefile
/usr/lib64/python2.7/site-packages/doc/cython/README.txt
/usr/lib64/python2.7/site-packages/doc/cython/c_numpy.pxd
/usr/lib64/python2.7/site-packages/doc/cython/c_python.pxd
/usr/lib64/python2.7/site-packages/doc/cython/numpyx.pyx
/usr/lib64/python2.7/site-packages/doc/cython/run_test.py
/usr/lib64/python2.7/site-packages/doc/cython/run_test.pyc
/usr/lib64/python2.7/site-packages/doc/cython/run_test.pyo
/usr/lib64/python2.7/site-packages/doc/cython/setup.py
/usr/lib64/python2.7/site-packages/doc/cython/setup.pyc
/usr/lib64/python2.7/site-packages/doc/cython/setup.pyo
/usr/lib64/python2.7/site-packages/doc/pyrex/MANIFEST
/usr/lib64/python2.7/site-packages/doc/pyrex/Makefile
/usr/lib64/python2.7/site-packages/doc/pyrex/README.txt
/usr/lib64/python2.7/site-packages/doc/pyrex/c_numpy.pxd
/usr/lib64/python2.7/site-packages/doc/pyrex/c_python.pxd
/usr/lib64/python2.7/site-packages/doc/pyrex/notes
/usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.c
/usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.pyx
/usr/lib64/python2.7/site-packages/doc/pyrex/run_test.py
/usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyc
/usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyo
/usr/lib64/python2.7/site-packages/doc/pyrex/setup.py
/usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyc
/usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyo
/usr/lib64/python2.7/site-packages/doc/swig/Array.i
/usr/lib64/python2.7/site-packages/doc/swig/Array1.cxx
/usr/lib64/python2.7/site-packages/doc/swig/Array1.h


-- 
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane  or...@cora.nwra.com
Boulder, CO 80301  http://www.cora.nwra.com
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Old tickets

[Christopher Barker]

On 3/31/11 8:41 AM, Benjamin Root wrote:
> The ticket is about the functions np.divide and np.power, not / and
> **. This currently does not work the same, unlike what's said above:

> Hmm, I didn't notice that distinction.  So long as there is a consistent
> difference between the function and the operator, I am fine with that.

> >  whether we want
> >  np.power to behave differently from ** (if they are internally handled
> >  separately at all)...

Please, no! the operator and functions should behave the same. I 
certainly always expect them to. For instance, I often write code with 
operators, then replace that with the functions so that i can add an 
"out" parameter for performance reasons -- I would be very surprised if 
the function were defined differently -- it would be a source of 
surprising bugs.

> So, in this case, the ** operator and np.power are identical, but
> diverges from the behavior of python's operator.

I think consistency within numpy is more important that consistency with 
python -- I expect differences like this from python (long integers, 
different data types, etc)

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Old tickets

[Charles R Harris]

On Thu, Mar 31, 2011 at 9:00 AM, Ralf Gommers
wrote:

> On Wed, Mar 30, 2011 at 11:52 PM, Derek Homeier
>  wrote:
> >
> > On 30 Mar 2011, at 23:26, Benjamin Root wrote:
> >
> >> Ticket 301: 'Make power and divide return floats from int inputs (like
> >> true_divide)'
> >> http://projects.scipy.org/numpy/ticket/301
> >> Invalid because the output dtype is the same as the input dtype unless
> >> you override using the dtype argument:
> >>  >>> np.power(3, 1, dtype=np.float128).dtype
> >> dtype('float128')
> >> Alternatively return a float and indicate in the docstring that the
> >> output dtype can be changed.
> >>
> >>
> >> FWIW,
> >>
> >> Just thought I'd note (on a python 2.6 system):
> >>
> >> >>> import numpy as np
> >> >>> a = np.array([1, 2, 3, 4])
> >> >>> a.dtype
> >> dtype('int32')
> >> >>> 2 / a
> >> array([2, 1, 0, 0])
> >> >>> from __future__ import division
> >> >>> 2 / a
> >> array([ 2.,  1.,  0.6667,  0.5   ])
> >>
> >> So, numpy already does this when true division is imported (and
> >> therefore consistent with whatever the python environment does), and
> >> python currently also returns integers for exponentials when both
> >> inputs are integers.
> >
> > I'd agree, and in my view power(3, -1) is well defined as 1 / 3 -
> > also, in future (or Python3)
> >
> >  >>> a/2
> > array([ 0.5,  1. ,  1.5,  2. ])
> >  >>> a//2
> > array([0, 1, 1, 2], dtype=int32)
>
> The ticket is about the functions np.divide and np.power, not / and
> **. This currently does not work the same, unlike what's said above:
>
> >>> from __future__ import division
> >>> x = np.arange(4) + 1
> >>> 2 / x
> array([ 2.,  1.,  0.6667,  0.5   ])
> >>> np.divide(2, x)
> array([2, 1, 0, 0])
>
> The power and divide functions should not check the "from __future__
> import division", just return floats IMHO. This is what's expected by
> most users, and much more useful than integer math


There is true_divide for that.

In [4]: np.true_divide(2, x)
Out[4]: array([ 2.,  1.,  0.6667,  0.5   ])

true_divide is what is called by the '/' operator in Python3.



Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Old tickets

[Benjamin Root]

On Thu, Mar 31, 2011 at 10:00 AM, Ralf Gommers
wrote:

> On Wed, Mar 30, 2011 at 11:52 PM, Derek Homeier
>  wrote:
> >
> > On 30 Mar 2011, at 23:26, Benjamin Root wrote:
> >
> >> Ticket 301: 'Make power and divide return floats from int inputs (like
> >> true_divide)'
> >> http://projects.scipy.org/numpy/ticket/301
> >> Invalid because the output dtype is the same as the input dtype unless
> >> you override using the dtype argument:
> >>  >>> np.power(3, 1, dtype=np.float128).dtype
> >> dtype('float128')
> >> Alternatively return a float and indicate in the docstring that the
> >> output dtype can be changed.
> >>
> >>
> >> FWIW,
> >>
> >> Just thought I'd note (on a python 2.6 system):
> >>
> >> >>> import numpy as np
> >> >>> a = np.array([1, 2, 3, 4])
> >> >>> a.dtype
> >> dtype('int32')
> >> >>> 2 / a
> >> array([2, 1, 0, 0])
> >> >>> from __future__ import division
> >> >>> 2 / a
> >> array([ 2.,  1.,  0.6667,  0.5   ])
> >>
> >> So, numpy already does this when true division is imported (and
> >> therefore consistent with whatever the python environment does), and
> >> python currently also returns integers for exponentials when both
> >> inputs are integers.
> >
> > I'd agree, and in my view power(3, -1) is well defined as 1 / 3 -
> > also, in future (or Python3)
> >
> >  >>> a/2
> > array([ 0.5,  1. ,  1.5,  2. ])
> >  >>> a//2
> > array([0, 1, 1, 2], dtype=int32)
>
> The ticket is about the functions np.divide and np.power, not / and
> **. This currently does not work the same, unlike what's said above:
>
> >>> from __future__ import division
> >>> x = np.arange(4) + 1
> >>> 2 / x
> array([ 2.,  1.,  0.6667,  0.5   ])
> >>> np.divide(2, x)
> array([2, 1, 0, 0])
>
> The power and divide functions should not check the "from __future__
> import division", just return floats IMHO. This is what's expected by
> most users, and much more useful than integer math.
>
>
Hmm, I didn't notice that distinction.  So long as there is a consistent
difference between the function and the operator, I am fine with that.


>
> > so I think at least a**n should follow integer math rules; depends on
> > whether we want
> > np.power to behave differently from ** (if they are internally handled
> > separately at all)...
> > Not sure if I understand the overload suggestion in the ticket
>
> I don't understand that either, it'll just lead to more confusion.
>
>
Did a little further investigation of this (I hadn't thought of negative
powers), and now I am getting thoroughly confused...

>>> import numpy as np
>>> a = np.array([1, 2])
>>> a.dtype
dtype('int32')
>>> 1**2
1
>>> 2**2
4
>>> 1**-2
1.0
>>> 2**-2
0.25
>>> a**2
array([1, 4])
>>> a**-2
array([1, 0])
>>> np.power(a, 2)
array([1, 4])
>>> np.power(a, -2)
array([1, 0])

So, in this case, the ** operator and np.power are identical, but diverges
from the behavior of python's operator.

FWIW,
Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Bruce Southey]

On 03/31/2011 10:08 AM, Ralf Gommers wrote:
> On Thu, Mar 31, 2011 at 5:03 PM, Bruce Southey  wrote:
>> On Wed, Mar 30, 2011 at 9:53 PM, Charles R Harris
>>   wrote:
>>>
>>> On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
>>>   wrote:
 On 26. mars 2011, at 21.44, Derek Homeier wrote:

> Hi Paul,
>
> having had a look at the other tickets you dug up,
>
>> [snip]
>> 1071:
>>   It is not clear to me whether loadtxt is supposed to support
>> missing values in the fashion indicated in the ticket.
> In principle it should at least allow you to, by the use of converters
> as described there.
> The problem is, the default delimiter is described as 'any
> whitespace', which in the
> present implementation obviously includes any number of blanks or
> tabs. These
> are therefore treated differently from delimiters like ',' or '&'. I'd
> reckon there are
> too many people actually relying on this behaviour to silently change it
> (e.g. I know plenty of tables with columns separated by either one or
> several
> tabs depending on the length of the previous entry). But the tab is
> apparently also
> treated differently if explicitly specified with "delimiter='\t'" -
> and in that case using
> a converter à la {2: lambda s: float(s or 'Nan')} is working for
> fields in the middle of
> the line, but not at the end - clearly warrants improvement. I've
> prepared a patch
> working for Python3 as well.
 Great!

>> This is an invalid ticket because the docstring clearly states that in
>> 3 different, yet critical places, that missing values are not handled
>> here:
>>
>> "Each row in the text file must have the same number of values."
>> "genfromtxt : Load data with missing values handled as specified."
>>   "   This function aims to be a fast reader for simply formatted files.  The
>> `genfromtxt` function provides more sophisticated handling of, e.g.,
>> lines with missing values."
>>
>> Really I am trying to separate the usage of loadtxt and genfromtxt to
>> avoid unnecessary duplication and confusion. Part of this is
>> historical because loadtxt was added in 2007 and genfromtxt was added
>> in 2009. So really certain features of loadtxt have been  'kept' for
>> backwards compatibility purposes yet these features can be 'abused' to
>> handle missing data. But I really consider that any missing values
>> should cause loadtxt to fail.
> I agree with you Bruce, but it would be easier to discuss this on the
> tickets instead of here. Could you add your comments there please?
>
> Ralf
>
'Easier' seems a contradiction when you have use captcha...
Sure I will add more comments there.

Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Ralf Gommers]

On Thu, Mar 31, 2011 at 5:03 PM, Bruce Southey  wrote:
> On Wed, Mar 30, 2011 at 9:53 PM, Charles R Harris
>  wrote:
>>
>>
>> On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
>>  wrote:
>>>
>>> On 26. mars 2011, at 21.44, Derek Homeier wrote:
>>>
>>> > Hi Paul,
>>> >
>>> > having had a look at the other tickets you dug up,
>>> >
> [snip]
>>>
>>> >> 1071:
>>> >>      It is not clear to me whether loadtxt is supposed to support
>>> >> missing values in the fashion indicated in the ticket.
>>> >
>>> > In principle it should at least allow you to, by the use of converters
>>> > as described there.
>>> > The problem is, the default delimiter is described as 'any
>>> > whitespace', which in the
>>> > present implementation obviously includes any number of blanks or
>>> > tabs. These
>>> > are therefore treated differently from delimiters like ',' or '&'. I'd
>>> > reckon there are
>>> > too many people actually relying on this behaviour to silently change it
>>> > (e.g. I know plenty of tables with columns separated by either one or
>>> > several
>>> > tabs depending on the length of the previous entry). But the tab is
>>> > apparently also
>>> > treated differently if explicitly specified with "delimiter='\t'" -
>>> > and in that case using
>>> > a converter à la {2: lambda s: float(s or 'Nan')} is working for
>>> > fields in the middle of
>>> > the line, but not at the end - clearly warrants improvement. I've
>>> > prepared a patch
>>> > working for Python3 as well.
>>>
>>> Great!
>>>
> This is an invalid ticket because the docstring clearly states that in
> 3 different, yet critical places, that missing values are not handled
> here:
>
> "Each row in the text file must have the same number of values."
> "genfromtxt : Load data with missing values handled as specified."
>  "   This function aims to be a fast reader for simply formatted files.  The
>    `genfromtxt` function provides more sophisticated handling of, e.g.,
>    lines with missing values."
>
> Really I am trying to separate the usage of loadtxt and genfromtxt to
> avoid unnecessary duplication and confusion. Part of this is
> historical because loadtxt was added in 2007 and genfromtxt was added
> in 2009. So really certain features of loadtxt have been  'kept' for
> backwards compatibility purposes yet these features can be 'abused' to
> handle missing data. But I really consider that any missing values
> should cause loadtxt to fail.

I agree with you Bruce, but it would be easier to discuss this on the
tickets instead of here. Could you add your comments there please?

Ralf


> The patch is incorrect because it should not include a space in the
> split() as indicated in the comment by the original reporter. Of
> course a corrected patch alone still is not sufficient to address the
> problem without the user providing the correct converter. Also you
> start to run into problems with multiple delimiters (such as one space
> versus two spaces) so you start down the path to add all the features
> that duplicate genfromtxt.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Bruce Southey]

On Wed, Mar 30, 2011 at 9:53 PM, Charles R Harris
 wrote:
>
>
> On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
>  wrote:
>>
>> On 26. mars 2011, at 21.44, Derek Homeier wrote:
>>
>> > Hi Paul,
>> >
>> > having had a look at the other tickets you dug up,
>> >
[snip]
>>
>> >> 1071:
>> >>      It is not clear to me whether loadtxt is supposed to support
>> >> missing values in the fashion indicated in the ticket.
>> >
>> > In principle it should at least allow you to, by the use of converters
>> > as described there.
>> > The problem is, the default delimiter is described as 'any
>> > whitespace', which in the
>> > present implementation obviously includes any number of blanks or
>> > tabs. These
>> > are therefore treated differently from delimiters like ',' or '&'. I'd
>> > reckon there are
>> > too many people actually relying on this behaviour to silently change it
>> > (e.g. I know plenty of tables with columns separated by either one or
>> > several
>> > tabs depending on the length of the previous entry). But the tab is
>> > apparently also
>> > treated differently if explicitly specified with "delimiter='\t'" -
>> > and in that case using
>> > a converter à la {2: lambda s: float(s or 'Nan')} is working for
>> > fields in the middle of
>> > the line, but not at the end - clearly warrants improvement. I've
>> > prepared a patch
>> > working for Python3 as well.
>>
>> Great!
>>
This is an invalid ticket because the docstring clearly states that in
3 different, yet critical places, that missing values are not handled
here:

"Each row in the text file must have the same number of values."
"genfromtxt : Load data with missing values handled as specified."
 "   This function aims to be a fast reader for simply formatted files.  The
`genfromtxt` function provides more sophisticated handling of, e.g.,
lines with missing values."

Really I am trying to separate the usage of loadtxt and genfromtxt to
avoid unnecessary duplication and confusion. Part of this is
historical because loadtxt was added in 2007 and genfromtxt was added
in 2009. So really certain features of loadtxt have been  'kept' for
backwards compatibility purposes yet these features can be 'abused' to
handle missing data. But I really consider that any missing values
should cause loadtxt to fail.

The patch is incorrect because it should not include a space in the
split() as indicated in the comment by the original reporter. Of
course a corrected patch alone still is not sufficient to address the
problem without the user providing the correct converter. Also you
start to run into problems with multiple delimiters (such as one space
versus two spaces) so you start down the path to add all the features
that duplicate genfromtxt.


Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Old tickets

[Ralf Gommers]

On Wed, Mar 30, 2011 at 11:52 PM, Derek Homeier
 wrote:
>
> On 30 Mar 2011, at 23:26, Benjamin Root wrote:
>
>> Ticket 301: 'Make power and divide return floats from int inputs (like
>> true_divide)'
>> http://projects.scipy.org/numpy/ticket/301
>> Invalid because the output dtype is the same as the input dtype unless
>> you override using the dtype argument:
>>  >>> np.power(3, 1, dtype=np.float128).dtype
>> dtype('float128')
>> Alternatively return a float and indicate in the docstring that the
>> output dtype can be changed.
>>
>>
>> FWIW,
>>
>> Just thought I'd note (on a python 2.6 system):
>>
>> >>> import numpy as np
>> >>> a = np.array([1, 2, 3, 4])
>> >>> a.dtype
>> dtype('int32')
>> >>> 2 / a
>> array([2, 1, 0, 0])
>> >>> from __future__ import division
>> >>> 2 / a
>> array([ 2.        ,  1.        ,  0.6667,  0.5       ])
>>
>> So, numpy already does this when true division is imported (and
>> therefore consistent with whatever the python environment does), and
>> python currently also returns integers for exponentials when both
>> inputs are integers.
>
> I'd agree, and in my view power(3, -1) is well defined as 1 / 3 -
> also, in future (or Python3)
>
>  >>> a/2
> array([ 0.5,  1. ,  1.5,  2. ])
>  >>> a//2
> array([0, 1, 1, 2], dtype=int32)

The ticket is about the functions np.divide and np.power, not / and
**. This currently does not work the same, unlike what's said above:

>>> from __future__ import division
>>> x = np.arange(4) + 1
>>> 2 / x
array([ 2.,  1.,  0.6667,  0.5   ])
>>> np.divide(2, x)
array([2, 1, 0, 0])

The power and divide functions should not check the "from __future__
import division", just return floats IMHO. This is what's expected by
most users, and much more useful than integer math.


> so I think at least a**n should follow integer math rules; depends on
> whether we want
> np.power to behave differently from ** (if they are internally handled
> separately at all)...
> Not sure if I understand the overload suggestion in the ticket

I don't understand that either, it'll just lead to more confusion.

Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] loadtxt/savetxt tickets

[Ralf Gommers]

On Thu, Mar 31, 2011 at 4:53 AM, Charles R Harris
 wrote:
>
>
> On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
>  wrote:
>>
>> On 26. mars 2011, at 21.44, Derek Homeier wrote:
>>
>> > Hi Paul,
>> >
>> > having had a look at the other tickets you dug up,
>> >
>> >> My opinions are my own, and in detail, they are:
>> >> 1752:
>> >>   I attach a possible patch. FWIW, I agree with the request. The
>> >> patch is written to be compatible with the fix in ticket #1562, but
>> >> I did not test that yet.
>> >
>> > Tested, see also my comments on Trac.
>>
>> Great!
>>
>> >> 1731:
>> >>   This seems like a rather trivial feature enhancement. I attach a
>> >> possible patch.
>> >
>> > Agreed. Haven't tested it though.
>>
>> Great!
>>
>> >> 1616:
>> >>   The suggested patch seems reasonable to me, but I do not have a
>> >> full list of what objects loadtxt supports today as opposed to what
>> >> this patch will support.
>>
>> Looks like you got this one. Just remember to make it compatible with
>> #1752. Should be easy.
>>
>> >> 1562:
>> >>   I attach a possible patch. This could also be the default
>> >> behavior to my mind, since the function caller can simply call
>> >> numpy.squeeze if needed. Changing default behavior would probably
>> >> break old code, however.
>> >
>> > See comments on Trac as well.
>>
>> Your patch is better, but there is one thing I disagree with.
>> 808    if X.ndim < ndmin:
>> 809        if ndmin == 1:
>> 810            X.shape = (X.size, )
>> 811        elif ndmin == 2:
>> 812            X.shape = (X.size, 1)
>> The last line should be:
>> 812            X.shape = (1, X.size)
>> If someone wants a 2D array out, they would most likely expect a one-row
>> file to come out as a one-row array, not the other way around. IMHO.
>>
>> >> 1458:
>> >>   The fix suggested in the ticket seems reasonable, but I have
>> >> never used record arrays, so I am not sure  of this.
>> >
>> > There were some issues with Python3, and I also had some general
>> > reservations
>> > as noted on Trac - basically, it makes 'unpack' equivalent to
>> > transposing for 2D-arrays,
>> > but to splitting into fields for 1D-recarrays. My question was, what's
>> > going to happen
>> > when you get to 2D-recarrays? Currently this is not an issue since
>> > loadtxt can only
>> > read 2D regular or 1D structured arrays. But this might change if the
>> > data block
>> > functionality (see below) were to be implemented - data could then be
>> > returned as
>> > 3D arrays or 2D structured arrays... Still, it would probably make
>> > most sense (or at
>> > least give the widest functionality) to have 'unpack=True' always
>> > return a list or iterator
>> > over columns.
>>
>> OK, I don't know recarrays, as I said.
>>
>> >> 1445:
>> >>   Adding this functionality could break old code, as some old
>> >> datafiles may have empty lines which are now simply ignored. I do
>> >> not think the feature is a good idea. It could rather be implemented
>> >> as a separate function.
>> >> 1107:
>> >>   I do not see the need for this enhancement. In my eyes, the
>> >> usecols kwarg does this and more. Perhaps I am misunderstanding
>> >> something here.
>> >
>> > Agree about #1445, and the bit about 'usecols' - 'numcols' would just
>> > provide a
>> > shorter call to e.g. read the first 20 columns of a file (well, not
>> > even that much
>> > over 'usecols=range(20)'...), don't think that justifies an extra
>> > argument.
>> > But the 'datablocks' provides something new, that a number of people
>> > seem
>> > to miss from e.g. gnuplot (including me, actually ;-). And it would
>> > also satisfy the
>> > request from #1445 without breaking backwards compatibility.
>> > I've been wondering if could instead specify the separator lines
>> > through the
>> > parameter, e.g. "blocksep=['None', 'blank','invalid']", not sure if
>> > that would make
>> > it more useful...
>>
>> What about writing a separate function, e.g. loadblocktxt, and have it
>> separate the chunks and call loadtxt for each chunk? Just a thought. Another
>> possibility would be to write a function that would let you load a set of
>> text files in a directory, and return a dict of datasets, one per file. One
>> could write a similar save-function, too. They would just need to call
>> loadtxt/savetxt on a per-file basis.
>>
>> >> 1071:
>> >>      It is not clear to me whether loadtxt is supposed to support
>> >> missing values in the fashion indicated in the ticket.
>> >
>> > In principle it should at least allow you to, by the use of converters
>> > as described there.
>> > The problem is, the default delimiter is described as 'any
>> > whitespace', which in the
>> > present implementation obviously includes any number of blanks or
>> > tabs. These
>> > are therefore treated differently from delimiters like ',' or '&'. I'd
>> > reckon there are
>> > too many people actually relying on this behaviour to silently change it
>> > (e.g. I know plenty of tables with columns separated by 

[Numpy-discussion] EuroPython 2011: call for paper is ending - Please spread the word

[Palla]

Hi all,
I'm Francesco and I am writing on behalf of "Python Italia APS", a no-profit
association promoting EuroPython conference. (www.europython.eu)

Europython End of Call for Presentations is April 6th. I'd like to ask to
you to forward this mail to anyone that you feel may be interested.

We're looking for proposals on every aspects of Python: programming from
novice to advanced levels, applications and frameworks, or how you have been
involved in introducing Python into your organisation.

**First-time speakers are especially welcome**; EuroPython is a community
conference and we are eager to hear about your experience. If you have
friends or colleagues who have something valuable to contribute, twist their
arms to tell us about it!


Presenting at EuroPython

We will accept a broad range of presentations, from reports on academic and
commercial projects to tutorials and case studies. As long as the
presentation is interesting and potentially useful to the Python community,
it will be considered for inclusion in the programme.

Can you show the conference-goers something new and useful? Can you show
attendees how to: use a module? Explore a Python language feature? Package
an application? If so, consider submitting a talk.


Talks and hands-on trainings

There are two different kind of presentations that you can give as a speaker
at EuroPython:

 * **Regular talk**. These are standard "talk with slides", allocated in
slots of 45, 60 or 90 minutes, depending on your preference and scheduling
constraints. A Q&A session is held at the end of the talk.
 * **Hands-on training**. These are advanced training sessions for a smaller
audience (10-20 people), to dive into the subject with all details. These
sessions are 4-hours long, and audience will be strongly encouraged to bring
a laptop to experiment. They should be prepared with less slides and more
source code. If possible, trainers will also give a short "teaser talk" of
30 minutes the day before the training, to tease delegates into attending
the training.

In the talk submission form, we assume that you intend to give a regular
talk on the subject, but you will be asked if you are available for also
doing a hands-on training on the very same subject.

Speakers that will give a hands-on training are rewarded with a **free
entrance** to EuroPython to compensate for the longer preparation required,
and might also be eligible for a speaking fee (which we cannot confirm at
the moment).

Topics and goals

Specific topics for EuroPython presentations include, but are not limited
to:

- Core Python
- Other implementations: Jython, IronPython, PyPy, and Stackless
- Python libraries and extensions
- Python 3.x migration
- Databases
- Documentation
- GUI Programming
- Game Programming
- Network Programming
- Open Source Python projects
- Packaging Issues
- Programming Tools
- Project Best Practices
- Embedding and Extending
- Science and Math
- Web-based Systems

Presentation goals usually are some of the following:

- Introduce audience to a new topic they are unaware of
- Introduce audience to new developments on a well-known topic
- Show audience real-world usage scenarios for a specific topic (case study)
- Dig into advanced and relatively-unknown details on a topic
- Compare different options in the market on a topic


Community-based talk voting
---
This year, for the first time in EuroPython history, the talk voting process
is fully public. Every partecipant gains the right to vote for talks
submitted during the Call For Papers, as soon as they commit to their
presence at the conference by buying a ticket. See all the details in the
talk voting[1] page.

Contacts

For any further question, feel free to contact the organizers at
i...@pycon.it. Thank you!

[1]: http://ep2011.europython.eu/talk-voting


-- 
->PALLA
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1

[Scott Sinclair]

On 31 March 2011 12:18, Pearu Peterson  wrote:
>
>
> On Thu, Mar 31, 2011 at 1:00 PM, Scott Sinclair
>  wrote:
>>
>> For what it's worth, the maintenance/1.6.x branch works for me on 64-bit
>> Ubuntu:
>>
>> (numpy-1.6.x)scott@godzilla:~$ python -c "import numpy; numpy.test()"
>
> You might want to run
>
>  python -c "import numpy; numpy.test('full')"
>
> as the corresponding test is decorated as slow.

python -c "import numpy; numpy.test('full')"
Running unit tests for numpy
NumPy version 1.6.0b2.dev-a172fd6
NumPy is installed in
/home/scott/.virtualenvs/numpy-1.6.x/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
nose version 1.0.0



Ran 3423 tests in 28.713s

OK (KNOWNFAIL=3, SKIP=4)

Cheers,
Scott
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.histogramdd of empty data

[Nils Becker]

Hi Ralf,

I cloned numpy/master and played around a little.

when giving the bins explicitely, now histogram2d and histogramdd work
as expected in all tests i tried.


However, some of the cases with missing bin specification appear
somewhat inconsistent.

The first question is if creating arbitrary bins for empty data and
empty bin specification is better than raising an Exception:

Specifically:

numpy.histogram2d([],[],bins=[0,0])
> (array([ 0.,  0.]), array([ 0.]), array([ 0.]))

numpy.histogram([],bins=0)
> ValueError: zero-size array to minimum.reduce without identity

so 1-d and 2-d behave not quite the same.

also, these work (although with arbitrary bin edges):

numpy.histogram2d([],[],bins=[1,1])
> (array([ 0.,  0.]), array([ 0.,  1.]), array([ 0.,  1.]))

numpy.histogram2d([],[],bins=[0,1])
> (array([ 0.,  0.]), array([ 0.]), array([ 0.,  1.]))

while this raises an error:

numpy.histogram([],bins=1)
> ValueError: zero-size array to minimum.reduce without identity

another thing with non-empty data:

numpy.histogram([1],bins=1)
> (array([1]), array([ 0.5,  1.5]))

numpy.histogram([1],bins=0)
> (array([], dtype=int64), array([ 0.5]))

while

numpy.histogram2d([1],[1],bins=A)
> ValueError: zero-size array to minimum.reduce without identity

(here A==[0,0] or A==[0,1] but not A==[1,1] which gives a result)

Nils
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1

[Pearu Peterson]

On Thu, Mar 31, 2011 at 1:00 PM, Scott Sinclair  wrote:

> On 31 March 2011 11:37, Pearu Peterson  wrote:
> >
> >
> > On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau 
> > wrote:
> >>
> >> On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen  wrote:
> >> > In article
> >> > ,
> >> >  Ralf Gommers  wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> I am pleased to announce the availability of the first beta of NumPy
> >> >> 1.6.0. Due to the extensive changes in the Numpy core for this
> >> >> release, the beta testing phase will last at least one month. Please
> >> >> test this beta and report any problems on the Numpy mailing list.
> >> >>
> >> >> Sources and binaries can be found at:
> >> >> http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/
> >> >> For (preliminary) release notes see below.
> >>
> >> I see a segfault on Ubuntu 64 bits for the test
> >> TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py.
> >> Am I the only one seeing it ?
> >>
> >
> > The test work here ok on Ubuntu 64 with numpy master. Could you try the
> > maintenance/1.6.x branch where the related bugs are fixed.
>
> For what it's worth, the maintenance/1.6.x branch works for me on 64-bit
> Ubuntu:
>
> (numpy-1.6.x)scott@godzilla:~$ python -c "import numpy; numpy.test()"
>

You might want to run

 python -c "import numpy; numpy.test('full')"

as the corresponding test is decorated as slow.

Pearu
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1

[Scott Sinclair]

On 31 March 2011 11:37, Pearu Peterson  wrote:
>
>
> On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau 
> wrote:
>>
>> On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen  wrote:
>> > In article
>> > ,
>> >  Ralf Gommers  wrote:
>> >
>> >> Hi,
>> >>
>> >> I am pleased to announce the availability of the first beta of NumPy
>> >> 1.6.0. Due to the extensive changes in the Numpy core for this
>> >> release, the beta testing phase will last at least one month. Please
>> >> test this beta and report any problems on the Numpy mailing list.
>> >>
>> >> Sources and binaries can be found at:
>> >> http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/
>> >> For (preliminary) release notes see below.
>>
>> I see a segfault on Ubuntu 64 bits for the test
>> TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py.
>> Am I the only one seeing it ?
>>
>
> The test work here ok on Ubuntu 64 with numpy master. Could you try the
> maintenance/1.6.x branch where the related bugs are fixed.

For what it's worth, the maintenance/1.6.x branch works for me on 64-bit Ubuntu:

(numpy-1.6.x)scott@godzilla:~$ python -c "import numpy; numpy.test()"
Running unit tests for numpy
NumPy version 1.6.0b2.dev-a172fd6
NumPy is installed in
/home/scott/.virtualenvs/numpy-1.6.x/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
nose version 1.0.0

--
Ran 3406 tests in 16.889s

OK (KNOWNFAIL=3, SKIP=4)

Cheers,
Scott
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1

[Pearu Peterson]

On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau wrote:

> On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen  wrote:
> > In article
> > ,
> >  Ralf Gommers  wrote:
> >
> >> Hi,
> >>
> >> I am pleased to announce the availability of the first beta of NumPy
> >> 1.6.0. Due to the extensive changes in the Numpy core for this
> >> release, the beta testing phase will last at least one month. Please
> >> test this beta and report any problems on the Numpy mailing list.
> >>
> >> Sources and binaries can be found at:
> >> http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/
> >> For (preliminary) release notes see below.
>
> I see a segfault on Ubuntu 64 bits for the test
> TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py.
> Am I the only one seeing it ?
>
>
The test work here ok on Ubuntu 64 with numpy master. Could you try the
maintenance/1.6.x branch where the related bugs are fixed.

Pearu
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] random number genration

[Alex Ter-Sarkissov]

thanks Sturl, your second code works well although I had to divide the
vector elements through 128 to get just the binaries
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.histogram on arrays.

[eat]

Hi,

On Wed, Mar 30, 2011 at 1:44 PM, Éric Depagne  wrote:

> >
> > Well I guess, for a slight performance improvement, you could create your
> > own streamlined histogrammer.
> >
> > But, in order to better grasp your situation it would be beneficial to
> know
> > how the counts and bounds are used later on. Just wondering if this kind
> > massive histogramming could be somehow avoided totally.
> Indeed.
> Here's what I do.
> My images come from CCD, and as such, the zero level in the image is not
> the
> true zero level, but is the true zero + the background noise of each
> pixels.
> By doing the histogram, I plan on detecting what is the most common value
> per
> row.
> Once I have the most common value, I can derive the interval where most of
> the
> values are (the index of the largest occurence is easily obtained by
> sorting
> the counts, and I take a slice [index_max_count,index_max_count+1] in the
> second array given by the histogram).
> Then, I  take the mean value of this interval and I assume it is the value
> of
> the bias for my row.
>
> I do this procedure both on the row and columns as a sanity check.
> And I know this procedure will not work if on any row/column there is a lot
> of
> signal and very little bias. I'll fix that afterwards ;-)
>
Perhaps something along these lines will help you:
from numpy import histogram, linspace, r_

def hist2(a, nob):
bins= linspace(a.min(), a.max(), nob+ 1)
d= linspace(0, bins[-1]* a.shape[0], a.shape[0])[:, None]
b= (a+ d).ravel()
bbins= (bins[:-1]+ d).ravel()
bbins= r_[bbins, bbins[-1]+ 1]
counts, _= histogram(b, bbins)
return counts.reshape(-1, nob), bins

It has two disadvantages 1) needs more memory and 2) "global" bins
(which although should be quite straightforward to enhance if needed).

Regards,
eat

>
> Éric.
>
>
> >
> > Regards,
> > eat
> >
>
> Un clavier azerty en vaut deux
> --
> Éric Depagnee...@depagne.org
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.6.0 beta 1

[David Cournapeau]

On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen  wrote:
> In article
> ,
>  Ralf Gommers  wrote:
>
>> Hi,
>>
>> I am pleased to announce the availability of the first beta of NumPy
>> 1.6.0. Due to the extensive changes in the Numpy core for this
>> release, the beta testing phase will last at least one month. Please
>> test this beta and report any problems on the Numpy mailing list.
>>
>> Sources and binaries can be found at:
>> http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/
>> For (preliminary) release notes see below.

I see a segfault on Ubuntu 64 bits for the test
TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py.
Am I the only one seeing it ?

David
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Willing to contribute to SciPy & NumPy ...

[Scott Sinclair]

On 31 March 2011 07:27, Sylvain Bellemare  wrote:
>  I would like to seriously start contributing to NumPy and/or SciPy, as much 
> as I possibly can.

I'm sure that your help will be welcomed!

A good place to get started is helping out with documentation (see
http://docs.scipy.org/numpy/Front%20Page/). SciPy has plenty of work
required - you'll probably learn your way into the code that way.
Another place to look is http://www.scipy.org/Developer_Zone. It's
worthwhile learning how to work with Git and Github if you want to get
patches accepted more easily.

Cheers,
Scott
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion