[Numpy-discussion] Windows build/distribute plan & MingwPy funding

[Ralf Gommers]

Hi all,

You probably know that building Numpy, Scipy and the rest of the Scipy
Stack on Windows is problematic. And that there are plans to adopt the
static MinGW-w64 based toolchain that Carl Kleffner has done a lot of work
on for the last two years to fix that situation.

The good news is: this has become a lot more concrete just now, with this
proposal for funding: http://mingwpy.github.io/proposal_december2015.html

Funding for phases 1 and 2 is already confirmed; the phase 3 part has been
submitted to the PSF. Phase 1 (of $1000) is funded by donations made to
Numpy and Scipy (through NumFOCUS), and phase 2 (of $4000) by NumFOCUS
directly. So a big thank you to everyone who made a donation to Numpy,
Scipy and NumFOCUS!

I hope that that proposal gives a clear idea of the work that's going to be
done over the next months. Note that the http://mingwpy.github.io contains
a lot more background info, description of technical issues, etc.

Feedback & ideas very welcome of course!

Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy funding update

[Ralf Gommers]

On Fri, Jan 1, 2016 at 12:17 AM, Robert Kern  wrote:

> On Wed, Dec 30, 2015 at 10:54 AM, Ralf Gommers 
> wrote:
> >
> > Hi all,
> >
> > A quick good news message: OSDC has made a $5k contribution to NumFOCUS,
> which is split between support for a women in technology workshop and
> support for Numpy:
> http://www.numfocus.org/blog/osdc-donates-5k-to-support-numpy-women-in-tech
> > This was a very nice surprise to me, and a first sign that the FSA
> (fiscal sponsorship agreement) we recently signed with NumFOCUS is going to
> yield significant benefits for Numpy.
> >
> > NumFOCUS is also doing a special end-of-year fundraiser. Funds donated
> (up to $5k) will be tripled by anonymous sponsors:
> http://www.numfocus.org/blog/numfocus-end-of-year-fundraising-drive-5000-matching-gift-challenge
> > So think of Numpy (or your other favorite NumFOCUS-sponsored project of
> course) if you're considering a holiday season charitable gift!
>
> That sounds great! Do we have any concrete plans for spending that money,
> yet?
>

There are:
  1. developer meetings
  2. MingwPy
  3. TBD - Numpy code/infrastructure work

Some initial plans were discussed and documented during the first Numpy
developer meeting at Scipy'15. See wiki page [1] and meeting minutes [2].

Funding for facilitating more developer meetings (~1 a year) is definitely
on the radar. Plans for funding development of new features or other
code/infrastructure work are still fuzzy (except for MingwPy, see my email
of today on that).

We need to transfer the relevant part of those meeting minutes [2] into the
docs, and expand upon those to get a clear roadmap online that we can build
on. This is work for the next months I think.

Cheers,
Ralf

[1] https://github.com/numpy/numpy/wiki/SciPy-2015-developer-meeting
[2]
https://docs.google.com/document/d/1IJcYdsHtk8MVAM4AZqFDBSf_nVG-mrB4Tv2bh9u1g4Y/edit?usp=sharing
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal for a new function: np.moveaxis

[Sebastian Berg]

Just a heads up, I am planning to put in Stephans pull request (more
info, see original mail below) as soon as some minor things are
cleared. So if you have any objections or better ideas for the name,
now is the time.

- Sebastian


On Mi, 2015-11-04 at 23:42 -0800, Stephan Hoyer wrote:
> I've put up a pull request implementing a new function, np.moveaxis,
> as an alternative to np.transpose and np.rollaxis:
> https://github.com/numpy/numpy/pull/6630
> 
> This functionality has been discussed (even the exact function name)
> several times over the years, but it never made it into a pull
> request. The most pressing issue is that the behavior of np.rollaxis
> is not intuitive to most users:
> https://mail.scipy.org/pipermail/numpy-discussion/2010-September/0528
> 82.html
> https://github.com/numpy/numpy/issues/2039
> http://stackoverflow.com/questions/29891583/reason-why-numpy-rollaxis
> -is-so-confusing
> 
> In this pull request, I also allow the source and destination axes to
> be sequences as well as scalars. This does not add much complexity to
> the code, solves some additional use cases and makes np.moveaxis a
> proper generalization of the other axes manipulation routines (see
> the pull requests for details).
> 
> Best of all, it already works on ndarray duck types (like masked
> array and dask.array), because they have already implemented
> transpose.
> 
> I think np.moveaxis would be a useful addition to NumPy -- I've found
> myself writing helper functions with a subset of its functionality
> several times over the past few years. What do you think?
> 
> Cheers,
> Stephan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.10.3 release.

[Charles R Harris]

On Sat, Jan 2, 2016 at 4:51 PM, Matthew Brett 
wrote:

> Hi,
>
> On Sat, Jan 2, 2016 at 9:47 PM, Charles R Harris
>  wrote:
> > Hi All,
> >
> > A significant segfault problem has been reported against Numpy 1.10.2
> and I
> > want to make a quick 1.10.3 release to get it fixed. Two questions
> >
> > What exactly is the release process that has been decided on? AFAIK, I
> > should just do a source release on Sourceforge, ping Matthew to produce
> > wheels for Mac and wait for him to put them on pypi, and then upload the
> > sources to pypi.
>
> You're welcome to ping me to do the Mac wheels, but I'd be even
> happier if you could have a go at triggering the build, to see if you
> can make it work:
>
> https://github.com/MacPython/numpy-wheels


Hmm... Since it needs to build off the tag to have the correct behavior, I
assume that that needs to be set explicitly. Also, it didn't look as if
your last pypi uploads were signed. Did you upload with twine?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Numpy 1.10.3 release.

[Charles R Harris]

Hi All,

A significant segfault problem has been reported against Numpy 1.10.2 and I
want to make a quick 1.10.3 release to get it fixed. Two questions


   - What exactly is the release process that has been decided on? AFAIK, I
   should just do a source release on Sourceforge, ping Matthew to produce
   wheels for Mac and wait for him to put them on pypi, and then upload the
   sources to pypi. No windows binaries are to be produced.
   - Is there anything else that needs fixing for 1.10.3?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.10.3 release.

[Matthew Brett]

Hi,

On Sat, Jan 2, 2016 at 9:47 PM, Charles R Harris
 wrote:
> Hi All,
>
> A significant segfault problem has been reported against Numpy 1.10.2 and I
> want to make a quick 1.10.3 release to get it fixed. Two questions
>
> What exactly is the release process that has been decided on? AFAIK, I
> should just do a source release on Sourceforge, ping Matthew to produce
> wheels for Mac and wait for him to put them on pypi, and then upload the
> sources to pypi.

You're welcome to ping me to do the Mac wheels, but I'd be even
happier if you could have a go at triggering the build, to see if you
can make it work:

https://github.com/MacPython/numpy-wheels

Cheers,

Mattthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.10.3 release.

[Matthew Brett]

On Sun, Jan 3, 2016 at 12:01 AM, Charles R Harris
 wrote:
>
>
> On Sat, Jan 2, 2016 at 4:51 PM, Matthew Brett 
> wrote:
>>
>> Hi,
>>
>> On Sat, Jan 2, 2016 at 9:47 PM, Charles R Harris
>>  wrote:
>> > Hi All,
>> >
>> > A significant segfault problem has been reported against Numpy 1.10.2
>> > and I
>> > want to make a quick 1.10.3 release to get it fixed. Two questions
>> >
>> > What exactly is the release process that has been decided on? AFAIK, I
>> > should just do a source release on Sourceforge, ping Matthew to produce
>> > wheels for Mac and wait for him to put them on pypi, and then upload the
>> > sources to pypi.
>>
>> You're welcome to ping me to do the Mac wheels, but I'd be even
>> happier if you could have a go at triggering the build, to see if you
>> can make it work:
>>
>> https://github.com/MacPython/numpy-wheels
>
>
> Hmm... Since it needs to build off the tag to have the correct behavior, I
> assume that that needs to be set explicitly.

No, the build system uses a script I wrote to find the tag closest in
development history to master, by default - see :
http://stackoverflow.com/a/24557377/1939576

So, if you push a tag, then trigger a build, there's a good chance
that will do what you want.  If not, then you can set the tag or
commit explicitly at :

https://github.com/MacPython/numpy-wheels/blob/master/.travis.yml#L5

> Also, it didn't look as if your
> last pypi uploads were signed. Did you upload with twine?

I did yes - with a custom uploader script [1], but I guess it would be
better to download, sign, then upload again using some other mechanism
that preserves the signature.

Cheers,

Matthew

[1] https://github.com/MacPython/terryfy/blob/master/wheel-uploader
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.10.3 release.

[josef.pktd]

On Sat, Jan 2, 2016 at 4:47 PM, Charles R Harris
 wrote:
> Hi All,
>
> A significant segfault problem has been reported against Numpy 1.10.2 and I
> want to make a quick 1.10.3 release to get it fixed. Two questions
>
> What exactly is the release process that has been decided on? AFAIK, I
> should just do a source release on Sourceforge, ping Matthew to produce
> wheels for Mac and wait for him to put them on pypi, and then upload the
> sources to pypi. No windows binaries are to be produced.
> Is there anything else that needs fixing for 1.10.3?


I'm running the 1.10.2 tests on Windows 10 in a virtualbox on Windows 8.1
using Gohlke binary for MKL on a fresh Python 3.5

This test
"Test workarounds for 32-bit limited fwrite, fseek, and ftell ..."
is taking a very long time. Is this expected?


I get the following errors on Windows 10, and also on Windows 8.1
Winpython 3.4 (except for the last "Unable to find vcvarsall.bat"
because it's set up for compiling with mingw)

Earlier I also got a ref count error message but I don't see it
anymore, so maybe I messed up when trying Ctrl+C to kill the tests.


==
ERROR: Failure: ImportError (cannot import name 'fib2')
--
Traceback (most recent call last):
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\failure.py",
line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\loader.py",
line 418, in loadTestsFromName
addr.filename, addr.module)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\importer.py",
line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\importer.py",
line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
  File "c:\users\josef\appdata\local\programs\python\python35\lib\imp.py",
line 234, in load_module
return load_source(name, filename, file)
  File "c:\users\josef\appdata\local\programs\python\python35\lib\imp.py",
line 172, in load_source
module = _load(spec)
  File "", line 693, in _load
  File "", line 673, in _load_unlocked
  File "", line 662, in exec_module
  File "", line 222, in _call_with_frames_removed
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\numpy\distutils\tests\f2py_ext\tests\test_fib2.py",
line 4, in 
from f2py_ext import fib2
ImportError: cannot import name 'fib2'

==
ERROR: Failure: ImportError (cannot import name 'foo')
--
Traceback (most recent call last):
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\failure.py",
line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\loader.py",
line 418, in loadTestsFromName
addr.filename, addr.module)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\importer.py",
line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\importer.py",
line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
  File "c:\users\josef\appdata\local\programs\python\python35\lib\imp.py",
line 234, in load_module
return load_source(name, filename, file)
  File "c:\users\josef\appdata\local\programs\python\python35\lib\imp.py",
line 172, in load_source
module = _load(spec)
  File "", line 693, in _load
  File "", line 673, in _load_unlocked
  File "", line 662, in exec_module
  File "", line 222, in _call_with_frames_removed
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\numpy\distutils\tests\f2py_f90_ext\tests\test_foo.py",
line 4, in 
from f2py_f90_ext import foo
ImportError: cannot import name 'foo'

==
ERROR: Failure: ImportError (cannot import name 'fib3')
--
Traceback (most recent call last):
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\failure.py",
line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\loader.py",
line 418, in loadTestsFromName
addr.filename, addr.module)
  File 
"c:\users\josef\appdata\local\programs\python\python35\lib\site-packages\nose\importer.py",
line 47, in importFromPath
return self.importFromDir(dir_path, fqname)