[Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Marten van Kerkwijk
Hi All,

I wondered if the move to python3-only starting with numpy 1.17 would
be a good reason to act on what we all seem to agree: that the matrix
class was a bad idea, with its overriding of multiplication and lack
of support for stacks of matrices. For 1.17, minimum python supposedly
is >=3.5, so we will be guaranteed to have the matrix multiply
operator @ available, and hence there is arguably even less of a case
for keeping the matrix class; removing it would allow taking out quite
a bit of accumulated special-casing (the immediate reasons for writing
this were gh-10123 and 10132).

What do people think? If we do go in this direction, we might want to
add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
python3; for python2 matrix would never disappear).

All the best,

Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Todd
On Nov 30, 2017 09:24, "Marten van Kerkwijk" 
wrote:

Hi All,

I wondered if the move to python3-only starting with numpy 1.17 would
be a good reason to act on what we all seem to agree: that the matrix
class was a bad idea, with its overriding of multiplication and lack
of support for stacks of matrices. For 1.17, minimum python supposedly
is >=3.5, so we will be guaranteed to have the matrix multiply
operator @ available, and hence there is arguably even less of a case
for keeping the matrix class; removing it would allow taking out quite
a bit of accumulated special-casing (the immediate reasons for writing
this were gh-10123 and 10132).

What do people think? If we do go in this direction, we might want to
add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
python3; for python2 matrix would never disappear).

All the best,

Marten


I still think moving it out into its own package would be better, making it
clear that anyone who cares about the class should step up because numpy
developers will not do any additional work on it. Similar to how weave was
handled with scipy.

So simultaneous with the deprecation you release a package with the matrix
class. Then people have until the deprecation period is over to port (which
should just be a matter of changing the imports).
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Chris Laumann
+1 (not that my lurking vote should necessarily carry much weight). Rip it out 
asap.

The existence of the matrix class has been literally the single biggest source 
of confusion and subtle bugs in my and my students' codes for years. 

Best, Chris

> On Nov 30, 2017, at 9:23 AM, Marten van Kerkwijk  
> wrote:
> 
> Hi All,
> 
> I wondered if the move to python3-only starting with numpy 1.17 would
> be a good reason to act on what we all seem to agree: that the matrix
> class was a bad idea, with its overriding of multiplication and lack
> of support for stacks of matrices. For 1.17, minimum python supposedly
> is >=3.5, so we will be guaranteed to have the matrix multiply
> operator @ available, and hence there is arguably even less of a case
> for keeping the matrix class; removing it would allow taking out quite
> a bit of accumulated special-casing (the immediate reasons for writing
> this were gh-10123 and 10132).
> 
> What do people think? If we do go in this direction, we might want to
> add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
> python3; for python2 matrix would never disappear).
> 
> All the best,
> 
> Marten
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Bryan Van de ven
This is exactly what we did with the bokeh.charts deprecation. Moving to a 
separate projects was both a huge relief for the developers as well as a great 
way to focus and clarify expectations for users. 

Bryan  

> On Nov 30, 2017, at 10:20, Chris Laumann  wrote:
> 
> +1 (not that my lurking vote should necessarily carry much weight). Rip it 
> out asap.
> 
> The existence of the matrix class has been literally the single biggest 
> source of confusion and subtle bugs in my and my students' codes for years. 
> 
> Best, Chris
> 
>> On Nov 30, 2017, at 9:23 AM, Marten van Kerkwijk  
>> wrote:
>> 
>> Hi All,
>> 
>> I wondered if the move to python3-only starting with numpy 1.17 would
>> be a good reason to act on what we all seem to agree: that the matrix
>> class was a bad idea, with its overriding of multiplication and lack
>> of support for stacks of matrices. For 1.17, minimum python supposedly
>> is >=3.5, so we will be guaranteed to have the matrix multiply
>> operator @ available, and hence there is arguably even less of a case
>> for keeping the matrix class; removing it would allow taking out quite
>> a bit of accumulated special-casing (the immediate reasons for writing
>> this were gh-10123 and 10132).
>> 
>> What do people think? If we do go in this direction, we might want to
>> add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
>> python3; for python2 matrix would never disappear).
>> 
>> All the best,
>> 
>> Marten
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Marten van Kerkwijk
Moving to a subpackage may indeed make more sense, though it might not
help as much with getting rid of the hacks inside other parts of numpy
to keep matrix working. In that respect it seems a bit different at
least from weave.

Then again, independently of whether we remove or release a separate
package, it is probably best to start by moving all tests involving
matrix to matrixlib/tests, so we can at least get a sense of what
hacks are actually present.

-- Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Ilhan Polat
This would be really good to remove the apparent confusion. Moreover, I
think cleanly explaining why using "np.matrix" is not a good idea *before*
announcing the news would encourage people to accept this decision along
the way. That would greatly reduce the sporadic "the devs are deprecating
stuff as they see fit without asking us" sentiment.

On Thu, Nov 30, 2017 at 6:00 PM, Marten van Kerkwijk <
m.h.vankerkw...@gmail.com> wrote:

> Moving to a subpackage may indeed make more sense, though it might not
> help as much with getting rid of the hacks inside other parts of numpy
> to keep matrix working. In that respect it seems a bit different at
> least from weave.
>
> Then again, independently of whether we remove or release a separate
> package, it is probably best to start by moving all tests involving
> matrix to matrixlib/tests, so we can at least get a sense of what
> hacks are actually present.
>
> -- Marten
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread josef . pktd
On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk <
m.h.vankerkw...@gmail.com> wrote:

> Hi All,
>
> I wondered if the move to python3-only starting with numpy 1.17 would
> be a good reason to act on what we all seem to agree: that the matrix
> class was a bad idea, with its overriding of multiplication and lack
> of support for stacks of matrices.


I don't think the matrix class was a bad idea at the time.

numpy was the underdog, I came from GAUSS and Matlab and numpy
arrays were just weird, especially loosing a dimension all the time
and the heavy required use of np.newaxis.
I guess nowadays kids don't learn `matrix` languages first anymore.

recarrays are another half-hearted feature in numpy that is mostly
obsolete with pandas and pandas_like DataFrames in other
packages.


(I don't mind the changes, but the deprecation cycle is often short,
especially for users like me that update numpy only about every 3 main
versions.)

Josef


> For 1.17, minimum python supposedly
> is >=3.5, so we will be guaranteed to have the matrix multiply
> operator @ available, and hence there is arguably even less of a case
> for keeping the matrix class; removing it would allow taking out quite
> a bit of accumulated special-casing (the immediate reasons for writing
> this were gh-10123 and 10132).
>
> What do people think? If we do go in this direction, we might want to
> add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
> python3; for python2 matrix would never disappear).
>
All the best,
>
> Marten
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Matthew Rocklin
How would the community handle the scipy.sparse matrix subclasses?  These
are still in common use.

Somewhat related: https://github.com/scipy/scipy/issues/8162

On Thu, Nov 30, 2017 at 1:13 PM,  wrote:

>
>
> On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk <
> m.h.vankerkw...@gmail.com> wrote:
>
>> Hi All,
>>
>> I wondered if the move to python3-only starting with numpy 1.17 would
>> be a good reason to act on what we all seem to agree: that the matrix
>> class was a bad idea, with its overriding of multiplication and lack
>> of support for stacks of matrices.
>
>
> I don't think the matrix class was a bad idea at the time.
>
> numpy was the underdog, I came from GAUSS and Matlab and numpy
> arrays were just weird, especially loosing a dimension all the time
> and the heavy required use of np.newaxis.
> I guess nowadays kids don't learn `matrix` languages first anymore.
>
> recarrays are another half-hearted feature in numpy that is mostly
> obsolete with pandas and pandas_like DataFrames in other
> packages.
>
>
> (I don't mind the changes, but the deprecation cycle is often short,
> especially for users like me that update numpy only about every 3 main
> versions.)
>
> Josef
>
>
>> For 1.17, minimum python supposedly
>> is >=3.5, so we will be guaranteed to have the matrix multiply
>> operator @ available, and hence there is arguably even less of a case
>> for keeping the matrix class; removing it would allow taking out quite
>> a bit of accumulated special-casing (the immediate reasons for writing
>> this were gh-10123 and 10132).
>>
>> What do people think? If we do go in this direction, we might want to
>> add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
>> python3; for python2 matrix would never disappear).
>>
> All the best,
>>
>> Marten
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Ralf Gommers
On Fri, Dec 1, 2017 at 7:17 AM, Matthew Rocklin  wrote:

> How would the community handle the scipy.sparse matrix subclasses?  These
> are still in common use.
>

They're not going anywhere for quite a while (until the sparse ndarrays
materialize at least). Hence np.matrix needs to be moved, not deleted. We
discussed this earlier this year:
https://mail.python.org/pipermail/numpy-discussion/2017-January/076332.html


> Somewhat related: https://github.com/scipy/scipy/issues/8162
>
> On Thu, Nov 30, 2017 at 1:13 PM,  wrote:
>
>>
>>
>> On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk <
>> m.h.vankerkw...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I wondered if the move to python3-only starting with numpy 1.17 would
>>> be a good reason to act on what we all seem to agree: that the matrix
>>> class was a bad idea, with its overriding of multiplication and lack
>>> of support for stacks of matrices.
>>
>>
I'd suggest any release in the next couple of years is fine,but the one
where we drop Python 2 support is probably the worst choice. That's one of
the few things the core Python devs got 100% right with the Python 3 move:
advocate that in the 2->3 transition packages would not make any API
changes in order to make porting the least painful.

Ralf



>
>> I don't think the matrix class was a bad idea at the time.
>>
>> numpy was the underdog, I came from GAUSS and Matlab and numpy
>> arrays were just weird, especially loosing a dimension all the time
>> and the heavy required use of np.newaxis.
>> I guess nowadays kids don't learn `matrix` languages first anymore.
>>
>> recarrays are another half-hearted feature in numpy that is mostly
>> obsolete with pandas and pandas_like DataFrames in other
>> packages.
>>
>>
>> (I don't mind the changes, but the deprecation cycle is often short,
>> especially for users like me that update numpy only about every 3 main
>> versions.)
>>
>> Josef
>>
>>
>>> For 1.17, minimum python supposedly
>>> is >=3.5, so we will be guaranteed to have the matrix multiply
>>> operator @ available, and hence there is arguably even less of a case
>>> for keeping the matrix class; removing it would allow taking out quite
>>> a bit of accumulated special-casing (the immediate reasons for writing
>>> this were gh-10123 and 10132).
>>>
>>> What do people think? If we do go in this direction, we might want to
>>> add PendingDeprecationWarning for 1.15 (maybe DeprecationWarning for
>>> python3; for python2 matrix would never disappear).
>>>
>> All the best,
>>>
>>> Marten
>>> ___
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>>
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Charles R Harris
On Thu, Nov 30, 2017 at 11:43 AM, Ralf Gommers 
wrote:

>
>
> On Fri, Dec 1, 2017 at 7:17 AM, Matthew Rocklin 
> wrote:
>
>> How would the community handle the scipy.sparse matrix subclasses?  These
>> are still in common use.
>>
>
> They're not going anywhere for quite a while (until the sparse ndarrays
> materialize at least). Hence np.matrix needs to be moved, not deleted. We
> discussed this earlier this year: https://mail.python.org/
> pipermail/numpy-discussion/2017-January/076332.html
>
>
>> Somewhat related: https://github.com/scipy/scipy/issues/8162
>>
>> On Thu, Nov 30, 2017 at 1:13 PM,  wrote:
>>
>>>
>>>
>>> On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk <
>>> m.h.vankerkw...@gmail.com> wrote:
>>>
 Hi All,

 I wondered if the move to python3-only starting with numpy 1.17 would
 be a good reason to act on what we all seem to agree: that the matrix
 class was a bad idea, with its overriding of multiplication and lack
 of support for stacks of matrices.
>>>
>>>
> I'd suggest any release in the next couple of years is fine,but the one
> where we drop Python 2 support is probably the worst choice. That's one of
> the few things the core Python devs got 100% right with the Python 3 move:
> advocate that in the 2->3 transition packages would not make any API
> changes in order to make porting the least painful.
>
> Ralf
>

Agree, we don't want to pile in too many changes at once. I think the big
sticking point is the sparse matrices in SciPy, even issuing a
DeprecationWarning could be problematic as long as there are sparse
matrices. May I suggest that we put together an NEP for the NumPy side of
things? Ralf, does SciPy have a mechanism for proposing such changes?



Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Stefan van der Walt
On Thu, Nov 30, 2017, at 10:13, josef.p...@gmail.com wrote:
> recarrays are another half-hearted feature in numpy that is mostly
> obsolete with pandas and pandas_like DataFrames in other
> packages.

I'm fully on board with factoring out np.matrix into a subpackage.
But I would not touch structured arrays; they are quite useful,
and sometimes perform surprisingly well compared to the other
solutions around.
Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Stefan van der Walt
On Thu, Nov 30, 2017, at 11:39, Charles R Harris wrote:
> Agree, we don't want to pile in too many changes at once. I think the big 
> sticking point is the sparse matrices in SciPy, even issuing a 
> DeprecationWarning could be problematic as long as there are sparse matrices.

Could you explain what you mean by SciPy sparse matrices being a big
sticking point?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Marten van Kerkwijk
Hi Ralf,

Sorry not to have recalled the previous thread.

Your point about not doing things in the python 2->3 move makes sense;
handy for me is no reason to give users an incentive not to move to
python3 because their matrix-dependent code breaks.

It does sound like, given the use of sparse, a separate package - or
perhaps (temporary) inclusion in scipy - would be the way to go. In
turn, collecting as much of the matrix tests and work-arounds together
in the `matrixlib` would be the right first step.

And, even better, to collect thoughts in a NEP.  Now if only I had not
written this while procrastinating on other things...

All the best,

Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Marten van Kerkwijk
On Thu, Nov 30, 2017 at 2:51 PM, Stefan van der Walt
 wrote:
> On Thu, Nov 30, 2017, at 10:13, josef.p...@gmail.com wrote:
>
> recarrays are another half-hearted feature in numpy that is mostly
> obsolete with pandas and pandas_like DataFrames in other
> packages.
>
>
> I'm fully on board with factoring out np.matrix into a subpackage.  But I
> would not touch structured arrays; they are quite useful, and sometimes
> perform surprisingly well compared to the other solutions around.

I think Josef specifically meant `recarrays`, which give access to
elements of a structured array via attribute access. I'd tend to agree
with him that those turned out not to be such a great idea. But (I
think) nobody is arguing we should get rid of arrays with structured
dtypes - I use them regularly myself too.

-- Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Stefan van der Walt
On Thu, Nov 30, 2017, at 12:02, Marten van Kerkwijk wrote:
> I think Josef specifically meant `recarrays`, which give access to
> elements of a structured array via attribute access. I'd tend to agree
> with him that those turned out not to be such a great idea. But (I
> think) nobody is arguing we should get rid of arrays with structured
> dtypes - I use them regularly myself too.

Ah, okay, that makes sense!

Which reminds me: while these are quite useful, they're not always
particularly pleasant to use.  A good first improvement would be to
allow columnar printing, and a few utility functions to give you some of
the basic functionality of pandas (calculating descriptive statistics
like mean, dropping NaN rows, some equivalent of groupby).  All these
are only a few lines of Python, but can be annoying to figure out.  If
this sounds appealing, I'd be willing to put together a small NEP.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Eric Firing

On 2017/11/30 12:00 PM, Stefan van der Walt wrote:

I think Josef specifically meant `recarrays`, which give access to
elements of a structured array via attribute access. I'd tend to agree
with him that those turned out not to be such a great idea. But (I


I have found recarrays to be useful, providing an alternative view that 
can be convenient.  What is the problem with them?


Eric
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Marten van Kerkwijk
Unlike for matrix, it is not so much a problem as an unclear use case
- the main thing they bring to structured dtype arrays is access by
attribute, which is slower than just doing getting the field by its
key.
Anyway, I don't think anybody is suggesting to remove them - they're
not a problem in the way matrix is, with its shape-mangling, etc.  --
Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Feng Yu
An NEP on utility functions for structured array definitely sounds
appealing to me.

On Thu, Nov 30, 2017 at 2:00 PM, Stefan van der Walt
 wrote:
> On Thu, Nov 30, 2017, at 12:02, Marten van Kerkwijk wrote:
>> I think Josef specifically meant `recarrays`, which give access to
>> elements of a structured array via attribute access. I'd tend to agree
>> with him that those turned out not to be such a great idea. But (I
>> think) nobody is arguing we should get rid of arrays with structured
>> dtypes - I use them regularly myself too.
>
> Ah, okay, that makes sense!
>
> Which reminds me: while these are quite useful, they're not always
> particularly pleasant to use.  A good first improvement would be to
> allow columnar printing, and a few utility functions to give you some of
> the basic functionality of pandas (calculating descriptive statistics
> like mean, dropping NaN rows, some equivalent of groupby).  All these
> are only a few lines of Python, but can be annoying to figure out.  If
> this sounds appealing, I'd be willing to put together a small NEP.
>
> Stéfan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Nathaniel Smith
On Thu, Nov 30, 2017 at 11:39 AM, Charles R Harris
 wrote:
>
>
> On Thu, Nov 30, 2017 at 11:43 AM, Ralf Gommers 
> wrote:
>> I'd suggest any release in the next couple of years is fine,but the one
>> where we drop Python 2 support is probably the worst choice. That's one of
>> the few things the core Python devs got 100% right with the Python 3 move:
>> advocate that in the 2->3 transition packages would not make any API changes
>> in order to make porting the least painful.
>
>
> Agree, we don't want to pile in too many changes at once. I think the big
> sticking point is the sparse matrices in SciPy, even issuing a
> DeprecationWarning could be problematic as long as there are sparse
> matrices. May I suggest that we put together an NEP for the NumPy side of
> things? Ralf, does SciPy have a mechanism for proposing such changes?

Agreed here as well... while I want to get rid of np.matrix as much as
anyone, doing that anytime soon would be *really* disruptive.

- There are tons of little scripts out there written by people who
didn't know better; we do want them to learn not to use np.matrix but
breaking all their scripts is a painful way to do that

- There are major projects like scikit-learn that simply have no
alternative to using np.matrix, because of scipy.sparse.

So I think the way forward is something like:

- Now or whenever someone gets together a PR: issue a
PendingDeprecationWarning in np.matrix.__init__ (unless it kills
performance for scikit-learn and friends), and put a big warning box
at the top of the docs. The idea here is to not actually break
anyone's code, but start to get out the message that we definitely
don't think anyone should use this if they have any alternative.

- After there's an alternative to scipy.sparse: ramp up the warnings,
possibly all the way to FutureWarning so that existing scripts don't
break but they do get noisy warnings

- Eventually, if we think it will reduce maintenance costs: split it
into a subpackage

I expect that one way or another we'll be maintaining matrix for quite
some time, and I agree with whoever said that most of the burden seems
to be in keeping the rest of numpy working sensibly with it, so I
don't think moving it into a subpackage is itself going to make a big
different either way. To me the logic is more like, if/when we decide
to actually break everyone's code by making `np.matrix` raise
AttributeError, then we should probably provide some package they can
import to get their code limping along again, and if we're going to do
that anyway then probably we should split it out first and shake out
any bugs before we make `np.matrix` start raising errors. But it's
going to be quite some time until we reach the "break everyone's code"
stage, given just how much code is out there using matrix, so there's
no point in making detailed plans right now.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Stefan van der Walt
On Thu, Nov 30, 2017, at 16:15, Nathaniel Smith wrote:
> PendingDeprecationWarning in np.matrix.__init__ (unless it kills
> performance for scikit-learn and friends), and put a big warning box
> at the top of the docs. The idea here is to not actually break
> anyone's code, but start to get out the message that we definitely
> don't think anyone should use this if they have any alternative.
> 
> - After there's an alternative to scipy.sparse: ramp up the warnings,
> possibly all the way to FutureWarning so that existing scripts don't
> break but they do get noisy warnings
> 
> - Eventually, if we think it will reduce maintenance costs: split it
> into a subpackage

Can't we make `np.matrix` into a new package right now, and have NumPy
depend on it internally?  At that point, start warning users that they
should also be using the external package, and eventually just remove
the shim in NumPy.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Marten van Kerkwijk
Hi Nathaniel,

Thanks for the concrete suggestion: see
https://github.com/numpy/numpy/pull/10142
I think this is useful independent of exactly how the eventual move to
a new package would work; next step might be to collect all matrix
tests in the `libmatrix` sub-module.

All the best,

Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17?

2017-11-30 Thread Gael Varoquaux
> - There are major projects like scikit-learn that simply have no
> alternative to using np.matrix, because of scipy.sparse.

Well, we have no love for np.matrix, we just use scipy.sparse and
np.ndarray.

> - After there's an alternative to scipy.sparse:

+1 for alternative to scipy.sparse before transition.

Gaël
___
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion