Re: [Numpy-discussion] @ operator
On Wed, Sep 10, 2014 at 4:53 PM, Charles R Harris
wrote:
>
> On Wed, Sep 10, 2014 at 10:52 AM, Pauli Virtanen wrote:
>>
>> 09.09.2014, 22:52, Charles R Harris kirjoitti:
>> > 1. Should the operator accept array_like for one of the arguments?
>> > 2. Does it need to handle __numpy_ufunc__, or will
>> > __array_priority__ serve?
>>
>> I think the __matmul__ operator implementation should follow that of
>> __mul__.
>>
>> [clip]
>> >3. Do we want PyArray_Matmul in the numpy API?
>> >4. Should a matmul function be supplied by the multiarray module?
>> >
>> > If 3 and 4 are wanted, should they use the __numpy_ufunc__ machinery, or
>> > will __array_priority__ serve?
>>
>> dot() function deals with __numpy_ufunc__, and the matmul() function
>> should behave similarly.
>>
>> It seems dot() uses __array_priority__ for selection of output return
>> subclass, so matmul() probably needs do the same thing.
>>
>> > Note that the type number operators, __add__ and such, currently use
>> > __numpy_ufunc__ in combination with __array_priority__, this in addition
>> > to
>> > the fact that they are by default using ufuncs that do the same. I'd
>> > rather
>> > that the __*__ operators simply rely on __array_priority__.
>>
>> The whole business of __array_priority__ and __numpy_ufunc__ in the
>> binary ops is solely about when should yield the execution to
>> __r__ of the other object.
>>
>> The rule of operation currently is: "__rmul__ before __numpy_ufunc__"
>>
>> If you remove the __numpy_ufunc__ handling, it becomes: "__rmul__ before
>> __numpy_ufunc__, except if array_priority happens to be smaller than
>> that of the other class and your class is not an ndarray subclass".
>>
>> The following binops also do not IIRC respect __array_priority__ in
>> preferring right-hand operand:
>>
>> - in-place operations
>> - comparisons
>>
>> One question here is whether it's possible to change the behavior of
>> __array_priority__ here at all, or whether changes are possible only in
>> the context of adding new attributes telling Numpy what to do.
>
> I was tempted to make it a generalized ufunc, which would take care of a lot
> of things, but there is a lot of overhead in those functions. Sounds like
> the easiest thing is to make it similar to dot, although having an inplace
> versions complicates the type selection a bit.
Can we please fix the overhead instead of adding more half-complete
implementations of the same concepts? I feel like this usually ends up
slowing things down in the end, as optimization efforts get divided...
My vote is:
__matmul__/__rmatmul__ do the standard dispatch stuff that all __op__
methods do (so I guess check __array_priority__ or whatever it is we
always do). I'd also be okay with ignoring __array_priority__ on the
grounds that __numpy_ufunc__ is better, and there's no existing code
relying on __array_priority__ support in __matmul__.
Having decided that we are actually going to run, they dispatch
unconditionally to np.newdot(a, b) (or newdot(a, b, out=a) for the
in-place version), similarly to how e.g. __add__ dispatches to np.add.
newdot acts like a standard gufunc with all the standard niceties,
including __numpy_ufunc__ dispatch.
("newdot" here is intended as a placeholder name, maybe it should be
np.linalg.matmul or something else to be bikeshed later. I also vote
that eventually 'dot' become an alias for this function, but whether
to do that is an orthogonal discussion for later.)
--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] @ operator
Charles R Harris wrote: > Note also that the dot cblas versions are not generally blocked, so the > size of the arrays is limited (and not checked). But it is possible to create a blocked dot function with the current cblas, even though they use C int for array dimensions. It would just further increase the complexity of dot (as if it's not bad enough already...) Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] @ operator
On Wed, Sep 10, 2014 at 2:53 PM, Charles R Harris wrote: > > > On Wed, Sep 10, 2014 at 10:52 AM, Pauli Virtanen wrote: > >> 09.09.2014, 22:52, Charles R Harris kirjoitti: >> > 1. Should the operator accept array_like for one of the arguments? >> > 2. Does it need to handle __numpy_ufunc__, or will >> > __array_priority__ serve? >> >> I think the __matmul__ operator implementation should follow that of >> __mul__. >> >> [clip] >> >3. Do we want PyArray_Matmul in the numpy API? >> >4. Should a matmul function be supplied by the multiarray module? >> > >> > If 3 and 4 are wanted, should they use the __numpy_ufunc__ machinery, or >> > will __array_priority__ serve? >> >> dot() function deals with __numpy_ufunc__, and the matmul() function >> should behave similarly. >> >> It seems dot() uses __array_priority__ for selection of output return >> subclass, so matmul() probably needs do the same thing. >> >> > Note that the type number operators, __add__ and such, currently use >> > __numpy_ufunc__ in combination with __array_priority__, this in >> addition to >> > the fact that they are by default using ufuncs that do the same. I'd >> rather >> > that the __*__ operators simply rely on __array_priority__. >> >> The whole business of __array_priority__ and __numpy_ufunc__ in the >> binary ops is solely about when should yield the execution to >> __r__ of the other object. >> >> The rule of operation currently is: "__rmul__ before __numpy_ufunc__" >> >> If you remove the __numpy_ufunc__ handling, it becomes: "__rmul__ before >> __numpy_ufunc__, except if array_priority happens to be smaller than >> that of the other class and your class is not an ndarray subclass". >> >> The following binops also do not IIRC respect __array_priority__ in >> preferring right-hand operand: >> >> - in-place operations >> - comparisons >> >> One question here is whether it's possible to change the behavior of >> __array_priority__ here at all, or whether changes are possible only in >> the context of adding new attributes telling Numpy what to do. >> >> > I was tempted to make it a generalized ufunc, which would take care of a > lot of things, but there is a lot of overhead in those functions. Sounds > like the easiest thing is to make it similar to dot, although having an > inplace versions complicates the type selection a bit. > Note also that the dot cblas versions are not generally blocked, so the size of the arrays is limited (and not checked). Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] @ operator
On Wed, Sep 10, 2014 at 10:52 AM, Pauli Virtanen wrote: > 09.09.2014, 22:52, Charles R Harris kirjoitti: > > 1. Should the operator accept array_like for one of the arguments? > > 2. Does it need to handle __numpy_ufunc__, or will > > __array_priority__ serve? > > I think the __matmul__ operator implementation should follow that of > __mul__. > > [clip] > >3. Do we want PyArray_Matmul in the numpy API? > >4. Should a matmul function be supplied by the multiarray module? > > > > If 3 and 4 are wanted, should they use the __numpy_ufunc__ machinery, or > > will __array_priority__ serve? > > dot() function deals with __numpy_ufunc__, and the matmul() function > should behave similarly. > > It seems dot() uses __array_priority__ for selection of output return > subclass, so matmul() probably needs do the same thing. > > > Note that the type number operators, __add__ and such, currently use > > __numpy_ufunc__ in combination with __array_priority__, this in addition > to > > the fact that they are by default using ufuncs that do the same. I'd > rather > > that the __*__ operators simply rely on __array_priority__. > > The whole business of __array_priority__ and __numpy_ufunc__ in the > binary ops is solely about when should yield the execution to > __r__ of the other object. > > The rule of operation currently is: "__rmul__ before __numpy_ufunc__" > > If you remove the __numpy_ufunc__ handling, it becomes: "__rmul__ before > __numpy_ufunc__, except if array_priority happens to be smaller than > that of the other class and your class is not an ndarray subclass". > > The following binops also do not IIRC respect __array_priority__ in > preferring right-hand operand: > > - in-place operations > - comparisons > > One question here is whether it's possible to change the behavior of > __array_priority__ here at all, or whether changes are possible only in > the context of adding new attributes telling Numpy what to do. > > I was tempted to make it a generalized ufunc, which would take care of a lot of things, but there is a lot of overhead in those functions. Sounds like the easiest thing is to make it similar to dot, although having an inplace versions complicates the type selection a bit. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] @ operator
09.09.2014, 22:52, Charles R Harris kirjoitti: > 1. Should the operator accept array_like for one of the arguments? > 2. Does it need to handle __numpy_ufunc__, or will > __array_priority__ serve? I think the __matmul__ operator implementation should follow that of __mul__. [clip] >3. Do we want PyArray_Matmul in the numpy API? >4. Should a matmul function be supplied by the multiarray module? > > If 3 and 4 are wanted, should they use the __numpy_ufunc__ machinery, or > will __array_priority__ serve? dot() function deals with __numpy_ufunc__, and the matmul() function should behave similarly. It seems dot() uses __array_priority__ for selection of output return subclass, so matmul() probably needs do the same thing. > Note that the type number operators, __add__ and such, currently use > __numpy_ufunc__ in combination with __array_priority__, this in addition to > the fact that they are by default using ufuncs that do the same. I'd rather > that the __*__ operators simply rely on __array_priority__. The whole business of __array_priority__ and __numpy_ufunc__ in the binary ops is solely about when should yield the execution to __r__ of the other object. The rule of operation currently is: "__rmul__ before __numpy_ufunc__" If you remove the __numpy_ufunc__ handling, it becomes: "__rmul__ before __numpy_ufunc__, except if array_priority happens to be smaller than that of the other class and your class is not an ndarray subclass". The following binops also do not IIRC respect __array_priority__ in preferring right-hand operand: - in-place operations - comparisons One question here is whether it's possible to change the behavior of __array_priority__ here at all, or whether changes are possible only in the context of adding new attributes telling Numpy what to do. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] SFMT (faster mersenne twister)
Julian Taylor wrote: > But as already mentioned by Robert, we know what we can do, what is > missing is someone writting the code. This is actually a part of NumPy I know in detail, so I will be able to contribute. Robert Kern's last post about objects like np.random.SFMT() working similar to RandomState should be doable and not break any backwards compatibility. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] SFMT (faster mersenne twister)
Pierre-Andre Noel wrote: > Why not do something like the C++11 ? In , a "generator" > is the engine producing randomness, and a "distribution" decides what is > the type of outputs that you want. This is what randomkit is doing internally, which is why it is so easy to plug in a different generator. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] @ operator
On Di, 2014-09-09 at 13:52 -0600, Charles R Harris wrote: > Hi All, > > > I'm in the midst of implementing the '@' operator (PEP 465), and there > are some behaviors that are unspecified by the PEP. > > 1. Should the operator accept array_like for one of the > arguments? To be in line with all the other operators, I would say yes > 1. Does it need to handle __numpy_ufunc__, or will > __array_priority__ serve? > 2. Do we want PyArray_Matmul in the numpy API? Don't care much either way, but I would say yes (why not). > 1. Should a matmul function be supplied by the multiarray module? > We could possibly put it to the linalg module. But I think we should provide such a function. > If 3 and 4 are wanted, should they use the __numpy_ufunc__ machinery, > or will __array_priority__ serve? > > Note that the type number operators, __add__ and such, currently use > __numpy_ufunc__ in combination with __array_priority__, this in > addition to the fact that they are by default using ufuncs that do the > same. I'd rather that the __*__ operators simply rely on > __array_priority__. > Hmmm, that is a difficult one. For the operators I agree, numpy_ufunc should not be necessary, since we have NotImplemented there, and the array priority does nothing except tell numpy to stop handling all array likes (i.e. other array-likes could actually say: you have an array-priority > 0 -> return NotImplemented). So yeah, why do the operators use numpy_ufunc at all? (unless due to implementation) If we have a function numpy_ufunc would probably make sense, since that circumvents the python dispatch mechanism. - Sebastian > > Thoughts? > > Chuck > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ANN: Bokeh 0.6 release
On behalf of the Bokeh team, I am very happy to announce the release of Bokeh version 0.6! Bokeh is a Python library for visualizing large and realtime datasets on the web. Its goal is to provide to developers (and domain experts) with capabilities to easily create novel and powerful visualizations that extract insight from local or remote (possibly large) data sets, and to easily publish those visualization to the web for others to explore and interact with. This release includes many bug fixes and improvements over our most recent 0.5.2 release: * Abstract Rendering recipes for large data sets: isocontour, heatmap * New charts in bokeh.charts: Time Series and Categorical Heatmap * Full Python 3 support for bokeh-server * Much expanded User and Dev Guides * Multiple axes and ranges capability * Plot object graph query interface * Hit-testing (hover tool support) for patch glyphs See the CHANGELOG for full details. I'd also like to announce a new Github Organization for Bokeh: https://github.com/bokeh. Currently it is home to Scala and and Julia language bindings for Bokeh, but the Bokeh project itself will be moved there before the next 0.7 release. Any implementors of new language bindings who are interested in hosting your project under this organization are encouraged to contact us. In upcoming releases, you should expect to see more new layout capabilities (colorbar axes, better grid plots and improved annotations), additional tools, even more widgets and more charts, R language bindings, Blaze integration and cloud hosting for Bokeh apps. Don't forget to check out the full documentation, interactive gallery, and tutorial at http://bokeh.pydata.org as well as the Bokeh IPython notebook nbviewer index (including all the tutorials) at: http://nbviewer.ipython.org/github/ContinuumIO/bokeh-notebooks/blob/master/index.ipynb If you are using Anaconda, you can install with conda: conda install bokeh Alternatively, you can install with pip: pip install bokeh BokehJS is also available by CDN for use in standalone javascript applications: http://cdn.pydata.org/bokeh-0.6.min.js http://cdn.pydata.org/bokeh-0.6.min.css Issues, enhancement requests, and pull requests can be made on the Bokeh Github page: https://github.com/continuumio/bokeh Questions can be directed to the Bokeh mailing list: bo...@continuum.io If you have interest in helping to develop Bokeh, please get involved! Thanks, Bryan Van de Ven Continuum Analytics http://continuum.io ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy ignores OPT/FOPT under Python3
On 9 September 2014 17:23, Thomas Unterthiner wrote: > > I want to use the OPT/FOPT environment viariables to set compiler flags > when compiling numpy. However it seems that they get ignored under > python3. Using Ubuntu 14.04 and numpy 1.9.0, I did the following: > > >export OPT="-march=native" > >export FOPT = "-march=native" > > python setup.py build # "python" executes python2.7 > [...snip...] > C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing > -march=native -fPIC > ^C Running the same on my computer (Fedora 20, python 2.7) doesn't seem to process the flags: C compiler: -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC Double-checking: $ echo $OPT -march=native ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
