On Feb 20, 2012, at 6:46 PM, Dag Sverre Seljebotn wrote: > On 02/20/2012 09:24 AM, Olivier Delalleau wrote: >> Hi Dag, >> >> Would you mind elaborating a bit on that example you mentioned at the >> end of your email? I don't quite understand what behavior you would like >> to achieve > > Sure, see below. I think we should continue discussion on numpy-discuss. > > I wrote: > >> You need at least a slightly different Python API to get anywhere, so >> numexpr/Theano is the right place to work on an implementation of this >> idea. Of course it would be nice if numexpr/Theano offered something as >> convenient as >> >> with lazy: >> arr = A + B + C # with all of these NumPy arrays >> # compute upon exiting... > > More information: > > The disadvantage today of using Theano (or numexpr) is that they require > using a different API, so that one has to learn and use Theano "from the > ground up", rather than just slap it on in an optimization phase. > > The alternative would require extensive changes to NumPy, so I guess > Theano authors or Francesc would need to push for this. > > The alternative would be (with A, B, C ndarray instances): > > with theano.lazy: > arr = A + B + C > > On __enter__, the context manager would hook into NumPy to override it's > arithmetic operators. Then it would build a Theano symbolic tree instead > of performing computations right away. > > In addition to providing support for overriding arithmetic operators, > slicing etc., it would be necesarry for "arr" to be an ndarray instance > which is "not yet computed" (data-pointer set to NULL, and store a > compute-me callback and some context information). > > Finally, the __exit__ would trigger computation. For other operations > which need the data pointer (e.g., single element lookup) one could > either raise an exception or trigger computation. > > This is just a rough sketch. It is not difficult "in principle", but of > course there's really a massive amount of work involved to work support > for this into the NumPy APIs. > > Probably, we're talking a NumPy 3.0 thing, after the current round of > refactorings have settled... > > Please: Before discussing this further one should figure out if there's > manpower available for it; no sense in hashing out a castle in the sky > in details.
I see. Mark Wiebe already suggested the same thing some time ago: https://github.com/numpy/numpy/blob/master/doc/neps/deferred-ufunc-evaluation.rst > Also it would be better to talk in person about this if > possible (I'm in Berkeley now and will attend PyData and PyCon). Nice. Most of Continuum crew (me included) will be attending to both conferences. Mark W. will make PyCon only, but will be a good occasion to discuss this further. See you, -- Francesc Alted _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
