Re: [Numpy-discussion] Looking for people interested in helping with Python compiler to LLVM
How about: * http://www.hotpy.org/ * http://pypy.org/numpydonate.html On Wed, Mar 21, 2012 at 11:14 PM, mark florisson wrote: > On 20 March 2012 20:49, Olivier Delalleau wrote: > > I doubt Theano is already as smart as you'd want it to be right now, > however > > the core mechanisms are there to perform graph optimizations and move > > computations to GPU. It may save time to start from there instead of > > starting all over from scratch. I'm not sure though, but it looks like it > > would be worth considering it at least. > > Thanks for the suggestion Olivier, as Dag said we discusses it, and > indeed we (or I) should look a lot deeper into it and see what > components are reusable there and discuss with the Theano community if > and how we can collaborate. > > > -=- Olivier > > > > Le 20 mars 2012 15:40, Dag Sverre Seljebotn > a > > écrit : > > > >> We talked some about Theano. There are some differences in project goals > >> which means that it makes sense to make this a seperate project: Cython > >> wants to use this to generate C code up front from the Cython AST at > >> compilation time; numba also has a different frontend (parsing of python > >> bytecode) and a different backend (LLVM). > >> > >> However, it may very well be possible that Theano could be refactored so > >> that the more essential algorithms working on the syntax tree could be > >> pulled out and shared with cython and numba. Then the question is > whether > >> the core of Theano is smart enough to compete with Fortran compilers and > >> support arbitraily strided inputs optimally. Otherwise one might as well > >> start from scratch. I'll leave that for Mark to figure out... > >> > >> Dag > >> -- > >> Sent from my Android phone with K-9 Mail. Please excuse my brevity. > >> > >> > >> Olivier Delalleau wrote: > >>> > >>> This sounds a lot like Theano, did you look into it? > >>> > >>> -=- Olivier > >>> > >>> Le 20 mars 2012 13:49, mark florisson a > écrit > >>> : > > On 13 March 2012 18:18, Travis Oliphant wrote: > >>> > >>> (Mark F., how does the above match how you feel about this?) > >> > >> I would like collaboration, but from a technical perspective I > think > >> this would be much more involved than just dumping the AST to an IR > >> and generating some code from there. For vector expressions I think > >> sharing code would be more feasible than arbitrary (parallel) > loops, > >> etc. Cython as a compiler can make many decisions that a Python > >> (bytecode) compiler can't make (at least without annotations and a > >> well-defined subset of the language (not so much the syntax as the > >> semantics)). I think in numba, if parallelism is to be supported, > you > >> will want a prange-like construct, as proving independence between > >> iterations can be very hard to near impossible for a compiler. > > > > I completely agree that you have to define some kind of syntax to > get > > parallelism. But, a prange construct would not be out of the > question, of > > course. > > > >> > >> As for code generation, I'm not sure how llvm would do things like > >> slicing arrays, reshaping, resizing etc (for vector expressions you > >> can first evaluate all slicing and indexing operations and then > >> compile the remaining vector expression), but for loops and array > >> reassignment within loops this would have to invoke the actual > >> slicing > >> code from the llvm code (I presume). > > > > There could be some analysis on the byte-code, prior to emitting the > > llvm code in order to handle lots of things. Basically, you have > to "play" > > the byte-code on a simple machine anyway in order to emit the > correct code. > > The big thing about Cython is you have to typedef too many things > that are > > really quite knowable from the code. If Cython could improve it's > type > > inference, then it would be a more suitable target. > > > >> There are many other things, like > >> bounds checking, wraparound, etc, that are all supported in both > >> numpy > >> and Cython, but going through an llvm layer would as far as I can > >> see, > >> require re-implementing those, at least if you want top-notch > >> performance. Personally, I think for non-trivial > performance-critical > >> code (for loops with indexing, slicing, function calls, etc) Cython > >> is > >> a better target. > > > > With libclang it is really quite possible to imagine a cython -> C > > target that itself compiles to llvm so that you can do everything > at that > > intermediate layer. However, LLVM is a much better layer for > optimization > > than C now that there are a lot of people collaborating on that > layer. I > > think it would be great if Cython targeted LLVM actually instead of > C. > >
Re: [Numpy-discussion] Looking for people interested in helping with Python compiler to LLVM
what is the difference to http://www.python.org/dev/peps/pep-3146/ ? On Sun, Mar 11, 2012 at 4:35 PM, Travis Oliphant wrote: > Hey all, > > I gave a lightning talk this morning on numba which is the start of a > Python compiler to machine code through the LLVM tool-chain. It is proof > of concept stage only at this point (use it only if you are interested in > helping develop the code at this point). The only thing that works is a > fast-vectorize capability on a few functions (without for-loops). But, it > shows how creating functions in Python that can be used by the NumPy > runtime in various ways. Several NEPS that will be discussed in the > coming months will use this concept. > > Right now there is very little design documentation, but I will be adding > some in the days ahead, especially if I get people who are interested in > collaborating on the project. I did talk to Fijal and Alex of the PyPy > project at PyCon and they both graciously suggested that I look at some of > the PyPy code which walks the byte-code and does translation to their > intermediate representation for inspiration. > > Again, the code is not ready for use, it is only proof of concept, but I > would like to get feedback and help especially from people who might have > written compilers before. The code lives at: > https://github.com/ContinuumIO/numba > > -Travis > > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] when numpy in Linux apt will be updated? It's still 1.3.0 with many bugs
> Thanks for all those instructions, however, personally I don't need them, I > have sucseeded with my own manipulations and even if I wouldn't I always can > build numpy/scipy from sources. > I mere care for quality and easibility of numpy installation for ordinary > non-skilled users. They may just try installation, see that it's buggy and > move away to use another soft. Rest assured that "non-ordinary" users like me also get irritated when we have to do extra work to get something to work/install when we want to try something out, its a waste of our time and resources if we have to do it often. Those problems might "always" be there in some degree, i have numpy installed on debian, fedora, and archlinux. Where there was no problems with the easy_install or package system install on those system (this time) But it will always happen at some time or another (as it did for me with install of other software via easy_install, rst2pdf and sphinx comes to mind) But that is a bug with the installation system not with the numpy package(you said 1.3.1 nupy had many bugs !?). A new user might move on to something other, but the reality is that problems like that crop up at some time, unless you have relatively static install. If the user moves on then he probably doesnt have a specific need for numpy, but just for something that can do the same job for his task. (it's unfortunate that he/we can't try it out without messing about). > D. mic ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] when numpy in Linux apt will be updated? It's still 1.3.0 with many bugs
Isnt [K]Ubuntu updated each 6 month? 2011/3/24 Dmitrey : > hi, > when numpy in Linux apt will be updated? It's still 1.3.0 with many bugs There will always be bugs, but numpy 1.3 is a stable release, unless there is a bug that affects what your doing right now? If you find a bug that prevents you from from your specific work, better report that bug, if you haven't already. > I tried to install numpy from PYPI where 1.5.1 seesm to be present, but > somehow it involves 1.3.0 instead: > > $ sudo easy_install numpy > install_dir /usr/local/lib/python2.6/dist-packages/ > Searching for numpy > Best match: numpy 1.3.0 > Adding numpy 1.3.0 to easy-install.pth file > > only after aptitude remove python-numpy version 1.5.1. is involved by > easy_install, but it fails: > > $ sudo easy_install numpy > > Adding numpy 1.5.1 to easy-install.pth file > Installing f2py script to /usr/local/bin > > Installed > /usr/local/lib/python2.6/dist-packages/numpy-1.5.1-py2.6-linux-x86_64.egg > Processing dependencies for numpy > Finished processing dependencies for numpy > /tmp/easy_install-QF6uJM/numpy-1.5.1/numpy/distutils/misc_util.py:251: > RuntimeWarning: Parent module 'numpy.distutils' not found while handling > absolute import I see an *absolute* import, maybe easy_install is different from where ubuntu expects numpy to be installed? i think the folder distutils is specific for ubuntu ? try and remove manually the egg in /usr/local/lib/python2.6/dist-packages/numpy-1.5.1-py2.6-linux-x86_64.egg then do $ sudo easy_install -U numpy -U is for update, maybe you have to run without -U first? > Error in atexit._run_exitfuncs: > Traceback (most recent call last): > File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/tmp/easy_install-QF6uJM/numpy-1.5.1/numpy/distutils/misc_util.py", > line 251, in clean_up_temporary_directory > ImportError: No module named numpy.distutils > Error in sys.exitfunc: > Traceback (most recent call last): > File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs > func(*targs, **kargs) > File "/tmp/easy_install-QF6uJM/numpy-1.5.1/numpy/distutils/misc_util.py", > line 251, in clean_up_temporary_directory > ImportError: No module named numpy.distutils > > I have Linux KUBUNTU 10.10 > > D. Good question.. ? easy_install should be distribution non-specific(as far as i know) see also this thread, even though its old. (late 2009) http://old.nabble.com/numpy-1.3.0-eggs-with-python2.6-seem-broken-on-osx,-and-linux-td26551531.html so maybe then download only the egg(with easy_install), then enter the folder and do sudo python2.6 setup.py install Dont you have python 2.7 also on ubuntu? regards mic ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] problem with array/ndarray/list parameter to function
Hi i have a problem with passing a parameter to a function xtickvals = np.arange(0, total_project_days+5, 5) its for setting ticks on an axis, the x-axis Ex1: This works xticks = pylab.setp(ax2, xticklabels=['0','10','20','30','40','50','60','70','80','90','100']) Ex2: This doesnt work xticklabels=['0', '5', '10', '15', '20', '25', '30', '35', '40', '45', '50', '55', '60', '65'] xticks = pylab.setp(ax2, xticklabels) I want to calculate how many tick is needed on that axis and then pass it to function. So that why i need something like the second example (or another way?) -- I couldnt se code tags? Code to calculate ticks and steps for output -- #xticksval = np.arange(0, total_project_days+stpsz=5, stepsz) xtickvals = np.arange(0, total_project_days+5, 5) # why do we need to define it here, instead of it being created in the for loop xticklabels = [] for i in xtickvals: xticklabels.append(str(i)) xticks = pylab.setp(ax2, xticklabels) - The above code gives the same problem as example2 regards mic ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
