Re: [Numpy-discussion] Looking for people interested in helping with Python compiler to LLVM
On 21 March 2012 05:20, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 03/20/2012 12:56 PM, Francesc Alted wrote: On Mar 20, 2012, at 2:29 PM, Dag Sverre Seljebotn wrote: Francesc Altedfranc...@continuum.io wrote: On Mar 20, 2012, at 12:49 PM, mark florisson wrote: Cython and Numba certainly overlap. However, Cython requires: 1) learning another language 2) creating an extension module --- loading bit-code files and dynamically executing (even on a different machine from the one that initially created them) can be a powerful alternative for run-time compilation and distribution of code. These aren't show-stoppers obviously. But, I think some users would prefer an even simpler approach to getting fast-code than Cython (which currently doesn't do enought type-inference and requires building a dlopen extension module). Dag and I have been discussing this at PyCon, and here is my take on it (at this moment :). Definitely, if you can avoid Cython then that is easier and more desirable in many ways. So perhaps we can create a third project called X (I'm not very creative, maybe ArrayExprOpt), that takes an abstract syntax tree in a rather simple form, performs code optimizations such as rewriting loops with array accesses to vector expressions, fusing vector expressions and loops, etc, and spits out a transformed AST containing these optimizations. If runtime information is given such as actual shape and stride information the transformations could figure out there and then whether to do things like collapsing, axes swapping, blocking (as in, introducing more axes or loops to retain discontiguous blocks in the cache), blocked memory copies to contiguous chunks, etc. The AST could then also say whether the final expressions are vectorizable. Part of this functionality is already in numpy's nditer, except that this would be implicit and do more (and hopefully with minimal overhead). So numba, Cython and maybe numexpr could use the functionality, simply by building the AST from Python and converting back (if necessary) to its own AST. As such, the AST optimizer would be only part of any (runtime) compiler's pipeline, and it should be very flexible to retain any information (metadata regarding actual types, control flow information, etc) provided by the original AST. It would not do control flow analysis, type inference or promotion, etc, but only deal with abstract types like integers, reals and arrays (C, Fortran or partly contiguous or strided). It would not deal with objects, but would allow to insert nodes like UnreorderableNode and SideEffectNode wrapping parts of the original AST. In short, it should be as easy as possible to convert from an original AST to this project's AST and back again afterwards. I think this is a very interesting project, and certainly projects like numba can benefit of it. So, in order to us have an idea on what you are after, can we assume that your project (call it X) would be kind of an compiler optimizer, and then the produced, optimized code could be feed into numba for optimized LLVM code generation (that on its turn, can be run on top of CPUs or GPUs or a combination)? Is that correct? I think so. Another way of thinking about it is that it is a reimplementation of the logic in the (good and closed source) Fortran 90 compilers, in a reusable component for inclusion in various compilers. Various c++ metaprogramming libraries (like Blitz++) are similar too. Aha, thanks. Giving that my interpretation above is correct, it is bit more difficult to me to see how your X project could be of benefit for numexpr. In fact, I actually see this the other way round: once the optimizer has discovered the vectorization parts, then go one step further and generate code that uses numexpr automatically (call this, vectorization through numexpr). This is what you mean, or I'm missing something? No. I think in some ways this is a competitor to numexpr -- you would gut out the middle of numexpr and keep the frontend and backend, but use this to optimize iteration order and blocking strategies. I see. Yes, I can easily see Mark's project X + numba more as a competitor (killer?) to numexpr too. I think the goal is for higher performance than what I understand numexpr can provide (in some cases, not all!). For instance, can numexpr deal well with a + a.T where a is a c-contiguous array? Any numpy-like iteration order will not work well, one needs to use higher-dimensional (eg 2D) blocking, not 1D blocking. No. numexpr cannot deal with the above problem efficiently. numexpr is about 1d blocking, so its approach is pretty naive (but effective for these 1d blocking tasks). (if numexpr can do this then great, the task might then reduce to refactoring numexpr so that cython and numba can use the same logic) Well, now that I think about it, the virtual
Re: [Numpy-discussion] Looking for people interested in helping with Python compiler to LLVM
On 20 March 2012 20:49, Olivier Delalleau sh...@keba.be wrote: I doubt Theano is already as smart as you'd want it to be right now, however the core mechanisms are there to perform graph optimizations and move computations to GPU. It may save time to start from there instead of starting all over from scratch. I'm not sure though, but it looks like it would be worth considering it at least. Thanks for the suggestion Olivier, as Dag said we discusses it, and indeed we (or I) should look a lot deeper into it and see what components are reusable there and discuss with the Theano community if and how we can collaborate. -=- Olivier Le 20 mars 2012 15:40, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no a écrit : We talked some about Theano. There are some differences in project goals which means that it makes sense to make this a seperate project: Cython wants to use this to generate C code up front from the Cython AST at compilation time; numba also has a different frontend (parsing of python bytecode) and a different backend (LLVM). However, it may very well be possible that Theano could be refactored so that the more essential algorithms working on the syntax tree could be pulled out and shared with cython and numba. Then the question is whether the core of Theano is smart enough to compete with Fortran compilers and support arbitraily strided inputs optimally. Otherwise one might as well start from scratch. I'll leave that for Mark to figure out... Dag -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Olivier Delalleau sh...@keba.be wrote: This sounds a lot like Theano, did you look into it? -=- Olivier Le 20 mars 2012 13:49, mark florisson markflorisso...@gmail.com a écrit : On 13 March 2012 18:18, Travis Oliphant tra...@continuum.io wrote: (Mark F., how does the above match how you feel about this?) I would like collaboration, but from a technical perspective I think this would be much more involved than just dumping the AST to an IR and generating some code from there. For vector expressions I think sharing code would be more feasible than arbitrary (parallel) loops, etc. Cython as a compiler can make many decisions that a Python (bytecode) compiler can't make (at least without annotations and a well-defined subset of the language (not so much the syntax as the semantics)). I think in numba, if parallelism is to be supported, you will want a prange-like construct, as proving independence between iterations can be very hard to near impossible for a compiler. I completely agree that you have to define some kind of syntax to get parallelism. But, a prange construct would not be out of the question, of course. As for code generation, I'm not sure how llvm would do things like slicing arrays, reshaping, resizing etc (for vector expressions you can first evaluate all slicing and indexing operations and then compile the remaining vector expression), but for loops and array reassignment within loops this would have to invoke the actual slicing code from the llvm code (I presume). There could be some analysis on the byte-code, prior to emitting the llvm code in order to handle lots of things. Basically, you have to play the byte-code on a simple machine anyway in order to emit the correct code. The big thing about Cython is you have to typedef too many things that are really quite knowable from the code. If Cython could improve it's type inference, then it would be a more suitable target. There are many other things, like bounds checking, wraparound, etc, that are all supported in both numpy and Cython, but going through an llvm layer would as far as I can see, require re-implementing those, at least if you want top-notch performance. Personally, I think for non-trivial performance-critical code (for loops with indexing, slicing, function calls, etc) Cython is a better target. With libclang it is really quite possible to imagine a cython - C target that itself compiles to llvm so that you can do everything at that intermediate layer. However, LLVM is a much better layer for optimization than C now that there are a lot of people collaborating on that layer. I think it would be great if Cython targeted LLVM actually instead of C. Finally, as for non-vector-expression code, I really believe Cython is a better target. cython.inline can have high overhead (at least the first time it has to compile), but with better (numpy-aware) type inference or profile guided optimizations (see recent threads on the cython-dev mailing list), in addition to things like prange, I personally believe Cython targets most of the use cases where numba would be able to generate performing code. Cython and Numba certainly overlap. However, Cython requires: 1) learning another language 2) creating an extension
[Numpy-discussion] Viewing a float64 array with a float32 array
Hi, Is it possible to have a view of a float64 array that is itself float32? So that: A = np.arange(5, dtype='d') A.view(dtype='f') would return a size 5 float32 array looking at A's data? Thanks, -- Hugo Gagnon ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Viewing a float64 array with a float32 array
Hi, Is it possible to have a view of a float64 array that is itself float32? So that: A = np.arange(5, dtype='d') A.view(dtype='f') would return a size 5 float32 array looking at A's data? I think not. The memory layout of a 32-bit IEEE float is not a subset of that of a 64-bit float -- see e.g. the first table in: http://steve.hollasch.net/cgindex/coding/ieeefloat.html This would work for int8/int16 or other int types (so long as the number doesn't exceed the range of the smaller type), but AFAIK not floats. Note how the subset relationship works for the int8/int16 case, but not float32/float64: str(numpy.array(100,dtype=numpy.int8).data) 'd' str(numpy.array(100,dtype=numpy.int16).data) 'd\x00' str(numpy.array(-100,dtype=numpy.int16).data) '\x9c\xff' str(numpy.array(-100,dtype=numpy.int8).data) '\x9c' str(numpy.array(100,dtype=numpy.float32).data) '\x00\x00\xc8B' str(numpy.array(100,dtype=numpy.float64).data) '\x00\x00\x00\x00\x00\x00Y@' Zach ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Viewing a float64 array with a float32 array
I'm not sure if you are referring to rounding errors but that's OK with me. I was thinking something along the lines of changing how numpy looks at the data of A's view by modifying say the stride attribute, etc. -- Hugo Gagnon On Wed, Mar 21, 2012, at 11:19, Zachary Pincus wrote: Hi, Is it possible to have a view of a float64 array that is itself float32? So that: A = np.arange(5, dtype='d') A.view(dtype='f') would return a size 5 float32 array looking at A's data? I think not. The memory layout of a 32-bit IEEE float is not a subset of that of a 64-bit float -- see e.g. the first table in: http://steve.hollasch.net/cgindex/coding/ieeefloat.html This would work for int8/int16 or other int types (so long as the number doesn't exceed the range of the smaller type), but AFAIK not floats. Note how the subset relationship works for the int8/int16 case, but not float32/float64: str(numpy.array(100,dtype=numpy.int8).data) 'd' str(numpy.array(100,dtype=numpy.int16).data) 'd\x00' str(numpy.array(-100,dtype=numpy.int16).data) '\x9c\xff' str(numpy.array(-100,dtype=numpy.int8).data) '\x9c' str(numpy.array(100,dtype=numpy.float32).data) '\x00\x00\xc8B' str(numpy.array(100,dtype=numpy.float64).data) '\x00\x00\x00\x00\x00\x00Y@' Zach ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Viewing a float64 array with a float32 array
I'm not sure if you are referring to rounding errors but that's OK with me. I was thinking something along the lines of changing how numpy looks at the data of A's view by modifying say the stride attribute, etc. Yes, so was I. As you can see in my example with ints below, you could skip every other byte of the int16 array to look at an int8 array. This is because the memory layout of an int8 is a proper subset of int16. (Modulo endian-concerns of course...) But looking at the link I provided, you can see that taking the first 32 bits of an float64 (or the last 32 or any 32) does not yield something that can be interpreted as a float32. So there's no subset relationship, and you can't do the strides-trick. To be extra clear, look at the memory layout of a float that's expressible without rounding error: str(numpy.array(128,dtype=numpy.float64).data) '\x00\x00\x00\x00\x00\x00`@' str(numpy.array(128,dtype=numpy.float32).data) '\x00\x00\x00C' There's obviously no stride trick whereby one will look like the other. Zach On Wed, Mar 21, 2012, at 11:19, Zachary Pincus wrote: Hi, Is it possible to have a view of a float64 array that is itself float32? So that: A = np.arange(5, dtype='d') A.view(dtype='f') would return a size 5 float32 array looking at A's data? I think not. The memory layout of a 32-bit IEEE float is not a subset of that of a 64-bit float -- see e.g. the first table in: http://steve.hollasch.net/cgindex/coding/ieeefloat.html This would work for int8/int16 or other int types (so long as the number doesn't exceed the range of the smaller type), but AFAIK not floats. Note how the subset relationship works for the int8/int16 case, but not float32/float64: str(numpy.array(100,dtype=numpy.int8).data) 'd' str(numpy.array(100,dtype=numpy.int16).data) 'd\x00' str(numpy.array(-100,dtype=numpy.int16).data) '\x9c\xff' str(numpy.array(-100,dtype=numpy.int8).data) '\x9c' str(numpy.array(100,dtype=numpy.float32).data) '\x00\x00\xc8B' str(numpy.array(100,dtype=numpy.float64).data) '\x00\x00\x00\x00\x00\x00Y@' Zach ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] About np array and recarray
Hi,
I am really confused on the np array or record array, and cannot understand
how it works.
What I want to do is that I have a normal python two dimensional array/list:
a = [['2000-01-01', 2],['2000-01-02', 3]]
I want to convert it to a recarray with this dtype [('date', 'object'),
('count', 'int')]. I tried multiple ways and none of them works. And some
of the tests show pretty odd behavior.
This is good, and it is almost what i want:
import numpy as np
a = [('2000-01-01', 2), ('2000-01-02', 3)]
np.array(a, dtype=[('date', 'object'), ('count', 'int')])
array([('2000-01-01', 2), ('2000-01-02', 3)],
dtype=[('date', '|O8'), ('count', 'i8')])
Why this doesn't work?!
a = [['2000-01-01', 2],['2000-01-02', 3]]
np.array(a, dtype=[('date', 'object'), ('count', 'int')])
Traceback (most recent call last):
File stdin, line 1, in module
ValueError: tried to set void-array with object members using buffer.
Why can this cause segmentation fault?!
a = [['2000-01-01', 2],['2000-01-02', 3]]
np.ndarray((len(a),), buffer=np.array(a), dtype=[('date', 'object'),
('count', 'int')])
Segmentation fault (And python quit!)
Python version 2.6.5
On this reference page,
http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html
x = np.array([(1,2),(3,4)])
x
array([[1, 2],
[3, 4]])
np.array([[1, 2], [3, 4]])
array([[1, 2],
[3, 4]])
Can anyone help me about this?
Thanks.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] About np array and recarray
On Mar 21, 2012, at 11:48 PM, Yan Tang wrote:
Hi,
I am really confused on the np array or record array, and cannot understand
how it works.
What I want to do is that I have a normal python two dimensional array/list:
a = [['2000-01-01', 2],['2000-01-02', 3]]
I want to convert it to a recarray with this dtype [('date', 'object'),
('count', 'int')]. I tried multiple ways and none of them works. And some
of the tests show pretty odd behavior.
This is good, and it is almost what i want:
import numpy as np
a = [('2000-01-01', 2), ('2000-01-02', 3)]
np.array(a, dtype=[('date', 'object'), ('count', 'int')])
array([('2000-01-01', 2), ('2000-01-02', 3)],
dtype=[('date', '|O8'), ('count', 'i8')])
This is the correct way to initiate the record array, or structured array, from
a Python object.
Why this doesn't work?!
a = [['2000-01-01', 2],['2000-01-02', 3]]
np.array(a, dtype=[('date', 'object'), ('count', 'int')])
Traceback (most recent call last):
File stdin, line 1, in module
ValueError: tried to set void-array with object members using buffer.
The error here could be more instructive, but the problems is that to simplify
the np.array factory function (which is already somewhat complex) it was
decided to force records to be input as tuples and not as lists. You *must*
use tuples to specify records for a structured array.
Why can this cause segmentation fault?!
a = [['2000-01-01', 2],['2000-01-02', 3]]
np.ndarray((len(a),), buffer=np.array(a), dtype=[('date', 'object'),
('count', 'int')])
Segmentation fault (And python quit!)
The np.ndarray constructor should not be used directly unless you know what you
are doing.
The np.array factory function is the standard way to create arrays. The
problem here is that you are explicitly asking NumPy to point to a particular
region of memory to use as it's data-buffer. This memory is the data buffer
of an array of strings. The np.array factory function will try and
auto-detect the data-type of the array if you do not specify it --- which in
this case results in an array of strings.Then, with the dtype specification
you are asking it to interpret a portion of that array of strings as a pointer
to a Python object. This will cause a segmentation fault when the printing
code tries to dereference a pointer which is actually 4 characters of a string.
This should probably be checked for in the ndarray constructor. I don't think
it ever really makes sense to use an object dtype when you also supply the
buffer unless that buffer actually held Python object pointers in the first
place. Even in this case you could do what you wanted without calling the
constructor. So, likely a check should be made so that you can't have an
object array and also supply a buffer.
Python version 2.6.5
On this reference page,
http://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html
x = np.array([(1,2),(3,4)])
x
array([[1, 2],
[3, 4]])
np.array([[1, 2], [3, 4]])
array([[1, 2],
[3, 4]])
Can anyone help me about this?
I'm not sure what you are asking for here? Yes, for arrays with
non-structured dtypes, numpy will treat tuples as lists.
Best regards,
-Travis
Thanks.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
