Hi, This is just to note that I did some optimization in Theano related to this. I did "easy" stuff, like adding c code to some "trivial" op that where bottleneck in our cases.
It is still slower then your cython example. Is it manually done cython or generated cython? Also, I fixed a crash due to the extra not used input to the theano function. If you use the linker=c Theano flag, with the development version of Theano. It will run faster then with the current config. thanks for the benchmark, it make it easy to see where is the bottleneck in Theano for you case. Fred On Tue, Nov 5, 2013 at 9:46 PM, Jason Moore <moorepa...@gmail.com> wrote: > Sure, I'll do some benchmarks on that soon. > > > Jason > moorepants.info > +01 530-601-9791 > > > On Tue, Nov 5, 2013 at 9:27 PM, Aaron S. Meurer <asmeu...@gmail.com> wrote: >> >> Assumedly if it doesn't, then you would be able to tell just by applying >> the sympy cse and seeing if it makes a significant difference in the run >> time. >> >> Aaron Meurer >> >> On Nov 5, 2013, at 6:45 PM, Matthew Rocklin <mrock...@gmail.com> wrote: >> >> Sadly I'm not an expert in C compilers and I no longer have an awesome >> compilers-post-doc-office-mate. I don't know what they'll do in this >> situation. His no-CSE statement would still hold if the input pointers >> point to dynamically allocated memory. I don't recall what he said if they >> point to statically allocated memory; I think it was a big nebulous "it >> depends." >> >> >> On Tue, Nov 5, 2013 at 5:35 PM, Jason Moore <moorepa...@gmail.com> wrote: >>> >>> Got it. >>> >>> SymPy's code generators in codegen and autowrap don't support evaluating >>> a SymPy matrix as the expression. I'd like to improve those so that you can >>> pass in a sequences of Matrices and it returns a function that evaluates all >>> of the matrices in a single C/Fortran/etc file. >>> >>> Here is a simple example of the c file I'm generating from two SymPy >>> matrices: >>> https://github.com/PythonDynamics/pydy-code-gen/blob/master/pydy_code_gen/tests/expected_cython/desired_mass_forcing_c.c >>> >>> >>> Jason >>> moorepants.info >>> +01 530-601-9791 >>> >>> >>> On Tue, Nov 5, 2013 at 5:41 PM, Matthew Rocklin <mrock...@gmail.com> >>> wrote: >>>> >>>> My previous statement was that Theano won over autowrap.ufuncify because >>>> theano did CSE internally (theano's basic data structure is a DAG). >>>> >>>> My understanding is that your C friend is correct IF the code is written >>>> in the standard way, declaring each variable individually. I remember >>>> looking at older code of your groups in which all of the variables were >>>> stored in a dynamically allocated array. My C friend mentioned that most >>>> compilers won't dare touch such a structure because it's verify that >>>> transformations are safe. I have no idea what your various code generators >>>> are doing under the hood. >>>> >>>> By the way I did not intend to raise the issue of CSE. I'm mostly >>>> curious about the poor performance of Theano and I don't think this is that >>>> issue. >>>> >>>> >>>> On Tue, Nov 5, 2013 at 10:48 AM, Aaron Meurer <asmeu...@gmail.com> >>>> wrote: >>>>> >>>>> I would still benchmark running cse and not. If cse() is fast enough, >>>>> it might out-balance the overhead of the code generation, especially >>>>> if it significantly reduces the resulting code size. >>>>> >>>>> Aaron Meurer >>>>> >>>>> On Tue, Nov 5, 2013 at 11:36 AM, Jason Moore <moorepa...@gmail.com> >>>>> wrote: >>>>> > My roommate who's a C guy suggested that CSE was a waste of time if >>>>> > you have >>>>> > a good compiler. He claimed that the compiler will do CSE anyways, so >>>>> > doing >>>>> > it in the C code was not useful. I need to compare my Cython code >>>>> > output >>>>> > with and without CSE (it currently uses CSE). >>>>> > >>>>> > How would I use CSE with theano_function? I'm not doing that >>>>> > currently and >>>>> > it wasn't apparent to me how to go about that. >>>>> > >>>>> > >>>>> > Jason >>>>> > moorepants.info >>>>> > +01 530-601-9791 >>>>> > >>>>> > >>>>> > On Tue, Nov 5, 2013 at 1:30 PM, Matthew Rocklin <mrock...@gmail.com> >>>>> > wrote: >>>>> >> >>>>> >> In my experiments with sympy-theano earlier this year I found that >>>>> >> it was >>>>> >> competitive with our Fortran code generation (autowrap.ufuncify) on >>>>> >> single >>>>> >> array outputs and outperformed it when CSE were involved. I don't >>>>> >> have >>>>> >> anything recorded that profiles scalar operations. >>>>> >> >>>>> >> Frederic might have some intuition on Theano's performance on large >>>>> >> scalar >>>>> >> computations. I've brought this up in the past and don't actually >>>>> >> remember >>>>> >> the answer. >>>>> >> >>>>> >> >>>>> >> On Tue, Nov 5, 2013 at 9:51 AM, Jason Moore <moorepa...@gmail.com> >>>>> >> wrote: >>>>> >>> >>>>> >>> Ok, so I think I'm comparing things correctly then. It is >>>>> >>> surprising that >>>>> >>> the compiled Theano code doesn't seem to execute that fast. The >>>>> >>> functions >>>>> >>> I'm generating are essentially evaluating very long sympy >>>>> >>> expressions, maybe >>>>> >>> Theano isn't great at that... >>>>> >>> >>>>> >>> >>>>> >>> Jason >>>>> >>> moorepants.info >>>>> >>> +01 530-601-9791 >>>>> >>> >>>>> >>> >>>>> >>> On Tue, Nov 5, 2013 at 12:48 PM, Matthew Rocklin >>>>> >>> <mrock...@gmail.com> >>>>> >>> wrote: >>>>> >>>> >>>>> >>>> `sympy.printing.theanocode.theano_function` calls >>>>> >>>> `theano.function` >>>>> >>>> >>>>> >>>> The result of `theano_function` is a compiled theano function. >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> >>>> On Tue, Nov 5, 2013 at 9:27 AM, Jason Moore <moorepa...@gmail.com> >>>>> >>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Jason >>>>> >>>>> moorepants.info >>>>> >>>>> +01 530-601-9791 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Nov 5, 2013 at 8:46 AM, Frédéric Bastien >>>>> >>>>> <no...@nouiz.org> >>>>> >>>>> wrote: >>>>> >>>>>> >>>>> >>>>>> Hi, >>>>> >>>>>> >>>>> >>>>>> Just to know, witch version of Theano did you used? I have speed >>>>> >>>>>> up a >>>>> >>>>>> little the optimization phase of the compilation and I have a PR >>>>> >>>>>> that >>>>> >>>>>> will optimize this more. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> I'm using the master branch of Theano as of last week. >>>>> >>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> Also, Theano cache the compilation of the c code. Where this >>>>> >>>>>> timming >>>>> >>>>>> done with a empty theano cache or a filled Theano cache? If you >>>>> >>>>>> ran >>>>> >>>>>> the same benchmark multiple time on the same computer, only the >>>>> >>>>>> first >>>>> >>>>>> time Theano will compile the c code, the other time it will >>>>> >>>>>> reuse what >>>>> >>>>>> is in the cache. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> I time two things wrt to Theano: >>>>> >>>>> >>>>> >>>>> 1. The time it takes to call >>>>> >>>>> sympy.printing.theanocode.theano_function >>>>> >>>>> >>>>> >>>>> and >>>>> >>>>> >>>>> >>>>> 2. The time it takes to call the function generated from 1. >>>>> >>>>> >>>>> >>>>> When does the Theano compilation happen? Is it in the call to >>>>> >>>>> theano_function or the first time I use the generated function? >>>>> >>>>> >>>>> >>>>> If it is the latter then my timing comparisons aren't really >>>>> >>>>> comparing >>>>> >>>>> apples to apples. >>>>> >>>>> >>>>> >>>>>> >>>>> >>>>>> >>>>> >>>>>> thanks >>>>> >>>>>> >>>>> >>>>>> Frédéric >>>>> >>>>>> >>>>> >>>>>> On Sun, Nov 3, 2013 at 3:22 PM, Ronan Lamy >>>>> >>>>>> <ronan.l...@gmail.com> >>>>> >>>>>> wrote: >>>>> >>>>>> > Le 03/11/13 14:19, Jason Moore a écrit : >>>>> >>>>>> > >>>>> >>>>>> >> Ronan, >>>>> >>>>>> >> >>>>> >>>>>> >> Thanks for looking at the derivation code. We haven't ever >>>>> >>>>>> >> really >>>>> >>>>>> >> had >>>>> >>>>>> >> any review of it outside of our mechanical engineer group, so >>>>> >>>>>> >> this >>>>> >>>>>> >> very >>>>> >>>>>> >> helpful. >>>>> >>>>>> >> >>>>> >>>>>> >> I'll review your PR and look into the second two items. >>>>> >>>>>> >> >>>>> >>>>>> >> Why do you think the dictionary representation will be so >>>>> >>>>>> >> much >>>>> >>>>>> >> faster? >>>>> >>>>>> > >>>>> >>>>>> > >>>>> >>>>>> > Using a dict would make Vector.__eq__ a lot less expensive, >>>>> >>>>>> > and make >>>>> >>>>>> > canonicalisation (as currently done in Vector.__init__) >>>>> >>>>>> > O(len(args)) >>>>> >>>>>> > instead >>>>> >>>>>> > of O(len(args)**2). >>>>> >>>>>> > >>>>> >>>>>> > However, it seems that the main performance issue is that >>>>> >>>>>> > instantiating >>>>> >>>>>> > Matrix objects is expensive. Using a dict isn't strictly >>>>> >>>>>> > required to >>>>> >>>>>> > solve >>>>> >>>>>> > this, but the structure I suggest would allow e.g. dot >>>>> >>>>>> > products of >>>>> >>>>>> > vectors >>>>> >>>>>> > to be decomposed as combinations of multiplications of >>>>> >>>>>> > components >>>>> >>>>>> > and dot >>>>> >>>>>> > products of base vectors, which can be optimised or cached >>>>> >>>>>> > separately. There >>>>> >>>>>> > would be no need for intermediate Matrix objects. >>>>> >>>>>> > >>>>> >>>>>> > >>>>> >>>>>> >> >>>>> >>>>>> >> On Sat, Nov 2, 2013 at 10:11 PM, Ronan Lamy >>>>> >>>>>> >> <ronan.l...@gmail.com >>>>> >>>>>> >> <mailto:ronan.l...@gmail.com>> wrote: >>>>> >>>>>> >> >>>>> >>>>>> >> Le 01/11/13 11:54, Jason Moore a écrit : >>>>> >>>>>> >> >>>>> >>>>>> >> I've been tinkering with code generation for ODE's >>>>> >>>>>> >> that >>>>> >>>>>> >> sympy.physics.mechanics spits out and have some >>>>> >>>>>> >> results: >>>>> >>>>>> >> >>>>> >>>>>> >> http://www.moorepants.info/__blog/pydy-code-gen.html >>>>> >>>>>> >> >>>>> >>>>>> >> <http://www.moorepants.info/blog/pydy-code-gen.html> >>>>> >>>>>> >> >>>>> >>>>>> >> Several people have posted topics on this recently. >>>>> >>>>>> >> We need >>>>> >>>>>> >> to >>>>> >>>>>> >> build in >>>>> >>>>>> >> a code generator for solving ODE's into SymPy that >>>>> >>>>>> >> would >>>>> >>>>>> >> play >>>>> >>>>>> >> well with >>>>> >>>>>> >> the codegen and autowrap modules. I think I can use >>>>> >>>>>> >> this >>>>> >>>>>> >> code I >>>>> >>>>>> >> wrote as >>>>> >>>>>> >> a base to start working on that but would need some >>>>> >>>>>> >> help >>>>> >>>>>> >> generalizing it >>>>> >>>>>> >> beyond our systems. Feedback is welcome. >>>>> >>>>>> >> >>>>> >>>>>> >> >>>>> >>>>>> >> I've only looked at the derivation part, it's an >>>>> >>>>>> >> interesting >>>>> >>>>>> >> real-world(ish) benchmark for expression manipulation. >>>>> >>>>>> >> However, >>>>> >>>>>> >> it >>>>> >>>>>> >> incurs a lot of avoidable overhead: >>>>> >>>>>> >> * Extracting some loop constants out of their loops cuts >>>>> >>>>>> >> down >>>>> >>>>>> >> the >>>>> >>>>>> >> run-time by 40%, cf. >>>>> >>>>>> >> https://github.com/sympy/__sympy/pull/2570 >>>>> >>>>>> >> >>>>> >>>>>> >> <https://github.com/sympy/sympy/pull/2570> >>>>> >>>>>> >> * The creation of temporary objects by the Vector class >>>>> >>>>>> >> takes >>>>> >>>>>> >> up >>>>> >>>>>> >> most of the run-time. >>>>> >>>>>> >> * The internal representation of Vector objects seems >>>>> >>>>>> >> inefficient. I >>>>> >>>>>> >> think it should be switched to a dict-based >>>>> >>>>>> >> representation as a >>>>> >>>>>> >> linear combination of base vectors (e.g. using {I.x: l0, >>>>> >>>>>> >> B.y: >>>>> >>>>>> >> -l1} >>>>> >>>>>> >> for l0 * I.x - l1 * B.y). >>>>> >>>>>> >> >>>>> >>>>>> >> >>>>> >>>>>> >> -- >>>>> >>>>>> >> You received this message because you are subscribed to >>>>> >>>>>> >> the >>>>> >>>>>> >> Google >>>>> >>>>>> >> Groups "sympy" group. >>>>> >>>>>> >> To unsubscribe from this group and stop receiving emails >>>>> >>>>>> >> from >>>>> >>>>>> >> it, >>>>> >>>>>> >> send an email to sympy+unsubscribe@__googlegroups.com >>>>> >>>>>> >> <mailto:sympy%2bunsubscr...@googlegroups.com>. >>>>> >>>>>> >> >>>>> >>>>>> >> To post to this group, send email to >>>>> >>>>>> >> sympy@googlegroups.com >>>>> >>>>>> >> <mailto:sympy@googlegroups.com>. >>>>> >>>>>> >> Visit this group at >>>>> >>>>>> >> http://groups.google.com/__group/sympy >>>>> >>>>>> >> <http://groups.google.com/group/sympy>. >>>>> >>>>>> >> For more options, visit >>>>> >>>>>> >> https://groups.google.com/__groups/opt_out >>>>> >>>>>> >> <https://groups.google.com/groups/opt_out>. >>>>> >>>>>> >> >>>>> >>>>>> >> >>>>> >>>>>> >> >>>>> >>>>>> >> -- >>>>> >>>>>> >> You received this message because you are subscribed to the >>>>> >>>>>> >> Google >>>>> >>>>>> >> Groups "sympy" group. >>>>> >>>>>> >> To unsubscribe from this group and stop receiving emails from >>>>> >>>>>> >> it, >>>>> >>>>>> >> send >>>>> >>>>>> >> an email to sympy+unsubscr...@googlegroups.com. >>>>> >>>>>> >> To post to this group, send email to sympy@googlegroups.com. >>>>> >>>>>> >> Visit this group at http://groups.google.com/group/sympy. >>>>> >>>>>> >> For more options, visit >>>>> >>>>>> >> https://groups.google.com/groups/opt_out. >>>>> >>>>>> > >>>>> >>>>>> > >>>>> >>>>>> > -- >>>>> >>>>>> > You received this message because you are subscribed to the >>>>> >>>>>> > Google >>>>> >>>>>> > Groups >>>>> >>>>>> > "sympy" group. >>>>> >>>>>> > To unsubscribe from this group and stop receiving emails from >>>>> >>>>>> > it, >>>>> >>>>>> > send an >>>>> >>>>>> > email to sympy+unsubscr...@googlegroups.com. >>>>> >>>>>> > To post to this group, send email to sympy@googlegroups.com. >>>>> >>>>>> > Visit this group at http://groups.google.com/group/sympy. >>>>> >>>>>> > For more options, visit >>>>> >>>>>> > https://groups.google.com/groups/opt_out. >>>>> >>>>>> >>>>> >>>>>> -- >>>>> >>>>>> You received this message because you are subscribed to the >>>>> >>>>>> Google >>>>> >>>>>> Groups "sympy" group. >>>>> >>>>>> To unsubscribe from this group and stop receiving emails from >>>>> >>>>>> it, send >>>>> >>>>>> an email to sympy+unsubscr...@googlegroups.com. >>>>> >>>>>> To post to this group, send email to sympy@googlegroups.com. >>>>> >>>>>> Visit this group at http://groups.google.com/group/sympy. >>>>> >>>>>> For more options, visit >>>>> >>>>>> https://groups.google.com/groups/opt_out. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> You received this message because you are subscribed to the >>>>> >>>>> Google >>>>> >>>>> Groups "sympy" group. >>>>> >>>>> To unsubscribe from this group and stop receiving emails from it, >>>>> >>>>> send >>>>> >>>>> an email to sympy+unsubscr...@googlegroups.com. >>>>> >>>>> To post to this group, send email to sympy@googlegroups.com. >>>>> >>>>> Visit this group at http://groups.google.com/group/sympy. >>>>> >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>>>> >>>> >>>>> >>>> -- >>>>> >>>> You received this message because you are subscribed to the Google >>>>> >>>> Groups "sympy" group. >>>>> >>>> To unsubscribe from this group and stop receiving emails from it, >>>>> >>>> send >>>>> >>>> an email to sympy+unsubscr...@googlegroups.com. >>>>> >>>> To post to this group, send email to sympy@googlegroups.com. >>>>> >>>> Visit this group at http://groups.google.com/group/sympy. >>>>> >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>> >>>>> >>> >>>>> >>> -- >>>>> >>> You received this message because you are subscribed to the Google >>>>> >>> Groups >>>>> >>> "sympy" group. >>>>> >>> To unsubscribe from this group and stop receiving emails from it, >>>>> >>> send an >>>>> >>> email to sympy+unsubscr...@googlegroups.com. >>>>> >>> To post to this group, send email to sympy@googlegroups.com. >>>>> >>> Visit this group at http://groups.google.com/group/sympy. >>>>> >>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> You received this message because you are subscribed to the Google >>>>> >> Groups >>>>> >> "sympy" group. >>>>> >> To unsubscribe from this group and stop receiving emails from it, >>>>> >> send an >>>>> >> email to sympy+unsubscr...@googlegroups.com. >>>>> >> To post to this group, send email to sympy@googlegroups.com. >>>>> >> Visit this group at http://groups.google.com/group/sympy. >>>>> >> For more options, visit https://groups.google.com/groups/opt_out. >>>>> > >>>>> > >>>>> > -- >>>>> > You received this message because you are subscribed to the Google >>>>> > Groups >>>>> > "sympy" group. >>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>> > send an >>>>> > email to sympy+unsubscr...@googlegroups.com. >>>>> > To post to this group, send email to sympy@googlegroups.com. >>>>> > Visit this group at http://groups.google.com/group/sympy. >>>>> > For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "sympy" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to sympy+unsubscr...@googlegroups.com. >>>>> To post to this group, send email to sympy@googlegroups.com. >>>>> Visit this group at http://groups.google.com/group/sympy. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "sympy" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to sympy+unsubscr...@googlegroups.com. >>>> To post to this group, send email to sympy@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/sympy. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "sympy" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to sympy+unsubscr...@googlegroups.com. >>> To post to this group, send email to sympy@googlegroups.com. >>> Visit this group at http://groups.google.com/group/sympy. >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sympy+unsubscr...@googlegroups.com. >> To post to this group, send email to sympy@googlegroups.com. >> Visit this group at http://groups.google.com/group/sympy. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sympy+unsubscr...@googlegroups.com. >> To post to this group, send email to sympy@googlegroups.com. >> Visit this group at http://groups.google.com/group/sympy. >> For more options, visit https://groups.google.com/groups/opt_out. > > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
