My previous statement was that Theano won over autowrap.ufuncify because theano did CSE internally (theano's basic data structure is a DAG).
My understanding is that your C friend is correct IF the code is written in the standard way, declaring each variable individually. I remember looking at older code of your groups in which all of the variables were stored in a dynamically allocated array. My C friend mentioned that most compilers won't dare touch such a structure because it's verify that transformations are safe. I have no idea what your various code generators are doing under the hood. By the way I did not intend to raise the issue of CSE. I'm mostly curious about the poor performance of Theano and I don't think this is that issue. On Tue, Nov 5, 2013 at 10:48 AM, Aaron Meurer <asmeu...@gmail.com> wrote: > I would still benchmark running cse and not. If cse() is fast enough, > it might out-balance the overhead of the code generation, especially > if it significantly reduces the resulting code size. > > Aaron Meurer > > On Tue, Nov 5, 2013 at 11:36 AM, Jason Moore <moorepa...@gmail.com> wrote: > > My roommate who's a C guy suggested that CSE was a waste of time if you > have > > a good compiler. He claimed that the compiler will do CSE anyways, so > doing > > it in the C code was not useful. I need to compare my Cython code output > > with and without CSE (it currently uses CSE). > > > > How would I use CSE with theano_function? I'm not doing that currently > and > > it wasn't apparent to me how to go about that. > > > > > > Jason > > moorepants.info > > +01 530-601-9791 > > > > > > On Tue, Nov 5, 2013 at 1:30 PM, Matthew Rocklin <mrock...@gmail.com> > wrote: > >> > >> In my experiments with sympy-theano earlier this year I found that it > was > >> competitive with our Fortran code generation (autowrap.ufuncify) on > single > >> array outputs and outperformed it when CSE were involved. I don't have > >> anything recorded that profiles scalar operations. > >> > >> Frederic might have some intuition on Theano's performance on large > scalar > >> computations. I've brought this up in the past and don't actually > remember > >> the answer. > >> > >> > >> On Tue, Nov 5, 2013 at 9:51 AM, Jason Moore <moorepa...@gmail.com> > wrote: > >>> > >>> Ok, so I think I'm comparing things correctly then. It is surprising > that > >>> the compiled Theano code doesn't seem to execute that fast. The > functions > >>> I'm generating are essentially evaluating very long sympy expressions, > maybe > >>> Theano isn't great at that... > >>> > >>> > >>> Jason > >>> moorepants.info > >>> +01 530-601-9791 > >>> > >>> > >>> On Tue, Nov 5, 2013 at 12:48 PM, Matthew Rocklin <mrock...@gmail.com> > >>> wrote: > >>>> > >>>> `sympy.printing.theanocode.theano_function` calls `theano.function` > >>>> > >>>> The result of `theano_function` is a compiled theano function. > >>>> > >>>> > >>>> > >>>> On Tue, Nov 5, 2013 at 9:27 AM, Jason Moore <moorepa...@gmail.com> > >>>> wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Jason > >>>>> moorepants.info > >>>>> +01 530-601-9791 > >>>>> > >>>>> > >>>>> On Tue, Nov 5, 2013 at 8:46 AM, Frédéric Bastien <no...@nouiz.org> > >>>>> wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> Just to know, witch version of Theano did you used? I have speed up > a > >>>>>> little the optimization phase of the compilation and I have a PR > that > >>>>>> will optimize this more. > >>>>> > >>>>> > >>>>> I'm using the master branch of Theano as of last week. > >>>>> > >>>>>> > >>>>>> > >>>>>> Also, Theano cache the compilation of the c code. Where this timming > >>>>>> done with a empty theano cache or a filled Theano cache? If you ran > >>>>>> the same benchmark multiple time on the same computer, only the > first > >>>>>> time Theano will compile the c code, the other time it will reuse > what > >>>>>> is in the cache. > >>>>> > >>>>> > >>>>> I time two things wrt to Theano: > >>>>> > >>>>> 1. The time it takes to call > sympy.printing.theanocode.theano_function > >>>>> > >>>>> and > >>>>> > >>>>> 2. The time it takes to call the function generated from 1. > >>>>> > >>>>> When does the Theano compilation happen? Is it in the call to > >>>>> theano_function or the first time I use the generated function? > >>>>> > >>>>> If it is the latter then my timing comparisons aren't really > comparing > >>>>> apples to apples. > >>>>> > >>>>>> > >>>>>> > >>>>>> thanks > >>>>>> > >>>>>> Frédéric > >>>>>> > >>>>>> On Sun, Nov 3, 2013 at 3:22 PM, Ronan Lamy <ronan.l...@gmail.com> > >>>>>> wrote: > >>>>>> > Le 03/11/13 14:19, Jason Moore a écrit : > >>>>>> > > >>>>>> >> Ronan, > >>>>>> >> > >>>>>> >> Thanks for looking at the derivation code. We haven't ever really > >>>>>> >> had > >>>>>> >> any review of it outside of our mechanical engineer group, so > this > >>>>>> >> very > >>>>>> >> helpful. > >>>>>> >> > >>>>>> >> I'll review your PR and look into the second two items. > >>>>>> >> > >>>>>> >> Why do you think the dictionary representation will be so much > >>>>>> >> faster? > >>>>>> > > >>>>>> > > >>>>>> > Using a dict would make Vector.__eq__ a lot less expensive, and > make > >>>>>> > canonicalisation (as currently done in Vector.__init__) > O(len(args)) > >>>>>> > instead > >>>>>> > of O(len(args)**2). > >>>>>> > > >>>>>> > However, it seems that the main performance issue is that > >>>>>> > instantiating > >>>>>> > Matrix objects is expensive. Using a dict isn't strictly required > to > >>>>>> > solve > >>>>>> > this, but the structure I suggest would allow e.g. dot products of > >>>>>> > vectors > >>>>>> > to be decomposed as combinations of multiplications of components > >>>>>> > and dot > >>>>>> > products of base vectors, which can be optimised or cached > >>>>>> > separately. There > >>>>>> > would be no need for intermediate Matrix objects. > >>>>>> > > >>>>>> > > >>>>>> >> > >>>>>> >> On Sat, Nov 2, 2013 at 10:11 PM, Ronan Lamy < > ronan.l...@gmail.com > >>>>>> >> <mailto:ronan.l...@gmail.com>> wrote: > >>>>>> >> > >>>>>> >> Le 01/11/13 11:54, Jason Moore a écrit : > >>>>>> >> > >>>>>> >> I've been tinkering with code generation for ODE's that > >>>>>> >> sympy.physics.mechanics spits out and have some results: > >>>>>> >> > >>>>>> >> http://www.moorepants.info/__blog/pydy-code-gen.html > >>>>>> >> > >>>>>> >> <http://www.moorepants.info/blog/pydy-code-gen.html> > >>>>>> >> > >>>>>> >> Several people have posted topics on this recently. We > need > >>>>>> >> to > >>>>>> >> build in > >>>>>> >> a code generator for solving ODE's into SymPy that would > >>>>>> >> play > >>>>>> >> well with > >>>>>> >> the codegen and autowrap modules. I think I can use this > >>>>>> >> code I > >>>>>> >> wrote as > >>>>>> >> a base to start working on that but would need some help > >>>>>> >> generalizing it > >>>>>> >> beyond our systems. Feedback is welcome. > >>>>>> >> > >>>>>> >> > >>>>>> >> I've only looked at the derivation part, it's an interesting > >>>>>> >> real-world(ish) benchmark for expression manipulation. > However, > >>>>>> >> it > >>>>>> >> incurs a lot of avoidable overhead: > >>>>>> >> * Extracting some loop constants out of their loops cuts down > >>>>>> >> the > >>>>>> >> run-time by 40%, cf. > https://github.com/sympy/__sympy/pull/2570 > >>>>>> >> > >>>>>> >> <https://github.com/sympy/sympy/pull/2570> > >>>>>> >> * The creation of temporary objects by the Vector class takes > >>>>>> >> up > >>>>>> >> most of the run-time. > >>>>>> >> * The internal representation of Vector objects seems > >>>>>> >> inefficient. I > >>>>>> >> think it should be switched to a dict-based representation > as a > >>>>>> >> linear combination of base vectors (e.g. using {I.x: l0, B.y: > >>>>>> >> -l1} > >>>>>> >> for l0 * I.x - l1 * B.y). > >>>>>> >> > >>>>>> >> > >>>>>> >> -- > >>>>>> >> You received this message because you are subscribed to the > >>>>>> >> Google > >>>>>> >> Groups "sympy" group. > >>>>>> >> To unsubscribe from this group and stop receiving emails from > >>>>>> >> it, > >>>>>> >> send an email to sympy+unsubscribe@__googlegroups.com > >>>>>> >> <mailto:sympy%2bunsubscr...@googlegroups.com>. > >>>>>> >> > >>>>>> >> To post to this group, send email to sympy@googlegroups.com > >>>>>> >> <mailto:sympy@googlegroups.com>. > >>>>>> >> Visit this group at http://groups.google.com/__group/sympy > >>>>>> >> <http://groups.google.com/group/sympy>. > >>>>>> >> For more options, visit > >>>>>> >> https://groups.google.com/__groups/opt_out > >>>>>> >> <https://groups.google.com/groups/opt_out>. > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> -- > >>>>>> >> You received this message because you are subscribed to the > Google > >>>>>> >> Groups "sympy" group. > >>>>>> >> To unsubscribe from this group and stop receiving emails from it, > >>>>>> >> send > >>>>>> >> an email to sympy+unsubscr...@googlegroups.com. > >>>>>> >> To post to this group, send email to sympy@googlegroups.com. > >>>>>> >> Visit this group at http://groups.google.com/group/sympy. > >>>>>> >> For more options, visit https://groups.google.com/groups/opt_out > . > >>>>>> > > >>>>>> > > >>>>>> > -- > >>>>>> > You received this message because you are subscribed to the Google > >>>>>> > Groups > >>>>>> > "sympy" group. > >>>>>> > To unsubscribe from this group and stop receiving emails from it, > >>>>>> > send an > >>>>>> > email to sympy+unsubscr...@googlegroups.com. > >>>>>> > To post to this group, send email to sympy@googlegroups.com. > >>>>>> > Visit this group at http://groups.google.com/group/sympy. > >>>>>> > For more options, visit https://groups.google.com/groups/opt_out. > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > >>>>>> Groups "sympy" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, > send > >>>>>> an email to sympy+unsubscr...@googlegroups.com. > >>>>>> To post to this group, send email to sympy@googlegroups.com. > >>>>>> Visit this group at http://groups.google.com/group/sympy. > >>>>>> For more options, visit https://groups.google.com/groups/opt_out. > >>>>> > >>>>> > >>>>> -- > >>>>> You received this message because you are subscribed to the Google > >>>>> Groups "sympy" group. > >>>>> To unsubscribe from this group and stop receiving emails from it, > send > >>>>> an email to sympy+unsubscr...@googlegroups.com. > >>>>> To post to this group, send email to sympy@googlegroups.com. > >>>>> Visit this group at http://groups.google.com/group/sympy. > >>>>> For more options, visit https://groups.google.com/groups/opt_out. > >>>> > >>>> > >>>> -- > >>>> You received this message because you are subscribed to the Google > >>>> Groups "sympy" group. > >>>> To unsubscribe from this group and stop receiving emails from it, send > >>>> an email to sympy+unsubscr...@googlegroups.com. > >>>> To post to this group, send email to sympy@googlegroups.com. > >>>> Visit this group at http://groups.google.com/group/sympy. > >>>> For more options, visit https://groups.google.com/groups/opt_out. > >>> > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups > >>> "sympy" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an > >>> email to sympy+unsubscr...@googlegroups.com. > >>> To post to this group, send email to sympy@googlegroups.com. > >>> Visit this group at http://groups.google.com/group/sympy. > >>> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "sympy" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to sympy+unsubscr...@googlegroups.com. > >> To post to this group, send email to sympy@googlegroups.com. > >> Visit this group at http://groups.google.com/group/sympy. > >> For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "sympy" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to sympy+unsubscr...@googlegroups.com. > > To post to this group, send email to sympy@googlegroups.com. > > Visit this group at http://groups.google.com/group/sympy. > > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
