On Tue, Aug 6, 2013 at 11:04 AM, Guy Parsey <guy.par...@gmail.com> wrote:
> Hello, > > Theano/Sympy questions > Fred, in terms of replacing sympy.Pieceiwse with a Theano equivalent, > since sympy.Piecewise attempts each condition (from ExprCondPair) until one > is valid, I would think that the closest equivalent would be a recursive > theano ifelse (ie. sympy.Piecewise((expr1,cond1),(expr2,cond2)) ~ > theano.ifelse(cond1,expr1,theano.ifelse(cond2,expr2,None))) statement so > that the expressions are not evaluated until the condition is achieved. > This might be too much of a patch though and I am not sure how to implement > it with the same argument structure as sympy.Pieceiwse. > It would work. But for the timming, I would also try with the switch op instead of ifelse. The ifelse currently have much bigger oveahead then the switch op, and even if the switch op evaluate both branch, it happen relatively frequently that it is faster then the ifelse. Especially if each expression have small size, like a scalar or a small vector. > With regards to using an externally defined theano graph or op (theano > wrapped sympy implementation), how would one pass them to theano_function? > It feels as though it would be analogous to the autowrap helpers argument, > or would this be an issue of merging graphs before calling theano.function? > If I can understand this step, I feel as though both of my problems would > be solvable (a wrapping of a piecewise along with a wrapping of an > interpolated function) > There is 2 steps. 1) make a theano op like descrided here: http://deeplearning.net/software/theano/tutorial/extending_theano.html 2) Modify theano_function to use the new Theano op for the equivalent SymPy op. It could be possible to make an autowrap for this, but I don't know enough of the detail to be sure. > In all of my simplified test cases (no pieceiwse or interpolations, 9 > heavily-linked non-linear ODEs), I also show that the lambdify function is > faster to evaluate than my created theano function. When passing a list of > expressions, if there is a commons subexpression that exists in two > expressions (not twice in the same expression), is it treated as such > during theano compilation or is each expression handled separately? > Theano will merge duplicated subexpression automatically by default. What is your function? If it is too "simple", for example just working on scalar, Theano have by default much overhead. As said for the other benchmark, there is tricks to remove a big part or even all of it, but it ask for some work on the developer. > > Codegen questions (maybe a different topic): > Jason, I have started playing with the sympy.printing.codegen (C for now, > Fortran later) as an alternative to theano (I would very much like to get > multiple methods working, and GPU acceleration makes me want to keep theano > implementation). There is a size-able speedup relative to lambdify or > theano_function, which is to be expected, but I arrive at the same problem > as above with regards to interpolated or piecewise functions. The initial > crash from using Piecewise functions comes from Routine calling > sympy.tensor.index_methods get_contraction_structure which explicitly > states no support for Piecewise function types. In principle, if one made a > symbolic implemented function (preferably without an analytical > representation), could one link the python object to the compiled code? > What attributes would said implemented_function need to have? > > Thank you for your responses, I am glad to hear that at least I am not > completely missing something. > Cheers, > Guy > > > On Tuesday, August 6, 2013 9:37:41 AM UTC-4, Jason Moore wrote: > >> Fred, >> >> I think on_used_input=ignore should be a default in Matthew's theano >> printing code or that arg needs to be pushed up to his layer. I hit that >> issue too. >> >> The code I have is here: https://github.com/** >> PythonDynamics/pydy-code-gen<https://github.com/PythonDynamics/pydy-code-gen> >> >> See the results.txt file for basic speed comparisons. I'm generating the >> ODEs for an n-link pendulum with mechanics and then see how fast it >> generates and simulates with scipy.odeint. >> >> The code doesn't work at the moment. I haven't touched it in a month and >> looks like some things have changed in sympy and/or theano. I'll work on >> the bugs now. >> >> But "python benchmark.py" should run it with sympy master and ?some? >> version of Theano. >> >> >> Jason >> moorepants.info >> +01 530-601-9791 >> >> >> On Tue, Aug 6, 2013 at 9:25 AM, Frédéric Bastien <no...@nouiz.org> wrote: >> >>> Hi, >>> >>> I don't know what is sympy.functions.elementary.**piecewise. Do Jason >>> answered that part? If not, I'll look into it to know how to make Theano >>> reproduce it. About converting any Sympy symbol to Theano symbol, when >>> there isn't a one to one matching, you can create a one to a full Theano >>> graph conversion. When this is possible, it is probably the best, as if you >>> make a new Theano op, it work work on the GPU. But if you make a Theano >>> graph, there is good change that the graph will already work on the GPU. >>> >>> If it is not possible to make a Theano graph for a sympy symbol, it is >>> possible to make a new Theano op that just wrap the sympy implementation. >>> Also, if this is a bottleneck, I recently added an example that show how to >>> use numba with Theano so speed up the python code in a Theano op. >>> >>> >>> Theano do not parallelize on the CPU, except for the call to BLAS, when >>> the BLAS library is parallel. On the GPU, it is parallel. >>> >>> Having on_unused_input=ignore is normal for complicated generated code. >>> That is why it was added. But when the code is simpler and not generated, >>> but all user manually coded, most of time it mean the user didn't do what >>> he wanted. If you know it is normal that you have unused input, there is no >>> problem to use that flag. >>> >>> >>> Jason, about the case where Theano is slower, can you send me the Theano >>> code? I would like to look at it. I'm very surprised that Theano is slower >>> then Sympy in this case and would like to know why it is like this. >>> >>> Fred >>> >>> >>> On Mon, Aug 5, 2013 at 9:03 PM, Jason Moore <moore...@gmail.com> wrote: >>> >>>> Guy, >>>> >>>> We're working on the same problem for sympy.physics.mechanics. Matthew >>>> Rocklin added support for matrix conversions in the theano code that is in >>>> SymPy and I used that, but found that theano was slower that lambdify for >>>> most of my cases (I only have two cores, so I'm not taking advantage of the >>>> Theano parallel stuff). I think writing specific code gen for ode >>>> integration is going to be the best bet. I'm happy to collaborate on this >>>> with you. >>>> >>>> >>>> Jason >>>> moorepants.info >>>> +01 530-601-9791 >>>> >>>> >>>> On Mon, Aug 5, 2013 at 3:13 PM, Guy Parsey <guy.p...@gmail.com> wrote: >>>> >>>>> Hello Everyone, >>>>> Thank you in advance for reading through my problem and for any input >>>>> you may have. I must say that I still feel like a novice programmer and my >>>>> problems may be easily solvable from a different mindset. My present >>>>> project entails time-integration of extremely stiff and non-linear ODEs >>>>> with regards to chemical kinetics (one derivative equation for each >>>>> variable species) and energy equations. Initially we were planning on >>>>> using >>>>> the sympy.lambdify function to create callable functions for the main >>>>> function along with the jacobian and passing said functions to >>>>> scipy.integrate.odeint, however this method only works for easier test >>>>> cases (fewer species and/or no energy equations) before being limited by >>>>> either the list recursion limit or segfaulting due to the limited stack >>>>> size. I know that both of these limits can be edited, but that fact that I >>>>> am reaching them makes me feel as though I am doing something extremely >>>>> inefficiently. Outside of the documentation of SymPy and Theano, I have >>>>> also been heavily using the BlogPost by Matthew Rocklin >>>>> http://matthewrocklin.**com/blog/work/2013/03/19/** >>>>> SymPy-Theano-part-1/<http://matthewrocklin.com/blog/work/2013/03/19/SymPy-Theano-part-1/>. >>>>> >>>>> Presently I am trying to use the mapping between Theano and SymPy >>>>> (sympy.printing.theanocode theano_function) to make my callable functions >>>>> and take advantage of the optimization routines. I have two major problems >>>>> and a few questions: >>>>> >>>>> 1st major problem: Though piecewise functions exist in SymPy >>>>> (sympy.functions.elementary.**piecewise) there is no counterpart in >>>>> Theano. Looking at the source of the inspiration for theanocode ( >>>>> https://github.com/nouiz/**theano_sympy/<https://github.com/nouiz/theano_sympy/> >>>>> graph_translation.py) I see that some of the SymPy equivalents were >>>>> defined as lambda functions. Is there an equivalent way to add Theano >>>>> conditional expressions wrapped into a function to add to the mapping >>>>> dictionary in theanocode.py? >>>>> >>>>> 2nd major problem: Similar to the problem above in that I am not sure >>>>> that the Theano counterpart is; some of the terms that I use are >>>>> interpolated functions (with one ODE variable as input) that we have >>>>> wrapped symbolically while providing a numerical implementation (so that >>>>> symbolic derivatives can be made, resulting in their own interpolations). >>>>> Is it possible to recreate the interpolation function as a Theano >>>>> operation >>>>> for use within the system of ODEs? >>>>> >>>>> Remain questions: >>>>> I presently have to flatten my input to theano_function to a list of >>>>> expressions and then wrap to return to a form (Jacobian is a matrix not a >>>>> vector); is it possible to have a matrix of different expressions as an >>>>> input to theano_function with a vector output? >>>>> >>>>> I know that a huge amount of Theano speed up is due to parallelization >>>>> of matrix operations (which I do not have), should I be focusing on SymPy >>>>> Autowrap/Ufuncify or my own code generation instead of trying to get >>>>> Theano >>>>> to play nicely? >>>>> >>>>> Stupid questions: >>>>> Does sympy.printing.theanocode.**theano_function automatically >>>>> optimize the compiled graph? >>>>> >>>>> Minor comment: >>>>> Perhaps unnecessary for most uses of the theano_function, but I needed >>>>> to modify function inputs so as to be able to use the keyword argument >>>>> 'on_unused_input=ignore' as opposed to 'raise' so that I did not need to >>>>> have all symbols in all equations. This may be avoided by having the >>>>> unused >>>>> symbols somehow (I don't know how) included in each expression. >>>>> >>>>> Thank you again for your time in reading my problems and any potential >>>>> help you may think of. I can attach code if necessary, I just didn't want >>>>> to make my post more confusing. >>>>> Have an excellent day. >>>>> Sincerely, >>>>> Guy Parsey >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "sympy" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to sympy+un...@**googlegroups.com. >>>>> To post to this group, send email to sy...@googlegroups.com. >>>>> >>>>> Visit this group at >>>>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>>>> . >>>>> For more options, visit >>>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>>> . >>>>> >>>>> >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "sympy" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to sympy+un...@**googlegroups.com. >>>> To post to this group, send email to sy...@googlegroups.com. >>>> >>>> Visit this group at >>>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>>> . >>>> For more options, visit >>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>> . >>>> >>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "sympy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to sympy+un...@**googlegroups.com. >>> To post to this group, send email to sy...@googlegroups.com. >>> >>> Visit this group at >>> http://groups.google.com/**group/sympy<http://groups.google.com/group/sympy> >>> . >>> For more options, visit >>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>> . >>> >>> >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
