It's because I didn't have fastcache installed.... After installing
it, by default I got:
certik@redhawk:~/repos/symengine/benchmarks(py)$ python kane2.py
Setup
Converting to SymEngine...
SymPy Jacobian:
Total time: 0.123499155045 s
SymEngine Jacobian:
Total time: 0.00305485725403 s
Speedup: 40.43x
(I am running some longer calculation at the same time, so the timing
is a bit different, but it got faster.)
But bigger cache doesn't seem to affect it, as you noted:
certik@redhawk:~/repos/symengine/benchmarks(py)$
SYMPY_CACHE_SIZE=50000 python kane2.py
Setup
Converting to SymEngine...
SymPy Jacobian:
Total time: 0.121557950974 s
SymEngine Jacobian:
Total time: 0.00302696228027 s
Speedup: 40.16x
More interestingly, I can now differentiate the actual bicycle
expression. So far I just took one element on the right hand side
vector, and differentiate with respect to q5. I substitute for symbols
first (in both SymPy and SymEngine). This is what I got:
certik@redhawk:~/repos/symengine/benchmarks(py)$ python kane3.py
Converting to SymPy...
Converting to SymEngine...
Done.
Subs
Eval initial expression
-14960.1119361
-14960.111936127382563475771853345057230700491739739146860248076 +
0.00000000000000000000000000000000000000000000000000000000000000*I
SymPy diff:
Total time: 3.90587186813 s
SymEngine diff:
Total time: 0.0883350372314 s
Speedup: 44.22x
Subs
Eval derivative
-31789.936485
-31789.936484889832248148585748003998439133612431284784993412159 +
0.00000000000000000000000000000000000000000000000000000000000000*I
Converting SymPy derivative to SymEngine...
Subs
Eval derivative
-31789.936485
-31789.936484889832248148585748003998439133612431284784993279022 +
0.00000000000000000000000000000000000000000000000000000000000000*I
The floating point numbers are calculated by substituting integers for
the symbols and evaluating. The shorter number is using machine double
precision, the longer number is using MPC (evaluates in complex plane,
to make sure we don't get any negative values under square roots).
They both agree as you can see. The initial expression number I
checked against sympy (substituting the numbers using sympy and
evaluating using sympy). For the derivatives, currently I just the
sympy derivative to symengine to evaluate as a floating point, since
sympy is just too slow (it's been running for 30 minutes -- it's stuck
at the .n() method in sympy). The SymEngine evaluation is immediate
for double precision and a fraction of a second for the MPC. But the
conversion between symengine and sympy works well and I tested the
initial expression in both, so I think this part works. And the fact
that sympy derivative calculated using sympy and converted to
symengine and symengine derivative agrees I think proves with pretty
high confidence that things work.
Now the next step is to calculate the whole jacobian and again test
that we get the same expression. It's very important to spend time and
check the numerical evaluation of the expression, since there could be
bugs in symengine. I just fixed one here:
https://github.com/sympy/symengine/pull/556 (it was giving me
different numbers for the initial expression, but now they agree).
This was a bug in subs, but there could be a bug in the derivative as
well.
After we get the same expressions, we can start benchmarking. I can
start profiling the derivative code in C++ and see if it is possible
to speed it up. I can also implement cache in symengine, though I
don't know if it would speed up anything here. Still, even if we only
get 45x speedup, then instead of 4 minutes, the whole bicycle should
take 5s. So I think that's a big deal. I will also try to benchmark
this against other software like Sage to get an idea how we are doing.
Ondrej
On Mon, Jul 20, 2015 at 11:03 PM, Jason Moore <moorepa...@gmail.com> wrote:
> Ondrej,
>
> I'm not sure why you don't see performance increase with increased cache.
> The following shows that the benchmarks do run faster with a large cache.
> Interestingly the memory doesn't seem to change (but I'm not sure I
> understand how they measure mem usage). Notice that the jacobian wrt to
> symbols is not affected by the cache size but the jacobians wrt to the
> functions is, as is substitution. It seems that .diff() calls subs a lot
> (see https://github.com/sympy/sympy/issues/9701), so maybe the slow down is
> all due to the caching used for subs.
>
> moorepants@moorepants-2170p:sympy_benchmarks_bjodah(larger-expr)$ asv
> --config asv.conf.conda.json run
> · Fetching recent changes.
> · Creating environments
> · Discovering benchmarks
> ·· Uninstalling from py2.7-fastcache-mpmath.
> ·· Building for py2.7-fastcache-mpmath
> ·· Installing into py2.7-fastcache-mpmath..
> · Running 9 total benchmarks (1 commits * 1 environments * 9 benchmarks)
> [ 0.00%] · For sympy commit hash 488f3c20:
> [ 0.00%] ·· Building for py2.7-fastcache-mpmath...
> [ 0.00%] ·· Benchmarking py2.7-fastcache-mpmath
> [ 11.11%] ··· Running integrate.TimeIntegration01.time_doit
> 365.73ms
> [ 22.22%] ··· Running integrate.TimeIntegration01.time_doit_meijerg
> 107.41ms
> [ 33.33%] ··· Running
> large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_functions
> 37M
> [ 44.44%] ··· Running
> large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_symbols
> 37M
> [ 55.56%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_subs
> 37M
> [ 66.67%] ··· Running
> large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_functions
> 1.75s
> [ 77.78%] ··· Running
> large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_symbols
> 35.49ms
> [ 88.89%] ··· Running
> large_exprs.TimeLargeExpressionOperations.time_manual_jacobian_wrt_functions
> 1.79s
> [100.00%] ··· Running large_exprs.TimeLargeExpressionOperations.time_subs
> 1.66s
> moorepants@moorepants-2170p:sympy_benchmarks_bjodah(larger-expr)$
> SYMPY_CACHE_SIZE=10000 asv --config asv.conf.conda.json run
> · Fetching recent changes.
> · Creating environments
> · Discovering benchmarks
> ·· Uninstalling from py2.7-fastcache-mpmath.
> ·· Building for py2.7-fastcache-mpmath
> ·· Installing into py2.7-fastcache-mpmath..
> · Running 9 total benchmarks (1 commits * 1 environments * 9 benchmarks)
> [ 0.00%] · For sympy commit hash 488f3c20:
> [ 0.00%] ·· Building for py2.7-fastcache-mpmath...
> [ 0.00%] ·· Benchmarking py2.7-fastcache-mpmath
> [ 11.11%] ··· Running integrate.TimeIntegration01.time_doit
> 361.81ms
> [ 22.22%] ··· Running integrate.TimeIntegration01.time_doit_meijerg
> 104.40ms
> [ 33.33%] ··· Running
> large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_functions
> 37M
> [ 44.44%] ··· Running
> large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_symbols
> 37M
> [ 55.56%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_subs
> 37M
> [ 66.67%] ··· Running
> large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_functions
> 56.62ms
> [ 77.78%] ··· Running
> large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_symbols
> 33.04ms
> [ 88.89%] ··· Running
> large_exprs.TimeLargeExpressionOperations.time_manual_jacobian_wrt_functions
> 55.25ms
> [100.00%] ··· Running large_exprs.TimeLargeExpressionOperations.time_subs
> 13.89ms
>
>
>
> Jason
> moorepants.info
> +01 530-601-9791
>
> On Mon, Jul 20, 2015 at 6:57 PM, Ondřej Čertík <ondrej.cer...@gmail.com>
> wrote:
>>
>> On Mon, Jul 20, 2015 at 7:48 PM, Ondřej Čertík <ondrej.cer...@gmail.com>
>> wrote:
>> > On Sun, Jul 19, 2015 at 4:57 PM, Jason Moore <moorepa...@gmail.com>
>> > wrote:
>> >> I just tried this out with jacobian() and subs() over the commits since
>> >> 0.7.3 to master. It's showing me that the new caching is the killer
>> >> slowdown:
>> >>
>> >> https://github.com/sympy/sympy/commit/a63005e4
>> >>
>> >> I've submitted a PR to Björn's repo:
>> >> https://github.com/bjodah/sympy_benchmarks_bjodah/pull/1/files
>> >>
>> >> I'm assuming it uses fastcache because I have it installed, but maybe
>> >> not.
>> >> I'm not sure how to control for dependencies yet.
>> >
>> > I modified your benchmark to first do the substitution for symbols,
>> > only then do the Jacobian:
>> >
>> >
>> > https://github.com/sympy/symengine/commit/2ababdb89e6c6db04d4a5df3e30185b824de9c0c
>> >
>> > Here are the results:
>> >
>> > certik@redhawk:~/repos/symengine/benchmarks(py)$ python kane2.py
>> > Setup
>> > Converting to SymEngine...
>> > SymPy Jacobian:
>> > Total time: 0.155253887177 s
>> > SymEngine Jacobian:
>> > Total time: 0.00277400016785 s
>> >
>> > Speedup: 55.97x
>> >
>> >
>> > I played with the cache size, it makes no difference. Your original
>> > benchmark ran about 1s on my machine, with SYMPY_CACHE_SIZE=5000 it
>> > took about 0.9s. So you can speedup SymPy itself just by using symbols
>> > instead of functions by a factor of 5.8x. Btw, I tested that the
>> > result that I get from symengine is exactly the same as in sympy
>> >
>> > (https://github.com/sympy/symengine/commit/2ababdb89e6c6db04d4a5df3e30185b824de9c0c#diff-e423f8edd8641c702513b2ea9c10eaecR32).
>> >
>> >
>> > How did you generate the benchmark? Can you generate a bit larger
>> > matrix? Once the symengine benchmark runs in about 1s, then I can try
>> > to speed it up (because I would see the result in the timing more
>> > easily).
>>
>> I actually found a much larger benchmark, where just a simple
>> differentiation of a single entry (after substituting functionsf or
>> symbols) takes 60s for sympy.
>>
>>
>> https://github.com/sympy/symengine/commit/31378da77e5463870b9a22fda7a17deb68068fa8
>>
>> Here is where I am saving the matrix:
>>
>>
>> https://github.com/sympy/symengine/commit/31378da77e5463870b9a22fda7a17deb68068fa8#diff-39829995ea91876984e13e3b828d740aR252
>>
>> Does this make sense? Is that a good benchmark? I love the size, I
>> just want to make sure I am saving the right thing. Is that the full
>> bicycle?
>>
>> Ondrej
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sympy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to sympy+unsubscr...@googlegroups.com.
>> To post to this group, send email to sympy@googlegroups.com.
>> Visit this group at http://groups.google.com/group/sympy.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sympy/CADDwiVDZ_JSgC-C_j5J0zmSOXS9TAQsGFv_XcmbBvx3ijPhPrg%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+unsubscr...@googlegroups.com.
> To post to this group, send email to sympy@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/CAP7f1AhavPNhXzdhUDUVH1ihgDgbc4FbdmRPBHrMW9HQ4ijZDA%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sympy/CADDwiVCNOioRCvXJ5SrdTn41m7R9RMpp_c9ASTY-oO5nVkR8LQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
