Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 23/04/2015 23:51, Stéphane Mottelet a écrit : Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] This kind of large vectors are the used to build a sparse matrix each time the vector v changes, but with a constant sparsity pattern. Actually, the time spent by Scilab in the statement M1=sparse(M1_ij,M1_v,[n1,n2]) is negligible compared to the time spent to build f M1_v... I have also noticed that if you need to define such a matrix with more that one column, the time elapsed is not linear with respect to the number of columns: typically 4 times slower for 2 columns. In fact the statement v=[1 1 ... 1000 1000] is even two times slower than v1=[1 ... 1000]; v2=[1 1000]; v=[v1 v2]; So my question to users who have the experience of dynamic link of user code : do you think that using dynamic link of compiled generated C code could improve the timings ? Mais apriori je ne fais rien qui soit OS dependant... As your code is generated it should effectively a good idea to generate C code and use incremental linking (once the code has been compiled and link you can expect a speed factor around 100 times. But the compilation may be slow. So using dynmaic linking is a very good idea if your generated code has to be run many times. Serge In advance, thanks for your help ! S. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
If A stays constant this should work well. Yes, doing this as a C call will speed things up, too, but if you can do it with matrices and no C calls your code may stand the test of time better. What seems to work for me is to keep in mind that Scilab tends to be slow with individual operations, but without much of a speed hit for matrix operations. So any place that you can replace a 'for' loop or string of statements with a matrix operation, you can save time. On Fri, 2015-04-24 at 14:50 +0200, Stéphane Mottelet wrote: > Hello Antoine > > > I think your idea is excellent in my context, I Will try it and keep > you informed of the actual improvement, > > > S. > > Le 24 avr. 2015 à 13:31, Antoine Monmayrant > a écrit : > > > > Hi again, > > > > Another way you might improve your code: can you write it as a > > matrix-vector product? > > Like > > > > M1_v=A*v; > > Where A is a matrix of size (839,172) that corresponds to your weird > > combination of v(i) to build M1_v. > > It seems to me that you should be able to write it that way, which > > would save a lot of time as matrix product are fast in Scilab. > > > > Antoine > > > > > > Le 04/23/2015 11:51 PM, Stéphane Mottelet a écrit : > > > > > Hello, > > > > > > I am currently working on a project where Scilab code is > > > automatically generated, and after many code optimization, the > > > remaining bottleneck is the time that Scilab spends to execute > > > simple code like this (full script (where the vector has 839 > > > lines) with timings is attached) : > > > > > > M1_v=[v(17) > > > v(104) > > > v(149) > > > -(v(18)+v(63)+v(103)) > > > -(v(18)+v(63)+v(103)) > > > v(17) > > > ... > > > v(104) > > > v(149) > > > ] > > > > > > This kind of large vectors are the used to build a sparse matrix > > > each time the vector v changes, but with a constant sparsity > > > pattern. Actually, the time spent by Scilab in the statement > > > > > > M1=sparse(M1_ij,M1_v,[n1,n2]) > > > > > > is negligible compared to the time spent to build f M1_v... > > > > > > I have also noticed that if you need to define such a matrix with > > > more that one column, the time elapsed is not linear with respect > > > to the number of columns: typically 4 times slower for 2 columns. > > > In fact the statement > > > > > > v=[1 1 > > > ... > > > 1000 1000] > > > > > > is even two times slower than > > > > > > v1=[1 > > > ... > > > 1000]; > > > v2=[1 > > > > > > 1000]; > > > v=[v1 v2]; > > > > > > So my question to users who have the experience of dynamic link of > > > user code : do you think that using dynamic link of compiled > > > generated C code could improve the timings ? > > > > > > In advance, thanks for your help ! > > > > > > S. > > > > > > > > > > > > > > > ___ > > > users mailing list > > > users@lists.scilab.org > > > http://lists.scilab.org/mailman/listinfo/users > > > > > > > > ___ > > users mailing list > > users@lists.scilab.org > > http://lists.scilab.org/mailman/listinfo/users > > > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users -- Tim Wescott www.wescottdesign.com Control & Communications systems, circuit & software design. Phone: 503.631.7815 Cell: 503.349.8432 ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hello,
You will find a description of the actual problem which is solved by the
generated code : http://arxiv.org/pdf/1206.5072v1.pdf
See particularly page 6, where I explain the way the code generation
works. However, be warned that the code snipplets are quite obsolete,
since the generated code is now more efficient.
For matrices M_k(v) (the ones the thread is about), Antoine's answer
give a solution because the element of this matrix are linear
combinations of the v(i)'s.
But for other matrices, wich are derivatives of the fonctions expressing
the non-linear system with respect to state or parameters, the terms to
assemble are non-linear algebraic expressions mixing states and v(i)'s.
S.
Le 24/04/2015 15:19, Brian Bouterse a écrit :
I don't know much about scilab optimizations, but trying to
hand-optimize auto-generated code almost certainly will result in
correctness problems. It's likely better to make your improvements in
the software that is generating the scilab code. I think the main
feature there is to have it generate a vectorized implementation
instead of a procedural one (as you have now).
-Brian
On Fri, Apr 24, 2015 at 8:34 AM, Stéphane Mottelet
mailto:stephane.motte...@utc.fr>> wrote:
Hello,
this is not trivial indexing, in fact some terms are linear
combination of v's components
M1_v=[v(17)
v(104)
v(149)
-(v(18)+v(63)+v(103))
-(v(18)+v(63)+v(103))
v(17)
...
v(104)
v(149)
]
How do you take this into account in your proposed method ? These
combinations are sums of influxes in a metabolic network, and the
code is automatically generated.
S.
Le 24/04/2015 13:48, Samuel Gougeon a écrit :
Le 24/04/2015 13:36, Samuel Gougeon a écrit :
Hello Stephane,
You can speed up by a factor larger than 100 just by calling v
once (or 3 times) instead of ~1000, as shown by this test:
Actually, to be more accurate, the right comparative test is the
following:
function test2()
v = rand(172,1);
p = grand(1,839,"unf",1,173)
// Part 1: 1 call to v()
tic()
for i=1:1000
m1_v = v(p)
end
disp(toc())
// Part 2 : 839 calls to v()
deff("test3()", "for i=1:1000, M1_v = ["+strcat("v("+string(p)+")")+"],
end")
tic()
test3()
disp(toc())
endfunction
In this version, the compilation time used by execstr() is no
longer taken into account.
The results are still explicit:
-->test2()
0.016
0.78
So, a speed-up by ~x 50
Samuel
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
--
Brian Bouterse
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 24/04/2015 15:35, Samuel Gougeon a écrit : Le 24/04/2015 14:50, Stéphane Mottelet a écrit : Hello Antoine I think your idea is excellent in my context, I Will try it and keep you informed of the actual improvement, S. Le 24 avr. 2015 à 13:31, Antoine Monmayrant mailto:antoine.monmayr...@laas.fr>> a écrit : Hi again, Another way you might improve your code: can you write it as a matrix-vector product? Like M1_v=A*v; Where A is a matrix of size (839,172) that corresponds to your weird combination of v(i) to build M1_v. It seems to me that you should be able to write it that way, which would save a lot of time as matrix product are fast in Scilab. The test: A = sign(sprand(839,172,2/172, "normal")); i = A>0; A(i) = 1; i = A<0; A(i) = -1; p = rand(172,1); tic() for i = 1:1000 M1_v = A*p; end toc() gives -->toc() ans = 0.156 But somewhere you will somehow have to actually preset A(1,17) = 1; A(2,104) = 1; A(3,149) = 1; A(4, [18 63 103]) = -1; A(5, [18 63 103]) = -1; A(6,17) = 1; ... A(838, 104) = 1; A(839, 149) = 1; A priori, this is a bit more human-readable, but also more time-consuming... I am afraid that doing this in this way you will loose the gain "got" with matrix multiplication. Samuel No it is not a problem because the A matrix has to be created only once, since its sparsity pattern won't change later on. S. Antoine Le 04/23/2015 11:51 PM, Stéphane Mottelet a écrit : Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] .../... ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 24/04/2015 15:35, Samuel Gougeon a écrit : .../... The test: A = sign(sprand(839,172,2/172, "normal")); Sorry for this bad mixing between an unworkable line and its workaround. Dropping sign() on the first line is OK for a copy/past: A = sprand(839,172,2/172, "normal"); i = A>0; A(i) = 1; i = A<0; A(i) = -1; p = rand(172,1); tic() for i = 1:1000 M1_v = A*p; end toc() ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 24/04/2015 14:50, Stéphane Mottelet a écrit : Hello Antoine I think your idea is excellent in my context, I Will try it and keep you informed of the actual improvement, S. Le 24 avr. 2015 à 13:31, Antoine Monmayrant mailto:antoine.monmayr...@laas.fr>> a écrit : Hi again, Another way you might improve your code: can you write it as a matrix-vector product? Like M1_v=A*v; Where A is a matrix of size (839,172) that corresponds to your weird combination of v(i) to build M1_v. It seems to me that you should be able to write it that way, which would save a lot of time as matrix product are fast in Scilab. The test: A = sign(sprand(839,172,2/172, "normal")); i = A>0; A(i) = 1; i = A<0; A(i) = -1; p = rand(172,1); tic() for i = 1:1000 M1_v = A*p; end toc() gives -->toc() ans = 0.156 But somewhere you will somehow have to actually preset A(1,17) = 1; A(2,104) = 1; A(3,149) = 1; A(4, [18 63 103]) = -1; A(5, [18 63 103]) = -1; A(6,17) = 1; ... A(838, 104) = 1; A(839, 149) = 1; A priori, this is a bit more human-readable, but also more time-consuming... I am afraid that doing this in this way you will loose the gain "got" with matrix multiplication. Samuel Antoine Le 04/23/2015 11:51 PM, Stéphane Mottelet a écrit : Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] .../... ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 04/24/2015 03:19 PM, Brian Bouterse a écrit :
I don't know much about scilab optimizations, but trying to
hand-optimize auto-generated code almost certainly will result in
correctness problems. It's likely better to make your improvements in
the software that is generating the scilab code. I think the main
feature there is to have it generate a vectorized implementation
instead of a procedural one (as you have now).
I agree with you: scilab is fast with linear algebra and slow for the
rest and so far you have not generated your scilab code with scilab
strength and weakness in mind.
So if you stick with Scilab, you should go for a matrix form (maybe a
sparse matrix) or follow Samuel's version.
Cheers,
Antoine
-Brian
On Fri, Apr 24, 2015 at 8:34 AM, Stéphane Mottelet
mailto:stephane.motte...@utc.fr>> wrote:
Hello,
this is not trivial indexing, in fact some terms are linear
combination of v's components
M1_v=[v(17)
v(104)
v(149)
-(v(18)+v(63)+v(103))
-(v(18)+v(63)+v(103))
v(17)
...
v(104)
v(149)
]
How do you take this into account in your proposed method ? These
combinations are sums of influxes in a metabolic network, and the
code is automatically generated.
S.
Le 24/04/2015 13:48, Samuel Gougeon a écrit :
Le 24/04/2015 13:36, Samuel Gougeon a écrit :
Hello Stephane,
You can speed up by a factor larger than 100 just by calling v
once (or 3 times) instead of ~1000, as shown by this test:
Actually, to be more accurate, the right comparative test is the
following:
function test2()
v = rand(172,1);
p = grand(1,839,"unf",1,173)
// Part 1: 1 call to v()
tic()
for i=1:1000
m1_v = v(p)
end
disp(toc())
// Part 2 : 839 calls to v()
deff("test3()", "for i=1:1000, M1_v = ["+strcat("v("+string(p)+")")+"],
end")
tic()
test3()
disp(toc())
endfunction
In this version, the compilation time used by execstr() is no
longer taken into account.
The results are still explicit:
-->test2()
0.016
0.78
So, a speed-up by ~x 50
Samuel
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
--
Brian Bouterse
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
I don't know much about scilab optimizations, but trying to hand-optimize
auto-generated code almost certainly will result in correctness problems.
It's likely better to make your improvements in the software that is
generating the scilab code. I think the main feature there is to have it
generate a vectorized implementation instead of a procedural one (as you
have now).
-Brian
On Fri, Apr 24, 2015 at 8:34 AM, Stéphane Mottelet wrote:
> Hello,
>
> this is not trivial indexing, in fact some terms are linear combination of
> v's components
>
> M1_v=[v(17)
> v(104)
> v(149)
> -(v(18)+v(63)+v(103))
> -(v(18)+v(63)+v(103))
> v(17)
> ...
> v(104)
> v(149)
> ]
>
> How do you take this into account in your proposed method ? These
> combinations are sums of influxes in a metabolic network, and the code is
> automatically generated.
>
>
> S.
>
>
>
> Le 24/04/2015 13:48, Samuel Gougeon a écrit :
>
> Le 24/04/2015 13:36, Samuel Gougeon a écrit :
>
> Hello Stephane,
>
> You can speed up by a factor larger than 100 just by calling v once (or 3
> times) instead of ~1000, as shown by this test:
>
>
> Actually, to be more accurate, the right comparative test is the following:
>
> function test2()
> v = rand(172,1);
> p = grand(1,839,"unf",1,173)
> // Part 1: 1 call to v()
> tic()
> for i=1:1000
> m1_v = v(p)
> end
> disp(toc())
>
> // Part 2 : 839 calls to v()
> deff("test3()", "for i=1:1000, M1_v = [ "+strcat("v("+string(p)+") ")+"],
> end")
> tic()
> test3()
> disp(toc())endfunction
>
>
> In this version, the compilation time used by execstr() is no longer taken
> into account.
>
> The results are still explicit:
> -->test2()
>
> 0.016
>
> 0.78
>
> So, a speed-up by ~x 50
>
> Samuel
>
>
>
> ___
> users mailing
> listusers@lists.scilab.orghttp://lists.scilab.org/mailman/listinfo/users
>
>
>
> ___
> users mailing list
> users@lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
>
>
--
Brian Bouterse
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 24/04/2015 14:34, Stéphane Mottelet a écrit : Hello, this is not trivial indexing, in fact some terms are linear combination of v's components M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] How do you take this into account in your proposed method ? These combinations are sums of influxes in a metabolic network, and the code is automatically generated. In this way: from 1> M1_v=[v(17) 2> v(104) 3> v(149) 4> -(v(18)+v(63)+v(103)) 5> -(v(18)+v(63)+v(103)) 6> v(17) ... 838> v(104) 839> v(149) ] set: i = [1 2 3 6 ... 838 839] iv = [17 104 149 17 ... 104 149] M1_v(i) = v(iv) im = [4 5 ] // or setdiff(1:839, i). ivm = [18 18 ... ] M1_v(im) = -v(ivm) // + loop over the rank of elements in the linear combination // + adapt according to the automatic generation of your code Samuel ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hello Antoine I think your idea is excellent in my context, I Will try it and keep you informed of the actual improvement, S. > Le 24 avr. 2015 à 13:31, Antoine Monmayrant a > écrit : > > Hi again, > > Another way you might improve your code: can you write it as a matrix-vector > product? > Like > > M1_v=A*v; > Where A is a matrix of size (839,172) that corresponds to your weird > combination of v(i) to build M1_v. > It seems to me that you should be able to write it that way, which would save > a lot of time as matrix product are fast in Scilab. > > Antoine > > > Le 04/23/2015 11:51 PM, Stéphane Mottelet a écrit : >> Hello, >> >> I am currently working on a project where Scilab code is automatically >> generated, and after many code optimization, the remaining bottleneck is the >> time that Scilab spends to execute simple code like this (full script (where >> the vector has 839 lines) with timings is attached) : >> >> M1_v=[v(17) >> v(104) >> v(149) >> -(v(18)+v(63)+v(103)) >> -(v(18)+v(63)+v(103)) >> v(17) >> ... >> v(104) >> v(149) >> ] >> >> This kind of large vectors are the used to build a sparse matrix each time >> the vector v changes, but with a constant sparsity pattern. Actually, the >> time spent by Scilab in the statement >> >> M1=sparse(M1_ij,M1_v,[n1,n2]) >> >> is negligible compared to the time spent to build f M1_v... >> >> I have also noticed that if you need to define such a matrix with more that >> one column, the time elapsed is not linear with respect to the number of >> columns: typically 4 times slower for 2 columns. In fact the statement >> >> v=[1 1 >> ... >> 1000 1000] >> >> is even two times slower than >> >> v1=[1 >> ... >> 1000]; >> v2=[1 >> >> 1000]; >> v=[v1 v2]; >> >> So my question to users who have the experience of dynamic link of user code >> : do you think that using dynamic link of compiled generated C code could >> improve the timings ? >> >> In advance, thanks for your help ! >> >> S. >> >> >> >> >> ___ >> users mailing list >> users@lists.scilab.org >> http://lists.scilab.org/mailman/listinfo/users > > > ___ > users mailing list > users@lists.scilab.org > http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hello,
this is not trivial indexing, in fact some terms are linear combination
of v's components
M1_v=[v(17)
v(104)
v(149)
-(v(18)+v(63)+v(103))
-(v(18)+v(63)+v(103))
v(17)
...
v(104)
v(149)
]
How do you take this into account in your proposed method ? These
combinations are sums of influxes in a metabolic network, and the code
is automatically generated.
S.
Le 24/04/2015 13:48, Samuel Gougeon a écrit :
Le 24/04/2015 13:36, Samuel Gougeon a écrit :
Hello Stephane,
You can speed up by a factor larger than 100 just by calling v once
(or 3 times) instead of ~1000, as shown by this test:
Actually, to be more accurate, the right comparative test is the
following:
function test2()
v = rand(172,1);
p = grand(1,839,"unf",1,173)
// Part 1: 1 call to v()
tic()
for i=1:1000
m1_v = v(p)
end
disp(toc())
// Part 2 : 839 calls to v()
deff("test3()", "for i=1:1000, M1_v = ["+strcat("v("+string(p)+")")+"],
end")
tic()
test3()
disp(toc())
endfunction
In this version, the compilation time used by execstr() is no longer
taken into account.
The results are still explicit:
-->test2()
0.016
0.78
So, a speed-up by ~x 50
Samuel
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 24/04/2015 13:36, Samuel Gougeon a écrit :
Hello Stephane,
You can speed up by a factor larger than 100 just by calling v once
(or 3 times) instead of ~1000, as shown by this test:
Actually, to be more accurate, the right comparative test is the following:
function test2()
v = rand(172,1);
p = grand(1,839,"unf",1,173)
// Part 1: 1 call to v()
tic()
for i=1:1000
m1_v = v(p)
end
disp(toc())
// Part 2 : 839 calls to v()
deff("test3()", "for i=1:1000, M1_v = ["+strcat("v("+string(p)+")")+"],
end")
tic()
test3()
disp(toc())
endfunction
In this version, the compilation time used by execstr() is no longer
taken into account.
The results are still explicit:
-->test2()
0.016
0.78
So, a speed-up by ~x 50
Samuel
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hello Stephane,
You can speed up by a factor larger than 100 just by calling v once (or
3 times) instead of ~1000, as shown by this test:
function test2()
// part 1 : only 1 call to v
v = rand(172,1);
p = grand(1,839,"unf",1,173)
tic()
for i=1:1000
m1_v = v(p)
end
disp(toc())
// part 2 : 839 calls to v
e = "M1_v = ["+strcat("v("+string(p)+")")+"]"
tic()
for i=1:1000
execstr(e)
end
disp(toc())
// part 3 : calls to "empty" execstr(), to be substrated from the previous
result
e = "M1_v = 0";
tic()
for i=1:1000
execstr(e)
end
disp(toc())
endfunction
On my PC, i get:
-->test2
0.015
1.888
0.015
So, you do not really need to go to compiled solutions, just to rewrite
839 lines into ~10 lines.
Regards
Samuel
Le 23/04/2015 23:51, Stéphane Mottelet a écrit :
Hello,
I am currently working on a project where Scilab code is automatically
generated, and after many code optimization, the remaining bottleneck
is the time that Scilab spends to execute simple code like this (full
script (where the vector has 839 lines) with timings is attached) :
M1_v=[v(17)
v(104)
v(149)
-(v(18)+v(63)+v(103))
-(v(18)+v(63)+v(103))
v(17)
...
v(104)
v(149)
]
This kind of large vectors are the used to build a sparse matrix each
time the vector v changes, but with a constant sparsity pattern.
Actually, the time spent by Scilab in the statement
M1=sparse(M1_ij,M1_v,[n1,n2])
is negligible compared to the time spent to build f M1_v...
I have also noticed that if you need to define such a matrix with more
that one column, the time elapsed is not linear with respect to the
number of columns: typically 4 times slower for 2 columns. In fact the
statement
v=[1 1
...
1000 1000]
is even two times slower than
v1=[1
...
1000];
v2=[1
1000];
v=[v1 v2];
So my question to users who have the experience of dynamic link of
user code : do you think that using dynamic link of compiled generated
C code could improve the timings ?
In advance, thanks for your help !
S.
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hi again, Another way you might improve your code: can you write it as a matrix-vector product? Like M1_v=A*v; Where A is a matrix of size (839,172) that corresponds to your weird combination of v(i) to build M1_v. It seems to me that you should be able to write it that way, which would save a lot of time as matrix product are fast in Scilab. Antoine Le 04/23/2015 11:51 PM, Stéphane Mottelet a écrit : Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] This kind of large vectors are the used to build a sparse matrix each time the vector v changes, but with a constant sparsity pattern. Actually, the time spent by Scilab in the statement M1=sparse(M1_ij,M1_v,[n1,n2]) is negligible compared to the time spent to build f M1_v... I have also noticed that if you need to define such a matrix with more that one column, the time elapsed is not linear with respect to the number of columns: typically 4 times slower for 2 columns. In fact the statement v=[1 1 ... 1000 1000] is even two times slower than v1=[1 ... 1000]; v2=[1 1000]; v=[v1 v2]; So my question to users who have the experience of dynamic link of user code : do you think that using dynamic link of compiled generated C code could improve the timings ? In advance, thanks for your help ! S. ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Le 04/24/2015 10:20 AM, Stéphane Mottelet a écrit : Hello, Ok, I will try dynamic link since I want the project to stay into the Scilab world, but another alternative is Julia (http://julialang.org/). I have made some tests this night and got a speedup of 25, the explanation is JIT compilation. Maybe will we have JIT compilation in Scilab 6 ? Hi Stéphane, JIT is not the only explanation for the speedup. (I also did some scilab->julia conversion earlier this year to see how much we could gain). It seems to me that your code is not particularly efficient for scilab, while it might be a bit less inefficient in julia. For example: - you are indexing a lot, which is not particularly fast in scilab. - you are re-indexing several times: ...,v(17),v(104),v(149),...,v(17),v(104),v(149),... -> part_v=[v(17),v(104),v(149)] ; ...,part_v,...,part_v, could be faster. - you might need to pre-allocate M1_v : M1_v=zeros(whatever_size_it_is) so that scilab won't have to re-allocate it while you make it grow by stacking scalar after scalar. - more generally, it seems your code is highly redundant, couldn't you save some time by reusing some precalculated vector bit, like [v(17),v(104),v(149)] ? Hope it helps, Antoine Best regards, S. Le 24/04/2015 09:30, awe...@hidglobal.com a écrit : Hello Stephane, We have a Scilab program which performs a numerical integration on data points in 3-dimensions - it has two nested loops. When the number of data points was large this was slow so we implemented the calculation function in C and got a speed improvement of about 24 times ! We also found three other improvements: using pointer arithmetic was faster than 'for' loops, 'pow(x, 2)' was faster than x*x, handling the data as 3 (N x 1) vectors was faster than using 1 (N x 3) matrix. each of these giving something like a 3-4% improvement - small compared to x24 but still worth having. If you don't mind tackling the dynamic linking it's probably worth the effort if you'll use this program a few times - good luck. Adrian. *Adrian Weeks * Development Engineer, Hardware Engineering EMEA Office: +44 (0)2920 528500 | Desk: +44 (0)2920 528523 | Fax: +44 (0)2920 520178_ __aweeks@hidglobal.com_ <mailto:awe...@hidglobal.com> HID Global Logo <http://www.hidglobal.com/> Unit 3, Cae Gwyrdd, Green meadow Springs, Cardiff, UK, CF15 7AB._ __www.hidglobal.com_ <http://www.hidglobal.com/> From: Stéphane Mottelet To: "International users mailing list for Scilab." Date: 23/04/2015 22:52 Subject:[Scilab-users] Ways to speed up simple things in Scilab ? Sent by:"users" Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] This kind of large vectors are the used to build a sparse matrix each time the vector v changes, but with a constant sparsity pattern. Actually, the time spent by Scilab in the statement M1=sparse(M1_ij,M1_v,[n1,n2]) is negligible compared to the time spent to build f M1_v... I have also noticed that if you need to define such a matrix with more that one column, the time elapsed is not linear with respect to the number of columns: typically 4 times slower for 2 columns. In fact the statement v=[1 1 ... 1000 1000] is even two times slower than v1=[1 ... 1000]; v2=[1 1000]; v=[v1 v2]; So my question to users who have the experience of dynamic link of user code : do you think that using dynamic link of compiled generated C code could improve the timings ? In advance, thanks for your help ! S. [attachment "test.sce" deleted by Adrian Weeks/CWL/EU/ITG] ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users -- Département de Génie Informatique EA 4297 Transformations Intégrées de la Matière Renouvelable Université de Technologie de Compiègne - CS 60319 60203 Compiègne cedex ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hello, Ok, I will try dynamic link since I want the project to stay into the Scilab world, but another alternative is Julia (http://julialang.org/). I have made some tests this night and got a speedup of 25, the explanation is JIT compilation. Maybe will we have JIT compilation in Scilab 6 ? Best regards, S. Le 24/04/2015 09:30, awe...@hidglobal.com a écrit : Hello Stephane, We have a Scilab program which performs a numerical integration on data points in 3-dimensions - it has two nested loops. When the number of data points was large this was slow so we implemented the calculation function in C and got a speed improvement of about 24 times ! We also found three other improvements: using pointer arithmetic was faster than 'for' loops, 'pow(x, 2)' was faster than x*x, handling the data as 3 (N x 1) vectors was faster than using 1 (N x 3) matrix. each of these giving something like a 3-4% improvement - small compared to x24 but still worth having. If you don't mind tackling the dynamic linking it's probably worth the effort if you'll use this program a few times - good luck. Adrian. *Adrian Weeks * Development Engineer, Hardware Engineering EMEA Office: +44 (0)2920 528500 | Desk: +44 (0)2920 528523 | Fax: +44 (0)2920 520178_ __aweeks@hidglobal.com_ <mailto:awe...@hidglobal.com> HID Global Logo <http://www.hidglobal.com/> Unit 3, Cae Gwyrdd, Green meadow Springs, Cardiff, UK, CF15 7AB._ __www.hidglobal.com_ <http://www.hidglobal.com/> From: Stéphane Mottelet To: "International users mailing list for Scilab." Date: 23/04/2015 22:52 Subject:[Scilab-users] Ways to speed up simple things in Scilab ? Sent by:"users" Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] This kind of large vectors are the used to build a sparse matrix each time the vector v changes, but with a constant sparsity pattern. Actually, the time spent by Scilab in the statement M1=sparse(M1_ij,M1_v,[n1,n2]) is negligible compared to the time spent to build f M1_v... I have also noticed that if you need to define such a matrix with more that one column, the time elapsed is not linear with respect to the number of columns: typically 4 times slower for 2 columns. In fact the statement v=[1 1 ... 1000 1000] is even two times slower than v1=[1 ... 1000]; v2=[1 1000]; v=[v1 v2]; So my question to users who have the experience of dynamic link of user code : do you think that using dynamic link of compiled generated C code could improve the timings ? In advance, thanks for your help ! S. [attachment "test.sce" deleted by Adrian Weeks/CWL/EU/ITG] ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users -- Département de Génie Informatique EA 4297 Transformations Intégrées de la Matière Renouvelable Université de Technologie de Compiègne - CS 60319 60203 Compiègne cedex ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
Re: [Scilab-users] Ways to speed up simple things in Scilab ?
Hello Stephane, We have a Scilab program which performs a numerical integration on data points in 3-dimensions - it has two nested loops. When the number of data points was large this was slow so we implemented the calculation function in C and got a speed improvement of about 24 times ! We also found three other improvements: using pointer arithmetic was faster than 'for' loops, 'pow(x, 2)' was faster than x*x, handling the data as 3 (N x 1) vectors was faster than using 1 (N x 3) matrix. each of these giving something like a 3-4% improvement - small compared to x24 but still worth having. If you don't mind tackling the dynamic linking it's probably worth the effort if you'll use this program a few times - good luck. Adrian. Adrian Weeks Development Engineer, Hardware Engineering EMEA Office: +44 (0)2920 528500 | Desk: +44 (0)2920 528523 | Fax: +44 (0)2920 520178 awe...@hidglobal.com Unit 3, Cae Gwyrdd, Green meadow Springs, Cardiff, UK, CF15 7AB. www.hidglobal.com From: Stéphane Mottelet To: "International users mailing list for Scilab." Date: 23/04/2015 22:52 Subject: [Scilab-users] Ways to speed up simple things in Scilab ? Sent by: "users" Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] This kind of large vectors are the used to build a sparse matrix each time the vector v changes, but with a constant sparsity pattern. Actually, the time spent by Scilab in the statement M1=sparse(M1_ij,M1_v,[n1,n2]) is negligible compared to the time spent to build f M1_v... I have also noticed that if you need to define such a matrix with more that one column, the time elapsed is not linear with respect to the number of columns: typically 4 times slower for 2 columns. In fact the statement v=[1 1 ... 1000 1000] is even two times slower than v1=[1 ... 1000]; v2=[1 1000]; v=[v1 v2]; So my question to users who have the experience of dynamic link of user code : do you think that using dynamic link of compiled generated C code could improve the timings ? In advance, thanks for your help ! S. [attachment "test.sce" deleted by Adrian Weeks/CWL/EU/ITG] ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users ___ users mailing list users@lists.scilab.org http://lists.scilab.org/mailman/listinfo/users
[Scilab-users] Ways to speed up simple things in Scilab ?
Hello, I am currently working on a project where Scilab code is automatically generated, and after many code optimization, the remaining bottleneck is the time that Scilab spends to execute simple code like this (full script (where the vector has 839 lines) with timings is attached) : M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) -(v(18)+v(63)+v(103)) v(17) ... v(104) v(149) ] This kind of large vectors are the used to build a sparse matrix each time the vector v changes, but with a constant sparsity pattern. Actually, the time spent by Scilab in the statement M1=sparse(M1_ij,M1_v,[n1,n2]) is negligible compared to the time spent to build f M1_v... I have also noticed that if you need to define such a matrix with more that one column, the time elapsed is not linear with respect to the number of columns: typically 4 times slower for 2 columns. In fact the statement v=[1 1 ... 1000 1000] is even two times slower than v1=[1 ... 1000]; v2=[1 1000]; v=[v1 v2]; So my question to users who have the experience of dynamic link of user code : do you think that using dynamic link of compiled generated C code could improve the timings ? In advance, thanks for your help ! S. function test() timer(); v=rand(172,1); for i=1:1000 M1_v=[v(17) v(104) v(149) -(v(18)+v(63)+v(103)) v(17) v(104) v(149) -(v(18)+v(63)+v(103)) v(17) v(104) v(149) -(v(18)+v(63)+v(103)) v(17) v(104) v(149) -(v(18)+v(63)+v(103)) v(17) v(104) v(149) -(v(18)+v(63)+v(103)) v(38) v(125) v(127) -(v(39)+v(41)+v(124)) v(38) v(125) v(127) -(v(39)+v(41)+v(124)) v(38) v(125) v(127) -(v(39)+v(41)+v(124)) v(38) v(125) v(127) -(v(39)+v(41)+v(124)) v(38) v(125) v(127) -(v(39)+v(41)+v(124)) v(15) v(102) v(147) -(v(16)+v(61)+v(85)+v(101)) v(15) v(102) v(147) -(v(16)+v(61)+v(85)+v(101)) v(36) -(v(37)+v(122)) v(36) -(v(37)+v(122)) v(36) -(v(37)+v(122)) v(67) -(v(68)+v(153)) v(67) -(v(68)+v(153)) v(67) -(v(68)+v(153)) v(67) -(v(68)+v(153)) v(67) -(v(68)+v(153)) v(67) -(v(68)+v(153)) v(82) -(v(83)+v(168)) v(82) -(v(83)+v(168)) v(82) -(v(83)+v(168)) v(82) -(v(83)+v(168)) v(70) -(v(71)+v(156)) v(70) -(v(71)+v(156)) v(70) -(v(71)+v(156)) v(70) -(v(71)+v(156)) v(63) v(150) -(v(64)+v(149)) v(63) v(150) -(v(64)+v(149)) v(63) v(150) -(v(64)+v(149)) v(63) v(150) -(v(64)+v(149)) v(63) v(150) -(v(64)+v(149)) v(61) v(127) -(v(41)+v(62)+v(147)) v(61) v(127) -(v(41)+v(62)+v(147)) v(43) v(135) -(v(49)+v(129)) v(43) v(135) -(v(49)+v(129)) v(43) v(135) -(v(49)+v(129)) v(43) v(135) -(v(49)+v(129)) v(69) v(156) v(160) v(161) v(164) v(166) v(168) -(v(70)+v(74)+v(75)+v(77)+v(78)+v(80)+v(82)+v(155)) v(69) v(156) v(160) v(161) v(164) v(166) v(168) -(v(70)+v(74)+v(75)+v(77)+v(78)+v(80)+v(82)+v(155)) v(69) v(156) v(160) v(161) v(164) v(166) v(168) -(v(70)+v(74)+v(75)+v(77)+v(78)+v(80)+v(82)+v(155)) v(69) v(156) v(160) v(161) v(164) v(166) v(168) -(v(70)+v(74)+v(75)+v(77)+v(78)+v(80)+v(82)+v(155)) v(48) v(135) v(136) -(v(49)+v(50)+v(57)+v(134)) v(48) v(135) v(136) -(v(49)+v(50)+v(57)+v(134)) v(48) v(135) v(136) -(v(49)+v(50)+v(57)+v(134)) v(26) v(114) -(v(27)+v(28)+v(112)) v(26) v(114) -(v(27)+v(28)+v(112)) v(26) v(114) -(v(27)+v(28)+v(112)) v(35) v(122) v(124) v(124) v(158) v(160) v(161) -(v(36)+v(38)+v(38)+v(72)+v(74)+v(75)+v(121)) v(35) v(122) v(124) v(124) v(158) v(160) v(161) -(v(36)+v(38)+v(38)+v(72)+v(74)+v(75)+v(121)) v(35) v(122) v(124) v(124) v(158) v(160) v(161) -(v(36)+v(38)+v(38)+v(72)+v(74)+v(75)+v(121)) v(44) v(131) v(144) -(v(45)+v(47)+v(58)+v(130)) v(44) v(131) v(144) -(v(45)+v(47)+v(58)+v(130)) v(44) v(131) v(144) -(v(45)+v(47)+v(58)+v(130)) v(44) v(131) v(144) -(v(45)+v(47)+v(58)+v(130)) v(44) v(131) v(144) -(v(45)+v(47)+v(58)+v(130)) v(10) v(15) v(17) v(18) v(23) v(38) v(41) v(51) v(52) v(54) v(55) v(60) v(72) v(74) v(75) v(108) v(153) -(v(22)+v(67)+v(84)+v(96)+v(101)+v(103)+v(104)+v(109)+v(124)+v(127)+v(137)+v(138)+v(140)+v(141)+v(146)+v(158)+v(160)+v(161)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(50) v(137) v(138) v(140) v(141) v(144) -(v(51)+v(52)+v(54)+v(55)+v(58)+v(136)) v(30) -(v(31)+v(116)) v(30) -(v(31)+v(116)) v(30) -(v(31)+v(116)) v(49) v(136) -(v(50)+v(135)) v(49) v(136) -(v(50)+v(135)) v(49) v(136) -(v(50)+v(135)) v(49) v(136) -(v(50)+v(135)) v(49) v(136) -(v(50)+v(135)) v(49) v(136) -(v(50)+v(135)) v(49) v(136) -(v(50)+v(135)) v(12) v(100) v(129) -(v
