Dear Juan Pablo, The email of the author of the paper is not included in your post, so please forward my replies below to him.
On 15 Sep 2010, at 10:51, Juan Pablo Carbajal wrote: > Moreover, about the online documentation, with online documentation we > do not mean that there is no available manual or tutorials on the > Internet. In contrast, we imply that there is not an online site with > the syntax of function of Octave, like R or Scilab. There is an "an online site with the syntax of function of Octave". To reach it, just go to http://www.octave.org and click "Docs" on the menu on the left. The direct link is: "http://www.gnu.org/software/octave/doc/ interpreter". The same documentation can be accessed from within Octave by typing the command "doc" at the Octave prompt. > About the outdated version of Octave I have to admit that our work > took place months ago and as you observe this does not have to do only > with Octave. Versions of other software are not the latest at this > time but it was when our work completed. [...snip...] > Furthermore, in our evaluation tests we do not optimize the codes but > we use the built-in functions of the software. I would like to ensure > you that all the comments and recommendations are taken seriously into > account and they will be included in our future work. Besides that, we > would not hesitate to contact the software representatives in future > for further advice and comments. It is against the intimate nature of scientific work to present results that are in a form that does not allow the community to reproduce them independently in order to assess their validity. In particular the information included in the section on performance evaluation is vague and incomplete if the code that has been run to get those timings is not distributed along with the paper. Indeed, by running what is my interpretation [attached below in Octave syntax] of the tests described in that section, I get results that differ by orders of magnitude to those presented in the paper. It would be a great contirbution to further development of free software to make the code of your benchmark tests publicly available to developers. Best regards, Carlo de Falco ----------8<----------- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %Miscellaneous operations freemat mathnium octave R Scilab %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %Loop test 10,000 × 10,000 601.606 798.788 1526.000 261.077 271.713 tic, for ii=1:1e4, for jj=1:1e4, end, end, toc() %Elapsed time is 14.674 seconds. % 2000 × 2000 random matrix^1000 1.573 3.886 0.592 0.745 29.398 a = rand(2000); tic, a^1e3; toc() %Elapsed time is 23.488 seconds. tic, a.^1e3; toc() %Elapsed time is 0.42365 seconds. % Sorting of 5,000,000 random values 4.545 94.692 1.581 1.449 2.300 a = rand (5e6, 1); tic (), sort (a); toc () %Elapsed time is 1.2075 seconds. % FFT over 220 random values 0.405 23.912 0.137 0.763 0.991 a = rand (220, 1); tic (), fft (a); toc () %Elapsed time is 0.00012302 seconds. % Calculation of 2,000,000 Fibonacci numbers 1.798 81.205 2.514 0.430 3.047 nf = 2e6; %with for loop tic; fib = ones (nf, 1); for ii=3:nf; fib(ii) = fib(ii-1)+fib(ii-2); end; toc() %Elapsed time is 39.529878 seconds. %with filter tic (); x = [1; zeros(nf-1, 1)]; a = [1 -1 -1]; b = 1; fibfil = filter(b, a, x); toc () %Elapsed time is 0.8 seconds. isequal (fib, fibfil) %ans = 1 % Factorial of a big integer (10 digits) 0.002 0.003 0.007 0.008 0.003 a = floor (rand (1)*1e10) %a = 8.2000e+09 tic, factorial (a); toc %Elapsed time is 0.000299 seconds. % Plot 2-D on 200,000 points 0.563 1.072 0.128 7.988 19.292 %with gnuplot a = rand (2e5, 1); tic (); plot (a); toc () %Elapsed time is 0.6472 seconds. (but the window takes much longer to show up) close all backend ('fltk') tic (); plot (a); toc () %Elapsed time is 0.04016 seconds. (and the window comes up very quickly) % Plot 3-D on 200,000 points 1.105 3.691 0.091 0.216 1.789 close all backend ('gnuplot') tic (); plot3 (a); toc () %Elapsed time is 0.4186 seconds. (but the window takes forever to show up) close all backend ('fltk') tic (); plot3 (a); toc () %Elapsed time is 0.04582 seconds. (and the window comes up very quickly) % Average performance for this group 37.71% 14.57% 68.49% 57.62% 31.01% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Matrix operations freemat mathnium octave R Scilab %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Matrix multiplication among two 2000 × 2000 random matrices 8.187 171.389 18.664 0.070 4.626 a = rand (2e3, 2e3); b = rand (2e3, 2e3); tic (); a*b; toc () %Elapsed time is 1.3 seconds. tic (); a.*b; toc () %Elapsed time is 0.06476 seconds. % Transpose of a 2000 × 2000 random matrix 0.311 2.494 0.110 1.853 0.127 tic (); a'; toc () %Elapsed time is 0.06137 seconds. tic (); a.'; toc () %Elapsed time is 0.066 seconds. % Creation of a 2000 × 2000 Hilbert matrix 0.042 - 0.351 0.519 0.229 tic (); a = hilb (2e3); toc () %Elapsed time is 0.3474 seconds. % Hessenberg form of a 2000 × 2000 random matrix - 501.603 1274.100 - 29.412 a = rand (2e3, 2e3); tic (); hess (a); toc () %Elapsed time is 11.886 seconds. % Rank of a 2000 × 2000 random matrix 32.150 308.015 27.225 15.597 29.273 tic (); rank (a); toc () %Elapsed time is 13.795 seconds. % Trace of a 2000 × 2000 random matrix 60.195 0.679 0.028 0.038 0.005 tic (); trace (a); toc () %Elapsed time is 0.003068 seconds. % Condition number of a 2000 × 2000 random matrix 491.47 3939.406 20.735 16.853 29.257 tic (); cond (a); toc () %Elapsed time is 13.394 seconds. % Kronecker product of two 200 × 200 random matrices - 20.367 0.210 0.337 0.102 a = rand (200); b = rand (200); tic; kron (a, b); toc () %error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt % Average performance for the tests of this group 31.35% 2.42% 39.75% 50.90% 64.68% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Basic algebra freemat mathnium octave R Scilab %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Determinant of a 2000 × 2000 random matrix 3.945 33.214 6.007 5.796 3.249 a = rand (2000); tic; det (a); toc () %Elapsed time is 0.739024 seconds. % Inverse of a 2000 × 2000 random matrix 533.880 78.364 18.991 24.409 9.489 tic; inv (a); toc () %Elapsed time is 2.23666 seconds. % Eigenvalues of a 2000 × 2000 random matrix 44.679 3645.349 58.462 59.596 46.147 tic; eig (a); toc () %Elapsed time is 47.4915 seconds. % Eigenvectors over a 2000 × 2000 random matrix 104.053 3787.40 125.200 126.105 540.456 tic; [v, l] = eig (a); toc () %Elapsed time is 120.715 seconds. % 2000 × 2000 dot product matrix 8.665 181.192 18.763 11.995 4.751 a = rand (2e3, 1); b = rand (1, 2e3); tic; a * b; toc () %% Is that what is meant by dot product matrix?? %Elapsed time is 0.0591681 seconds. % Norm of a 2000 × 2000 random matrix 30.000 6.623 27.196 0.180 29.240 a = rand (2e3); tic; norm (a); toc () %Elapsed time is 13.2002 seconds. % Linear system solve of 1500 equations 1.903 7.151 2.764 2.677 76.925 a = rand (1.5e3); b = rand (1.5e3, 1); tic; a \b; toc () %Elapsed time is 0.40785 seconds. % Average performance for the tests of this group 62.80% 8.26% 51.20% 66.16% 59.88% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Advanced algebra freemat mathnium octave R Scilab %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Cholesky decomposition of a 2000 × 2000 random matrix – - 51.931 2.843 2.920 1.716 a = rand (2e3); a = a*a'; tic; chol (a); toc () %Elapsed time is 0.388352 seconds. % Lu decomposition of a 1500 × 1500 random matrix 1.667 19.799 6.709 0.003 1.687 a = rand (1500); tic; lu (a); toc () %Elapsed time is 0.689038 seconds. % Qr decomposition of a 1200 × 1200 random matrix 3.055 54.817 24.710 3.234 2.970 a = rand (1200); tic; qr (a); toc () %Elapsed time is 0.819952 seconds. % Singular value decomposition of a 2000 × 2000 random matrix 59.016 3871.344 205.740 86.328 29.167 a = rand (2000); tic; svd (a); toc () %Elapsed time is 13.3108 seconds. % Schur decomposition of a 1500 × 1500 random matrix - 325.062 32.601 51.172 30.227 a = rand (1500); tic; schur (a); toc () %Elapsed time is 29.2586 seconds. % Reduced Row Echelon Form of a 2000 × 2000 random matrix 311.442 21.890 269.770 – - 144.047 a = rand (2000); tic; rref (a); toc () %Elapsed time is 196.287 seconds. % Average performance for the tests of this group 38.47% 19.80% 31.24% 68.69% 69.23% % Statistics % PCA over a 3000 × 300 random matrix – – 1.5395 1.7020 11.5281 % Gaussian error function on a 2000 × 2000 random matrix 0.110 1.003 0.571 11.202 0.161 % Linear regression over a 2000 × 2000 random matrix 3.717 172.287 6.025 17.848 3.412 % Arithmetic mean over a 2 × 106 random vector 0.034 4.686 0.153 0.057 0.001 % Geometric mean over a 2 × 106 random vector – – 1.284 4.937 0.002 % Harmonic mean over a 2 × 106 random vector – – 0.599 0.224 0.002 % Standard deviation over a 2 × 106 random vector 0.128 1.156 0.383 0.151 1.178 % Variance of a 1200 × 1200 random matrix 0.041 – 0.448 3.145 2.945 % Skewness of 5 × 106 random values 0.325 106.453 1.211 0.674 2.297 % Kurtosis of 5 × 106 random values 0.329 153.826 1.205 0.771 2.250 % Range values of a 2000 × 2000 random array 0.123 0.594 0.249 0.102 0.087 % Average performance for the tests of this group 83.19% 5.61% 28.07% 34.19% 56.59% ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Octave-dev mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/octave-dev
