On Friday, August 28, 2020, Ahmed Karaman <ahmedkhaledkara...@gmail.com> wrote:
> A nightly performance testing system to monitor any change in QEMU > performance across seventeen different targets. > > Reviewed-by: Aleksandar Markovic <aleksandar.qemu.de...@gmail.com> > The system includes eight different benchmarks to provide a variety > of testing workloads. > > dijkstra_double: > Find the shortest path between the source node and all other nodes > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > distances are double values. The value of n can be specified using > the -n flag. The default value is 2000. > > dijkstra_int32: > Find the shortest path between the source node and all other nodes > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > distances are int32 values. The value of n can be specified using > the -n flag. The default value is 2000. > > matmult_double: > Standard matrix multiplication of an n*n matrix of randomly generated > double numbers from 0 to 100. The value of n is passed as an argument > with the -n flag. The default value is 200. > > matmult_int32: > Standard matrix multiplication of an n*n matrix of randomly generated > integer numbers from 0 to 100. The value of n is passed as an > argument with the -n flag. The default value is 200. > > qsort_double: > Quick sort of an array of n randomly generated double numbers from 0 > to 1000. The value of n is passed as an argument with the -n flag. > The default value is 300000. > > qsort_int32: > Quick sort of an array of n randomly generated integer numbers from 0 > to 50000000. The value of n is passed as an argument with the -n > flag.The default value is 300000. > > qsort_string: > Quick sort of an array of 10000 randomly generated strings of size 8 > (including null terminating character). The sort process is repeated > n number of times. The value of n is passed as an argument with the > -n flag. The default value is 20. > > search_string: > Search for the occurrence of a small string in a much larger random > string (“needle in a hay”). The search process is repeated n number > of times and each time, a different large random string (“hay”) is > generated. The value of n can be specified using the -n flag. The > default value is 20. > > Syntax: > nightly_tests_core.py [-h] [-r REF] > Optional arguments: > -h, --help Show this help message and exit > -r REF, --reference REF > Reference QEMU version - Default is v5.1.0 > Example of usage: > nightly_tests_core.py -r v5.1.0 2>log.txt > > The following report includes detailed setup and execution details > of the system: > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/ > QEMU-Nightly-Performance-Tests/ > > Signed-off-by: Ahmed Karaman <ahmedkhaledkara...@gmail.com> > --- > tests/performance/nightly-tests/README.md | 243 +++++ > .../source/dijkstra_double/dijkstra_double.c | 194 ++++ > .../source/dijkstra_int32/dijkstra_int32.c | 192 ++++ > .../source/matmult_double/matmult_double.c | 123 +++ > .../source/matmult_int32/matmult_int32.c | 121 +++ > .../source/qsort_double/qsort_double.c | 104 ++ > .../source/qsort_int32/qsort_int32.c | 103 ++ > .../source/qsort_string/qsort_string.c | 122 +++ > .../source/search_string/search_string.c | 110 +++ > .../scripts/nightly_tests_core.py | 920 ++++++++++++++++++ > .../scripts/run_nightly_tests.py | 135 +++ > .../nightly-tests/scripts/send_email.py | 56 ++ > 12 files changed, 2423 insertions(+) > create mode 100644 tests/performance/nightly-tests/README.md > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > dijkstra_double/dijkstra_double.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > dijkstra_int32/dijkstra_int32.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > matmult_double/matmult_double.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > matmult_int32/matmult_int32.c > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_double/qsort_double.c > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_int32/qsort_int32.c > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_string/qsort_string.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > search_string/search_string.c > create mode 100755 tests/performance/nightly-tests/scripts/nightly_tests_ > core.py > create mode 100755 tests/performance/nightly-tests/scripts/run_nightly_ > tests.py > create mode 100644 tests/performance/nightly-tests/scripts/send_email.py > > diff --git a/tests/performance/nightly-tests/README.md > b/tests/performance/nightly-tests/README.md > new file mode 100644 > index 0000000000..6db3b351b3 > --- /dev/null > +++ b/tests/performance/nightly-tests/README.md > @@ -0,0 +1,243 @@ > +### QEMU Nightly Tests > + > +**Required settings:** > + > +Update the `GMAIL_USER` object in `send_email.py` with your credentials. > + > +For more details on how the system works, please check the [eighth > report](https://ahmedkrmn.github.io/TCG-Continuous- > Benchmarking/QEMU-Nightly-Performance-Tests/) of the "TCG Continuos > Benchmarking" series. > + > +**Running the System:** > + > +The default reference version is v5.1.0. To specify a custom version, > please use the `-r, --reference` flag. > + > +```bash > +./run_nightly_tests.py > +``` > + > +**Output:** > + > +``` > +Host CPU : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz > +Host Memory : 15.49 GB > + > +Start Time (UTC) : 2020-08-25 21:30:01 > +End Time (UTC) : 2020-08-25 22:02:37 > +Execution Time : 0:32:35.896990 > + > +Status : SUCCESS > + > +Note: > +Changes denoted by '-----' are less than 0.01%. > + > +-------------------------------------------------------- > + SUMMARY REPORT - COMMIT d1a2b51f > +-------------------------------------------------------- > + AVERAGE RESULTS > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 2 158 355 274 ----- +1.693% > +alpha 1 914 967 171 ----- +3.524% > +arm 8 076 402 940 ----- +2.304% > +hppa 4 261 685 987 -0.182% +3.164% > +m68k 2 690 273 044 ----- +7.131% > +mips 1 862 033 667 ----- +2.494% > +mipsel 2 008 211 069 ----- +2.674% > +mips64 1 918 635 565 ----- +2.818% > +mips64el 2 051 565 677 ----- +3.026% > +ppc 2 480 141 217 ----- +3.107% > +ppc64 2 576 713 959 ----- +3.143% > +ppc64le 2 558 853 539 ----- +3.173% > +riscv64 1 406 704 050 ----- +2.65% > +s390x 3 158 140 046 ----- +3.118% > +sh4 2 364 449 748 ----- +3.33% > +sparc64 3 318 544 783 ----- +3.851% > +x86_64 1 775 844 158 ----- +2.156% > +-------------------------------------------------------- > + > + DETAILED RESULTS > +-------------------------------------------------------- > +Test Program: dijkstra_double > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 3 062 583 464 ----- +1.424% > +alpha 3 191 864 698 ----- +3.696% > +arm 16 357 157 526 ----- +2.347% > +hppa 7 228 376 315 -0.139% +3.086% > +m68k 4 294 016 587 ----- +9.692% > +mips 3 051 419 166 ----- +2.427% > +mipsel 3 231 509 618 ----- +2.869% > +mips64 3 245 837 754 ----- +2.596% > +mips64el 3 414 195 796 ----- +3.021% > +ppc 4 914 520 972 -0.041% +4.74% > +ppc64 5 098 154 311 ----- +4.565% > +ppc64le 5 082 419 054 ----- +4.58% > +riscv64 2 192 294 915 ----- +1.955% > +s390x 4 584 503 977 ----- +2.896% > +sh4 3 949 036 447 ----- +3.464% > +sparc64 4 586 203 546 ----- +4.237% > +x86_64 2 484 092 105 ----- +1.75% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: dijkstra_int32 > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 2 210 194 577 ----- +1.493% > +alpha 1 494 133 274 ----- +2.15% > +arm 8 262 935 967 ----- +2.665% > +hppa 5 207 318 306 ----- +3.047% > +m68k 1 725 856 962 ----- +2.527% > +mips 1 495 227 032 ----- +1.492% > +mipsel 1 497 147 869 ----- +1.479% > +mips64 1 715 388 570 ----- +1.892% > +mips64el 1 695 276 864 ----- +1.913% > +ppc 2 014 557 389 ----- +1.819% > +ppc64 2 206 267 901 ----- +2.139% > +ppc64le 2 197 998 781 ----- +2.146% > +riscv64 1 354 912 745 ----- +2.396% > +s390x 2 916 247 062 ----- +1.241% > +sh4 1 990 532 533 ----- +2.669% > +sparc64 2 872 231 051 ----- +3.758% > +x86_64 1 553 981 241 ----- +2.12% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: matmult_double > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 1 412 273 223 ----- +0.302% > +alpha 3 233 991 649 ----- +7.473% > +arm 8 545 173 979 ----- +1.088% > +hppa 3 483 597 802 -1.267% +4.468% > +m68k 3 919 065 529 ----- +18.431% > +mips 2 344 774 894 ----- +4.091% > +mipsel 3 329 886 464 ----- +5.177% > +mips64 2 359 046 988 ----- +4.076% > +mips64el 3 343 664 785 ----- +5.167% > +ppc 3 209 457 051 ----- +3.246% > +ppc64 3 287 503 981 ----- +3.173% > +ppc64le 3 287 189 065 ----- +3.173% > +riscv64 1 221 603 682 ----- +0.277% > +s390x 2 874 199 923 ----- +5.827% > +sh4 3 543 943 634 ----- +6.416% > +sparc64 3 426 150 004 ----- +7.139% > +x86_64 1 248 917 276 ----- +0.322% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: matmult_int32 > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 598 681 621 ----- +0.585% > +alpha 372 437 418 ----- +0.677% > +arm 746 583 193 ----- +1.462% > +hppa 674 278 359 ----- +1.183% > +m68k 410 495 553 ----- +0.9% > +mips 499 698 837 ----- +0.531% > +mipsel 499 500 429 ----- +0.497% > +mips64 481 554 664 ----- +0.599% > +mips64el 465 057 054 ----- +0.619% > +ppc 341 334 603 ----- +0.944% > +ppc64 393 796 203 ----- +0.966% > +ppc64le 393 977 298 ----- +0.965% > +riscv64 351 709 769 ----- +0.785% > +s390x 494 427 384 ----- +0.599% > +sh4 402 668 444 ----- +0.899% > +sparc64 495 952 959 ----- +1.192% > +x86_64 402 928 461 ----- +0.833% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: qsort_double > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 2 709 683 624 ----- +2.417% > +alpha 1 969 460 172 ----- +3.68% > +arm 8 322 998 390 ----- +2.587% > +hppa 3 188 301 995 -0.047% +2.9% > +m68k 4 953 930 065 ----- +15.153% > +mips 2 123 919 587 ----- +3.055% > +mipsel 2 124 212 187 ----- +3.048% > +mips64 1 999 047 826 ----- +3.405% > +mips64el 1 996 426 772 ----- +3.409% > +ppc 2 819 267 902 -0.021% +5.435% > +ppc64 2 768 186 548 ----- +5.513% > +ppc64le 2 724 803 772 -0.011% +5.603% > +riscv64 1 638 328 937 ----- +4.021% > +s390x 2 519 081 708 ----- +3.362% > +sh4 2 595 545 154 ----- +2.994% > +sparc64 3 988 986 587 ----- +2.747% > +x86_64 2 033 468 588 ----- +3.234% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: qsort_int32 > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 2 193 392 565 ----- +2.916% > +alpha 1 521 291 933 ----- +4.193% > +arm 3 465 445 043 ----- +2.756% > +hppa 2 280 034 340 ----- +3.821% > +m68k 1 843 189 041 ----- +3.583% > +mips 1 558 024 873 ----- +3.863% > +mipsel 1 560 583 980 ----- +3.846% > +mips64 1 563 415 749 ----- +4.412% > +mips64el 1 542 677 320 ----- +4.474% > +ppc 1 728 698 880 ----- +3.665% > +ppc64 1 842 444 545 ----- +3.555% > +ppc64le 1 791 822 067 ----- +3.661% > +riscv64 1 348 866 430 ----- +4.654% > +s390x 2 184 073 151 ----- +3.319% > +sh4 1 946 492 539 ----- +3.624% > +sparc64 3 452 215 585 ----- +2.937% > +x86_64 1 813 544 414 ----- +3.537% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: qsort_string > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 2 592 218 418 ----- +2.468% > +alpha 1 855 834 626 ----- +3.487% > +arm 7 347 721 165 ----- +2.682% > +hppa 4 758 753 926 ----- +3.543% > +m68k 2 376 811 462 ----- +3.567% > +mips 2 166 608 045 ----- +2.532% > +mipsel 2 163 392 541 ----- +2.528% > +mips64 2 029 251 969 ----- +3.117% > +mips64el 2 011 628 621 ----- +3.145% > +ppc 2 492 942 463 ----- +2.673% > +ppc64 2 464 702 554 ----- +2.488% > +ppc64le 2 445 253 307 ----- +2.505% > +riscv64 1 625 053 328 ----- +3.953% > +s390x 4 194 608 798 ----- +6.623% > +sh4 2 164 142 539 ----- +3.166% > +sparc64 4 299 516 539 ----- +4.065% > +x86_64 2 940 456 780 ----- +2.649% > +-------------------------------------------------------- > +-------------------------------------------------------- > +Test Program: search_string > +-------------------------------------------------------- > +Target Instructions Latest v5.1.0 > +---------- -------------------- ---------- ---------- > +aarch64 2 487 814 704 ----- +1.94% > +alpha 1 680 723 605 ----- +2.835% > +arm 11 563 208 260 ----- +2.848% > +hppa 7 272 826 858 ----- +3.263% > +m68k 1 998 819 159 ----- +3.198% > +mips 1 656 596 909 ----- +1.959% > +mipsel 1 659 455 464 ----- +1.947% > +mips64 1 955 541 001 ----- +2.447% > +mips64el 1 943 598 207 ----- +2.462% > +ppc 2 320 350 477 ----- +2.332% > +ppc64 2 552 655 634 ----- +2.742% > +ppc64le 2 547 364 971 ----- +2.748% > +riscv64 1 520 862 601 ----- +3.159% > +s390x 5 497 978 370 ----- +1.078% > +sh4 2 323 236 696 ----- +3.41% > +sparc64 3 427 101 999 ----- +4.73% > +x86_64 1 729 364 402 ----- +2.806% > +-------------------------------------------------------- > +``` > diff --git a/tests/performance/nightly-tests/benchmarks/source/ > dijkstra_double/dijkstra_double.c b/tests/performance/nightly- > tests/benchmarks/source/dijkstra_double/dijkstra_double.c > new file mode 100644 > index 0000000000..9c0bb804ac > --- /dev/null > +++ b/tests/performance/nightly-tests/benchmarks/source/ > dijkstra_double/dijkstra_double.c > @@ -0,0 +1,194 @@ > +/* > + * Source file of a benchmark program involving calculations of the > + * shortest distances between a source node and all other nodes in a > + * graph of n nodes in which all nxn distances are defined as "double". > + * The number n can be given via command line, and the default is 2000. > + * The algorithm used is Dijsktra's. > + * > + * This file is a part of the project "TCG Continuous Benchmarking". > + * > + * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkara...@gmail.com> > + * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail. > com> > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see <https://www.gnu.org/licenses/>. > + * > + */ > + > +#include <float.h> > +#include <limits.h> > +#include <stdbool.h> > +#include <stdio.h> > +#include <stdlib.h> > +#include <unistd.h> > + > +/* Number of columns and rows in all matrixes*/ > +#define DEFAULT_NODE_COUNT 2000 > +#define MIN_NODE_COUNT 3 > +#define MAX_NODE_COUNT 10000 > + > + > +int32_t closest_index(int32_t count, double *distances, bool *flags) > +{ > + int32_t closest; > + double minimum = DBL_MAX; > + > + for (size_t i = 0; i < count; i++) { > + if (flags[i] == false && distances[i] <= minimum) { > + closest = i; > + minimum = distances[i]; > + } > + } > + > + return closest; > +} > + > +/** > + * Calculate the shortest distances from the source node using Dijkstra > method. > + * @param (out) distances An array of shortest distances from the source > node. > + * @param (out) via An array of nodes needed to be taken as the the last > + * before destination, for each destination. > + * @param (out) eccent Eccentricity of the source node. > + * @param (in) count The number of nodes. > + * @param (in) source Source node. > + * @param (in) matrix Distance matrix. > + */ > +void find_shortest_distances(double *distances, int32_t *via, double > *eccent, > + int32_t count, int32_t source, double > **matrix) > +{ > + bool *flags; > + > + flags = (bool *)malloc(count * sizeof(bool)); > + > + for (size_t i = 0; i < count; i++) { > + distances[i] = DBL_MAX; > + flags[i] = false; > + } > + > + distances[source] = 0.0; > + via[source] = source; > + > + for (size_t i = 0; i < count - 1; i++) { > + int32_t closest = closest_index(count, distances, flags); > + flags[closest] = true; > + for (size_t j = 0; j < count; j++) { > + if ((!flags[j]) && > + (matrix[closest][j]) && > + (distances[closest] != DBL_MAX) && > + (distances[j] > distances[closest] + > matrix[closest][j])) { > + distances[j] = distances[closest] + matrix[closest][j]; > + via[j] = closest; > + } > + } > + } > + > + *eccent = 0; > + for (size_t i = 0; i < count; i++) { > + if (*eccent < distances[i]) { > + *eccent = distances[i]; > + } > + } > + > + free(flags); > +} > + > + > +void main(int argc, char *argv[]) > +{ > + double **distance_matrix; > + double *shortest_distances; > + int32_t *via_node; > + int32_t node_count = DEFAULT_NODE_COUNT; > + int32_t source_node = 0; > + double node_eccentricity = 0.0; > + double range_factor = 999.0 / (double)(RAND_MAX); > + int32_t option; > + > + /* Parse command line options */ > + while ((option = getopt(argc, argv, "n:")) != -1) { > + if (option == 'n') { > + int32_t user_node_count = atoi(optarg); > + > + /* Check if the value is a string or zero */ > + if (user_node_count == 0) { > + fprintf(stderr, "Error ... Invalid value for option > '-n'.\n"); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is a negative number */ > + if (user_node_count < MIN_NODE_COUNT) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be a " > + "number less than %d.\n", MIN_NODE_COUNT); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is too large */ > + if (user_node_count > MAX_NODE_COUNT) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be " > + "more than %d.\n", MAX_NODE_COUNT); > + exit(EXIT_FAILURE); > + } > + node_count = user_node_count; > + } else { > + exit(EXIT_FAILURE); > + } > + } > + > + /* Allocate the memory space for all matrixes */ > + distance_matrix = (double **)malloc(node_count * sizeof(double *)); > + for (size_t i = 0; i < node_count; i++) { > + distance_matrix[i] = (double *)malloc(node_count * > sizeof(double)); > + } > + shortest_distances = (double *)malloc(node_count * sizeof(double)); > + via_node = (int32_t *)malloc(node_count * sizeof(int32_t)); > + > + /* Initialize helper arrays and populate distance_matrix */ > + srand(1); > + for (size_t i = 0; i < node_count; i++) { > + shortest_distances[i] = 0.0; > + via_node[i] = -1; > + distance_matrix[i][i] = 0.0; > + } > + for (size_t i = 0; i < node_count; i++) { > + for (size_t j = i + 1; j < node_count; j++) { > + distance_matrix[i][j] = 1.0 + range_factor * (double)rand(); > + distance_matrix[j][i] = distance_matrix[i][j]; > + } > + } > + > + find_shortest_distances(shortest_distances, via_node, > &node_eccentricity, > + node_count, source_node, distance_matrix); > + > + /* Control printing */ > + printf("CONTROL RESULT:\n"); > + printf(" Distance matrix (top left part):\n"); > + for (size_t i = 0; i < 3; i++) { > + for (size_t j = 0; j < 3; j++) { > + printf(" %7.2f", distance_matrix[i][j]); > + } > + printf("\n"); > + } > + printf(" Source: %d (eccentricity: %f)\n", > + source_node, node_eccentricity); > + printf(" Destination Distance Via Node\n"); > + for (size_t i = 0; i < 3; i++) { > + printf(" %5d %7.2f %4d\n", > + i, shortest_distances[i], via_node[i]); > + } > + > + /* Free all previously allocated space */ > + for (size_t i = 0; i < node_count; i++) { > + free(distance_matrix[i]); > + } > + free(distance_matrix); > + free(shortest_distances); > + free(via_node); > +} > diff --git a/tests/performance/nightly-tests/benchmarks/source/ > dijkstra_int32/dijkstra_int32.c b/tests/performance/nightly- > tests/benchmarks/source/dijkstra_int32/dijkstra_int32.c > new file mode 100644 > index 0000000000..2663cde943 > --- /dev/null > +++ b/tests/performance/nightly-tests/benchmarks/source/ > dijkstra_int32/dijkstra_int32.c > @@ -0,0 +1,192 @@ > +/* > + * Source file of a benchmark program involving calculations of the > + * shortest distances between a source node and all other nodes in a > + * graph of n nodes in which all nxn distances are defined as "int32". > + * The number n can be given via command line, and the default is 2000. > + * The algorithm used is Dijsktra's. > + * > + * This file is a part of the project "TCG Continuous Benchmarking". > + * > + * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkara...@gmail.com> > + * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail. > com> > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see <https://www.gnu.org/licenses/>. > + * > + */ > + > +#include <limits.h> > +#include <stdbool.h> > +#include <stdio.h> > +#include <stdlib.h> > +#include <unistd.h> > + > +/* Number of columns and rows in all matrixes*/ > +#define DEFAULT_NODE_COUNT 2000 > +#define MIN_NODE_COUNT 3 > +#define MAX_NODE_COUNT 10000 > + > + > +int32_t closest_index(int32_t count, int32_t *distances, bool *flags) > +{ > + int32_t closest; > + int32_t minimum = INT_MAX; > + > + for (size_t i = 0; i < count; i++) { > + if (flags[i] == false && distances[i] <= minimum) { > + closest = i; > + minimum = distances[i]; > + } > + } > + > + return closest; > +} > + > +/** > + * Calculate the shortest distances from the source node using Dijkstra > method. > + * @param (out) distances An array of shortest distances from the source > node. > + * @param (out) via An array of nodes needed to be taken as the the last > + * before destination, for each destination. > + * @param (out) eccent Eccentricity of the source node. > + * @param (in) count The number of nodes. > + * @param (in) source Source node. > + * @param (in) matrix Distance matrix. > + */ > +void find_shortest_distances(int32_t *distances, int32_t *via, int32_t > *eccent, > + int32_t count, int32_t source, int32_t > **matrix) > +{ > + bool *flags; > + > + flags = (bool *)malloc(count * sizeof(bool)); > + > + for (size_t i = 0; i < count; i++) { > + distances[i] = INT_MAX; > + flags[i] = false; > + } > + > + distances[source] = 0; > + via[source] = source; > + > + for (size_t i = 0; i < count - 1; i++) { > + int32_t closest = closest_index(count, distances, flags); > + flags[closest] = true; > + for (size_t j = 0; j < count; j++) { > + if ((!flags[j]) && > + (matrix[closest][j]) && > + (distances[closest] != INT_MAX) && > + (distances[j] > distances[closest] + > matrix[closest][j])) { > + distances[j] = distances[closest] + matrix[closest][j]; > + via[j] = closest; > + } > + } > + } > + > + *eccent = 0; > + for (size_t i = 0; i < count; i++) { > + if (*eccent < distances[i]) { > + *eccent = distances[i]; > + } > + } > + > + free(flags); > +} > + > + > +void main(int argc, char *argv[]) > +{ > + int32_t **distance_matrix; > + int32_t *shortest_distances; > + int32_t *via_node; > + int32_t node_count = DEFAULT_NODE_COUNT; > + int32_t source_node = 0; > + int32_t node_eccentricity = 0; > + int32_t option; > + > + /* Parse command line options */ > + while ((option = getopt(argc, argv, "n:")) != -1) { > + if (option == 'n') { > + int32_t user_node_count = atoi(optarg); > + > + /* Check if the value is a string or zero */ > + if (user_node_count == 0) { > + fprintf(stderr, "Error ... Invalid value for option > '-n'.\n"); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is a negative number */ > + if (user_node_count < MIN_NODE_COUNT) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be a " > + "number less than %d.\n", MIN_NODE_COUNT); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is too large */ > + if (user_node_count > MAX_NODE_COUNT) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be " > + "more than %d.\n", MAX_NODE_COUNT); > + exit(EXIT_FAILURE); > + } > + node_count = user_node_count; > + } else { > + exit(EXIT_FAILURE); > + } > + } > + > + /* Allocate the memory space for all matrixes */ > + distance_matrix = (int32_t **)malloc(node_count * sizeof(int32_t *)); > + for (size_t i = 0; i < node_count; i++) { > + distance_matrix[i] = (int32_t *)malloc(node_count * > sizeof(int32_t)); > + } > + shortest_distances = (int32_t *)malloc(node_count * sizeof(int32_t)); > + via_node = (int32_t *)malloc(node_count * sizeof(int32_t)); > + > + /* Initialize helper arrays and populate distance_matrix */ > + srand(1); > + for (size_t i = 0; i < node_count; i++) { > + shortest_distances[i] = 0; > + via_node[i] = -1; > + distance_matrix[i][i] = 0; > + } > + for (size_t i = 0; i < node_count; i++) { > + for (size_t j = i + 1; j < node_count; j++) { > + distance_matrix[i][j] = 1 + (rand()) / (RAND_MAX / 999); > + distance_matrix[j][i] = distance_matrix[i][j]; > + } > + } > + > + find_shortest_distances(shortest_distances, via_node, > &node_eccentricity, > + node_count, source_node, distance_matrix); > + > + /* Control printing */ > + printf("CONTROL RESULT:\n"); > + printf(" Distance matrix (top left part):\n"); > + for (size_t i = 0; i < 3; i++) { > + for (size_t j = 0; j < 3; j++) { > + printf(" %6d", distance_matrix[i][j]); > + } > + printf("\n"); > + } > + printf(" Source: %d (eccentricity: %d)\n", > + source_node, node_eccentricity); > + printf(" Destination Distance Via Node\n"); > + for (size_t i = 0; i < 3; i++) { > + printf(" %5d %3d %4d\n", > + i, shortest_distances[i], via_node[i]); > + } > + > + /* Free all previously allocated space */ > + for (size_t i = 0; i < node_count; i++) { > + free(distance_matrix[i]); > + } > + free(distance_matrix); > + free(shortest_distances); > + free(via_node); > +} > diff --git a/tests/performance/nightly-tests/benchmarks/source/ > matmult_double/matmult_double.c b/tests/performance/nightly- > tests/benchmarks/source/matmult_double/matmult_double.c > new file mode 100644 > index 0000000000..42bbb4717a > --- /dev/null > +++ b/tests/performance/nightly-tests/benchmarks/source/ > matmult_double/matmult_double.c > @@ -0,0 +1,123 @@ > +/* > + * Source file of a benchmark program involving calculations of > + * a product of two matrixes nxn whose elements are "double". The > + * number n can be given via command line, and the default is 200. > + * > + * This file is a part of the project "TCG Continuous Benchmarking". > + * > + * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkara...@gmail.com> > + * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail. > com> > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see <https://www.gnu.org/licenses/>. > + * > + */ > + > +#include <stdio.h> > +#include <stdlib.h> > +#include <unistd.h> > + > +/* Number of columns and rows in all matrixes*/ > +#define DEFAULT_MATRIX_SIZE 200 > +#define MIN_MATRIX_SIZE 2 > +#define MAX_MATRIX_SIZE 200000 > + > +void main(int argc, char *argv[]) > +{ > + double **matrix_a; > + double **matrix_b; > + double **matrix_res; > + size_t i; > + size_t j; > + size_t k; > + int32_t matrix_size = DEFAULT_MATRIX_SIZE; > + int32_t option; > + double range_factor = 100.0 / (double)(RAND_MAX); > + > + > + /* Parse command line options */ > + while ((option = getopt(argc, argv, "n:")) != -1) { > + if (option == 'n') { > + int32_t user_matrix_size = atoi(optarg); > + > + /* Check if the value is a string or zero */ > + if (user_matrix_size == 0) { > + fprintf(stderr, "Error ... Invalid value for option > '-n'.\n"); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is a negative number */ > + if (user_matrix_size < MIN_MATRIX_SIZE) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be a " > + "number less than %d.\n", > MIN_MATRIX_SIZE); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is too large */ > + if (user_matrix_size > MAX_MATRIX_SIZE) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be " > + "more than %d.\n", MAX_MATRIX_SIZE); > + exit(EXIT_FAILURE); > + } > + matrix_size = user_matrix_size; > + } else { > + exit(EXIT_FAILURE); > + } > + } > + > + /* Allocate the memory space for all matrixes */ > + matrix_a = (double **)malloc(matrix_size * sizeof(double *)); > + for (i = 0; i < matrix_size; i++) { > + matrix_a[i] = (double *)malloc(matrix_size * sizeof(double)); > + } > + matrix_b = (double **)malloc(matrix_size * sizeof(double *)); > + for (i = 0; i < matrix_size; i++) { > + matrix_b[i] = (double *)malloc(matrix_size * sizeof(double)); > + } > + matrix_res = (double **)malloc(matrix_size * sizeof(double *)); > + for (i = 0; i < matrix_size; i++) { > + matrix_res[i] = (double *)malloc(matrix_size * sizeof(double)); > + } > + > + /* Populate matrix_a and matrix_b with random numbers */ > + srand(1); > + for (i = 0; i < matrix_size; i++) { > + for (j = 0; j < matrix_size; j++) { > + matrix_a[i][j] = range_factor * (double)rand(); > + matrix_b[i][j] = range_factor * (double)rand(); > + } > + } > + > + /* Calculate the product of two matrixes */ > + for (i = 0; i < matrix_size; i++) { > + for (j = 0; j < matrix_size; j++) { > + matrix_res[i][j] = 0.0; > + for (k = 0; k < matrix_size; k++) { > + matrix_res[i][j] += matrix_a[i][k] * matrix_b[k][j]; > + } > + } > + } > + > + /* Control printing */ > + printf("CONTROL RESULT:\n"); > + printf(" %f %f\n", matrix_res[0][0], matrix_res[0][1]); > + printf(" %f %f\n", matrix_res[1][0], matrix_res[1][1]); > + > + /* Free all previously allocated space */ > + for (i = 0; i < matrix_size; i++) { > + free(matrix_a[i]); > + free(matrix_b[i]); > + free(matrix_res[i]); > + } > + free(matrix_a); > + free(matrix_b); > + free(matrix_res); > +} > diff --git > a/tests/performance/nightly-tests/benchmarks/source/matmult_int32/matmult_int32.c > b/tests/performance/nightly-tests/benchmarks/source/ > matmult_int32/matmult_int32.c > new file mode 100644 > index 0000000000..29a6eb000d > --- /dev/null > +++ b/tests/performance/nightly-tests/benchmarks/source/ > matmult_int32/matmult_int32.c > @@ -0,0 +1,121 @@ > +/* > + * Source file of a benchmark program involving calculations of > + * a product of two matrixes nxn whose elements are "int32_t". The > + * number n can be given via command line, and the default is 200. > + * > + * This file is a part of the project "TCG Continuous Benchmarking". > + * > + * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkara...@gmail.com> > + * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail. > com> > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see <https://www.gnu.org/licenses/>. > + * > + */ > + > +#include <stdio.h> > +#include <stdlib.h> > +#include <unistd.h> > + > +/* Number of columns and rows in all matrixes*/ > +#define DEFAULT_MATRIX_SIZE 200 > +#define MIN_MATRIX_SIZE 2 > +#define MAX_MATRIX_SIZE 200000 > + > +void main(int argc, char *argv[]) > +{ > + int32_t **matrix_a; > + int32_t **matrix_b; > + int32_t **matrix_res; > + size_t i; > + size_t j; > + size_t k; > + int32_t matrix_size = DEFAULT_MATRIX_SIZE; > + int32_t option; > + > + /* Parse command line options */ > + while ((option = getopt(argc, argv, "n:")) != -1) { > + if (option == 'n') { > + int32_t user_matrix_size = atoi(optarg); > + > + /* Check if the value is a string or zero */ > + if (user_matrix_size == 0) { > + fprintf(stderr, "Error ... Invalid value for option > '-n'.\n"); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is a negative number */ > + if (user_matrix_size < MIN_MATRIX_SIZE) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be a " > + "number less than %d.\n", > MIN_MATRIX_SIZE); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is too large */ > + if (user_matrix_size > MAX_MATRIX_SIZE) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be " > + "more than %d.\n", MAX_MATRIX_SIZE); > + exit(EXIT_FAILURE); > + } > + matrix_size = user_matrix_size; > + } else { > + exit(EXIT_FAILURE); > + } > + } > + > + /* Allocate the memory space for all matrixes */ > + matrix_a = (int32_t **)malloc(matrix_size * sizeof(int32_t *)); > + for (i = 0; i < matrix_size; i++) { > + matrix_a[i] = (int32_t *)malloc(matrix_size * sizeof(int32_t)); > + } > + matrix_b = (int32_t **)malloc(matrix_size * sizeof(int32_t *)); > + for (i = 0; i < matrix_size; i++) { > + matrix_b[i] = (int32_t *)malloc(matrix_size * sizeof(int32_t)); > + } > + matrix_res = (int32_t **)malloc(matrix_size * sizeof(int32_t *)); > + for (i = 0; i < matrix_size; i++) { > + matrix_res[i] = (int32_t *)malloc(matrix_size * sizeof(int32_t)); > + } > + > + /* Populate matrix_a and matrix_b with random numbers */ > + srand(1); > + for (i = 0; i < matrix_size; i++) { > + for (j = 0; j < matrix_size; j++) { > + matrix_a[i][j] = (rand()) / (RAND_MAX / 100); > + matrix_b[i][j] = (rand()) / (RAND_MAX / 100); > + } > + } > + > + /* Calculate the product of two matrixes */ > + for (i = 0; i < matrix_size; i++) { > + for (j = 0; j < matrix_size; j++) { > + matrix_res[i][j] = 0; > + for (k = 0; k < matrix_size; k++) { > + matrix_res[i][j] += matrix_a[i][k] * matrix_b[k][j]; > + } > + } > + } > + > + /* Control printing */ > + printf("CONTROL RESULT:\n"); > + printf(" %d %d\n", matrix_res[0][0], matrix_res[0][1]); > + printf(" %d %d\n", matrix_res[1][0], matrix_res[1][1]); > + > + /* Free all previously allocated space */ > + for (i = 0; i < matrix_size; i++) { > + free(matrix_a[i]); > + free(matrix_b[i]); > + free(matrix_res[i]); > + } > + free(matrix_a); > + free(matrix_b); > + free(matrix_res); > +} > diff --git > a/tests/performance/nightly-tests/benchmarks/source/qsort_double/qsort_double.c > b/tests/performance/nightly-tests/benchmarks/source/qsort_ > double/qsort_double.c > new file mode 100644 > index 0000000000..efc1b2eee1 > --- /dev/null > +++ b/tests/performance/nightly-tests/benchmarks/source/qsort_ > double/qsort_double.c > @@ -0,0 +1,104 @@ > +/* > + * Source file of a benchmark program involving sorting of an array > + * of length n whose elements are "double". The default value for n > + * is 300000, and it can be set via command line as well. > + * > + * This file is a part of the project "TCG Continuous Benchmarking". > + * > + * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkara...@gmail.com> > + * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail. > com> > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see <https://www.gnu.org/licenses/>. > + * > + */ > + > +#include <stdio.h> > +#include <stdlib.h> > +#include <unistd.h> > + > +/* Number of elements in the array to be sorted */ > +#define DEFAULT_ARRAY_LEN 300000 > +#define MIN_ARRAY_LEN 3 > +#define MAX_ARRAY_LEN 30000000 > + > +/* Upper limit used for generation of random numbers */ > +#define UPPER_LIMIT 1000.0 > + > +/* Comparison function passed to qsort() */ > +static int compare(const void *a, const void *b) > +{ > + if (*(const double *)a > *(const double *)b) { > + return 1; > + } else if (*(const double *)a < *(const double *)b) { > + return -1; > + } > + return 0; > +} > + > +void main(int argc, char *argv[]) > +{ > + double *array_to_be_sorted; > + int32_t array_len = DEFAULT_ARRAY_LEN; > + int32_t option; > + double range_factor = UPPER_LIMIT / (double)(RAND_MAX); > + > + /* Parse command line options */ > + while ((option = getopt(argc, argv, "n:")) != -1) { > + if (option == 'n') { > + int32_t user_array_len = atoi(optarg); > + > + /* Check if the value is a string or zero */ > + if (user_array_len == 0) { > + fprintf(stderr, "Error ... Invalid value for option > '-n'.\n"); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is a negative number */ > + if (user_array_len < MIN_ARRAY_LEN) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be a " > + "number less than %d.\n", MIN_ARRAY_LEN); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is too large */ > + if (user_array_len > MAX_ARRAY_LEN) { > + fprintf(stderr, "Error ... Value for option '-n' cannot > be " > + "more than %d.\n", MAX_ARRAY_LEN); > + exit(EXIT_FAILURE); > + } > + array_len = user_array_len; > + } else { > + exit(EXIT_FAILURE); > + } > + } > + > + /* Allocate the memory space for the array */ > + array_to_be_sorted = (double *) malloc(array_len * sizeof(double)); > + > + /* Populate the_array with random numbers */ > + srand(1); > + for (size_t i = 0; i < array_len; i++) { > + array_to_be_sorted[i] = range_factor * (double)rand(); > + } > + > + /* Sort the_array using qsort() */ > + qsort(array_to_be_sorted, array_len, sizeof(array_to_be_sorted[0]), > + compare); > + > + /* Control printing */ > + printf("CONTROL RESULT:\n"); > + printf("%14.10f %14.10f %14.10f\n", > + array_to_be_sorted[0], array_to_be_sorted[1], > array_to_be_sorted[2]); > + > + /* Free all previously allocated space */ > + free(array_to_be_sorted); > +} > diff --git > a/tests/performance/nightly-tests/benchmarks/source/qsort_int32/qsort_int32.c > b/tests/performance/nightly-tests/benchmarks/source/qsort_ > int32/qsort_int32.c > new file mode 100644 > index 0000000000..76ca9c3490 > --- /dev/null > +++ b/tests/performance/nightly-tests/benchmarks/source/qsort_ > int32/qsort_int32.c > @@ -0,0 +1,103 @@ > +/* > + * Source file of a benchmark program involving sorting of an array > + * of length n whose elements are "int32_t". The default value for n > + * is 300000, and it can be set via command line as well. > + * > + * This file is a part of the project "TCG Continuous Benchmarking". > + * > + * Copyright (C) 2020 Ahmed Karaman <ahmedkhaledkara...@gmail.com> > + * Copyright (C) 2020 Aleksandar Markovic <aleksandar.qemu.devel@gmail. > com> > + * > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program. If not, see <https://www.gnu.org/licenses/>. > + * > + */ > + > +#include <stdio.h> > +#include <stdlib.h> > +#include <unistd.h> > + > +/* Number of elements in the array to be sorted */ > +#define DEFAULT_ARRAY_LEN 300000 > +#define MIN_ARRAY_LEN 3 > +#define MAX_ARRAY_LEN 30000000 > + > +/* Upper limit used for generation of random numbers */ > +#define UPPER_LIMIT 50000000 > + > +/* Comparison function passed to qsort() */ > +static int compare(const void *a, const void *b) > +{ > + if (*(const int32_t *)a > *(const int32_t *)b) { > + return 1; > + } else if (*(const int32_t *)a < *(const int32_t *)b) { > + return -1; > + } > + return 0; > +} > + > +void main(int argc, char *argv[]) > +{ > + int32_t *array_to_be_sorted; > + int32_t array_len = DEFAULT_ARRAY_LEN; > + int32_t option; > + > + /* Parse command line options */ > + while ((option = getopt(argc, argv, "n:")) != -1) { > + if (option == 'n') { > + int32_t user_array_len = atoi(optarg); > + > + /* Check if the value is a string or zero */ > + if (user_array_len == 0) { > + fprintf(stderr, "Error ... Invalid value for option > '-n'.\n"); > + exit(EXIT_FAILURE); > + } > + /* Check if the value is a negative number */ > + if (user_array_len < MIN_ARRAY_LEN) { > +
