Re: [PATCH 7/9] tests/performance: Add nightly tests
Aleksandar Markovic writes: > On Wednesday, September 2, 2020, Alex Bennée wrote: > >> >> Ahmed Karaman writes: >> >> > A nightly performance testing system to monitor any change in QEMU >> > performance across seventeen different targets. >> > >> > The system includes eight different benchmarks to provide a variety >> > of testing workloads. >> > >> > dijkstra_double: >> > Find the shortest path between the source node and all other nodes >> > using Dijkstra’s algorithm. The graph contains n nodes where all nxn >> > distances are double values. The value of n can be specified using >> > the -n flag. The default value is 2000. >> > >> > dijkstra_int32: >> > Find the shortest path between the source node and all other nodes >> > using Dijkstra’s algorithm. The graph contains n nodes where all nxn >> > distances are int32 values. The value of n can be specified using >> > the -n flag. The default value is 2000. >> > >> > matmult_double: >> > Standard matrix multiplication of an n*n matrix of randomly generated >> > double numbers from 0 to 100. The value of n is passed as an argument >> > with the -n flag. The default value is 200. >> > >> > matmult_int32: >> > Standard matrix multiplication of an n*n matrix of randomly generated >> > integer numbers from 0 to 100. The value of n is passed as an >> > argument with the -n flag. The default value is 200. >> > >> > qsort_double: >> > Quick sort of an array of n randomly generated double numbers from 0 >> > to 1000. The value of n is passed as an argument with the -n flag. >> > The default value is 30. >> > >> > qsort_int32: >> > Quick sort of an array of n randomly generated integer numbers from 0 >> > to 5000. The value of n is passed as an argument with the -n >> > flag.The default value is 30. >> > >> > qsort_string: >> > Quick sort of an array of 1 randomly generated strings of size 8 >> > (including null terminating character). The sort process is repeated >> > n number of times. The value of n is passed as an argument with the >> > -n flag. The default value is 20. >> > >> > search_string: >> > Search for the occurrence of a small string in a much larger random >> > string (“needle in a hay”). The search process is repeated n number >> > of times and each time, a different large random string (“hay”) is >> > generated. The value of n can be specified using the -n flag. The >> > default value is 20. >> > >> > Syntax: >> > nightly_tests_core.py [-h] [-r REF] >> > Optional arguments: >> > -h, --helpShow this help message and exit >> > -r REF, --reference REF >> > Reference QEMU version - Default is v5.1.0 >> > Example of usage: >> > nightly_tests_core.py -r v5.1.0 2>log.txt >> > >> > The following report includes detailed setup and execution details >> > of the system: >> > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/ >> QEMU-Nightly-Performance-Tests/ >> > >> > Signed-off-by: Ahmed Karaman >> > --- >> > tests/performance/nightly-tests/README.md | 243 + >> > .../source/dijkstra_double/dijkstra_double.c | 194 >> > .../source/dijkstra_int32/dijkstra_int32.c| 192 >> > .../source/matmult_double/matmult_double.c| 123 +++ >> > .../source/matmult_int32/matmult_int32.c | 121 +++ >> > .../source/qsort_double/qsort_double.c| 104 ++ >> > .../source/qsort_int32/qsort_int32.c | 103 ++ >> > .../source/qsort_string/qsort_string.c| 122 +++ >> > .../source/search_string/search_string.c | 110 +++ >> > .../scripts/nightly_tests_core.py | 920 ++ >> > .../scripts/run_nightly_tests.py | 135 +++ >> > .../nightly-tests/scripts/send_email.py | 56 ++ >> > 12 files changed, 2423 insertions(+) >> > create mode 100644 tests/performance/nightly-tests/README.md >> > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ >> dijkstra_double/dijkstra_double.c >> > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ >> dijkstra_int32/dijkstra_int32.c >> > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ >> matmult_double/matmult_double.c >> > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ >> matmult_int32/matmult_int32.c >> > create mode 100644 tests/performance/nightly- >> tests/benchmarks/source/qsort_double/qsort_double.c >> > create mode 100644 tests/performance/nightly- >> tests/benchmarks/source/qsort_int32/qsort_int32.c >> > create mode 100644 tests/performance/nightly- >> tests/benchmarks/source/qsort_string/qsort_string.c >> > create mode 100644 >> > tests/performance/nightly-tests/benchmarks/source/ >> search_string/search_string.c >> >> Perhaps we could compress these paths down to: >> >> tests/tcg/benchmarks/foo.c >> tests/tcg/benchmarks/bar.c >> >> and then we can also ensure they are built using the existing TCG tests >> cross compile framework. >> >> > Hi, > >
Re: [PATCH 7/9] tests/performance: Add nightly tests
On Wednesday, September 2, 2020, Alex Bennée wrote: > > Ahmed Karaman writes: > > > A nightly performance testing system to monitor any change in QEMU > > performance across seventeen different targets. > > > > The system includes eight different benchmarks to provide a variety > > of testing workloads. > > > > dijkstra_double: > > Find the shortest path between the source node and all other nodes > > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > > distances are double values. The value of n can be specified using > > the -n flag. The default value is 2000. > > > > dijkstra_int32: > > Find the shortest path between the source node and all other nodes > > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > > distances are int32 values. The value of n can be specified using > > the -n flag. The default value is 2000. > > > > matmult_double: > > Standard matrix multiplication of an n*n matrix of randomly generated > > double numbers from 0 to 100. The value of n is passed as an argument > > with the -n flag. The default value is 200. > > > > matmult_int32: > > Standard matrix multiplication of an n*n matrix of randomly generated > > integer numbers from 0 to 100. The value of n is passed as an > > argument with the -n flag. The default value is 200. > > > > qsort_double: > > Quick sort of an array of n randomly generated double numbers from 0 > > to 1000. The value of n is passed as an argument with the -n flag. > > The default value is 30. > > > > qsort_int32: > > Quick sort of an array of n randomly generated integer numbers from 0 > > to 5000. The value of n is passed as an argument with the -n > > flag.The default value is 30. > > > > qsort_string: > > Quick sort of an array of 1 randomly generated strings of size 8 > > (including null terminating character). The sort process is repeated > > n number of times. The value of n is passed as an argument with the > > -n flag. The default value is 20. > > > > search_string: > > Search for the occurrence of a small string in a much larger random > > string (“needle in a hay”). The search process is repeated n number > > of times and each time, a different large random string (“hay”) is > > generated. The value of n can be specified using the -n flag. The > > default value is 20. > > > > Syntax: > > nightly_tests_core.py [-h] [-r REF] > > Optional arguments: > > -h, --helpShow this help message and exit > > -r REF, --reference REF > > Reference QEMU version - Default is v5.1.0 > > Example of usage: > > nightly_tests_core.py -r v5.1.0 2>log.txt > > > > The following report includes detailed setup and execution details > > of the system: > > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/ > QEMU-Nightly-Performance-Tests/ > > > > Signed-off-by: Ahmed Karaman > > --- > > tests/performance/nightly-tests/README.md | 243 + > > .../source/dijkstra_double/dijkstra_double.c | 194 > > .../source/dijkstra_int32/dijkstra_int32.c| 192 > > .../source/matmult_double/matmult_double.c| 123 +++ > > .../source/matmult_int32/matmult_int32.c | 121 +++ > > .../source/qsort_double/qsort_double.c| 104 ++ > > .../source/qsort_int32/qsort_int32.c | 103 ++ > > .../source/qsort_string/qsort_string.c| 122 +++ > > .../source/search_string/search_string.c | 110 +++ > > .../scripts/nightly_tests_core.py | 920 ++ > > .../scripts/run_nightly_tests.py | 135 +++ > > .../nightly-tests/scripts/send_email.py | 56 ++ > > 12 files changed, 2423 insertions(+) > > create mode 100644 tests/performance/nightly-tests/README.md > > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > dijkstra_double/dijkstra_double.c > > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > dijkstra_int32/dijkstra_int32.c > > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > matmult_double/matmult_double.c > > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > matmult_int32/matmult_int32.c > > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_double/qsort_double.c > > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_int32/qsort_int32.c > > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_string/qsort_string.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/ > search_string/search_string.c > > Perhaps we could compress these paths down to: > > tests/tcg/benchmarks/foo.c > tests/tcg/benchmarks/bar.c > > and then we can also ensure they are built using the existing TCG tests > cross compile framework. > > Hi, May I just bring some alternative views on this topic of benchmarks/multiple cross-compiling/nightly tests? Having a known source code of any benchmark
Re: [PATCH 7/9] tests/performance: Add nightly tests
Thanks Mr. Alex, On Wed, Sep 2, 2020 at 3:26 PM Alex Bennée wrote: > > > Ahmed Karaman writes: > > > A nightly performance testing system to monitor any change in QEMU > > performance across seventeen different targets. > > > > The system includes eight different benchmarks to provide a variety > > of testing workloads. > > > > dijkstra_double: > > Find the shortest path between the source node and all other nodes > > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > > distances are double values. The value of n can be specified using > > the -n flag. The default value is 2000. > > > > dijkstra_int32: > > Find the shortest path between the source node and all other nodes > > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > > distances are int32 values. The value of n can be specified using > > the -n flag. The default value is 2000. > > > > matmult_double: > > Standard matrix multiplication of an n*n matrix of randomly generated > > double numbers from 0 to 100. The value of n is passed as an argument > > with the -n flag. The default value is 200. > > > > matmult_int32: > > Standard matrix multiplication of an n*n matrix of randomly generated > > integer numbers from 0 to 100. The value of n is passed as an > > argument with the -n flag. The default value is 200. > > > > qsort_double: > > Quick sort of an array of n randomly generated double numbers from 0 > > to 1000. The value of n is passed as an argument with the -n flag. > > The default value is 30. > > > > qsort_int32: > > Quick sort of an array of n randomly generated integer numbers from 0 > > to 5000. The value of n is passed as an argument with the -n > > flag.The default value is 30. > > > > qsort_string: > > Quick sort of an array of 1 randomly generated strings of size 8 > > (including null terminating character). The sort process is repeated > > n number of times. The value of n is passed as an argument with the > > -n flag. The default value is 20. > > > > search_string: > > Search for the occurrence of a small string in a much larger random > > string (“needle in a hay”). The search process is repeated n number > > of times and each time, a different large random string (“hay”) is > > generated. The value of n can be specified using the -n flag. The > > default value is 20. > > > > Syntax: > > nightly_tests_core.py [-h] [-r REF] > > Optional arguments: > > -h, --helpShow this help message and exit > > -r REF, --reference REF > > Reference QEMU version - Default is v5.1.0 > > Example of usage: > > nightly_tests_core.py -r v5.1.0 2>log.txt > > > > The following report includes detailed setup and execution details > > of the system: > > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/QEMU-Nightly-Performance-Tests/ > > > > Signed-off-by: Ahmed Karaman > > --- > > tests/performance/nightly-tests/README.md | 243 + > > .../source/dijkstra_double/dijkstra_double.c | 194 > > .../source/dijkstra_int32/dijkstra_int32.c| 192 > > .../source/matmult_double/matmult_double.c| 123 +++ > > .../source/matmult_int32/matmult_int32.c | 121 +++ > > .../source/qsort_double/qsort_double.c| 104 ++ > > .../source/qsort_int32/qsort_int32.c | 103 ++ > > .../source/qsort_string/qsort_string.c| 122 +++ > > .../source/search_string/search_string.c | 110 +++ > > .../scripts/nightly_tests_core.py | 920 ++ > > .../scripts/run_nightly_tests.py | 135 +++ > > .../nightly-tests/scripts/send_email.py | 56 ++ > > 12 files changed, 2423 insertions(+) > > create mode 100644 tests/performance/nightly-tests/README.md > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/dijkstra_double/dijkstra_double.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/dijkstra_int32/dijkstra_int32.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/matmult_double/matmult_double.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/matmult_int32/matmult_int32.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/qsort_double/qsort_double.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/qsort_int32/qsort_int32.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/qsort_string/qsort_string.c > > create mode 100644 > > tests/performance/nightly-tests/benchmarks/source/search_string/search_string.c > > Perhaps we could compress these paths down to: > > tests/tcg/benchmarks/foo.c > tests/tcg/benchmarks/bar.c > > and then we can also ensure they are built using the existing TCG tests > cross compile framework. > Can you provide the commands for compiling the benchmarks for the 17 targets? > > create mode 100755 > >
Re: [PATCH 7/9] tests/performance: Add nightly tests
Ahmed Karaman writes: > A nightly performance testing system to monitor any change in QEMU > performance across seventeen different targets. > > The system includes eight different benchmarks to provide a variety > of testing workloads. > > dijkstra_double: > Find the shortest path between the source node and all other nodes > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > distances are double values. The value of n can be specified using > the -n flag. The default value is 2000. > > dijkstra_int32: > Find the shortest path between the source node and all other nodes > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > distances are int32 values. The value of n can be specified using > the -n flag. The default value is 2000. > > matmult_double: > Standard matrix multiplication of an n*n matrix of randomly generated > double numbers from 0 to 100. The value of n is passed as an argument > with the -n flag. The default value is 200. > > matmult_int32: > Standard matrix multiplication of an n*n matrix of randomly generated > integer numbers from 0 to 100. The value of n is passed as an > argument with the -n flag. The default value is 200. > > qsort_double: > Quick sort of an array of n randomly generated double numbers from 0 > to 1000. The value of n is passed as an argument with the -n flag. > The default value is 30. > > qsort_int32: > Quick sort of an array of n randomly generated integer numbers from 0 > to 5000. The value of n is passed as an argument with the -n > flag.The default value is 30. > > qsort_string: > Quick sort of an array of 1 randomly generated strings of size 8 > (including null terminating character). The sort process is repeated > n number of times. The value of n is passed as an argument with the > -n flag. The default value is 20. > > search_string: > Search for the occurrence of a small string in a much larger random > string (“needle in a hay”). The search process is repeated n number > of times and each time, a different large random string (“hay”) is > generated. The value of n can be specified using the -n flag. The > default value is 20. > > Syntax: > nightly_tests_core.py [-h] [-r REF] > Optional arguments: > -h, --helpShow this help message and exit > -r REF, --reference REF > Reference QEMU version - Default is v5.1.0 > Example of usage: > nightly_tests_core.py -r v5.1.0 2>log.txt > > The following report includes detailed setup and execution details > of the system: > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/QEMU-Nightly-Performance-Tests/ > > Signed-off-by: Ahmed Karaman > --- > tests/performance/nightly-tests/README.md | 243 + > .../source/dijkstra_double/dijkstra_double.c | 194 > .../source/dijkstra_int32/dijkstra_int32.c| 192 > .../source/matmult_double/matmult_double.c| 123 +++ > .../source/matmult_int32/matmult_int32.c | 121 +++ > .../source/qsort_double/qsort_double.c| 104 ++ > .../source/qsort_int32/qsort_int32.c | 103 ++ > .../source/qsort_string/qsort_string.c| 122 +++ > .../source/search_string/search_string.c | 110 +++ > .../scripts/nightly_tests_core.py | 920 ++ > .../scripts/run_nightly_tests.py | 135 +++ > .../nightly-tests/scripts/send_email.py | 56 ++ > 12 files changed, 2423 insertions(+) > create mode 100644 tests/performance/nightly-tests/README.md > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/dijkstra_double/dijkstra_double.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/dijkstra_int32/dijkstra_int32.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/matmult_double/matmult_double.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/matmult_int32/matmult_int32.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/qsort_double/qsort_double.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/qsort_int32/qsort_int32.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/qsort_string/qsort_string.c > create mode 100644 > tests/performance/nightly-tests/benchmarks/source/search_string/search_string.c Perhaps we could compress these paths down to: tests/tcg/benchmarks/foo.c tests/tcg/benchmarks/bar.c and then we can also ensure they are built using the existing TCG tests cross compile framework. > create mode 100755 > tests/performance/nightly-tests/scripts/nightly_tests_core.py > create mode 100755 > tests/performance/nightly-tests/scripts/run_nightly_tests.py > create mode 100644 tests/performance/nightly-tests/scripts/send_email.py > > diff --git a/tests/performance/nightly-tests/README.md > b/tests/performance/nightly-tests/README.md > new file mode 100644 > index 00..6db3b351b3 > ---
Re: [PATCH 7/9] tests/performance: Add nightly tests
On Friday, August 28, 2020, Ahmed Karaman wrote: > A nightly performance testing system to monitor any change in QEMU > performance across seventeen different targets. > > Reviewed-by: Aleksandar Markovic > The system includes eight different benchmarks to provide a variety > of testing workloads. > > dijkstra_double: > Find the shortest path between the source node and all other nodes > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > distances are double values. The value of n can be specified using > the -n flag. The default value is 2000. > > dijkstra_int32: > Find the shortest path between the source node and all other nodes > using Dijkstra’s algorithm. The graph contains n nodes where all nxn > distances are int32 values. The value of n can be specified using > the -n flag. The default value is 2000. > > matmult_double: > Standard matrix multiplication of an n*n matrix of randomly generated > double numbers from 0 to 100. The value of n is passed as an argument > with the -n flag. The default value is 200. > > matmult_int32: > Standard matrix multiplication of an n*n matrix of randomly generated > integer numbers from 0 to 100. The value of n is passed as an > argument with the -n flag. The default value is 200. > > qsort_double: > Quick sort of an array of n randomly generated double numbers from 0 > to 1000. The value of n is passed as an argument with the -n flag. > The default value is 30. > > qsort_int32: > Quick sort of an array of n randomly generated integer numbers from 0 > to 5000. The value of n is passed as an argument with the -n > flag.The default value is 30. > > qsort_string: > Quick sort of an array of 1 randomly generated strings of size 8 > (including null terminating character). The sort process is repeated > n number of times. The value of n is passed as an argument with the > -n flag. The default value is 20. > > search_string: > Search for the occurrence of a small string in a much larger random > string (“needle in a hay”). The search process is repeated n number > of times and each time, a different large random string (“hay”) is > generated. The value of n can be specified using the -n flag. The > default value is 20. > > Syntax: > nightly_tests_core.py [-h] [-r REF] > Optional arguments: > -h, --helpShow this help message and exit > -r REF, --reference REF > Reference QEMU version - Default is v5.1.0 > Example of usage: > nightly_tests_core.py -r v5.1.0 2>log.txt > > The following report includes detailed setup and execution details > of the system: > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/ > QEMU-Nightly-Performance-Tests/ > > Signed-off-by: Ahmed Karaman > --- > tests/performance/nightly-tests/README.md | 243 + > .../source/dijkstra_double/dijkstra_double.c | 194 > .../source/dijkstra_int32/dijkstra_int32.c| 192 > .../source/matmult_double/matmult_double.c| 123 +++ > .../source/matmult_int32/matmult_int32.c | 121 +++ > .../source/qsort_double/qsort_double.c| 104 ++ > .../source/qsort_int32/qsort_int32.c | 103 ++ > .../source/qsort_string/qsort_string.c| 122 +++ > .../source/search_string/search_string.c | 110 +++ > .../scripts/nightly_tests_core.py | 920 ++ > .../scripts/run_nightly_tests.py | 135 +++ > .../nightly-tests/scripts/send_email.py | 56 ++ > 12 files changed, 2423 insertions(+) > create mode 100644 tests/performance/nightly-tests/README.md > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > dijkstra_double/dijkstra_double.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > dijkstra_int32/dijkstra_int32.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > matmult_double/matmult_double.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > matmult_int32/matmult_int32.c > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_double/qsort_double.c > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_int32/qsort_int32.c > create mode 100644 tests/performance/nightly- > tests/benchmarks/source/qsort_string/qsort_string.c > create mode 100644 tests/performance/nightly-tests/benchmarks/source/ > search_string/search_string.c > create mode 100755 tests/performance/nightly-tests/scripts/nightly_tests_ > core.py > create mode 100755 tests/performance/nightly-tests/scripts/run_nightly_ > tests.py > create mode 100644 tests/performance/nightly-tests/scripts/send_email.py > > diff --git a/tests/performance/nightly-tests/README.md > b/tests/performance/nightly-tests/README.md > new file mode 100644 > index 00..6db3b351b3 > --- /dev/null > +++ b/tests/performance/nightly-tests/README.md > @@ -0,0 +1,243 @@ > +### QEMU Nightly Tests > + > +**Required settings:** > +
[PATCH 7/9] tests/performance: Add nightly tests
A nightly performance testing system to monitor any change in QEMU performance across seventeen different targets. The system includes eight different benchmarks to provide a variety of testing workloads. dijkstra_double: Find the shortest path between the source node and all other nodes using Dijkstra’s algorithm. The graph contains n nodes where all nxn distances are double values. The value of n can be specified using the -n flag. The default value is 2000. dijkstra_int32: Find the shortest path between the source node and all other nodes using Dijkstra’s algorithm. The graph contains n nodes where all nxn distances are int32 values. The value of n can be specified using the -n flag. The default value is 2000. matmult_double: Standard matrix multiplication of an n*n matrix of randomly generated double numbers from 0 to 100. The value of n is passed as an argument with the -n flag. The default value is 200. matmult_int32: Standard matrix multiplication of an n*n matrix of randomly generated integer numbers from 0 to 100. The value of n is passed as an argument with the -n flag. The default value is 200. qsort_double: Quick sort of an array of n randomly generated double numbers from 0 to 1000. The value of n is passed as an argument with the -n flag. The default value is 30. qsort_int32: Quick sort of an array of n randomly generated integer numbers from 0 to 5000. The value of n is passed as an argument with the -n flag.The default value is 30. qsort_string: Quick sort of an array of 1 randomly generated strings of size 8 (including null terminating character). The sort process is repeated n number of times. The value of n is passed as an argument with the -n flag. The default value is 20. search_string: Search for the occurrence of a small string in a much larger random string (“needle in a hay”). The search process is repeated n number of times and each time, a different large random string (“hay”) is generated. The value of n can be specified using the -n flag. The default value is 20. Syntax: nightly_tests_core.py [-h] [-r REF] Optional arguments: -h, --helpShow this help message and exit -r REF, --reference REF Reference QEMU version - Default is v5.1.0 Example of usage: nightly_tests_core.py -r v5.1.0 2>log.txt The following report includes detailed setup and execution details of the system: https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/QEMU-Nightly-Performance-Tests/ Signed-off-by: Ahmed Karaman --- tests/performance/nightly-tests/README.md | 243 + .../source/dijkstra_double/dijkstra_double.c | 194 .../source/dijkstra_int32/dijkstra_int32.c| 192 .../source/matmult_double/matmult_double.c| 123 +++ .../source/matmult_int32/matmult_int32.c | 121 +++ .../source/qsort_double/qsort_double.c| 104 ++ .../source/qsort_int32/qsort_int32.c | 103 ++ .../source/qsort_string/qsort_string.c| 122 +++ .../source/search_string/search_string.c | 110 +++ .../scripts/nightly_tests_core.py | 920 ++ .../scripts/run_nightly_tests.py | 135 +++ .../nightly-tests/scripts/send_email.py | 56 ++ 12 files changed, 2423 insertions(+) create mode 100644 tests/performance/nightly-tests/README.md create mode 100644 tests/performance/nightly-tests/benchmarks/source/dijkstra_double/dijkstra_double.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/dijkstra_int32/dijkstra_int32.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/matmult_double/matmult_double.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/matmult_int32/matmult_int32.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/qsort_double/qsort_double.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/qsort_int32/qsort_int32.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/qsort_string/qsort_string.c create mode 100644 tests/performance/nightly-tests/benchmarks/source/search_string/search_string.c create mode 100755 tests/performance/nightly-tests/scripts/nightly_tests_core.py create mode 100755 tests/performance/nightly-tests/scripts/run_nightly_tests.py create mode 100644 tests/performance/nightly-tests/scripts/send_email.py diff --git a/tests/performance/nightly-tests/README.md b/tests/performance/nightly-tests/README.md new file mode 100644 index 00..6db3b351b3 --- /dev/null +++ b/tests/performance/nightly-tests/README.md @@ -0,0 +1,243 @@ +### QEMU Nightly Tests + +**Required settings:** + +Update the `GMAIL_USER` object in `send_email.py` with your credentials. + +For more details on how the system works, please check the [eighth report](https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/QEMU-Nightly-Performance-Tests/) of the "TCG Continuos Benchmarking" series. + +**Running the
