> On 4 Apr 2019, at 20:11, Gilles Sadowski <gillese...@gmail.com> wrote: > > Hi Alex. > > Le mer. 3 avr. 2019 à 17:55, Alex Herbert <alex.d.herb...@gmail.com > <mailto:alex.d.herb...@gmail.com>> a écrit : >> >> I have decided to demonstrate where I am with the stress test >> application. A small project to add command-line support using Picocli >> ballooned into a lot of new code. Here's a PR with the new code: >> >> https://github.com/apache/commons-rng/pull/34 >> <https://github.com/apache/commons-rng/pull/34> >> >> Everything is still in the examples-stress module. All new files. >> However the output is a command-line program that can be extended to do >> other things. Perhaps this justifies renaming the module to >> examples-rng-utils or creating a new module in the same vein that can be >> used for utilities and this one dedicated to the stress test. > > "utils" is not very telling. > Do you mean separating the output of integers (in binary format) to > console and the application that starts the processes to read them?
Previous discussion had mentioned another feature so when I wrote that I was thinking more generic. Hence the ‘utilities' name. However that other feature was the ability to write out the numbers from a RNG. And now I realise that only came about because I was writing out the numbers to file so I could pass them to DieHarder as a file input and avoid the direct sub-process link. So I’ll drop the ’utils’ name and go back to the stress name and concentrate the program on things to do with stress testing. > >> One such >> utility previously discussed was to produce output from a named >> generator for a chosen method, e.g. nextLong, nextInt, nextDouble, etc. > > Is it really useful (other than for accommodating the interface > of the test suites)? > For java applications, usage of "RandomSource" seems the > recommended way to get the above. Agreed. As I just explained above the idea for this is to allow the output to be dumped to a file for use in a stress test application that requires file input rather than stdin. So it could just be made to output nextInt as unsigned values in the Dieharder format or as binary (with the endianness option). Output of floats and doubles and other formats can be left until they are needed for a specific purpose. At least with this it serves to demonstrate that Dieharder can be run using either the bridge or via file input. I’ll update so that you can seed the RandomSource and then output the files needed for Dieharder. Then add a bit of documentation in the endianness.md file to run through an example of using Dieharder with files and the direct bridge. > >> >> Back to the application as it stands. It achieves the goal of: >> >> - Having command-line help >> >> - Listing all known generators >> >> - Running a stress test for a list of generators, passing the output to >> an external application (e.g. BigCrush) >> >> - Allowing arguments to be passed to generators that need them, e.g. >> TWO_CMRES_SELECT >> >> - Allowing a variable number of stress test trials to be run >> >> - Options for overwriting existing results files >> >> - Support for big/little-endian platforms (this is natively detected) >> >> - Other stuff too >> >> >> Starting in the root directory: >> >>> cd commons-rng-examples/examples-stress >> >>> gcc src/main/c/stdin2testu01.c -o stdin2testu01 -ltestu01 \ >> -ltestu01probdist -ltestu01mylib -lm >> >>> mvn package -P examples-stress >> >>> java -jar target/rng-utils.jar >> >> rng-utils [-hV] [--logging=<logLevel>] [COMMAND] >> >> Apache Commons Random Number Generators Utilities. >> >> Options: >> --logging=<logLevel> Specify the logging level (default: INFO). >> Valid values: ERROR, INFO, DEBUG >> -h, --help Show this help message and exit. >> -V, --version Print version information and exit. >> >> Commands: >> stress Run repeat trials of random data generators using a provided test >> application. >> list List random generators. >> >>> java -jar target/rng-utils.jar list -h >> >> rng-utils list [-hV] [--logging=<logLevel>] [-f=<format>] [-p=<idPrefix>] >> [-t=<trials>] >> >> List random generators. >> >> Options: >> --logging=<logLevel> Specify the logging level (default: INFO). >> Valid values: ERROR, INFO, DEBUG >> -h, --help Show this help message and exit. >> -V, --version Print version information and exit. >> -f, --format=<format> The list format (default: STRESS_TEST). >> Valid values: STRESS_TEST, PLAIN >> -p, --prefix=<idPrefix> The ID prefix. >> Used for the stress test format. >> -t, --trials=<trials> The number of trials for each random generator. >> Used for the stress test format. >> >>> java -jar target/rng-utils.jar list >> >> # Random generators list. >> # Any generator with no trials is ignored during testing. >> # >> # ID RandomSource trials [constructor arguments ...] >> 1 JDK 1 >> 2 WELL_512_A 1 >> 3 WELL_1024_A 1 >> 4 WELL_19937_A 1 >> 5 WELL_19937_C 1 >> 6 WELL_44497_A 1 >> 7 WELL_44497_B 1 >> 8 MT 1 >> 9 ISAAC 1 >> 10 SPLIT_MIX_64 1 >> 11 XOR_SHIFT_1024_S 1 >> 12 TWO_CMRES 1 >> 13 TWO_CMRES_SELECT 0 [1, 2] >> 14 MT_64 1 >> 15 MWC_256 1 >> 16 KISS 1 >> 17 XOR_SHIFT_1024_S_PHI 1 >> >>> java -jar target/rng-utils.jar stress -h >> >> rng-utils stress [-hrV] [--dry-run] [--logging=<logLevel>] >> [--prefix=<fileOutputPrefix>] [-b=<byteOrder>] >> [-l=<genList>] >> [-n=<taskCount>] [-o=<outputMode>] [-t=<trials>] >> <executable> >> [<argument>...] >> >> Run repeat trials of random data generators using a provided test >> application. >> Data is transferred to the application sub-process via standard input. >> >> Parameters: >> <executable> The stress test executable. >> [<argument>...] The arguments to pass to the executable. >> >> Options: >> --logging=<logLevel> Specify the logging level (default: INFO). >> Valid values: ERROR, INFO, DEBUG >> -h, --help Show this help message and exit. >> -V, --version Print version information and exit. >> --prefix=<fileOutputPrefix> >> Results file prefix (default: stress_). >> -o, --output-mode=<outputMode> >> Output mode for existing files (default: ERROR). >> Valid values: ERROR, SKIP, APPEND, OVERWRITE >> -l, --list=<genList> List of random generators. >> The default list is all known generators. >> -t, --trials=<trials> The number of trials for each random generator. >> Used only for the default list (default: 1). >> -n, --tasks=<taskCount> Number of concurrent tasks (default: 4). >> Two threads are required per task. >> -b, --byte-order=<byteOrder> >> Byte-order of the transferred data (default: >> LITTLE_ENDIAN). >> Valid values: BIG_ENDIAN, LITTLE_ENDIAN. >> -r, --reverse-bits Reverse the bits in the data (default: false). >> Note: Generators may fail tests for a reverse >> sequence >> when passing using the standard sequence. >> --dry-run Perform a dry run where the generators and >> output files >> are created but the stress test is not >> executed. >> >>> java -jar target/rng-utils.jar stress ./stdin2testu01 SmallCrush >> >> [INFO] Running stress test ... >> [INFO] Progress 0 / 16 (0.00%) >> [INFO] Progress 1 / 16 (6.25%) >> [INFO] Progress 2 / 16 (12.50%) >> [INFO] Progress 3 / 16 (18.75%) >> [INFO] Progress 4 / 16 (25.00%) >> [INFO] Progress 5 / 16 (31.25%) >> [INFO] Progress 7 / 16 (43.75%) >> [INFO] Progress 9 / 16 (56.25%) >> [INFO] Progress 10 / 16 (62.50%) >> [INFO] Progress 12 / 16 (75.00%) >> [INFO] Progress 13 / 16 (81.25%) >> [INFO] Progress 14 / 16 (87.50%) >> [INFO] Progress 15 / 16 (93.75%) >> [INFO] Progress 16 / 16 (100%) >> >> >> Output files will be in 'stress_X_N' where X is the identifier for the >> RNG and N is the trial number. The default identifier X is the order in >> the RandomSource enum. >> >> I have updated the results files to have some more information in the >> header and footer including: start and end times; the byte/bit reversed >> state is prepended to the RNG name; the exit value of the sub-process; >> the count of numbers used for the test: >> >>> grep '^#' stress_4_1 >> >> # >> # RNG: Byte-reversed org.apache.commons.rng.core.source32.Well19937a >> # >> # Java: 1.8.0_191 >> # Runtime: 1.8.0_191-8u191-b12-2ubuntu0.16.04.1-b12 >> # JVM: OpenJDK 64-Bit Server VM 25.191-b12 >> # OS: Linux 4.4.0-143-generic amd64 >> # >> # Analyzer: >> /home/ah403/git/commons-rng/commons-rng-examples/examples-stress/stdin2testu01 >> SmallCrush >> # >> # Start: 2019-04-03 16:40:30 >> # >> # >> # End: 2019-04-03 16:40:55 >> # >> # Exit value: 0 >> # Numbers used: 226897920 >= 2^27 (907.6 MB) >> # >> # Test duration: 0.42309478473333334 minutes >> # >> >> So the current stress test results for all generators can be achieved using: >> >>> java -jar target/rng-utils.jar stress --trials 3 --prefix tu_ >>> ./stdin2testu01 BigCrush >> >>> java -jar target/rng-utils.jar stress --trials 3 --prefix dh_ >>> /usr/bin/dieharder -a -g 200 -Y 1 -k 2 >> >> >> Please try it out using the above PR and provide some feedback. > > The above listing of new features looks quite fine. > Don't hesitate to commit to "master". We can always > make adjustments afterwards if needs be. OK. I’ll clean it up to be more stress-test focussed and commit it. I did have a look at getting more hardware information to add to the output reports. With this library: https://github.com/oshi/oshi <https://github.com/oshi/oshi> I added a command to output system information about the processor: java -jar target/rng-utils.jar system Java: 1.8.0_191 OpenJDK 64-Bit Server VM 25.191-b12 OS: GNU/Linux Ubuntu 16.04.6 LTS (Xenial Xerus) build 4.4.0-145-generic (Arch amd64) Processor: Intel(R) Xeon(R) CPU E5-1680 v3 @ 3.20GHz (1 physical CPU, 8 physical CPU core(s), 16 logical CPU(s)) Memory: 31.3 GiB java -jar target/rng-utils.jar system Java: 1.8.0_131 Java HotSpot(TM) 64-Bit Server VM 25.131-b11 OS: Apple macOS 10.14.3 (Mojave) build 18D109 (Arch x86_64) Processor: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (1 physical CPU, 4 physical CPU core(s), 8 logical CPU(s)) Memory: 16 GiB It works nicely on the two machines I usually work on. The API can access a lot more information but this seems to be the essentials. It may be useful to add this info to all the performance/benchmarking reports in the user manual. But I am not sure how relevant the processor and clock speed are. The JVM is the machine and the hardware to a certain extent is not relevant. I’ve noticed differences when benchmarking between the two machines for speed but that may be due to the use of Hotspot verses OpenJDK rather than the hardware. For the stress test the processor speed is not relevant (output of an RNG should be the same) unless studying the run times for the test. Thoughts? Alex > > Thank you, > Gilles > >> >> Alex > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > <mailto:dev-unsubscr...@commons.apache.org> > For additional commands, e-mail: dev-h...@commons.apache.org > <mailto:dev-h...@commons.apache.org>
