jiridanek commented on pull request #1292: URL: https://github.com/apache/qpid-dispatch/pull/1292#issuecomment-877771444
The benchmark library automatically adjusts number of iterations of each test based on iteration duration and on the variance of previous iteration durations. On GHA that seems to translate to runtime of between 6 to 60 seconds for the single test I have there. This is obviously unacceptably too long going forward. I would not be too hard to split the benchmarks to a separate job and that then gives ~10 minute budget to run (similar to other test jobs). Running the benchmarks without ASan would help to speed it up and make the result more relevant (for performance), too. For now, this PR sets short min_time for the benchmark, so it only runs just about once. Enough to check it is capable of running, does not trigger ASan errors, and that the measured time is not way too outlandish. https://github.com/apache/qpid-dispatch/runs/3039601651?check_suite_focus=true#step:9:4704 ``` 76: Test command: /usr/bin/bwrap "--bind" "/" "/" "--unshare-net" "--dev" "/dev" "--die-with-parent" "--" "/opt/hostedtoolcache/Python/3.7.10/x64/bin/python3.7" "/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests/run.py" "/home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests/c_benchmarks/c-benchmarks" "--benchmark_min_time=0.001" 76: Test timeout computed to be: 1200 76: 2021-07-11 09:47:42 76: Running /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/build/tests/c_benchmarks/c-benchmarks 76: Run on (2 X 2294.69 MHz CPU s) 76: CPU Caches: 76: L1 Data 32K (x2) 76: L1 Instruction 32K (x2) 76: L2 Unified 256K (x2) 76: L3 Unified 51200K (x1) 76: Load Average: 2.30, 1.65, 0.79 76: --------------------------------------------------------------------------- 76: Benchmark Time CPU Iterations 76: --------------------------------------------------------------------------- 76: BM_RouterInitializeMinimalConfig 371 ms 4.25 ms 1 76: ----------------------------------------------------- 76: Suppressions used: 76: count bytes template 76: 1 56 ^IoAdapter_init$ 76: 10 5456 ^_PyObject_Realloc 76: 533 852035 ^PyObject_Malloc$ 76: 1 32 ^PyThread_allocate_lock$ 76: 4 9897 ^PyMem_Malloc$ 76: 2 1528 ^_PyObject_GC_Resize$ 76: ----------------------------------------------------- 76: 37/38 Test #76: c-benchmarks ...................................... Passed 0.73 sec ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
