jeremiaswerner commented on issue #3942: make error count and percentage configurable for gatling tests URL: https://github.com/apache/incubator-openwhisk/pull/3942#issuecomment-411029337 I've extended the PR to make all thresholds configurable via CLI invocation. ``` # API_KEY=<mykey> OPENWHISK_HOST=<myhost> USERS="10" REQUESTS_PER_SEC="10" MEAN_RESPONSE_TIME="500" SWIFT_MEAN_RESPONSE_TIME="1000" JAVA_MAX_ERRORS_ALLOWS=5 MAX_ERRORS_ALLOWED="1" ./gradlew gatlingRun-LatencySimulation ``` Result looks like ``` ================================================================================ ---- Global Information -------------------------------------------------------- > request count 412 (OK=412 KO=0 ) > min response time 174 (OK=174 KO=- ) > max response time 823 (OK=823 KO=- ) > mean response time 199 (OK=199 KO=- ) > std deviation 69 (OK=69 KO=- ) > response time 50th percentile 180 (OK=180 KO=- ) > response time 75th percentile 192 (OK=192 KO=- ) > response time 95th percentile 302 (OK=302 KO=- ) > response time 99th percentile 520 (OK=520 KO=- ) > mean requests/sec 4.964 (OK=4.964 KO=- ) ---- Response Time Distribution ------------------------------------------------ > t < 800 ms 411 (100%) > 800 ms < t < 1200 ms 1 ( 0%) > t > 1200 ms 0 ( 0%) > failed 0 ( 0%) ================================================================================ Reports generated in 0s. Please open the following file: /Users/jerewern/repos/bluewhisk/open/tests/performance/gatling_tests/build/reports/gatling/latencysimulation-1533642179855/index.html Warm nodejs:default invocation: mean of response time is less than or equal to 500.0 : true Warm nodejs:default invocation: mean of response time is less than 500.0 : true Warm nodejs:default invocation: count of failed requests is less than or equal to 1.0 : true Warm nodejs:default invocation: percentage of failed requests is less than or equal to 0.1 : true Warm python:default invocation: mean of response time is less than or equal to 500.0 : true Warm python:default invocation: mean of response time is less than 500.0 : true Warm python:default invocation: count of failed requests is less than or equal to 1.0 : true Warm python:default invocation: percentage of failed requests is less than or equal to 0.1 : true Warm swift:default invocation: mean of response time is less than or equal to 1000.0 : true Warm swift:default invocation: mean of response time is less than 500.0 : true Warm swift:default invocation: count of failed requests is less than or equal to 1.0 : true Warm swift:default invocation: percentage of failed requests is less than or equal to 0.1 : true Warm java:default invocation: mean of response time is less than or equal to 500.0 : true Warm java:default invocation: mean of response time is less than 500.0 : true Warm java:default invocation: count of failed requests is less than or equal to 1.0 : true Warm java:default invocation: percentage of failed requests is less than or equal to 0.1 : true BUILD SUCCESSFUL in 1m 25s 3 actionable tasks: 1 executed, 2 up-to-date ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
