[GitHub] markusthoemmes commented on a change in pull request #3942: make error count and percentage configurable for gatling tests

[GitBox]

markusthoemmes commented on a change in pull request #3942: make error count 
and percentage configurable for gatling tests
URL: 
https://github.com/apache/incubator-openwhisk/pull/3942#discussion_r208202442
 
 

 ##
 File path: 
tests/performance/gatling_tests/src/gatling/scala/LatencySimulation.scala
 ##
 @@ -91,14 +99,20 @@ class LatencySimulation extends Simulation {
 .protocols(openWhiskProtocol)
 
   actions
-.map { case (kind, _, _, _) => s"Warm $kind invocation" }
-.foldLeft(testSetup) { (agg, cur) =>
+.map { case (kind, _, _, _) => kind }
+.foldLeft(testSetup) { (agg, kind) =>
+  val cur = s"Warm $kind invocation"
   // One failure will make the build yellow
+  val mrt : Int = sys.env.getOrElse(kind.split(":").head.toUpperCase + "_" 
+ MEAN_RESPONSE_TIME, meanResponseTime.toString).toInt
+  val maxrt = sys.env.getOrElse(kind.split(":").head.toUpperCase + "_" + 
MAX_MEAN_RESPONSE_TIME, maximalMeanResponseTime.toString).toInt
+  val maxerr = sys.env.getOrElse(kind.split(":").head.toUpperCase + "_" + 
MAX_ERRORS_ALLOWED, maxErrorsAllowed.toString).toInt
+  val maxerrp = sys.env.getOrElse(kind.split(":").head.toUpperCase + "_" + 
MAX_ERRORS_ALLOWED_PERCENTAGE, maxErrorsAllowedPercentage.toString).toDouble
 
 Review comment:
   Does it make sense to exclude this into a function to prevent duplication, 
like:
   
   ```scala
   def toKindSpecificKey(kind: String, suffix: String) = 
kind.split(':').head.toUpperCase + "_" + suffix
   ```
   
   To be used like:
   
   ```scala
   val mrt: Int = sys.env.getOrElse(toKindSpecificKey(kind, MEAN_RESPONSE_TIME) 
, meanResponseTime.toString).toInt
   ```
   
   Also: No need to abbreviate the constant names here. Please make them "fully 
qualified" to be readable more easily, like: `specificMeanResponseTime`, 
`specificMaxResponseTime` etc.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] markusthoemmes commented on a change in pull request #3942: make error count and percentage configurable for gatling tests

[GitBox]

markusthoemmes commented on a change in pull request #3942: make error count 
and percentage configurable for gatling tests
URL: 
https://github.com/apache/incubator-openwhisk/pull/3942#discussion_r208200918
 
 

 ##
 File path: tests/performance/README.md
 ##
 @@ -101,14 +103,20 @@ The comparison of the thresholds is against the mean 
response times of the warm
 Available environment variables:
 
 ```
-OPENWHISK_HOST  (required)
-API_KEY (required, format: UUID:KEY)
-PAUSE_BETWEEN_INVOKES   (default: 0)
-MEAN_RESPONSE_TIME  (required)
-MAX_MEAN_RESPONSE_TIME  (default: MEAN_RESPONSE_TIME)
-EXCLUDED_KINDS  (default: "", format: 
"python:default,java:default,swift:default")
+OPENWHISK_HOST(required)
+API_KEY   (required, format: UUID:KEY)
+PAUSE_BETWEEN_INVOKES (default: 0)
+MEAN_RESPONSE_TIME(required)
+MAX_MEAN_RESPONSE_TIME(default: MEAN_RESPONSE_TIME)
+EXCLUDED_KINDS(default: "", format: 
"python:default,java:default,swift:default")
+MAX_ERRORS_ALLOWED(default: 0)
+MAX_ERRORS_ALLOWED_PERCENTAGE (default: 0)
 ```
 
+It is possible to override the `MEAN_RESPONSE_TIME`, `MAX_MEAN_RESPONSE_TIME`, 
`MAX_ERRORS_ALLOWED` and `MAX_ERRORS_ALLOWED_PERCENTAGE`
+for each kind by adding the kind as perfix in upper case, like 
`JAVA_MEAN_RESPONSE_TIME`.
 
 Review comment:
   typo: prefix.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services