[GitHub] [apisix] Yiyiyimu commented on issue #3391: [proposal]: add performance testing in CI

2021-01-21 Thread GitBox
Yiyiyimu commented on issue #3391: URL: https://github.com/apache/apisix/issues/3391#issuecomment-765199553 Waiting for the test result! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [apisix] Yiyiyimu commented on issue #3391: [proposal]: add performance testing in CI

2021-01-21 Thread GitBox
Yiyiyimu commented on issue #3391: URL: https://github.com/apache/apisix/issues/3391#issuecomment-764518459 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [apisix] Yiyiyimu commented on issue #3391: [proposal]: add performance testing in CI

2021-01-21 Thread GitBox
Yiyiyimu commented on issue #3391: URL: https://github.com/apache/apisix/issues/3391#issuecomment-765098590 Cool! I think we need some data to prove the point This is an automated message from the Apache Git Service. To

[GitHub] [apisix] Yiyiyimu commented on issue #3391: [proposal]: add performance testing in CI

2021-01-21 Thread GitBox
Yiyiyimu commented on issue #3391: URL: https://github.com/apache/apisix/issues/3391#issuecomment-764605634 @idbeta How could we know that "not differ too much" would happen, or "they are within an order of magnitude". Maybe we need some methods to get over those false positives, but not

[GitHub] [apisix] Yiyiyimu commented on issue #3391: [proposal]: add performance testing in CI

2021-01-21 Thread GitBox
Yiyiyimu commented on issue #3391: URL: https://github.com/apache/apisix/issues/3391#issuecomment-764530183 > ok, good idea, the performance of Github action will change a lot, I tested it several times, QPS from `13300` to `21433`, so different. Wow that indeed differs a lot. Would

[GitHub] [apisix] Yiyiyimu commented on issue #3391: [proposal]: add performance testing in CI

2021-01-21 Thread GitBox
Yiyiyimu commented on issue #3391: URL: https://github.com/apache/apisix/issues/3391#issuecomment-764518459 The lowest result out of 5 test seems very easy to be surpassed. Maybe we could test for like 100 times, get its distribution and try to set the benchmark value