Hello everyone
We are planning to add integration testing framework and initial test cases
in https://issues.apache.org/jira/browse/YUNIKORN-29, general thoughts are
as follows.
Basic testing framework includes:
1. AppInfo struct: define basic information, requests and status of an
application
2. AppManager interface and its implementations like DeploymentAppManager:
support useful operations like create/delete/refresh(status)/wait(for apps
to be satisfied or cleaned up) for testing.
3. CommonConfig struct: keep several common configurations for testing like
schedulerName, outputRootPath, namespace, queue etc.
4. Output tools like chart painter and file writer with specific format
(like csv).
Initial test cases:
1. Throughput : Request a certain number of pods via different schedulers
and then record the distributions of scheduled pods, draw results of
different schedulers on the same chart or write results into a file.
2. Node Fairness: Request a certain number of pods via different schedulers
and then record the distributions of node usage, draw results on charts
separately for different schedulers and write results into a file.
3. Queue Fairness (Only for YuniKorn since K8s scheduler doesn't support it
yet): Prepare queues with different capacities, request pods with different
number or resource for these queues, then record the usage of queues, draw
results on the same chart and write results into a file.
Implementation:
1. Package structure: add test/integration package in yunikorn-k8shim
project, the source files with structures or tools of testing framework
will be directly putted in this package, and test cases will be seperately
managed by case package in sub-directories: cases/<special-case> such as
cases/throughput, cases/node-fairness, cases/queue-fairness
2. Configurations: cases keep their specific configurations themselves and
all configurations can be maintained in a single yaml file together with
common configurations, the config structure like this:
```
common:
schedulerName: yunikorn
maxWaitSeconds: 60
queue: root.default
namespace: default
outputRootPath: /tmp
cases:
throughput:
schedulerNames:
- yunikorn
- default-scheduelr
appName: throughput-test
numPods: 10000
...
node-fairness:
...
queue-fairness:
...
```
3. Executions: all cases have main function themselves and need one
command-line argument: configPath, so that we can directly run specified
case and easily perform integration testing on different scenarios.
Any comments and suggestions are welcome!
Thanks,
Tao
