Hi Brooklyners,
We’ve been thinking on how we can make it easy to write tests for entities in
policies. There follows a proposal for a new feature in Brooklyn to provide
such tests directly in YAML Blueprints. We request comments from the community
on the idea, and would very much welcome suggestions for developing it.
Description of Proposal
To ensure the quality of blueprints in the Brooklyn ecosystem, we propose some
additional test support functionality in Brooklyn. This consists of additional
entities which, through composition and configuration, allow for thorough
coverage of blueprint test scenarios. In addition, these tests will provide a
good illustration of how blueprints are intended be used.
For ease of use, we are looking to support the specification of test cases as
YAML blueprints which refer to the system under test and define one or more
test scenarios. These tests could then be invoked manually, simply by
deploying them, or by the maven plugin or an automated test framework.
Over time, this test functionality should include:
- Ability to operate directly upon the entities in Blueprints, testing sensor
values, invoking effectors, checking policies, etc.
- Support for Rebind, i.e. the ability to restart the Brooklyn server, such
that the persisted state of running instances is correctly restored.
- Backwards compatibility is also important - persisted state from previous
versions should work with future versions of Brooklyn.
- Blueprint upgrade: for a running instance of the blueprint, upgrade from one
version of the blueprint to another should be tested.
- Extensibility: A test framework must be extensible: as well as generic
assertions (e.g. service reports as up), there needs to be more specific health
checks (e.g. port is reachable and returns correct HTTP code).
These tests will also help with the QA when releasing a new version of
Brooklyn, but their primary purpose is to provide for QA of individual
entities. Non-goal: these tests are not intended to duplicate or be a
substitute for the unit tests and integration tests that should be written/run
in Brooklyn and in other downstream projects.
The work must be incremental: the first early deliverables should provide
immediate value, while allowing more advanced features to be added over time.
Draft Implementation
To make the ideas above concrete, we have created example code [1].
This includes the following new entities:
- org.apache.brooklyn.test.framework.TestSensor: Checks the value of a sensor
against an expected value.
- org.apache.brooklyn.test.framework.TestEffector: Invoke an effector on an
entity.
- org.apache.brooklyn.test.framework.TestCase: A ‘container’ class for Test
entities. TestCases have brooklyn.children that are started in order.
- org.apache.brooklyn.test.framework.ParallelTestCase: Same as TestCase but
runs its children in parallel.
- org.apache.brooklyn.test.framework.TestHttpCall: Do an HTTP GET and check the
results.
Example Blueprint
This is an example of a blueprint that contains a test. It deploys a Tomcat
Server, then uses TestSensor to check it is running (sensor service.isUp).
Next it tests that the main URI of the Tomcat front page becomes accessible,
with TestHttpCall. It then stops the Tomcat using a TestEffector, and then
again invokes the sensor test, this time with an expectation that the service
is not running (service.isUp = false).
name: Webapp Test
location: localhost
services:
- type: org.apache.brooklyn.test.framework.TestCase
brooklyn.children:
- type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
id: tomcat
brooklyn.config:
war:
"http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war";
- type: org.apache.brooklyn.test.framework.TestSensor
name: tomcat comes up
target: $brooklyn:component("tomcat")
sensor: service.isUp
timeout: 5m
assert:
equals: true
- type: org.apache.brooklyn.test.framework.TestHttpCall
name: tomcat root context accessible
url: url: $brooklyn:component("tomcat").attributeWhenReady("main.uri")
assert:
status: 200
- type: org.apache.brooklyn.test.framework.TestEffector
name: stop tomcat
target: $brooklyn:component("tomcat")
effector: stop
- type: org.apache.brooklyn.test.framework.TestSensor
name: tomcat goes down
target: $brooklyn:component("tomcat")
sensor: service.isUp
assert:
equals: false
TestCase will invoke its children sequentially. The types may be added to the
catalog as `test-case`, `test-sensor`, and `test-effector`; and we might
support `target` and `tests` as more descriptive alternatives to
`brooklyn.children`, with a `target` entity becoming the default target for
sub-tests. Thus the above test case could also in future be written as:
name: Webapp Test
location: localhost
services:
- type: test-case
name: Stop Test
target:
type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
id: tomcat
brooklyn.config:
war:
"http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war";
tests:
- type: test-sensor
name: tomcat comes up
sensor: service.isUp
timeout: 5m
assert:
equals: true
- type: test-httpcall
name: tomcat root context accessible
url: url: $brooklyn:component("tomcat").attributeWhenReady("main.uri")
assert:
status: 200
- type: test-effector
name: stop tomcat
effector: stop
- type: test-sensor
name: tomcat goes down
sensor: service.isUp
assert:
equals: false
Because the individual tests are normal entities, they could be added to the
catalog in the normal way, so that for instance new types test-service-is-up
and test-service-is-down could easily be defined to replace the first and last
tests with a single line.
More complex test scenarios, such as launching a load test script, could be
achieved simply by writing your own entity to conduct the scenario; it could be
pointed at the target, given its own ID, and that entity referenced as part of
subsequent test-sensor and test-effector tests.
Over time we see the library of test entities growing, including actions like
testing HTTP calls and performing rebind into different servers, and we see the
library of tests growing, so that tests become a critical part of any new
blueprint.
References
[1] https://github.com/m4rkmckenna/incubator-brooklyn/tree/test-framework
Chris Burke ([email protected])
David Lloyd ([email protected])
John McCabe ([email protected])
Mark McKenna ([email protected])
Geoff Macartney ([email protected])
