+100 :) Particularly I like the incremental approach and the clear mid-term roadmap.
Let's do it ;) Andrea Il giorno gio 29 ott 2015 21:16 Geoff Macartney < [email protected]> ha scritto: > Hi Brooklyners, > > We’ve been thinking on how we can make it easy to write tests for entities > in policies. There follows a proposal for a new feature in Brooklyn to > provide such tests directly in YAML Blueprints. We request comments from > the community on the idea, and would very much welcome suggestions for > developing it. > > > > Description of Proposal > > To ensure the quality of blueprints in the Brooklyn ecosystem, we propose > some additional test support functionality in Brooklyn. This consists of > additional entities which, through composition and configuration, allow for > thorough coverage of blueprint test scenarios. In addition, these tests > will provide a good illustration of how blueprints are intended be used. > > For ease of use, we are looking to support the specification of test cases > as YAML blueprints which refer to the system under test and define one or > more test scenarios. These tests could then be invoked manually, simply by > deploying them, or by the maven plugin or an automated test framework. > > Over time, this test functionality should include: > > - Ability to operate directly upon the entities in Blueprints, testing > sensor values, invoking effectors, checking policies, etc. > - Support for Rebind, i.e. the ability to restart the Brooklyn server, > such that the persisted state of running instances is correctly restored. > - Backwards compatibility is also important - persisted state from > previous versions should work with future versions of Brooklyn. > - Blueprint upgrade: for a running instance of the blueprint, upgrade from > one version of the blueprint to another should be tested. > - Extensibility: A test framework must be extensible: as well as generic > assertions (e.g. service reports as up), there needs to be more specific > health checks (e.g. port is reachable and returns correct HTTP code). > > These tests will also help with the QA when releasing a new version of > Brooklyn, but their primary purpose is to provide for QA of individual > entities. Non-goal: these tests are not intended to duplicate or be a > substitute for the unit tests and integration tests that should be > written/run in Brooklyn and in other downstream projects. > > The work must be incremental: the first early deliverables should provide > immediate value, while allowing more advanced features to be added over > time. > > > > Draft Implementation > > To make the ideas above concrete, we have created example code [1]. > > This includes the following new entities: > > - org.apache.brooklyn.test.framework.TestSensor: Checks the value of a > sensor against an expected value. > - org.apache.brooklyn.test.framework.TestEffector: Invoke an effector on > an entity. > - org.apache.brooklyn.test.framework.TestCase: A ‘container’ class for > Test entities. TestCases have brooklyn.children that are started in order. > - org.apache.brooklyn.test.framework.ParallelTestCase: Same as TestCase > but runs its children in parallel. > - org.apache.brooklyn.test.framework.TestHttpCall: Do an HTTP GET and > check the results. > > > > Example Blueprint > > This is an example of a blueprint that contains a test. It deploys a > Tomcat Server, then uses TestSensor to check it is running (sensor > service.isUp). Next it tests that the main URI of the Tomcat front page > becomes accessible, with TestHttpCall. It then stops the Tomcat using a > TestEffector, and then again invokes the sensor test, this time with an > expectation that the service is not running (service.isUp = false). > > > name: Webapp Test > location: localhost > services: > - type: org.apache.brooklyn.test.framework.TestCase > brooklyn.children: > - type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer > id: tomcat > brooklyn.config: > war: " > http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war > " > - type: org.apache.brooklyn.test.framework.TestSensor > name: tomcat comes up > target: $brooklyn:component("tomcat") > sensor: service.isUp > timeout: 5m > assert: > equals: true > - type: org.apache.brooklyn.test.framework.TestHttpCall > name: tomcat root context accessible > url: url: $brooklyn:component("tomcat").attributeWhenReady("main.uri") > assert: > status: 200 > - type: org.apache.brooklyn.test.framework.TestEffector > name: stop tomcat > target: $brooklyn:component("tomcat") > effector: stop > - type: org.apache.brooklyn.test.framework.TestSensor > name: tomcat goes down > target: $brooklyn:component("tomcat") > sensor: service.isUp > assert: > equals: false > > > TestCase will invoke its children sequentially. The types may be added to > the catalog as `test-case`, `test-sensor`, and `test-effector`; and we > might support `target` and `tests` as more descriptive alternatives to > `brooklyn.children`, with a `target` entity becoming the default target for > sub-tests. Thus the above test case could also in future be written as: > > > name: Webapp Test > location: localhost > services: > - type: test-case > name: Stop Test > target: > type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer > id: tomcat > brooklyn.config: > war: " > http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war > " > tests: > - type: test-sensor > name: tomcat comes up > sensor: service.isUp > timeout: 5m > assert: > equals: true > - type: test-httpcall > name: tomcat root context accessible > url: url: $brooklyn:component("tomcat").attributeWhenReady("main.uri") > assert: > status: 200 > - type: test-effector > name: stop tomcat > effector: stop > - type: test-sensor > name: tomcat goes down > sensor: service.isUp > assert: > equals: false > > Because the individual tests are normal entities, they could be added to > the catalog in the normal way, so that for instance new types > test-service-is-up and test-service-is-down could easily be defined to > replace the first and last tests with a single line. > > More complex test scenarios, such as launching a load test script, could > be achieved simply by writing your own entity to conduct the scenario; it > could be pointed at the target, given its own ID, and that entity > referenced as part of subsequent test-sensor and test-effector tests. > > Over time we see the library of test entities growing, including actions > like testing HTTP calls and performing rebind into different servers, and > we see the library of tests growing, so that tests become a critical part > of any new blueprint. > > References > > [1] https://github.com/m4rkmckenna/incubator-brooklyn/tree/test-framework > > > Chris Burke ([email protected]) > David Lloyd ([email protected]) > John McCabe ([email protected]) > Mark McKenna ([email protected]) > Geoff Macartney ([email protected]) > >
