http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/about-chef.md ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/about-chef.md b/guide/yaml/chef/about-chef.md new file mode 100644 index 0000000..db3ec3c --- /dev/null +++ b/guide/yaml/chef/about-chef.md @@ -0,0 +1,50 @@ +--- +title: About Chef +title_in_menu: About Chef +layout: website-normal +--- + +## What you need to know about Chef + +Chef works in two different modes, *server* and *solo*. *Server* is where the Chef client talks to a central server +to retrieve information about its roles, policies and cookbooks (where a cookbook defines how to install and +configure a particular piece of software). With *solo*, the client works in isolation, therefore its configuration +and cookbooks must be supplied by another means. + +Chef *client* is the Chef agent. This is a Ruby application which is installed on each and every managed host. When +invoked in server mode, it will contact the Chef server to check for updates to cookbooks and policy; it then "runs" +the recipes in its run lists, to converge the machine to a known state. In solo mode, it reads the locally-maintained +cookbooks and policies. The client may be run as a daemon that checks the server regularly, or it could merely be +run manually when required. + +The *policy* is a set of rules on the Chef server. A client starts with a set of *attributes*, which could be as +simple as its name and a recipe runlist, or which may involve a more complex set of attributes about how it is to be +configured. The client then augments this with auto-detected metadata - a tool called `ohai` is run that collects +detailed information about the host. Next, the policy on the server modifies these attributes - overriding some, +setting defaults for others - to produce a final set of attributes. It is these which are the input to the recipes. +Finally, the attributes are uploaded to the server where they are stored as metadata for the node, where they can be +inspected and modified by the system operator. + +Also of interest is `knife`, which is the workstation toolkit for Chef. Typically this would be installed on the +operation engineer's workstation, where it would be used to interact with the Chef server and clients. Of particular +interest to us is the *bootstrap* operation, which is used for setting up new Chef clients - given a virtual machine, +it will install the Chef client on it, configure it with enough information to find the Chef server and performs its +first run, and then kicks off the Chef client for the first time. + +There is often a preconception about how a Chef client is bootstrapped; mistakenly, there is the belief that the +`knife` tool configures the Chef server with information about the client, and the client finds out about itself from +the server. This is not the case - the bootstrap operation does not involve `knife` talking to the server. Instead, +`knife` packages up all of the required information and sends it to the client - the client will then introduce +itself to the server, passing on its configuration. + +This diagram summarises the interaction between Brooklyn, the new node, and the various Chef tools. Note that there +is no interaction between the AMP Server and the Chef Server. + +[](chef-call-flow.png) + +### How Brooklyn interacts with Chef + +Brooklyn understands both the *server* and *solo* modes of operation. Server mode utilises the `knife` toolkit, and +therefore `knife` must be installed onto the AMP server and configured appropriately. Solo mode does not have any +special requirements; when running in solo mode, Brooklyn will install and configure the Chef client over SSH, just +like it does most other kinds of entities.
http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/advanced-chef-integration.md ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/advanced-chef-integration.md b/guide/yaml/chef/advanced-chef-integration.md new file mode 100644 index 0000000..0d65286 --- /dev/null +++ b/guide/yaml/chef/advanced-chef-integration.md @@ -0,0 +1,48 @@ +--- +title: Advanced Chef Integration +title_in_menu: Advanced Chef Integration +layout: website-normal +--- + +### Adding Sensors and Effectors + +Custom sensors and effectors can be added using an `entity.initializer` section in the YAML blueprint. + +One common pattern is to have sensors which extract information from Ohai. +Another common pattern is to install a monitoring agent as part of the run list, +configured to talk to a monitoring store, and then to add a sensor feed which reads data from that store. + +On the effector side, you can add SSH-based effectors in the usual way. +You can also describe additional chef converge targets following the pattern set down in +`ChefLifecycleEffectorTasks`, making use of conveniences in `ChefSoloTasks` and `ChefServerTasks`, +or provide effectors which invoke network API's of the systems under management +(for example to supply the common `executeScript` effector as on the standard `MySqlNode`). + + +### Next Steps: Simpifying sensors and effectors, transferring files, and configuring ports + +The Brooklyn-Chef integration is work in progress, with a few open issues we'd still like to add. +Much of the thinking for this is set forth in the [Google document](https://docs.google.com/a/cloudsoftcorp.com/document/d/18ZwzmncbJgJeQjnSvMapTWg6N526cvGMz5jaqdkxMf8) +indicated earlier. If you'd like to work with us to implement these, please let us know. + + +## Reference + +A general schema for the supported YAML is below: + +{% highlight yaml %} +- type: chef:cookbook_name + cookbook_urls: + cookbook_name: url://for/cookbook.tgz + dependency1: url://for/dependency1.tgz + launch_run_list: [ "cookbook_name::start" ] + launch_attributes: # map of arguments to set in the chef node + service_name: cookbook_service + pid_file: /var/run/cookbook.pid +{% endhighlight %} + +If you are interested in exploring the Java code for creating blueprints, +start with the `TypedToyMySqlEntiyChef` class, which essentially does what this tutorial has shown; +and then move on to the `DynamicToyMySqlEntiyChef` which starts to look at more sophisticated constructs. +(Familiarity with BASH and basic Java blueprints may be useful at that stage.) + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/chef-call-flow.png ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/chef-call-flow.png b/guide/yaml/chef/chef-call-flow.png new file mode 100644 index 0000000..d899de2 Binary files /dev/null and b/guide/yaml/chef/chef-call-flow.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/creating-blueprints.md ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/creating-blueprints.md b/guide/yaml/chef/creating-blueprints.md new file mode 100644 index 0000000..8d30131 --- /dev/null +++ b/guide/yaml/chef/creating-blueprints.md @@ -0,0 +1,105 @@ +--- +title: Creating Blueprints from Chef +title_in_menu: Creating Blueprints from Chef +layout: website-normal +--- + +In a nutshell, a new Chef-based entity can be defined as a service by specifying +`chef:cookbook_name` as the `service_type`, along with a collection of optional configuration. +An illustrative example is below: + +{% highlight yaml %} +{% readj example_yaml/mysql-chef-1.yaml %} +{% endhighlight %} + +*This works without any installation: try it now, copying-and-pasting to the Brooklyn console. +(Don't forget to add your preferred `location: some-cloud` to the spec.)* + +Notice, if you target `google-compute-engine` location, you may need to specify `bind_address: 0.0.0.0` for the `mysql` cookbook, as described [here](https://github.com/chef-cookbooks/mysql/blob/46dccac22d282a05ee6a401e10ae8f5f8114fd66/README.md#parameters). + +We'll now walk through the important constituent parts, +and then proceed to describing things which can be done to simplify the deployment. + + +### Cookbook Primary Name + +The first thing to note is the type definition: + + - type: chef:mysql + +This indicates that the Chef entity should be used (`org.apache.brooklyn.entity.chef.ChefEntity`) +to interpret and pass the configuration, +and that it should be parameterised with a `brooklyn.chef.cookbook.primary.name` of `mysql`. +This is the cookbook namespace used by default for determining what to install and run. + + +### Importing Cookbooks + +Next we specify which cookbooks are required and where they can be pulled from: + + cookbook_urls: + mysql: https://github.com/opscode-cookbooks/mysql/archive/v4.0.12.tar.gz + openssl: https://github.com/opscode-cookbooks/openssl/archive/v1.1.0.tar.gz + build-essential: https://github.com/opscode-cookbooks/build-essential/archive/v1.4.4.tar.gz + +Here, specific versions are being downloaded from the canonical github repository. +Any URL can be used, so long as it is resolvable on either the target machine or the +Brooklyn server; this includes `file:` and `classpath:` URLs. + +The archive can be ZIP or TAR or TGZ. + +The structure of the archive must be that a single folder is off the root, +and in that folder contains the usual Chef recipe and auxiliary files. +For example, the archive might contain `mysql-master/recipes/server.rb`. +Archives such as those above from github match this format. +The name of that folder does not matter, as often they contain version information. +When deployed, these will be renamed to match the short name (the key in the `cookbooks_url` map, +for instance `mysql` or `openssl`). + +If Chef server is configured (see below), this section can be omitted. + + +### Launch Run List and Attributes + +The next part is to specify the Chef run list and attributes to store when launching the entity: + + launch_run_list: + - mysql::server + + launch_attributes: + mysql: + server_root_password: p4ssw0rd + server_repl_password: p4ssw0rd + server_debian_password: p4ssw0rd + +For the `launch_run_list`, you can use either the YAML `- recipe` syntax or the JSON `[ "recipe" ]` syntax. + +The `launch_attributes` key takes a map which will be stored against the `node` object in Chef. +Thus in this example, the parameter `node['mysql']['server_root_password']` required by the mysql blueprint +is set as specified. + +You can of course set many other attributes in this manner, in addition to those that are required! + + +### Simple Monitoring + +The final section determines how Brooklyn confirms that the service is up. +Sophisticated solutions may install monitoring agents as part of the `launch_run_list`, +with Brooklyn configured to read monitoring information to confirm the launch was successful. +However for convenience, two common mechanisms are available out of the box: + + #service_name: mysqld + pid_file: /var/run/mysqld/mysqld.pid + +If `service_name` is supplied, Brooklyn will check the return code of the `status` command +run against that service, ensuring it is 0. (Note that this is not universally reliable, +although it is the same mechanism which Chef typically uses to test status when determining +whether to start a service. Some services, e.g. postgres, will return 0 even if the service +is not running.) + +If a `pid_file` is supplied, Brooklyn will check whether a process with the PID specified in that +file is running. This has been selected for mysql because it appears to be more portable: +the service name varies among OS's: it is `mysqld` on CentOS but `mysql` on Ubuntu! + + + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/example_yaml/mysql-chef-1.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/example_yaml/mysql-chef-1.yaml b/guide/yaml/chef/example_yaml/mysql-chef-1.yaml new file mode 100644 index 0000000..bdac530 --- /dev/null +++ b/guide/yaml/chef/example_yaml/mysql-chef-1.yaml @@ -0,0 +1,24 @@ +name: chef-mysql-sample +services: +- type: chef:mysql + + cookbook_urls: + # only needed for chef solo; URL can be local to brooklyn, or github, etc... + mysql: https://github.com/opscode-cookbooks/mysql/archive/v4.0.12.tar.gz + openssl: https://github.com/opscode-cookbooks/openssl/archive/v1.1.0.tar.gz + build-essential: https://github.com/opscode-cookbooks/build-essential/archive/v1.4.4.tar.gz + + launch_run_list: [ "mysql::server" ] + launch_attributes: + mysql: + # these attrs are required by the mysql cookbook under node['mysql'] + server_root_password: p4ssw0rd + server_repl_password: p4ssw0rd + server_debian_password: p4ssw0rd + # many others are attrs are supported by the cookbook and can be passed here... + + # how to determine if the process is running and how to kill it + # (supported options are `service_name` and `pid_file`; normally you should just pick one. + # here we use the pid_file because the service_name varies, mysql on centos, mysqld on ubuntu!) + #service_name: mysqld + pid_file: /var/run/mysqld/mysqld.pid http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/example_yaml/mysql-chef-2.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/example_yaml/mysql-chef-2.yaml b/guide/yaml/chef/example_yaml/mysql-chef-2.yaml new file mode 100644 index 0000000..ced2dbe --- /dev/null +++ b/guide/yaml/chef/example_yaml/mysql-chef-2.yaml @@ -0,0 +1,28 @@ +name: chef-mysql-sample +services: +- type: chef:mysql + id: db + + cookbook_urls: + # only needed for chef solo; URL can be local to brooklyn, or github, etc... + mysql: https://github.com/opscode-cookbooks/mysql/archive/v4.0.12.tar.gz + openssl: https://github.com/opscode-cookbooks/openssl/archive/v1.1.0.tar.gz + build-essential: https://github.com/opscode-cookbooks/build-essential/archive/v1.4.4.tar.gz + + launch_run_list: [ "mysql::server" ] + launch_attributes: + mysql: + # these attrs are required by the mysql cookbook under node['mysql'] + server_root_password: $brooklyn:component("db").config("mysql.password") + server_repl_password: $brooklyn:component("db").config("mysql.password") + server_debian_password: $brooklyn:component("db").config("mysql.password") + # many others are attrs are supported by the cookbook and can be passed here... + + # how to determine if the process is running and how to kill it + # (supported options are `service_name` and `pid_file`; normally you should just pick one. + # here we use the pid_file because the service_name varies, mysql on centos, mysqld on ubuntu!) + #service_name: mysqld + pid_file: /var/run/mysqld/mysqld.pid + +brooklyn.config: + mysql.password: p4ssw0rd http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/index.md ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/index.md b/guide/yaml/chef/index.md new file mode 100644 index 0000000..f0fa3d0 --- /dev/null +++ b/guide/yaml/chef/index.md @@ -0,0 +1,18 @@ +--- +title: Chef in YAML Blueprints +layout: website-normal +children: +- about-chef.md +- creating-blueprints.md +- writing-chef.md +- advanced-chef-integration.md +--- + +This guide describes how Brooklyn entities can be easily created from Chef cookbooks. +As of this writing (May 2014) some of the integration points are under active development, +and comments are welcome. +A plan for the full integration is online [here](https://docs.google.com/a/cloudsoftcorp.com/document/d/18ZwzmncbJgJeQjnSvMapTWg6N526cvGMz5jaqdkxMf8). + +This guide assumes you are familiar with the basics of [creating YAML blueprints](../). + +{% include list-children.html %} http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/chef/writing-chef.md ---------------------------------------------------------------------- diff --git a/guide/yaml/chef/writing-chef.md b/guide/yaml/chef/writing-chef.md new file mode 100644 index 0000000..0593837 --- /dev/null +++ b/guide/yaml/chef/writing-chef.md @@ -0,0 +1,79 @@ +--- +title: Writing Chef for Blueprints +title_in_menu: Writing Chef for Blueprints +layout: website-normal +--- + +## Making it Simpler + +The example we've just seen shows how existing Chef cookbooks can be +used as the basis for entities. If you're *writing* the Chef recipes, +there are a few simple techniques we've established with the Chef community +which make blueprints literally as simple as: + + - type: chef:mysql + brooklyn.config: + mysql_password: p4ssw0rd + pid_file: /var/run/mysqld/mysqld.pid + + +### Some Basic Conventions + +* **A `start` recipe**: + The first step is to provide a `start` recipe in `recipes/start.rb`; + if no `launch_run_list` is supplied, this is what will be invoked to launch the entity. + It can be as simple as a one-line file: + + include_recipe 'mysql::server' + +* **Using `brooklyn.config`**: + All the `brooklyn.config` is passed to Chef as node attributes in the `node['brooklyn']['config']` namespace. + Thus if the required attributes in the mysql recipe are set to take a value set in + `node['brooklyn']['config']['mysql_password']`, you can dispense with the `launch_attributes` section. + + +## Using Chef Server + +The examples so far have not required Chef Server, so they will work without any external +Chef dependencies (besides the built-in install from `https://www.opscode.com/chef/install.sh` +and the explicitly referenced cookbooks). If you use Chef Server, however, you'll want your +managed nodes to be integrated with it. This is easy to set up, with a few options: + +If you have `knife` set up in your shell environment, the Brooklyn Chef support will use it +by default. If the recipes are installed in your Chef server, you can go ahead and remove +the `cookbooks_url` section! + +Use of `solo` or `knife` can be forced by setting the `chef_mode` flag (`brooklyn.chef.mode` config key) +to either of those values. (It defaults to `autodetect`, which will use `knife` if it is on the path and satisfies +sanity checks). + +If you want to specify a different configuration, there are a number of config keys you can use: + +* `brooklyn.chef.knife.executableFile`: this should be point to the knife binary to use +* `brooklyn.chef.knife.configFile`: this should point to the knife configuration to use +* `brooklyn.chef.knife.setupCommands`: an optional set of commands to run prior to invoking knife, + for example to run `rvm` to get the right ruby version on the Brooklyn server + +If you're interested in seeing the Chef REST API be supported directly (without knife), +please let us know. We'd like to see this too, and we'll help you along the way! + + +## Tips and Tricks + +To help you on your way writing Chef blueprints, here are a handful of pointers +particularly useful in this context: + +* Configuration keys can be inherited from the top-level and accessed using `$brooklyn:component('id').config('key_name')`. + An example of this is shown in the `mysql-chef.yaml` sample recipe contained in the Brooklyn code base + and [here](example_yaml/mysql-chef-2.yaml) for convenience. + Here, `p4ssw0rd` is specified only once and then used for all the attributes required by the stock mysql cookbook. + +* Github tarball downloads! You'll have noticed these in the example already, but they are so useful we thought + we'd call them out again. Except when you're developing, we recommend using specific tagged versions rather than master. + +* The usual machine `provisioning.properties` are supported with Chef blueprints, + so you can set things like `minRam` and `osFamily` + +* To see more configuration options, and understand the ones presented here in more detail, see the javadoc or + the code for the class `ChefConfig` in the Brooklyn code base. + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/clusters-and-policies.md ---------------------------------------------------------------------- diff --git a/guide/yaml/clusters-and-policies.md b/guide/yaml/clusters-and-policies.md new file mode 100644 index 0000000..fc65179 --- /dev/null +++ b/guide/yaml/clusters-and-policies.md @@ -0,0 +1,42 @@ +--- +title: Clusters and Policies +layout: website-normal +toc: ../guide_toc.json +categories: [use, guide, defining-applications] +--- + +Now let's bring the concept of the "cluster" back in. +We could wrap our appserver in the same `DynamicCluster` we used earlier, +although then we'd need to define and configure the load balancer. +But another blueprint, the `ControlledDynamicWebAppCluster`, does this for us. +It takes the same `memberSpec`, so we can build a fully functional elastic 3-tier +deployment of our `hello-world-sql` application as follows: + +{% highlight yaml %} +{% readj example_yaml/appserver-clustered-w-db.yaml %} +{% endhighlight %} + +This sets up Nginx as the controller by default, but that can be configured +using the `controllerSpec` key. In fact, JBoss is the default appserver, +and because configuration in Brooklyn is inherited by default, +the same blueprint can be expressed more concisely as: + +{% highlight yaml %} +{% readj example_yaml/appserver-clustered-w-db-concise.yaml %} +{% endhighlight %} + +The other nicety supplied by the `ControlledDynamicWebAppCluster` blueprint is that +it aggregates sensors from the appserver, so we have access to things like +`webapp.reqs.perSec.windowed.perNode`. +These are convenient for plugging in to policies! +We can set up our blueprint to do autoscaling based on requests per second +(keeping it in the range 10..100, with a maximum of 5 appserver nodes) +as follows: + +{% highlight yaml %} +{% readj example_yaml/appserver-w-policy.yaml %} +{% endhighlight %} + +Use your favorite load-generation tool (`jmeter` is one good example) to send a huge +volume of requests against the server and see the policies kick in to resize it. + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/clusters.md ---------------------------------------------------------------------- diff --git a/guide/yaml/clusters.md b/guide/yaml/clusters.md new file mode 100644 index 0000000..78b1df9 --- /dev/null +++ b/guide/yaml/clusters.md @@ -0,0 +1,34 @@ +--- +title: Clusters, Specs, and Composition +layout: website-normal +toc: ../guide_toc.json +categories: [use, guide, defining-applications] +--- + +What if you want multiple machines? + +One way is just to repeat the `- type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess` block, +but there's another way which will keep your powder [DRY](http://en.wikipedia.org/wiki/Don't_repeat_yourself): + +{% highlight yaml %} +{% readj example_yaml/cluster-vm.yaml %} +{% endhighlight %} + +Here we've composed the previous blueprint introducing some new important concepts, the `DynamicCluster` +the `$brooklyn` DSL, and the "entity-spec". Let's unpack these. + +The `DynamicCluster` creates a set of homogeneous instances. +At design-time, you specify an initial size and the specification for the entity it should create. +At runtime you can restart and stop these instances as a group (on the `DynamicCluster`) or refer to them +individually. You can resize the cluster, attach enrichers which aggregate sensors across the cluster, +and attach policies which, for example, replace failed members or resize the cluster dynamically. + +The specification is defined in the `memberSpec` key. As you can see it looks very much like the +previous blueprint, with one extra line. Entries in the blueprint which start with `$brooklyn:` +refer to the Brooklyn DSL and allow a small amount of logic to be embedded +(if there's a lot of logic, it's recommended to write a blueprint YAML plugin or write the blueprint itself +as a plugin, in Java or a JVM-supported language). + +In this case we want to indicate that the parameter to `memberSpec` is an entity specification +(`EntitySpec` in the underlying type system); the `entitySpec` DSL command will do this for us. +The example above thus gives us 5 VMs identical to the one we created in the previous section. http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/configuring-vms.md ---------------------------------------------------------------------- diff --git a/guide/yaml/configuring-vms.md b/guide/yaml/configuring-vms.md new file mode 100644 index 0000000..97fb9d0 --- /dev/null +++ b/guide/yaml/configuring-vms.md @@ -0,0 +1,31 @@ +--- +title: Configuring VMs +layout: website-normal +toc: ../guide_toc.json +categories: [use, guide, defining-applications] +--- + +Another simple blueprint will just create a VM which you can use, without any software installed upon it: + +{% highlight yaml %} +{% readj example_yaml/simple-vm.yaml %} +{% endhighlight %} + + +*We've omitted the `location` section here and in many of the examples below; +add the appropriate choice when you paste your YAML. Note that the `provisioning.properties` will be +ignored if deploying to `localhost` or `byon` fixed-IP machines.* + +This will create a VM with the specified parameters in your choice of cloud. +In the GUI (and in the REST API), the entity is called "VM", +and the hostname and IP address(es) are reported as [sensors]({{ site.path.guide }}/concepts/entities.html). +There are many more `provisioning.properties` supported here, +including: + +* a `user` to create (if not specified it creates the same username as `brooklyn` is running under) +* a `password` for him or a `publicKeyFile` and `privateKeyFile` (defaulting to keys in `~/.ssh/id_rsa{.pub,}` and no password, + so if you have keys set up you can immediately ssh in!) +* `machineCreateAttempts` (for dodgy clouds, and they nearly all fail occasionally!) +* and things like `imageId` and `userMetadata` and disk and networking options (e.g. `autoAssignFloatingIp` for private clouds) + +For more information, see [Operations: Locations](../ops/locations/). http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/creating-yaml.md ---------------------------------------------------------------------- diff --git a/guide/yaml/creating-yaml.md b/guide/yaml/creating-yaml.md new file mode 100644 index 0000000..ffb1e36 --- /dev/null +++ b/guide/yaml/creating-yaml.md @@ -0,0 +1,78 @@ +--- +title: The Basic Structure +layout: website-normal +toc: ../guide_toc.json +categories: [use, guide, defining-applications] +--- + +## A First Blueprint + +The easiest way to write a blueprint is as a YAML file. +This follows the <a href="https://www.oasis-open.org/committees/camp/";>OASIS CAMP</a> plan specification, +with some extensions described below. +(A [YAML reference](yaml-reference.html) has more information, +and if the YAML doesn't yet do what you want, +it's easy to add new extensions using your favorite JVM language.) + +### The Basic Structure + +Here's a very simple YAML blueprint plan, to explain the structure: + +{% highlight yaml %} +{% readj example_yaml/simple-appserver.yaml %} +{% endhighlight %} + +* The `name` is just for the benefit of us humans. + +* The `location` specifies where this should be deployed. + If you've [set up passwordless localhost SSH access](../ops/locations/#localhost) + you can use `localhost` as above, but if not, just wait ten seconds for the next example. + +* The `services` block takes a list of the typed services we want to deploy. + This is the meat of the blueprint plan, as you'll see below. + +Finally, the clipboard in the top-right corner of the example plan box above (hover your cursor over the box) lets you easily copy-and-paste into the web-console: +simply [download and launch]({{ site.path.guide }}/start/running.html) Brooklyn, +then in the "Create Application" dialog at the web console +(usually [http://127.0.0.1:8081/](http://127.0.0.1:8081/), paste the copied YAML into the "Yaml" tab of the dialog and press "Finish". +There are several other ways to deploy, including `curl` and via the command-line, +and you can configure users, https, persistence, and more, +as described [in the ops guide](../ops/). + +[](web-console-yaml.png) + + + +<!-- +TODO building up children entities + +--> + + + +### More Information + +Topics to explore next on the topic of YAML blueprints are: + +{% include list-children.html %} + +Plenty of examples of blueprints exist in the Brooklyn codebase, +so another starting point is to [`git clone`]({{ site.path.guide }}/dev/code/index.html) it +and search for `*.yaml` files therein. + +Brooklyn lived as a Java framework for many years before we felt confident +to make a declarative front-end, so you can do pretty much anything you want to +by dropping to the JVM. For more information on Java: + +* start with a [Maven archetype]({{site.path.guide}}/java/archetype.html) +* see all [Brooklyn Java guide]({{site.path.guide}}/java/) topics +* look at test cases in the [codebase](https://github.com/apache/incubator-brooklyn) + +<!-- +TODO +* review some [examples]({{site.path.guide}}/use/examples/index.html) +--> + +You can also come talk to us, on IRC (#brooklyncentral on Freenode) or +any of the usual [hailing frequencies]({{site.path.website}}/community/), +as these documents are a work in progress. http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/custom-entities.md ---------------------------------------------------------------------- diff --git a/guide/yaml/custom-entities.md b/guide/yaml/custom-entities.md new file mode 100644 index 0000000..9698aa8 --- /dev/null +++ b/guide/yaml/custom-entities.md @@ -0,0 +1,238 @@ +--- +title: Custom Entities +layout: website-normal +toc: ../guide_toc.json +categories: [use, guide, defining-applications] +--- + +So far we've covered how to configure and compose entities. +There's a large library of blueprints available, but +there are also times when you'll want to write your own. + +For complex use cases, you can write JVM, but for many common situations, +some of the highly-configurable blueprints make it easy to write in YAML, +including `bash` and Chef. + + +### Vanilla Software using `bash` + +The following blueprint shows how a simple script can be embedded in the YAML +(the `|` character is special YAML which makes it easier to insert multi-line text): + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat.yaml %} +{% endhighlight %} + +This starts a simple `nc` listener on port 4321 which will respond `hello` to the first +session which connects to it. Test it by running `telnet localhost 4321` +or opening `http://localhost:4321` in a browser. + +Note that it only allows you connect once, and after that it fails. +This is deliberate! We'll repair this later in this example. +Until then however, in the *Applications* view you can click the server, +go to the `Effectors` tab, and click `restart` to bring if back to life. + +This is just a simple script, but it shows how any script can be easily embedded here, +including a script to download and run other artifacts. +Many artifacts are already packaged such that they can be downloaded and launched +with a simple script, and `VanillaSoftwareProcess` can also be used for them. + + +#### Downloading Files + +We can specify a `download.url` which downloads an artifact +(and automatically unpacking TAR, TGZ, and ZIP archives) +before running `launch.command` relative to where that file is installed (or unpacked), +with the default `launch.command` being `./start.sh`. + +So if we create a file `/tmp/netcat-server.tgz` containing just `start.sh` in the root +which consists of the two lines in the previous example, +we can instead write our example as: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-file.yaml %} +{% endhighlight %} + + +#### Port Inferencing + +If you're deploying to a cloud machine, a firewall might block the port 4321. +We can tell Brooklyn to open this port explicitly by specifying `inboundPorts: [ 4321 ]`; +however a more idiomatic way is to specify a config ending with `.port`, +such as: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-port.yaml %} +{% endhighlight %} + +The regex for ports to be opened can be configured using +the config `inboundPorts.configRegex` (which has `.*\.port` as the default value). + +Config keys of type `org.apache.brooklyn.api.location.PortRange` (aka `port`) +have special behaviour: when configuring, you can use range notation `8000-8100` or `8000+` to tell Brooklyn +to find **one** port matching; this is useful when ports might be in use. +In addition, any such config key will be opened, +irrespective of whether it matches the `inboundPorts.configRegex`. +To prevent any inferencing of ports to open, you can set the config `inboundPorts.autoInfer` to `false`. + +Note that in the example above, `netcat.port` must be specified in a `brooklyn.config` block. +This block can be used to hold any config (including for example the `launch.command`), +but for convenience Brooklyn allows config keys declared on the underlying type +to be specified up one level, alongside the type. +However config keys which are *not* declared on the type *must* be declared in the `brooklyn.config` block. + + +#### Declaring New Config Keys + +We can define config keys to be presented to the user +using the `brooklyn.parameters` block: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-port-parameter.yaml %} +{% endhighlight %} + +The example above will allow a user to specify a message to send back +and the port where netcat will listen. +The metadata on these parameters is available at runtime in the UI +and through the API, and is used when populating a catalog. + +The example also shows how these values can be passed as environment variables to the launch command. +The `$brooklyn:config(...)` function returns the config value supplied or default. +For the type `port`, an attribute sensor is also created to report the *actual* port used after port inference, +and so the `$brooklyn:attributeWhenReady(...)` function is used. +(If `$brooklyn:config("netcat.port")` had been used, `4321+` would be passed as `NETCAT_PORT`.) + +This gives us quite a bit more power in writing our blueprint: + +* Multiple instances of the server can be launched simultaneously on the same host, + as the `4321+` syntax enables Brooklyn to assign them different ports +* If this type is added to the catalog, a user can configure the message and the port; + we'll show this in the next section + + +#### Using the Catalog and Clustering + +The *Catalog* tab allows you to add blueprints which you can refer to in other blueprints. +In that tab, click *+* then *YAML*, and enter the following: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-catalog.bom %} +{% endhighlight %} + +This is the same example as in the previous section, wrapped according to the catalog YAML requirements, +with one new block added defining an enricher. An enricher creates a new sensor from other values; +in this case it will create a `main.uri` sensor by populating a `printf`-style string `"http://%s:%s"` +with the sensor values. + +With this added to the catalog, we can reference the type `netcat-example` when we deploy an application. +Return to the *Home* or *Applications* tab, click *+*, and submit this YAML blueprint: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-reference.yaml %} +{% endhighlight %} + +This extends the previous blueprint which we registered in the catalog, +meaning that we don't need to include it each time. +Here, we've elected to supply our own message, but we'll use the default port. +More importantly, we can package it for others to consume -- or take items others have built. + +We can go further and use this to deploy a cluster, +this time giving a custom port as well as a custom message: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-cluster.yaml %} +{% endhighlight %} + +In either of the above examples, if you explore the tree in the *Applications* view +and look at the *Summary* tab of any of the server instances, you'll now see the URL where netcat is running. +But remember, netcat will stop after one run, so you'll only be able to use each link once +before you have to restart it. You can also run `restart` on the cluster, +and if you haven't yet experimented with `resize` on the cluster you might want to do that. + + +#### Determining Successful Launch + +One requirement of the launch script is that it store the process ID (PID) in the file +pointed to by `$PID_FILE`, hence the second line of the script. +This is because Brooklyn wants to monitor the services under management. +You'll observe this if you connect to one of the netcat services, +as the process exits afterwards and Brooklyn sets the entity to an `ON_FIRE` state. +(You can also test this with a `killall nc` before connecting +or issueing a `stop` command on the server -- but not on the example, +as stopping an application root causes it to be removed altogether!) + +There are other options for determining launch: you can set `checkRunning.command` and `stop.command` instead, +as documented on the javadoc and config keys of the {% include java_link.html class_name="VanillaSoftwareProcess" package_path="org/apache/brooklyn/entity/software/base" project_subpath="software/base" %} class, +and those scripts will be used instead of checking and stopping the process whose PID is in `$PID_FILE`. + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-more-commands.yaml %} +{% endhighlight %} + +And indeed, once you've run one `telnet` to the server, you'll see that the +service has gone "on fire" in Brooklyn -- because the `nc` process stops after one run. + + +#### Attaching Policies + +Besides detecting this failure, Brooklyn policies can be added to the YAML to take appropriate +action. A simple recovery here might just to automatically restart the process: + +{% highlight yaml %} +{% readj example_yaml/vanilla-bash-netcat-restarter.yaml %} +{% endhighlight %} + +Autonomic management in Brooklyn often follows the principle that complex behaviours emerge +from composing simple policies. +The blueprint above uses one policy to triggering a failure sensor when the service is down, +and another responds to such failures by restarting the service. +This makes it easy to configure various aspects, such as to delay to see if the service itself recovers +(which here we've set to 15 seconds) or to bail out on multiple failures within a time window (which again we are not doing). +Running with this blueprint, you'll see that the service shows as on fire for 15s after a `telnet`, +before the policy restarts it. + + +### Sensors and Effectors + +For an even more interesting way to test it, look at the blueprint defining +[a netcat server and client](example_yaml/vanilla-bash-netcat-w-client.yaml). +This uses `initializers` to define an effector to `sayHiNetcat` on the `Simple Pinger` client, +using `env` variables to inject the `netcat-server` location and +`parameters` to pass in per-effector data: + + env: + TARGET_HOSTNAME: $brooklyn:component("netcat-server").attributeWhenReady("host.name") + brooklyn.initializers: + - type: org.apache.brooklyn.core.effector.ssh.SshCommandEffector + brooklyn.config: + name: sayHiNetcat + description: Echo a small hello string to the netcat entity + command: | + echo $message | nc $TARGET_HOSTNAME 4321 + parameters: + message: + description: The string to pass to netcat + defaultValue: hi netcat + +This blueprint also uses initializers to define sensors on the `netcat-server` entity +so that the `$message` we passed above gets logged and reported back: + + launch.command: | + echo hello | nc -l 4321 >> server-input & + echo $! > $PID_FILE + brooklyn.initializers: + - type: org.apache.brooklyn.core.sensor.ssh.SshCommandSensor + brooklyn.config: + name: output.last + period: 1s + command: tail -1 server-input + + +#### Summary + +These examples are relatively simple example, but they +illustrate many of the building blocks used in real-world blueprints, +and how they can often be easily described and combined in Brooklyn YAML blueprints. +Next, if you need to drive off-piste, or you want to write tests against these blueprints, +have a look at, for example, `VanillaBashNetcatYamlTest.java` in the Brooklyn codebase, +or follow the other references below. http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-clustered-w-db-concise.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-clustered-w-db-concise.yaml b/guide/yaml/example_yaml/appserver-clustered-w-db-concise.yaml new file mode 100644 index 0000000..0fd8759 --- /dev/null +++ b/guide/yaml/example_yaml/appserver-clustered-w-db-concise.yaml @@ -0,0 +1,15 @@ +name: appserver-clustered-w-db-concise +services: +- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster + initialSize: 2 + brooklyn.config: + wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + http.port: 8080+ + java.sysprops: + brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", + component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") +- type: org.apache.brooklyn.entity.database.mysql.MySqlNode + id: db + name: DB HelloWorld Visitors + brooklyn.config: + datastore.creation.script.url: https://github.com/apache/incubator-brooklyn/raw/master/usage/launcher/src/test/resources/visitors-creation-script.sql http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-clustered-w-db.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-clustered-w-db.yaml b/guide/yaml/example_yaml/appserver-clustered-w-db.yaml new file mode 100644 index 0000000..8bbb14f --- /dev/null +++ b/guide/yaml/example_yaml/appserver-clustered-w-db.yaml @@ -0,0 +1,18 @@ +name: appserver-clustered-w-db +services: +- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster + initialSize: 2 + memberSpec: + $brooklyn:entitySpec: + type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + brooklyn.config: + wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + http.port: 8080+ + java.sysprops: + brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", + component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") +- type: org.apache.brooklyn.entity.database.mysql.MySqlNode + id: db + name: DB HelloWorld Visitors + brooklyn.config: + datastore.creation.script.url: https://github.com/apache/incubator-brooklyn/raw/master/usage/launcher/src/test/resources/visitors-creation-script.sql http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-configured-in-config.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-configured-in-config.yaml b/guide/yaml/example_yaml/appserver-configured-in-config.yaml new file mode 100644 index 0000000..f817dd7 --- /dev/null +++ b/guide/yaml/example_yaml/appserver-configured-in-config.yaml @@ -0,0 +1,6 @@ +name: appserver-configured-in-config +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + brooklyn.config: + wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + http.port: 8080 http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-configured.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-configured.yaml b/guide/yaml/example_yaml/appserver-configured.yaml new file mode 100644 index 0000000..58416ca --- /dev/null +++ b/guide/yaml/example_yaml/appserver-configured.yaml @@ -0,0 +1,5 @@ +name: appserver-configured +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + war: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + httpPort: 8080 http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-w-db-other-flavor.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-w-db-other-flavor.yaml b/guide/yaml/example_yaml/appserver-w-db-other-flavor.yaml new file mode 100644 index 0000000..ac1106e --- /dev/null +++ b/guide/yaml/example_yaml/appserver-w-db-other-flavor.yaml @@ -0,0 +1,17 @@ +name: appserver-w-db-other-flavor +services: +- type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer + name: AppServer HelloWorld + brooklyn.config: + wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + http.port: 8080+ + java.sysprops: + brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", + component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") +- type: org.apache.brooklyn.entity.database.mariadb.MariaDbNode + id: db + name: DB HelloWorld Visitors + brooklyn.config: + datastore.creation.script.url: https://github.com/apache/incubator-brooklyn/raw/master/usage/launcher/src/test/resources/visitors-creation-script.sql + provisioning.properties: + minRam: 8192 http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-w-db.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-w-db.yaml b/guide/yaml/example_yaml/appserver-w-db.yaml new file mode 100644 index 0000000..0e9d959 --- /dev/null +++ b/guide/yaml/example_yaml/appserver-w-db.yaml @@ -0,0 +1,15 @@ +name: appserver-w-db +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + name: AppServer HelloWorld + brooklyn.config: + wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + http.port: 8080+ + java.sysprops: + brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", + component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") +- type: org.apache.brooklyn.entity.database.mysql.MySqlNode + id: db + name: DB HelloWorld Visitors + brooklyn.config: + datastore.creation.script.url: https://github.com/apache/incubator-brooklyn/raw/master/usage/launcher/src/test/resources/visitors-creation-script.sql http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/appserver-w-policy.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/appserver-w-policy.yaml b/guide/yaml/example_yaml/appserver-w-policy.yaml new file mode 100644 index 0000000..52bc3b8 --- /dev/null +++ b/guide/yaml/example_yaml/appserver-w-policy.yaml @@ -0,0 +1,26 @@ +name: appserver-w-policy +services: +- type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster + initialSize: 1 + memberSpec: + $brooklyn:entitySpec: + type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + brooklyn.config: + wars.root: http://search.maven.org/remotecontent?filepath=io/brooklyn/example/brooklyn-example-hello-world-sql-webapp/0.6.0/brooklyn-example-hello-world-sql-webapp-0.6.0.war + http.port: 8080+ + java.sysprops: + brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s", + component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") + brooklyn.policies: + - policyType: org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy + brooklyn.config: + metric: $brooklyn:sensor("brooklyn.entity.webapp.DynamicWebAppCluster", "webapp.reqs.perSec.windowed.perNode") + metricLowerBound: 10 + metricUpperBound: 100 + minPoolSize: 1 + maxPoolSize: 5 +- type: org.apache.brooklyn.entity.database.mysql.MySqlNode + id: db + name: DB HelloWorld Visitors + brooklyn.config: + datastore.creation.script.url: https://github.com/apache/incubator-brooklyn/raw/master/usage/launcher/src/test/resources/visitors-creation-script.sql http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/brooklyn-elasticsearch-catalog.bom ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/brooklyn-elasticsearch-catalog.bom b/guide/yaml/example_yaml/brooklyn-elasticsearch-catalog.bom new file mode 100644 index 0000000..bbed240 --- /dev/null +++ b/guide/yaml/example_yaml/brooklyn-elasticsearch-catalog.bom @@ -0,0 +1,124 @@ +brooklyn.catalog: + id: elasticsearch + version: 1.0 + iconUrl: https://avatars0.githubusercontent.com/u/6764390?v=3&s=400 + name: Elasticsearch + license: Apache-2.0 + issues_url: https://github.com/Graeme-Miller/brooklyn-elasticsearch/issues + item: + type: brooklyn.entity.group.DynamicCluster + initialSize: 2 + name: Elasticsearch Cluster + id: elasticsearch-cluster + description: A cluster of Elasticsearch nodes + + brooklyn.config: + install.version: 1.7.1 + elasticsearch.http.port: 9220 + elasticsearch.tcp.port: 9330 + + brooklyn.enrichers: + - type: org.apache.brooklyn.enricher.stock.Aggregator + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("url.tcp") + enricher.targetSensor: $brooklyn:sensor("urls.tcp.list") + enricher.aggregating.fromMembers: true + - type: org.apache.brooklyn.enricher.stock.Joiner + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("urls.tcp.list") + enricher.targetSensor: $brooklyn:sensor("urls.tcp.string") + uniqueTag: urls.quoted.string + - type: org.apache.brooklyn.enricher.stock.Transformer + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("urls.tcp.string") + enricher.targetSensor: $brooklyn:sensor("urls.tcp.withBrackets") + enricher.targetValue: $brooklyn:formatString("[%s]", $brooklyn:attributeWhenReady("urls.tcp.string")) + - type: org.apache.brooklyn.enricher.stock.Aggregator + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("url.http") + enricher.targetSensor: $brooklyn:sensor("urls.http.list") + enricher.aggregating.fromMembers: true + - type: org.apache.brooklyn.enricher.stock.Joiner + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("urls.http.list") + enricher.targetSensor: $brooklyn:sensor("urls.http.string") + uniqueTag: urls.http.quoted.string + - type: org.apache.brooklyn.enricher.stock.Transformer + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("urls.http.string") + enricher.targetSensor: $brooklyn:sensor("urls.http.withBrackets") + enricher.targetValue: $brooklyn:formatString("[%s]", $brooklyn:attributeWhenReady("urls.http.string")) + - type: org.apache.brooklyn.enricher.stock.Aggregator + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("host.address") + enricher.targetSensor: $brooklyn:sensor("host.address.first") + enricher.aggregating.fromMembers: true + enricher.transformation: + $brooklyn:object: + type: "org.apache.brooklyn.util.collections.CollectionFunctionals$FirstElementFunction" + + memberSpec: + $brooklyn:entitySpec: + - type: brooklyn.entity.basic.VanillaSoftwareProcess + name: Elasticsearch Node + + provisioning.properties: + osFamily: ubuntu + inboundPorts: + - $brooklyn:config("elasticsearch.http.port") + - $brooklyn:config("elasticsearch.tcp.port") + + install.command: | + $brooklyn:formatString(" + sudo apt-get update + sudo apt-get install -y openjdk-7-jdk wget + + wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - + echo \"deb http://packages.elastic.co/elasticsearch/1.7/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.7.list + echo \"deb http://packages.elastic.co/elasticsearch/1.6/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.6.list + echo \"deb http://packages.elastic.co/elasticsearch/1.5/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.5.list + echo \"deb http://packages.elastic.co/elasticsearch/1.4/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.4.list + echo \"deb http://packages.elastic.co/elasticsearch/1.3/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.3.list + echo \"deb http://packages.elastic.co/elasticsearch/1.2/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.2.list + echo \"deb http://packages.elastic.co/elasticsearch/1.1/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.1.list + echo \"deb http://packages.elastic.co/elasticsearch/1.0/debian stable main\" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-1.0.list + sudo apt-get update && sudo apt-get -y install elasticsearch=%s + ", + $brooklyn:config("install.version") + ) + + customize.command: | + $brooklyn:formatString(" + sudo rm -fr sudo tee /etc/elasticsearch/elasticsearch.yml; + echo discovery.zen.ping.multicast.enabled: false | sudo tee -a /etc/elasticsearch/elasticsearch.yml; + echo discovery.zen.ping.unicast.enabled: true | sudo tee -a /etc/elasticsearch/elasticsearch.yml; + echo 'discovery.zen.ping.unicast.hosts: %s' | sudo tee -a /etc/elasticsearch/elasticsearch.yml; + echo http.port: %s | sudo tee -a /etc/elasticsearch/elasticsearch.yml; + echo transport.tcp.port: %s | sudo tee -a /etc/elasticsearch/elasticsearch.yml; + ", + $brooklyn:component("parent", "").attributeWhenReady("urls.tcp.withBrackets"), + $brooklyn:config("elasticsearch.http.port"), + $brooklyn:config("elasticsearch.tcp.port") + ) + + + launch.command: sudo service elasticsearch start + + stop.command: sudo service elasticsearch stop + + checkRunning.command: | + $brooklyn:formatString("counter=`wget -T 15 -q -O- %s:%s | grep -c \"status. : 200\"`; if [ $counter -eq 0 ]; then exit 1; fi", + $brooklyn:attributeWhenReady("host.address"), + $brooklyn:config("elasticsearch.http.port")) + + brooklyn.enrichers: + - type: org.apache.brooklyn.enricher.stock.Transformer + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("host.address") + enricher.targetSensor: $brooklyn:sensor("url.tcp") + enricher.targetValue: $brooklyn:formatString("%s:%s", $brooklyn:attributeWhenReady("host.address"), $brooklyn:config("elasticsearch.tcp.port")) + - type: org.apache.brooklyn.enricher.stock.Transformer + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("host.address") + enricher.targetSensor: $brooklyn:sensor("url.http") + enricher.targetValue: $brooklyn:formatString("%s:%s", $brooklyn:attributeWhenReady("host.address"), $brooklyn:config("elasticsearch.http.port")) http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/brooklyn-elk-catalog.bom ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/brooklyn-elk-catalog.bom b/guide/yaml/example_yaml/brooklyn-elk-catalog.bom new file mode 100644 index 0000000..6fc11f4 --- /dev/null +++ b/guide/yaml/example_yaml/brooklyn-elk-catalog.bom @@ -0,0 +1,35 @@ +brooklyn.catalog: + version: 1.0 + iconUrl: https://avatars0.githubusercontent.com/u/6764390?v=3&s=400 + license: Apache-2.0 + issues_url: https://github.com/Graeme-Miller/brooklyn-elk/issues + itemType: template + item: + type: org.apache.brooklyn.entity.stock.BasicApplication + name: ELK Stack + id: ELK-Stack + description: | + Simple ELK stack deployment: provisions Elasticsearch, Kibana and Logtash as a child of Apache Tomcat 8 + services: + - type: elasticsearch + id: es + name: Elasticsearch Cluster + brooklyn.config: + install.version: 1.4.4 + - type: kibana-standalone + id: kibana + name: Kibana Server + customize.latch: $brooklyn:component("es").attributeWhenReady("service.isUp") + brooklyn.config: + kibana.elasticsearch.ip: $brooklyn:component("es").attributeWhenReady("host.address.first") + kibana.elasticsearch.port: $brooklyn:component("es").config("elasticsearch.http.port") + - type: org.apache.brooklyn.entity.webapp.tomcat.Tomcat8Server + id: tomcat + customize.latch: $brooklyn:component("es").attributeWhenReady("service.isUp") + brooklyn.config: + children.startable.mode: background_late + brooklyn.children: + - type: logstash-child + name: Logstash Child + brooklyn.config: + logstash.elasticsearch.host: $brooklyn:component("es").attributeWhenReady("urls.http.withBrackets") http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/brooklyn-kibana-catalog.bom ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/brooklyn-kibana-catalog.bom b/guide/yaml/example_yaml/brooklyn-kibana-catalog.bom new file mode 100644 index 0000000..ee006d2 --- /dev/null +++ b/guide/yaml/example_yaml/brooklyn-kibana-catalog.bom @@ -0,0 +1,52 @@ +brooklyn.catalog: + version: 0.0.2 + iconUrl: http://blog.arungupta.me/wp-content/uploads/2015/07/kibana-logo.png + items: + - id: kibana-standalone + name: "Kibana server" + description: "Single Kibana server" + description: | + Kibana server. Callers should configure 'kibana.elasticsearch.ip' + item: + type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + provisioning.properties: + osFamily: ubuntu + + brooklyn.config: + install.version: "4.1.1" + kibana.elasticsearch.ip: 127.0.0.1 # must be supplied by caller! + kibana.elasticsearch.port: 9200 # must be supplied by caller! + kibana.port: 5601 + + shell.env: + KIBANA_VERSION: $brooklyn:config("install.version") + ELASTICSEARCH_IP: $brooklyn:config("kibana.elasticsearch.ip") + ELASTICSEARCH_PORT: $brooklyn:config("kibana.elasticsearch.port") + KIBANA_PORT: $brooklyn:config("kibana.port") + + install.command: | + # Install required dependencies + sudo apt-get update + sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated tzdata openjdk-7-jdk wget + + # download kibana + sudo mkdir -p /opt/kibana/ + cd /opt/kibana + sudo chmod -R 777 /opt/kibana + wget -qO - https://download.elastic.co/kibana/kibana/kibana-${KIBANA_VERSION}-linux-x64.tar.gz | tar xz + + # customize config file for kibana + sed -i 's|http://localhost:9200|http://'"${ELASTICSEARCH_IP}"':'"${ELASTICSEARCH_PORT}"'|g' /opt/kibana/kibana-${KIBANA_VERSION}-linux-x64/config/kibana.yml + sed -i 's|5601|'"${KIBANA_PORT}"'|g' /opt/kibana/kibana-${KIBANA_VERSION}-linux-x64/config/kibana.yml + sed -i 's|# log_file: ./kibana.log|log_file: ./kibana.log|g' /opt/kibana/kibana-${KIBANA_VERSION}-linux-x64/config/kibana.yml + + launch.command: | + nohup /opt/kibana/kibana-${KIBANA_VERSION}-linux-x64/bin/kibana & + echo $! > $PID_FILE + + brooklyn.enrichers: + - type: org.apache.brooklyn.enricher.stock.Transformer + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("host.address") + enricher.targetSensor: $brooklyn:sensor("main.uri") + enricher.targetValue: $brooklyn:formatString("http://%s:%s";, $brooklyn:attributeWhenReady("host.address"), $brooklyn:config("kibana.port")) http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/brooklyn-logstash-catalog.bom ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/brooklyn-logstash-catalog.bom b/guide/yaml/example_yaml/brooklyn-logstash-catalog.bom new file mode 100644 index 0000000..7c431f5 --- /dev/null +++ b/guide/yaml/example_yaml/brooklyn-logstash-catalog.bom @@ -0,0 +1,59 @@ +brooklyn.catalog: + version: 1.5.5 + iconUrl: http://logstash.net/images/logstash.png + items: + - id: logstash-standalone + name: "Logstash server" + description: "Single Logstash server" + item: + type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + provisioning.properties: + osFamily: ubuntu + + brooklyn.config: + install.version: "1.5.4" + logstash.config.input: input { file { path => "/var/log/*" } } + logstash.config.filter: '' + logstash.config.output: output { stdout { } } + logstash.config.dir: /etc/logstash/conf.d/ + + shell.env: + VERSION: $brooklyn:config("install.version") + INPUT_CONFIG: $brooklyn:config("logstash.config.input") + FILTER_CONFIG: $brooklyn:config("logstash.config.filter") + OUTPUT_CONFIG: $brooklyn:config("logstash.config.output") + CONFIG_DIR: $brooklyn:config("logstash.config.dir") + + install.command: | + # Install required dependencies + sudo apt-get update + sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated tzdata openjdk-7-jdk wget + + # download logstash + sudo mkdir -p /opt/logstash/ + cd /opt/logstash + sudo chmod -R 777 /opt/logstash + wget -qO - https://download.elastic.co/logstash/logstash/logstash-${VERSION}.tar.gz | tar xz + + # add a config file for logstash + sudo mkdir -p ${CONFIG_DIR} + echo "${INPUT_CONFIG} ${FILTER_CONFIG} ${OUTPUT_CONFIG}" | sudo tee ${CONFIG_DIR}/01-basic.conf + + launch.command: | + sudo mkdir -p /var/log/logstash/ + sudo chmod -R 777 /var/log/logstash/ + nohup /opt/logstash/logstash-${VERSION}/bin/logstash agent -f ${CONFIG_DIR} -l /var/log/logstash/logstash.log & + echo $! > $PID_FILE + - id: logstash-child + name: "Logstash Child" + description: | + Logstash server to be embedded as a child of a SoftwareProcess who + publishes his 'log.location' as a sensor. + Callers should configure 'logstash.elasticsearch.host' (if using ES) + or 'logstash.config.output'. + item: + type: logstash-standalone + brooklyn.config: + logstash.config.input: $brooklyn:formatString("input { file { path => \"%s\" start_position => \"beginning\" } }", $brooklyn:parent().attributeWhenReady("log.location")) + logstash.elasticsearch.host: 127.0.0.1:9200 # must be supplied by caller! + logstash.config.output: $brooklyn:formatString("output { elasticsearch { host => %s protocol => \"http\" embedded => false } }", $brooklyn:config("logstash.elasticsearch.host")) http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/cluster-vm.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/cluster-vm.yaml b/guide/yaml/example_yaml/cluster-vm.yaml new file mode 100644 index 0000000..2b57f4e --- /dev/null +++ b/guide/yaml/example_yaml/cluster-vm.yaml @@ -0,0 +1,12 @@ +name: cluster-vm +services: +- type: org.apache.brooklyn.entity.group.DynamicCluster + initialSize: 5 + memberSpec: + $brooklyn:entitySpec: + type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess + name: VM + provisioning.properties: + minRam: 8g + minCores: 4 + minDisk: 100g http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/simple-appserver-with-location-byon.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/simple-appserver-with-location-byon.yaml b/guide/yaml/example_yaml/simple-appserver-with-location-byon.yaml new file mode 100644 index 0000000..8e9c951 --- /dev/null +++ b/guide/yaml/example_yaml/simple-appserver-with-location-byon.yaml @@ -0,0 +1,12 @@ +name: simple-appserver-with-location-byon +location: + byon: + user: brooklyn + privateKeyFile: ~/.ssh/brooklyn.pem + hosts: + - 192.168.0.18 + - 192.168.0.19 +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + location: + byon:(hosts="127.0.0.1") http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/simple-appserver-with-location.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/simple-appserver-with-location.yaml b/guide/yaml/example_yaml/simple-appserver-with-location.yaml new file mode 100644 index 0000000..79344c8 --- /dev/null +++ b/guide/yaml/example_yaml/simple-appserver-with-location.yaml @@ -0,0 +1,8 @@ +name: simple-appserver-with-location +location: + jclouds:aws-ec2: + region: us-east-1 + identity: AKA_YOUR_ACCESS_KEY_ID + credential: <access-key-hex-digits> +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/simple-appserver.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/simple-appserver.yaml b/guide/yaml/example_yaml/simple-appserver.yaml new file mode 100644 index 0000000..8e9d76a --- /dev/null +++ b/guide/yaml/example_yaml/simple-appserver.yaml @@ -0,0 +1,4 @@ +name: simple-appserver +location: localhost +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/simple-vm.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/simple-vm.yaml b/guide/yaml/example_yaml/simple-vm.yaml new file mode 100644 index 0000000..8f6447c --- /dev/null +++ b/guide/yaml/example_yaml/simple-vm.yaml @@ -0,0 +1,8 @@ +name: simple-vm +services: +- type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess + name: VM + provisioning.properties: + minRam: 8192mb + minCores: 4 + minDisk: 100gb http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/test-app-with-enrichers-slightly-simpler.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/test-app-with-enrichers-slightly-simpler.yaml b/guide/yaml/example_yaml/test-app-with-enrichers-slightly-simpler.yaml new file mode 100644 index 0000000..55d5b87 --- /dev/null +++ b/guide/yaml/example_yaml/test-app-with-enrichers-slightly-simpler.yaml @@ -0,0 +1,57 @@ +# +# example showing how enrichers can be set +# +name: test-app-with-enrichers +description: Testing many enrichers +services: +- type: org.apache.brooklyn.entity.group.DynamicCluster + id: cluster + initialSize: 3 + location: localhost + memberSpec: + $brooklyn:entitySpec: + type: org.apache.brooklyn.core.test.entity.TestEntity + brooklyn.enrichers: + - type: org.apache.brooklyn.enricher.stock.Transformer + # transform "ip" (which we expect a feed, not shown here, to set) to a URL; + # you can curl an address string to the sensors/ip endpoint an entity to trigger these enrichers + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("ip") + enricher.targetSensor: $brooklyn:sensor("url") + enricher.targetValue: $brooklyn:formatString("http://%s/";, $brooklyn:attributeWhenReady("ip")) + - type: org.apache.brooklyn.enricher.stock.Propagator + # use propagator to duplicate one sensor as another, giving the supplied sensor mapping; + # the other use of Propagator is where you specify a producer (using $brooklyn:entity(...) as below) + # from which to take sensors; in that mode you can specify `propagate` as a list of sensors whose names are unchanged, + # instead of (or in addition to) this map + brooklyn.config: + sensorMapping: + $brooklyn:sensor("url"): $brooklyn:sensor("org.apache.brooklyn.core.entity.Attributes", "main.uri") + brooklyn.enrichers: + - type: org.apache.brooklyn.enricher.stock.Aggregator + # aggregate `url` sensors from children into a list + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("url") + enricher.targetSensor: $brooklyn:sensor("urls.list") + enricher.aggregating.fromMembers: true + - type: org.apache.brooklyn.enricher.stock.Joiner + # create a string from that list, for use e.g. in bash scripts + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("urls.list") + enricher.targetSensor: $brooklyn:sensor("urls.list.comma_separated.max_2") + maximum: 2 + # TODO infer uniqueTag, name etc + uniqueTag: urls.list.comma_separated.max_2 + - type: org.apache.brooklyn.enricher.stock.Joiner + # pick one uri as the main one to use + brooklyn.config: + enricher.sourceSensor: $brooklyn:sensor("urls.list") + enricher.targetSensor: $brooklyn:sensor("org.apache.brooklyn.core.entity.Attributes", "main.uri") + quote: false + maximum: 1 +brooklyn.enrichers: +- type: org.apache.brooklyn.enricher.stock.Propagator + # if nothing specified for `propagating` or `sensorMapping` then + # Propagator will do all but the usual lifecycle defaults, handy at the root! + brooklyn.config: + producer: $brooklyn:entity("cluster") http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-catalog.bom ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-catalog.bom b/guide/yaml/example_yaml/vanilla-bash-netcat-catalog.bom new file mode 100644 index 0000000..7551818 --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-catalog.bom @@ -0,0 +1,35 @@ +brooklyn.catalog: + id: netcat-example + version: "1.0" + item: + type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Netcat Server + + launch.command: | + echo $MESSAGE | nc -l $NETCAT_PORT & + echo $! > $PID_FILE + + env: + MESSAGE: $brooklyn:config("message") + NETCAT_PORT: $brooklyn:attributeWhenReady("netcat.port") + + brooklyn.parameters: + - name: message + description: a message to send to the caller + default: hello + - name: netcat.port + type: port + description: the port netcat should run on + default: 4321+ + + brooklyn.enrichers: + - type: org.apache.brooklyn.enricher.stock.Transformer + brooklyn.config: + uniqueTag: main-uri-generator + enricher.sourceSensor: $brooklyn:sensor("host.address") + enricher.targetSensor: $brooklyn:sensor("main.uri") + enricher.targetValue: + $brooklyn:formatString: + - "http://%s:%s/"; + - $brooklyn:attributeWhenReady("host.address") + - $brooklyn:attributeWhenReady("netcat.port") http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-cluster.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-cluster.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-cluster.yaml new file mode 100644 index 0000000..70b69af --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-cluster.yaml @@ -0,0 +1,11 @@ +name: Netcat Cluster Example +location: localhost +services: +- type: org.apache.brooklyn.entity.group.DynamicCluster + memberSpec: + $brooklyn:entitySpec: + type: netcat-example + message: hello from cluster member + netcat.port: 8000+ + initialSize: 3 + restartMode: parallel http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-file.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-file.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-file.yaml new file mode 100644 index 0000000..d768739 --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-file.yaml @@ -0,0 +1,6 @@ +name: Simple Netcat Example From File +location: localhost +services: +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Netcat Server + download.url: file:///tmp/netcat-server.tgz http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-more-commands.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-more-commands.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-more-commands.yaml new file mode 100644 index 0000000..f4e894f --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-more-commands.yaml @@ -0,0 +1,16 @@ +name: Netcat Example with Explicit Check and Stop Commands +location: localhost +services: +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Netcat Server + launch.command: | + echo hello | nc -l 4321 & + echo $! > $PID_FILE + + # The following overrides demonstrate the use of a custom shell environment as well as + # check-running and stop commands. These are optional; default behavior will "do the + # right thing" with the pid file automatically. + + env: { CHECK_MARKER: "checkRunning", STOP_MARKER: "stop" } + checkRunning.command: echo $CHECK_MARKER >> DATE && test -f "$PID_FILE" && ps -p `cat $PID_FILE` >/dev/null + stop.command: echo $STOP_MARKER >> DATE && test -f "$PID_FILE" && { kill -9 `cat $PID_FILE`; rm /tmp/vanilla.pid; } http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-port-parameter.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-port-parameter.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-port-parameter.yaml new file mode 100644 index 0000000..90f83b4 --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-port-parameter.yaml @@ -0,0 +1,21 @@ +name: Netcat Example with Parameter +location: localhost +services: +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Netcat Server + launch.command: | + echo $MESSAGE | nc -l $NETCAT_PORT & + echo $! > $PID_FILE + + env: + MESSAGE: $brooklyn:config("message") + NETCAT_PORT: $brooklyn:attributeWhenReady("netcat.port") + + brooklyn.parameters: + - name: message + description: a message to send to the caller + default: hello + - name: netcat.port + type: port + description: the port netcat should run on + default: 4321+ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-port.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-port.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-port.yaml new file mode 100644 index 0000000..3ec0212 --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-port.yaml @@ -0,0 +1,13 @@ +name: Netcat Example with Port Opened +location: localhost +services: +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Netcat Server + launch.command: | + echo hello | nc -l 4321 & + echo $! > $PID_FILE + + brooklyn.config: + # matching the regex `.*\.port` will cause the port to be opened + # if in a cloud where configurable security groups are available + netcat.port: 4321 http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-reference.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-reference.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-reference.yaml new file mode 100644 index 0000000..0f10c55 --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-reference.yaml @@ -0,0 +1,5 @@ +name: Netcat Type Reference Example +location: localhost +services: +- type: netcat-example + message: hello from netcat using a registered type http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-restarter.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-restarter.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-restarter.yaml new file mode 100644 index 0000000..47e54ab --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-restarter.yaml @@ -0,0 +1,20 @@ +name: Netcat Example with Restarter Policy +location: localhost +services: +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + id: netcat-server + name: Simple Netcat Server + launch.command: | + echo hello | nc -l 4321 & + echo $! > $PID_FILE + brooklyn.enrichers: + - type: org.apache.brooklyn.policy.ha.ServiceFailureDetector + brooklyn.config: + # wait 15s after service fails before propagating failure + serviceFailedStabilizationDelay: 15s + brooklyn.policies: + - type: org.apache.brooklyn.policy.ha.ServiceRestarter + brooklyn.config: + # repeated failures in a time window can cause the restarter to abort, + # propagating the failure; a time window of 0 will mean it always restarts! + failOnRecurringFailuresInThisDuration: 0 http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat-w-client.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat-w-client.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat-w-client.yaml new file mode 100644 index 0000000..78ed99e --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat-w-client.yaml @@ -0,0 +1,78 @@ +name: Netcat Example with Client + +location: localhost + +services: + +# the netcat server instance, running in listener mode (-l) +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + id: netcat-server + name: Simple Netcat Server + launch.command: | + echo hello | nc -l 4321 >> server-input & + echo $! > $PID_FILE + + # a failure detector and a service restarter work together + brooklyn.enrichers: + - type: org.apache.brooklyn.policy.ha.ServiceFailureDetector + brooklyn.config: + # wait 15s after service fails before propagating failure + serviceFailedStabilizationDelay: 15s + + brooklyn.policies: + - type: org.apache.brooklyn.policy.ha.ServiceRestarter + brooklyn.config: + # repeated failures in a time window can cause the restarter to abort, + # propagating the failure; a time window of 0 will mean it always restarts! + failOnRecurringFailuresInThisDuration: 0 + + brooklyn.initializers: + # two sensors, recording the data sent to this netcat server: + + - type: org.apache.brooklyn.core.sensor.ssh.SshCommandSensor + brooklyn.config: + name: output.last + period: 1s + command: tail -1 server-input + + - type: org.apache.brooklyn.core.sensor.ssh.SshCommandSensor + brooklyn.config: + name: output.all + period: 1s + command: cat server-input + +# a client to hit netcat +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Pinger + + # set the hostname of the netcat instance as an env var for the scripts + env: + TARGET_HOSTNAME: + $brooklyn:component("netcat-server"). + attributeWhenReady("host.name") + + # start/check/stop are no-op + launch.command: "" + checkRunning.command: "" + stop.command: "" + + brooklyn.initializers: + # but there is a sample effector which runs nc in client mode + + - type: org.apache.brooklyn.core.effector.ssh.SshCommandEffector + brooklyn.config: + name: sayHiNetcat + description: Echo a small hello string to the netcat entity + command: | + echo $message | nc $TARGET_HOSTNAME 4321 + parameters: + message: + description: The string to pass to netcat + defaultValue: hi netcat + +# and add an enricher at the root so all sensors from netcat-server are visible on the root +brooklyn.enrichers: +- enricherType: org.apache.brooklyn.enricher.stock.Propagator + brooklyn.config: + enricher.producer: $brooklyn:component("netcat-server") + enricher.propagating.propagatingAll: true http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/example_yaml/vanilla-bash-netcat.yaml ---------------------------------------------------------------------- diff --git a/guide/yaml/example_yaml/vanilla-bash-netcat.yaml b/guide/yaml/example_yaml/vanilla-bash-netcat.yaml new file mode 100644 index 0000000..df616af --- /dev/null +++ b/guide/yaml/example_yaml/vanilla-bash-netcat.yaml @@ -0,0 +1,8 @@ +name: Simple Netcat Server Example +location: localhost +services: +- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess + name: Simple Netcat Server + launch.command: | + echo hello | nc -l 4321 & + echo $! > $PID_FILE http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/index.md ---------------------------------------------------------------------- diff --git a/guide/yaml/index.md b/guide/yaml/index.md new file mode 100644 index 0000000..792d9a0 --- /dev/null +++ b/guide/yaml/index.md @@ -0,0 +1,22 @@ +--- +title: YAML Blueprints +layout: website-normal +children: +- creating-yaml.md +- setting-locations.md +- configuring-vms.md +- clusters.md +- multiple-services.md +- clusters-and-policies.md +- blueprinting-tips.md +- custom-entities.md +- winrm/ +- chef/ +- test/ +- advanced-example.md +- { path: yaml-reference.md, title: YAML Blueprint Reference } +- { link: 'https://github.com/brooklyncentral/blueprint-library', title: 'GitHub Blueprint Library' } +--- + + +{% include list-children.html %} http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/yaml/multiple-services.md ---------------------------------------------------------------------- diff --git a/guide/yaml/multiple-services.md b/guide/yaml/multiple-services.md new file mode 100644 index 0000000..81ddf07 --- /dev/null +++ b/guide/yaml/multiple-services.md @@ -0,0 +1,97 @@ +--- +title: Multiple Services and Dependency Injection +layout: website-normal +--- + +We've seen the configuration of machines and how to build up clusters. +Now let's return to our app-server example and explore how more interesting +services can be configured, composed, and combined. + + +### Service Configuration + +We'll begin by using more key-value pairs to configure the JBoss server to run a real app: + +{% highlight yaml %} +{% readj example_yaml/appserver-configured.yaml %} +{% endhighlight %} + +(As before, you'll need to add the `location` info; `localhost` will work for these and subsequent examples.) + +When this is deployed, you can see management information in the Brooklyn Web Console, +including a link to the deployed application (downloaded to the target machine from the `hello-world` URL), +running on port 8080. + +**Tip**: If port 8080 might be in use, you can specify `8080+` to take the first available port >= 8080; +the actual port will be reported as a sensor by Brooklyn. + +It's also worth indicating an alternate, more formal syntax. +Not all configuration on entities is supported at the top level of the service specification +(only those which are defined as "flags" in the underlying blueprint, +e.g. the `@SetFromFlag("war")` in the `WebAppServiceConstants` parent of `JBoss7Server`). +All configuration has a formal qualified name, and this can be supplied even where flags or config keys are not +explicitly defined, by placing it into a `brooklyn.config` section: + +{% highlight yaml %} +{% readj example_yaml/appserver-configured-in-config.yaml %} +{% endhighlight %} + + +### Multiple Services + +If you explored the `hello-world-sql` application we just deployed, +you'll have noticed it tries to access a database. +And it fails, because we have not set one up. Let's do that now: + +{% highlight yaml %} +{% readj example_yaml/appserver-w-db.yaml %} +{% endhighlight %} + +Here there are a few things going on: + +* We've added a second service, which will be the database; + you'll note the database has been configured to run a custom setup script +* We've injected the URL of the second service into the appserver as a Java system property + (so our app knows where to find the database) + +**Caution: Be careful if you write your YAML in an editor which attempts to put "smart-quotes" in. +All quote characters must be plain ASCII, not fancy left-double-quotes and right-double-quotes!** + +There are as many ways to do dependency injection as there are developers, +it sometimes seems; our aim in Brooklyn is not to say this has to be done one way, +but to support the various mechanisms people might need, for whatever reasons. +(We each have our opinions about what works well, of course; +the one thing we do want to call out is that being able to dynamically update +the injection is useful in a modern agile application -- so we are definitively **not** +recommending this Java system property approach ... but it is an easy one to demo!) + +The way the dependency injection works is again by using the `$brooklyn:` DSL, +this time referring to the `component("db")` (looked up by the `id` field on our DB component), +and then to a sensor emitted by that component. +All the database entities emit a `database.url` sensor when they are up and running; +the `attributeWhenReady` DSL method will store a pointer to that sensor (a Java Future under the covers) +in the Java system properties map which the JBoss entity reads at launch time, blocking if needed. + +This means that the deployment occurs in parallel, and if the database comes up first, +there is no blocking; but if the JBoss entity completes its installation and +downloading the WAR, it will wait for the database before it launches. +At that point the URL is injected, first passing it through `formatString` +to include the credentials for the database (which are defined in the database creation script). + + +### An Aside: Substitutability + +Don't like JBoss? Is there something about Maria? +One of the modular principles we follow in Brooklyn is substitutability: +in many cases, the config keys, sensors, and effectors are defined +in superclasses and are portable across multiple implementations. + +Here's an example deploying the same application but with different flavors of the components: + +{% highlight yaml %} +{% readj example_yaml/appserver-w-db-other-flavor.yaml %} +{% endhighlight %} + +We've also brought in the `provisioning.properties` from the VM example earlier +so our database has 8GB RAM. +Any of those properties, including `imageId` and `user`, can be defined on a per-entity basis.
