http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-db-activities-step3.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-db-activities-step3.png b/guide/ops/gui/images/my-db-activities-step3.png new file mode 100644 index 0000000..0e1b508 Binary files /dev/null and b/guide/ops/gui/images/my-db-activities-step3.png differ
http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-cluster-starting.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-cluster-starting.png b/guide/ops/gui/images/my-web-cluster-starting.png new file mode 100644 index 0000000..c389b0b Binary files /dev/null and b/guide/ops/gui/images/my-web-cluster-starting.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-cluster-stop-confirm-large.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-cluster-stop-confirm-large.png b/guide/ops/gui/images/my-web-cluster-stop-confirm-large.png new file mode 100644 index 0000000..c9bdab6 Binary files /dev/null and b/guide/ops/gui/images/my-web-cluster-stop-confirm-large.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-cluster-stop-confirm.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-cluster-stop-confirm.png b/guide/ops/gui/images/my-web-cluster-stop-confirm.png new file mode 100644 index 0000000..179b00a Binary files /dev/null and b/guide/ops/gui/images/my-web-cluster-stop-confirm.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-large.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-large.png b/guide/ops/gui/images/my-web-large.png new file mode 100644 index 0000000..8954441 Binary files /dev/null and b/guide/ops/gui/images/my-web-large.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-summary-large.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-summary-large.png b/guide/ops/gui/images/my-web-summary-large.png new file mode 100644 index 0000000..fc4bffe Binary files /dev/null and b/guide/ops/gui/images/my-web-summary-large.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-summary.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-summary.png b/guide/ops/gui/images/my-web-summary.png new file mode 100644 index 0000000..e85752f Binary files /dev/null and b/guide/ops/gui/images/my-web-summary.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-validating-app-endpoint-large.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-validating-app-endpoint-large.png b/guide/ops/gui/images/my-web-validating-app-endpoint-large.png new file mode 100644 index 0000000..69005c4 Binary files /dev/null and b/guide/ops/gui/images/my-web-validating-app-endpoint-large.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web-validating-app-endpoint.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web-validating-app-endpoint.png b/guide/ops/gui/images/my-web-validating-app-endpoint.png new file mode 100644 index 0000000..d775717 Binary files /dev/null and b/guide/ops/gui/images/my-web-validating-app-endpoint.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/images/my-web.png ---------------------------------------------------------------------- diff --git a/guide/ops/gui/images/my-web.png b/guide/ops/gui/images/my-web.png new file mode 100644 index 0000000..a370606 Binary files /dev/null and b/guide/ops/gui/images/my-web.png differ http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/index.md ---------------------------------------------------------------------- diff --git a/guide/ops/gui/index.md b/guide/ops/gui/index.md new file mode 100644 index 0000000..48bd620 --- /dev/null +++ b/guide/ops/gui/index.md @@ -0,0 +1,11 @@ +--- +layout: website-normal +title: GUI Guide +children: +- running.md +- blueprints.md +- managing.md +- policies.md +--- + +{% include list-children.html %} http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/managing.md ---------------------------------------------------------------------- diff --git a/guide/ops/gui/managing.md b/guide/ops/gui/managing.md new file mode 100644 index 0000000..a972622 --- /dev/null +++ b/guide/ops/gui/managing.md @@ -0,0 +1,70 @@ +--- +title: Monitoring and Managing Applications +title_in_menu: Monitoring and Managing Applications +layout: website-normal +menu_parent: index.md +--- + +From the Home page, click on the application name or open the Applications tab. + +We can explore the management hierarchy of the application, which will show us the entities it is composed of. Starting from the application use the arrows to expand out the list of entities, or hover over the arrow until a menu popup is displayed so that you can select `Expand All`. + + * My Web Cluster (A `BasicApplication`) + * My DB (A `MySqlNode`) + * My Web (A `ControlledDynamicWebAppCluster`) + * Cluster of TomcatServer (A `DynamicWebAppCluster`) + * quarantine (A `QuarantineGroup`) + * TomcatServer (A `TomcatServer`) + * NginxController (An `NginxController`) + +Clicking on the "My Web Cluster" entity will show the "Summary" tab, +giving a very high level of what that component is doing. +Click on each of the child components in turn for more detail on that component. +Note that the cluster of web servers includes a "quarantine group", to which members of the +cluster that fail will be added. These are excluded from the load-balancer's targets. + +[](images/my-web-large.png) + + +## Activities + +The Activity tab allows us to drill down into the tasks each entity is currently executing or has recently completed. It is possible to drill down through all child tasks, and view the commands issued, along with any errors or warnings that occurred. + +For example clicking on the NginxController in the left hand tree and opening its Activity tab you can observe the 'start' task is 'In progress'. + +**Note**: You may observe different tasks depending on how far your deployment has progressed). + +[](images/my-db-activities-step1-large.png) + +Clicking on the 'start' task you can discover more details on the actions being carried out by that task (a task may consist of additional subtasks). + +[](images/my-db-activities-step2-large.png) + +Continuing to drill down into the 'In progress' tasks you will eventually reach the currently active task where you can investigate the ssh command executed on the target node including the current stdin, stdout and stderr output. + +[](images/my-db-activities-step3-large.png) + + +## Sensors + +Now click on the "Sensors" tab: +these data feeds drive the real-time picture of the application. +As you navigate in the tree at the left, you can see more targeted statistics coming in in real-time. + +Explore the sensors and the tree to find the URL where the _NginxController_ for the webapp we just deployed is running. This can be found in '**My Web Cluster** -> **My Web** -> **NginxController** -> **_main.uri_**'. + +Quickly return to the **âBrooklyn JS REST clientâ** web browser +tab showing the "Sensors" and observe the '**My Web Cluster** -> **My Web** -> **Cluster of TomcatServer** -> **_webapp.reqs.perSec.last_**' sensor value increase. + + + +## Stopping the Application + +To stop an application, select the application in the tree view (the top/root entity), click on the Effectors tab, and invoke the "Stop" effector. This will cleanly shutdown all components in the application and return any cloud machines that were being used. + +[](images/my-web-cluster-stop-confirm-large.png) + + +## Next + +Brooklyn's real power is in using **[Policies](policies.html)** to automatically *manage* applications. http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/policies.md ---------------------------------------------------------------------- diff --git a/guide/ops/gui/policies.md b/guide/ops/gui/policies.md new file mode 100644 index 0000000..e35c6f2 --- /dev/null +++ b/guide/ops/gui/policies.md @@ -0,0 +1,49 @@ +--- +title: Using Policies +title_in_menu: Using Policies +layout: website-normal +--- + +## Exploring and Testing Policies + +To see an example of policy based management, please deploy the following blueprint (changing +the location details as for the example shown earlier): + +{% highlight yaml %} +{% readj _my-web-cluster2.yaml %} +{% endhighlight %} + +The app server cluster has an `AutoScalerPolicy`, and the loadbalancer has a `targets` policy. + +Use the Applications tab in the web console to drill down into the Policies section of the ControlledDynamicWebAppCluster. You will see that the `AutoScalerPolicy` is running. + + +This policy automatically scales the cluster up or down to be the right size for the cluster's current load. One server is the minimum size allowed by the policy. + +The loadbalancer's `targets` policy ensures that the loadbalancer is updated as the cluster size changes. + +Sitting idle, this cluster will only contain one server, but you can use a tool like [jmeter](http://jmeter.apache.org/) pointed at the nginx endpoint to create load on the cluster. Download a jmeter test plan [here](https://github.com/apache/incubator-brooklyn/blob/master/examples/simple-web-cluster/resources/jmeter-test-plan.jmx). + +As load is added, Apache Brooklyn requests a new cloud machine, creates a new app server, and adds it to the cluster. As load is removed, servers are removed from the cluster, and the infrastructure is handed back to the cloud. + + +## Under the Covers + +The `AutoScalerPolicy` here is configured to respond to the sensor +reporting requests per second per node, invoking the default `resize` effector. +By clicking on the policy, you can configure it to respond to a much lower threshhold +or set long stabilization delays (the period before it scales out or back). + +An even simpler test is to manually suspend the policy, by clicking "Suspend" in the policies list. +You can then switch to the "Effectors" tab and manually trigger a `resize`. +On resize, new nodes are created and configured, +and in this case a policy on the nginx node reconfigures nginx whenever the set of active +targets changes. + + +## Next + +This guide has given a quick overview of using the Apache Brooklyn GUI to deploy, monitor and manage applications. The GUI also allows you to perform various Advanced management tasks and to explore and use the REST API (from the Script tab). Please take some time now to become more familiar with the GUI. + +Then continue to read through the [Operations Guide](../). + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/gui/running.md ---------------------------------------------------------------------- diff --git a/guide/ops/gui/running.md b/guide/ops/gui/running.md new file mode 100644 index 0000000..ef8edca --- /dev/null +++ b/guide/ops/gui/running.md @@ -0,0 +1,50 @@ +--- +title: Launching +title_in_menu: Launching +layout: website-normal +menu_parent: index.md +--- + +This guide will walk you through connecting to the Brooklyn Server Graphical User Interface and performing various tasks. + +For an explanation of common Brooklyn Concepts see the [Brooklyn Concepts Quickstart](../../start/concept-quickstart.html) or see the full guide in the [Brooklyn Concepts](../../concepts) chapter of the [User Guide](../../). + +This guide assumes that you are using Linux or Mac OS X and that Brooklyn Server will be running on your local system. + +## Launch Apache Brooklyn + +If you haven't already done so, you will need to start Brooklyn Server using the commands shown below. +It is not necessary at this time, but depending on what you are going to do, +you may wish to set up some other configuration options first, + +* [Security](../brooklyn_properties.html) +* [Persistence](../persistence/) + +Now start Brooklyn with the following command: + +{% highlight bash %} +$ cd apache-brooklyn-{{ site.brooklyn.version }} +$ bin/brooklyn launch +{% endhighlight %} + +Please refer to the [Server CLI Reference](../server-cli-reference.html) for details of other possible command line options. + +Brooklyn will output the address of the management interface: + +<pre> +INFO No security provider options specified. ... +INFO Starting Brooklyn web-console with passwordless access on localhost ... +INFO Starting brooklyn web-console on loopback interface because no security config is set +INFO Started Brooklyn console at http://127.0.0.1:8081/, running classpath://brooklyn.war +</pre> + +## Connect with Browser + +Next, open the web console on [http://127.0.0.1:8081](http://127.0.0.1:8081). +No applications have been deployed yet, so the "Create Application" dialog opens automatically. + +[](images/add-application-catalog-web-cluster-with-db-large.png) + + +## Next +The next section will show how to **[deploy a blueprint](blueprints.html)**. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/high-availability.md ---------------------------------------------------------------------- diff --git a/guide/ops/high-availability.md b/guide/ops/high-availability.md new file mode 100644 index 0000000..5f05cca --- /dev/null +++ b/guide/ops/high-availability.md @@ -0,0 +1,51 @@ +--- +title: High Availability +layout: website-normal +--- + +Brooklyn will automatically run in HA mode if multiple Brooklyn instances are started +pointing at the same persistence store. One Brooklyn node (e.g. the first one started) +is elected as HA master: all *write operations* against Brooklyn entities, such as creating +an application or invoking an effector, should be directed to the master. + +Once one node is running as `MASTER`, other nodes start in either `STANDBY` or `HOT_STANDBY` mode: + +* In `STANDBY` mode, a Brooklyn instance will monitor the master and will be a candidate + to become `MASTER` should the master fail. Standby nodes do *not* attempt to rebind + until they are elected master, so the state of existing entities is not available at + the standby node. However a standby server consumes very little resource until it is + promoted. + +* In `HOT_STANDBY` mode, a Brooklyn instance will read and make available the live state of + entities. Thus a hot-standby node is available as a read-only copy. + As with the standby node, if a hot-standby node detects that the master fails, + it will be a candidate for promotion to master. + +* In `HOT_BACKUP` mode, a Brooklyn instance will read and make available the live state of + entities, as a read-only copy. However this node is not able to become master, + so it can safely be used to test compatibility across different versions. + +To explicitly specify what HA mode a node should be in, the following CLI options are available +for the parameter `--highAvailability`: + +* `disabled`: management node works in isolation; it will not cooperate with any other standby/master nodes in management plane +* `auto`: will look for other management nodes, and will allocate itself as standby or master based on other nodes' states +* `master`: will startup as master; if there is already a master then fails immediately +* `standby`: will start up as lukewarm standby; if there is not already a master then fails immediately +* `hot_standby`: will start up as hot standby; if there is not already a master then fails immediately +* `hot_backup`: will start up as hot backup; this can be done even if there is not already a master; this node will not be a master + +The REST API offers live detection and control of the HA mode, +including setting priority to control which nodes will be promoted on master failure: + +* `/server/ha/state`: Returns the HA state of a management node (GET), + or changes the state (POST) +* `/server/ha/states`: Returns the HA states and detail for all nodes in a management plane +* `/server/ha/priority`: Returns the HA node priority for MASTER failover (GET), + or sets that priority (POST) + +Note that when POSTing to a non-master server it is necessary to pass a `Brooklyn-Allow-Non-Master-Access: true` header. +For example, the following cURL command could be used to change the state of a `STANDBY` node on `localhost:8082` to `HOT_STANDBY`: + + curl -v -X POST -d mode=HOT_STANDBY -H "Brooklyn-Allow-Non-Master-Access: true" http://localhost:8082/v1/server/ha/state + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/index.md ---------------------------------------------------------------------- diff --git a/guide/ops/index.md b/guide/ops/index.md new file mode 100644 index 0000000..78644d9 --- /dev/null +++ b/guide/ops/index.md @@ -0,0 +1,22 @@ +--- +title: Operations +layout: website-normal +children: +- server-cli-reference.md +- cli/ +- gui/ +- brooklyn_properties.md +- locations/ +- persistence/ +- high-availability.md +- catalog/ +- rest.md +- logging.md +- externalized-configuration.md +- requirements.md +- production-installation.md +- security-guidelines.md +- troubleshooting/ +--- + +{% include list-children.html %} http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/locations/cloud-credentials.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/cloud-credentials.md b/guide/ops/locations/cloud-credentials.md new file mode 100644 index 0000000..40cc108 --- /dev/null +++ b/guide/ops/locations/cloud-credentials.md @@ -0,0 +1,85 @@ +--- +title: Cloud Setup +layout: website-normal +--- + +To connect to a Cloud, Brooklyn requires appropriate credentials. These comprise the "identity" and +"credential" in Brooklyn terminology. + +For private clouds (and for some clouds being targeted using a standard API), the "endpoint" +must also be specified, which is the cloud's URL. + +The [jclouds guides](https://jclouds.apache.org/guides) includes documentation on configuring +different clouds. + + +## AWS + +### Credentials + +AWS has an "access key" and a "secret key", which correspond to Brooklyn's identity and credential +respectively. + +These keys are the way for any programmatic mechanism to access the AWS API. + +To generate an access key and a secret key, see [jclouds instructions](http://jclouds.apache.org/guides/aws) +and [AWS IAM instructions](http://docs.aws.amazon.com/IAM/latest/UserGuide/ManagingCredentials.html). + +An example of the expected format is shown below: + + brooklyn.location.jclouds.aws-ec2.identity=ABCDEFGHIJKLMNOPQRST + brooklyn.location.jclouds.aws-ec2.credential=abcdefghijklmnopqrstu+vwxyzabcdefghijklm + + +### Tidying up after jclouds + +Security groups are not always deleted by jclouds. This is due to a limitation in AWS (see +https://issues.apache.org/jira/browse/JCLOUDS-207). In brief, AWS prevents the security group +being deleted until there are no VMs using it. However, there is eventual consistency for +recording which VMs still reference those security groups. After deleting the VM, it can sometimes +take several minutes before the security group can be deleted. jclouds retries for 3 seconds, but +does not block for longer. + +There is utility written by Cloudsoft for deleting these unused resources: +http://www.cloudsoftcorp.com/blog/2013/03/tidying-up-after-jclouds. + + +## Google Compute Engine + +### Credentials + +Google Compute Engine (GCE) uses a service account e-mail address for the identity, and a private key +as the credential. + +To obtain these from GCE, see the [jclouds instructions](https://jclouds.apache.org/guides/google). + +An example of the expected format is shown below (note the credential is one long line, +with `\n` to represent the new line characters): + + brooklyn.location.jclouds.google-compute-engine.identity=123456789...@developer.gserviceaccount.com + brooklyn.location.jclouds.google-compute-engine.credential=-----BEGIN RSA PRIVATE KEY-----\nabcdefghijklmnopqrstuvwxyznabcdefghijk/lmnopqrstuvwxyzabcdefghij\nabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghij+lm\nnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklm\nnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxy\nzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijk\nlmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvw\nxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghi\njklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstu\nvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefg\nhijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrs\ntuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcde\nfghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvw\n-----END RSA PRIVATE KEY----- + + +### Quotas + +GCE accounts can have low default [quotas](https://cloud.google.com/compute/docs/resource-quotas). + +It is easy to requesta quota increase by submitting a [quota increase form](https://support.google.com/cloud/answer/6075746?hl=en). + + +### Networks + +GCE accounts often have a limit to the number of networks that can be created. One work around +is to manually create a network with the required open ports, and to refer to that named network +in Brooklyn's location configuration. + +To create a network, see [GCE network instructions](https://cloud.google.com/compute/docs/networking#networks_1). + +For example, for dev/demo purposes an "everything" network could be created that opens all ports. + +| Name | everything | +| Description | opens all tcp ports | +| Source IP Ranges | 0.0.0.0/0 | +| Allowed protocols and ports | tcp:0-65535 and udp:0-65535 | + + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/locations/index.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/index.md b/guide/ops/locations/index.md new file mode 100644 index 0000000..490d4c7 --- /dev/null +++ b/guide/ops/locations/index.md @@ -0,0 +1,420 @@ +--- +title: Locations +layout: website-normal +children: +- { section: Clouds } +- { section: Inheritance and Named Locations, title: Named Locations } +- { section: Localhost } +- { section: BYON } +- cloud-credentials.md +- more-locations.md +- location-customizers.md +- ssh-keys.md +--- + +Locations are the environments to which Brooklyn deploys applications, including: + +Brooklyn supports a wide range of locations: + +* <a href="#clouds">Clouds</a>, where it will provision machines +* <a href="#localhost">Localhost</a> (e.g. your laptop), + where it will deploy via `ssh` to `localhost` for rapid testing +* <a href="#byon">BYON</a>, where you "bring your own nodes", + specifying already-existing hosts to use +* And many others, including object stores and online services + +Configuration can be set in `~/.brooklyn/brooklyn.properties` +or directly in YAML when specifying a location. +On some entities, config keys determining maching selection and provisioning behavior +can also be set `in `provisioning.properties`. + + +### Clouds + +For most cloud provisioning tasks, Brooklyn uses +<a href="http://jclouds.org";>Apache jclouds</a>. +The identifiers for some of the most commonly used jclouds-supported clouds are +(or [see the full list](http://jclouds.apache.org/reference/providers/)): + +* `jclouds:aws-ec2:<region>`: Amazon EC2, where `:<region>` might be `us-east-1` or `eu-west-1` (or omitted) +* `jclouds:softlayer:<region>`: IBM Softlayer, where `:<region>` might be `dal05` or `ams01` (or omitted) +* `jclouds:google-compute-engine`: Google Compute Engine +* `jclouds:openstack-nova:<endpoint>`: OpenStack, where `:<endpoint>` is the access URL (required) +* `jclouds:cloudstack:<endpoint>`: Apache CloudStack, where `:<endpoint>` is the access URL (required) + +For any of these, of course, Brooklyn needs to be configured with an `identity` and a `credential`: + +{% highlight yaml %} +location: + jclouds:aws-ec2: + identity: ABCDEFGHIJKLMNOPQRST + credential: s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1rr3l +{% endhighlight %} + +The above YAML can be embedded directly in blueprints, either at the root or on individual services. +If you prefer to keep the credentials separate, these can be set instead in `brooklyn.properties` +in the `jclouds.<provider>` namespace: + +{% highlight bash %} +brooklyn.location.jclouds.aws-ec2.identity=ABCDEFGHIJKLMNOPQRST +brooklyn.location.jclouds.aws-ec2.credential=s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1rr3l +{% endhighlight %} + +And in this case you can reference the location in YAML with `location: jclouds:aws-ec2`. + +The Getting Started [template brooklyn.properties]({{ site.path.guide }}/start/brooklyn.properties) contains more examples +of configuring cloud endpoints, including information on credential types used in different clouds. + + +#### OS Initial Login and Setup + +Once a machine is provisioned, Brooklyn will normally attempt to log in via SSH and configure the machine sensibly. + +The credentials for the initial OS log on are typically discovered from the cloud, +but in some environments this is not possible. +The keys `loginUser` and either `loginUser.password` or `loginUser.privateKeyFile` can be used to force +Brooklyn to use specific credentials for the initial login to a cloud-provisioned machine. + +(This custom login is particularly useful when using a custom image templates where the cloud-side account +management logic is not enabled. For example, a vCloud (vCD) template can have guest customization that will change +the root password. This setting tells AMP to only use the given password, rather than the initial +randomly generated password that vCD returns. Without this property, there is a race for such templates: +does Brooklyn manage to create the admin user before the guest customization changes the login and reboots, +or is the password reset first (the latter means Brooklyn can never ssh to the VM). With this property, +Brooklyn will always wait for guest customization to complete before it is able to ssh at all. In such +cases, it is also recommended to use `useJcloudsSshInit=false`.) + +Following a successful logon, Brooklyn performs the following steps to configure the machine: + +1. creates a new user with the same name as the user `brooklyn` is running as locally + (this can be overridden with `user`, below). + +1. install the local user's `~/.ssh/id_rsa.pub` as an `authorized_keys` on the new machine, + to make it easy for the operator to `ssh` in + (override with `privateKeyFile`; or if there is no `id_{r,d}sa{,.pub}` an ad hoc keypair will be generated; + if there is a passphrase on the key, this must be supplied) + +1. give `sudo` access to the newly created user (override with `grantUserSudo: false`) + +1. disable direct `root` login to the machine + +These steps can be skipped or customized as described below. + + + +#### jclouds Config Keys + +The following is a subset of the most commonly used configuration keys used to customize +cloud provisioning. +For more keys and more detail on the keys below, see +{% include java_link.html class_name="JcloudsLocationConfig" package_path="org/apache/brooklyn/location/jclouds" project_subpath="locations/jclouds" %}. + +###### VM Creation + +- Most providers require exactly one of either `region` (e.g. `us-east-1`) or `endpoint` (the URL, usually for private cloud deployments) + +- Hardware requirements can be specified, including + `minRam`, `minCores`, and `os64Bit`; or as a specific `hardwareId` + +- VM image constraints can be set using `osFamily` (e.g. `Ubuntu`, `CentOS`, `Debian`, `RHEL`) + and `osVersionRegex`, or specific VM images can be specified using `imageId` or `imageNameRegex` + +- Specific VM images can be specified using `imageId` or `imageNameRegex` + +- Specific Security Groups can be specified using `securityGroups`, as a list of strings (the existing security group names), + or `inboundPorts` can be set, as a list of numeric ports (selected clouds only) + +- A specific existing key pair known at the cloud to use can be specified with `keyPair` + (selected clouds only) + +- A specific VM name (often the hostname) base to be used can be specified by setting `groupId`. + By default, this name is constructed based on the entity which is creating it, + including the ID of the app and of the entity. + (As many cloud portals let you filter views, this can help find a specific entity or all machines for a given application.) + For more sophisticated control over host naming, you can supply a custom + {% include java_link.html class_name="CloudMachineNamer" package_path="org/apache/brooklyn/core/location/cloud/names" project_subpath="core" %}, + for example + `cloudMachineNamer: CustomMachineNamer`. + {% include java_link.html class_name="CustomMachineNamer" package_path="org/apache/brooklyn/core/location/cloud/names" project_subpath="core" %} + will use the entity's name or following a template you supply. + On many clouds, a random suffix will be appended to help guarantee uniqueness; + this can be removed by setting `vmNameSaltLength: 0` (selected clouds only). + <!-- TODO jclouds softlayer includes a 3-char hex suffix --> + +- A DNS domain name where this host should be placed can be specified with `domainName` + (in selected clouds only) + +- User metadata can be attached using the syntax `userMetadata: { key: value, key2: "value 2" }` + (or `userMetadata=key=value,key2="value 2"` in a properties file) + +- By default, several pieces of user metadata are set to correlate VMs with Brooklyn entities, + prefixed with `brooklyn-`. + This user metadata can be omitted by setting `includeBrooklynUserMetadata: false`. + +- You can specify the number of attempts Brooklyn should make to create + machines with `machineCreateAttempts` (jclouds only). This is useful as an efficient low-level fix + for those occasions when cloud providers give machines that are dead on arrival. + You can of course also resolve it at a higher level with a policy such as + {% include java_link.html class_name="ServiceRestarter" package_path="org/apache/brooklyn/policy/ha" project_subpath="policy" %}. + +- If you want to investigate failures, set `destroyOnFailure: false` + to keep failed VM's around. (You'll have to manually clean them up.) + The default is false: if a VM fails to start, or is never ssh'able, then the VM will be terminated. + + +###### OS Setup + +- `user` and `password` can be used to configure the operating user created on cloud-provisioned machines + +- The `loginUser` config key (and subkeys) control the initial user to log in as, + in cases where this cannot be discovered from the cloud provider + +- Private keys can be specified using `privateKeyFile`; + these are not copied to provisioned machines, but are required if using a local public key + or a pre-defined `authorized_keys` on the server. + (For more information on SSH keys, see [here](ssh-keys.html).) + +- If there is a passphrase on the key file being used, you must supply it to Brooklyn for it to work, of course! + `privateKeyPassphrase` does the trick (as in `brooklyn.location.jclouds.privateKeyPassphrase`, or other places + where `privateKeyFile` is valid). If you don't like keys, you can just use a plain old `password`. + +- Public keys can be specified using `publicKeyFile`, + although these can usually be omitted if they follow the common pattern of being + the private key file with the suffix `.pub` appended. + (It is useful in the case of `loginUser.publicKeyFile`, where you shouldn't need, + or might not even have, the private key of the `root` user when you log in.) + +- Use `dontCreateUser` to have Brooklyn run as the initial `loginUser` (usually `root`), + without creating any other user. + +- A post-provisioning `setup.script` can be specified (as a URL) to run an additional script, + before making the `Location` available to entities, + optionally also using `setup.script.vars` (set as `key1:value1,key2:value2`) + +- Use `openIptables: true` to automatically configure `iptables`, to open the TCP ports required by + the software process. One can alternatively use `stopIptables: true` to entirely stop the + iptables service. + +- Use `installDevUrandom: true` to fall back to using `/dev/urandom` rather than `/dev/random`. This setting + is useful for cloud VMs where there is not enough random entropy, which can cause `/dev/random` to be + extremely slow (causing `ssh` to be extremely slow to respond). + +- Use `useJcloudsSshInit: false` to disable the use of the native jclouds support for initial commands executed + on the VM (e.g. for creating new users, setting root passwords, etc.). Instead, Brooklyn's ssh support will + be used. Timeouts and retries are more configurable within Brooklyn itself. Therefore this option is particularly + recommended when the VM startup is unusual (for example, if guest customizations will cause reboots and/or will + change login credentials). + +- Use `brooklyn.ssh.config.noDeleteAfterExec: true` to keep scripts on the server after execution. + The contents of the scripts and the stdout/stderr of their execution are available in the Brooklyn web console, + but sometimes it can also be useful to have them on the box. + This setting prevents scripts executed on the VMs from being deleted on completion. + Note that some scripts run periodically so this can eventually fill a disk; it should only be used for dev/test. + +###### Custom template options + +jclouds supports many additional options for configuring how a virtual machine is created and deployed, many of which +are for cloud-specific features and enhancements. Brooklyn supports some of these, but if what you are looking for is +not supported directly by Brooklyn, we instead offer a mechanism to set any parameter that is supported by the jclouds +template options for your cloud. + +Part of the process for creating a virtual machine is the creation of a jclouds `TemplateOptions` object. jclouds +providers extends this with extra options for each cloud - so when using the AWS provider, the object will be of +type `AWSEC2TemplateOptions`. By [examining the source code](https://github.com/jclouds/jclouds/blob/jclouds-1.9.0/providers/aws-ec2/src/main/java/org/jclouds/aws/ec2/compute/AWSEC2TemplateOptions.java), +you can see all of the options available to you. + +The `templateOptions` config key takes a map. The keys to the map are method names, and Brooklyn will find the method on +the `TemplateOptions` instance; it then invokes the method with arguments taken from the map value. If a method takes a +single parameter, then simply give the argument as the value of the key; if the method takes multiple parameters, the +value of the key should be an array, containing the argument for each parameter. + +For example, here is a complete blueprint that sets some AWS EC2 specific options: + + location: AWS_eu-west-1 + services: + - type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess + provisioningProperties: + templateOptions: + subnetId: subnet-041c8373 + mapNewVolumeToDeviceName: ["/dev/sda1", 100, true] + securityGroupIds: ['sg-4db68928'] + +Here you can see that we set three template options: + +- `subnetId` is an example of a single parameter method. Brooklyn will effectively try to run the statement + `templateOptions.subnetId("subnet-041c88373");` +- `mapNewVolumeToDeviceName` is an example of a multiple parameter method, so the value of the key is an array. + Brooklyn will effectively true to run the statement `templateOptions.mapNewVolumeToDeviceName("/dev/sda1", 100, true);` +- `securityGroupIds` demonstrates an ambiguity between the two types; Brooklyn will first try to parse the value as + a multiple parameter method, but there is no method that matches this parameter. In this case, Brooklyn will next try + to parse the value as a single parameter method which takes a parameter of type `List`; such a method does exist so + the operation will succeed. + +If the method call cannot be matched to the template options available - for example if you are trying to set an AWS EC2 +specific option but your location is an OpenStack cloud - then a warning is logged and the option is ignored. + + + + +See the following resources for more information: + +- [AWS VPC issues which may affect users with older AWS accounts](vpc-issues.html) +- [Amazon EC2 and Amazon Virtual Private Cloud](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html#vpc-only-instance-types) +- [Your Default VPC and Subnets](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/default-vpc.html) +- [Amazon VPC FAQs](http://aws.amazon.com/vpc/faqs/#Default_VPCs) + + +### Inheritance and Named Locations + +Named locations can be defined for commonly used groups of properties, +with the syntax `brooklyn.location.named.your-group-name.` +followed by the relevant properties. +These can be accessed at runtime using the syntax `named:your-group-name` as the deployment location. + +Some illustrative examples using named locations and +showing the syntax and properties above are as follows: + +{% highlight bash %} +# Production pool of machines for my application (deploy to named:prod1) +brooklyn.location.named.prod1=byon:(hosts="10.9.1.1,10.9.1.2,[email protected].{10,11,20-29}") +brooklyn.location.named.prod1.user=produser1 +brooklyn.location.named.prod1.privateKeyFile=~/.ssh/produser_id_rsa +brooklyn.location.named.prod1.privateKeyPassphrase=s3cr3tCOMPANYpassphrase + +# AWS using my company's credentials and image standard, then labelling images so others know they're mine +brooklyn.location.named.company-jungle=jclouds:aws-ec2:us-west-1 +brooklyn.location.named.company-jungle.identity=BCDEFGHIJKLMNOPQRSTU +brooklyn.location.named.company-jungle.privateKeyFile=~/.ssh/public_clouds/company_aws_id_rsa +brooklyn.location.named.company-jungle.imageId=ami-12345 +brooklyn.location.named.company-jungle.minRam=2048 +brooklyn.location.named.company-jungle.userMetadata=application=my-jungle-app,owner="Bob Johnson" +brooklyn.location.named.company-jungle.machineCreateAttempts=2 + +brooklyn.location.named.AWS\ Virginia\ Large\ Centos = jclouds:aws-ec2 +brooklyn.location.named.AWS\ Virginia\ Large\ Centos.region = us-east-1 +brooklyn.location.named.AWS\ Virginia\ Large\ Centos.imageId=us-east-1/ami-7d7bfc14 +brooklyn.location.named.AWS\ Virginia\ Large\ Centos.user=root +brooklyn.location.named.AWS\ Virginia\ Large\ Centos.minRam=4096 +{% endhighlight %} + +The precedence for configuration defined at different levels is that the most value +defined in the most specific context will apply. + +For example, in the example below the config key is repeatedly overridden. If you deploy +`location: named:my-aws`, Brooklyn will get `VAL5` or `KEY`: + +{% highlight bash %} +brooklyn.location.KEY=VAL1 +brooklyn.location.jclouds.KEY=VAL2 +brooklyn.location.jclouds.aws-ec2.KEY=VAL3 [email protected]=VAL4 +brooklyn.location.named.my-aws=jclouds:aws-ec2:us-west-1 +brooklyn.location.named.my-aws.KEY=VAL5 +{% endhighlight %} + + +### Localhost + +If passwordless ssh login to `localhost` and passwordless `sudo` is enabled on your +machine, you should be able to deploy blueprints with no special configuration, +just by specifying `location: localhost` in YAML. + +If you use a passpharse or prefer a different key, these can be configured as follows: + +{% highlight bash %} +brooklyn.location.localhost.privateKeyFile=~/.ssh/brooklyn_key +brooklyn.location.localhost.privateKeyPassphrase=s3cr3tPASSPHRASE +{% endhighlight %} + +If you encounter issues or for more information, see [SSH Keys Localhost Setup](ssh-keys.html#localhost-setup). + +If you are normally prompted for a password when executing `sudo` commands, passwordless `sudo` must also be enabled. To enable passwordless `sudo` for your account, a line must be added to the system `/etc/sudoers` file. To edit the file, use the `visudo` command: +{% highlight bash %} +sudo visudo +{% endhighlight %} +Add this line at the bottom of the file, replacing `username` with your own user: +{% highlight bash %} +username ALL=(ALL) NOPASSWD: ALL +{% endhighlight %} +If executing the following command does not ask for your password, then `sudo` should be setup correctly: +{% highlight bash %} +sudo ls +{% endhighlight %} + + +### BYON + +"Bring-your-own-nodes" mode is useful in production, where machines have been provisioned by someone else, +and during testing, to cut down provisioning time. + +Your nodes must meet the following prerequisites: + +- A suitable OS must have been installed on all nodes +- The node must be running sshd (or similar) +- the brooklyn user must be able to ssh to each node as root or as a user with passwordless sudo permission. (For more information on SSH keys, see [here](ssh-keys.html).) + +To deploy to machines with known IP's in a blueprint, use the following syntax: + +{% highlight yaml %} +location: + byon: + user: brooklyn + privateKeyFile: ~/.ssh/brooklyn.pem + hosts: + - 192.168.0.18 + - 192.168.0.19 +{% endhighlight %} + +Some of the login properties as described above for jclouds are supported, +but not `loginUser` (as no users are created), and not any of the +VM creation parameters such as `minRam` and `imageId`. +(These clearly do not apply in the same way, and they are *not* +by default treated as constraints, although an entity can confirm these +where needed.) +As before, if the brooklyn user and its default key are authorized for the hosts, +those fields can be omitted. + +Named locations can also be configured in your `brooklyn.properties`, +using the format `byon:(key=value,key2=value2)`. +For convenience, for hosts wildcard globs are supported. + +{% highlight bash %} +brooklyn.location.named.On-Prem\ Iron\ Example=byon:(hosts="10.9.1.1,10.9.1.2,[email protected].{10,11,20-29}") +brooklyn.location.named.On-Prem\ Iron\ Example.user=produser1 +brooklyn.location.named.On-Prem\ Iron\ Example.privateKeyFile=~/.ssh/produser_id_rsa +brooklyn.location.named.On-Prem\ Iron\ Example.privateKeyPassphrase=s3cr3tpassphrase +{% endhighlight %} + +For more complex host configuration, one can define custom config values per machine. In the example +below, there will be two machines. The first will be a machine reachable on +`ssh -i ~/.ssh/brooklyn.pem -p 8022 [email protected]`. The second is a windows machine, reachable +over WinRM. Each machine has also has a private address (e.g. for within a private network). + +{% highlight yaml %} +location: + byon: + hosts: + - ssh: 50.51.52.53:8022 + privateAddresses: [10.0.0.1] + privateKeyFile: ~/.ssh/brooklyn.pem + user: myuser + - winrm: 50.51.52.54:8985 + privateAddresses: [10.0.0.2] + password: mypassword + user: myuser + osFamily: windows +{% endhighlight %} + +The BYON location also supports a machine chooser, using the config key `byon.machineChooser`. +This allows one to plugin logic to choose from the set of available machines in the pool. For +example, additional config could be supplied for each machine. This could be used (during the call +to `location.obtain()`) to find the config that matches the requirements of the entity being +provisioned. See `FixedListMachineProvisioningLocation.MACHINE_CHOOSER`. + + +### Other Location Topics + +* [Cloud Credentials](cloud-credentials.html) +* [More Locations](more-locations.html) +* [Location Customizers](location-customizers.html) +* [SSH Keys](ssh-keys.html) http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/locations/location-customizers.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/location-customizers.md b/guide/ops/locations/location-customizers.md new file mode 100644 index 0000000..ad1f17e --- /dev/null +++ b/guide/ops/locations/location-customizers.md @@ -0,0 +1,152 @@ +--- +title: Location customizers +layout: website-normal +--- + +Apache Brooklyn supports a number of ways to configure and customize locations. These include +the `JcloudsLocationCustomizer`, which is for advanced customization of VM provisioning through jclouds. +There is also a `MachineLocationCustomizer`, which allows customization of machines being obtained +from any kind of location (including [Bring Your Own Nodes](index.html#byon)). + + +## Usage Guidelines + +Clearly there is an overlap for where things can be done. This section describes the recommended +separation of responsibilities. + +These are guidelines only - users are obviously free to make alternative usage decisions based on +their particular use-cases. + +### Responsibilities of Entity versus Location + +From an entity's perspective, it calls `location.obtain(options)` and gets back a usable +`MachineLocation` that has a standard base operating system that gives remote access +(e.g. for Linux it expects credentials for a user with `sudo` rights, and ssh access). + +However, there are special cases - for example the `location.obtain(options)` could return +a Docker container with the software pre-installed, and no remote access (see the +[Clocker project](http://clocker.io) for more information on use of Docker with Brooklyn). + +The entity is then responsible for configuring that machine according to the needs of the software +to be installed. + +For example, the entity may install software packages, upload/update configuration files, launch +processes, etc. + +The entity may also configure `iptables`. This is also possible through the `JcloudsLocation` +configuration. However, it is preferable to do this in the entity because it is part of +configuring the machine in the way required for the given software component. + +The entity may also perform custom OS setup, such as installing security patches. However, whether +this is appropriate depends on the nature of the security patch: if the security patch is specific +to the entity type, then it should be done within the entity; but if it is to harden the base OS +to make it comply with an organisation's standards (e.g. to overcome shortcomings of the base +image, or to install security patches) then a `MachineLocationCustomizer` is more appropriate. + +### Location Configuration Options + +This refers to standard location configuration: explicit config keys, and explicit jclouds template +configuration that can be passed through. + +This kind of configuration is simplest to use. It is the favoured mechanism when it comes to VM +provisioning, and should be used wherever possible. + +Note that a jclouds `TemplateBuilder` and cloud-specific `TemplateOptions` are the generic mechanisms +within jclouds for specifying the details of the compute resource to be provisioned. + +### Jclouds Location Customizer +A `JcloudsLocationCustomizer` has customization hooks to execute code at the various points of building +up the jclouds template and provisioning the machine. Where jclouds is being used and where the required +use of jclouds goes beyond simple configuration, this is an appropriate solution. + +For example, there is a `org.apache.brooklyn.location.jclouds.networking.JcloudsLocationSecurityGroupCustomizer` +which gives more advanced support for setting up security groups (e.g. in AWS-EC2). + +### Machine Customizer + +The `MachineLocationCustomizer` allows customization of machines being obtained from any kind of location. +For example, this includes for jclouds and for Bring Your Own Nodes (BYON). + +It provides customization hooks for when the machine has been provisioned (before it is returned by the location) +and when the machine is about to be released by the location. + +An example use would be to register (and de-register) the machine in a CMDB. + + +## Jclouds Location Customizers + +*Warning: additional methods (i.e. customization hooks) may be added to the `JcloudsLocationCustomizer` +interface in future releases. Users are therefore strongly encouraged to sub-class +`BasicJcloudsLocationCustomizer`, rather than implementing JcloudsLocationCustomizer directly.* + +The `JcloudsLocationCustomizer` provides customization hooks at various points of the Brooklyn's +use of jclouds. These can be used to adjust the configuration, to do additional setup, to do +custom logging, etc. + +* Customize the `org.jclouds.compute.domain.TemplateBuilder`, before it is used to build the template. + This is used to influence the choice of VM image, hardware profile, etc. This hook is not normally + required as the location configuration options can be used in instead. + +* Customize the `org.jclouds.compute.domain.Template`, to be used when creating the machine. This + hook is most often used for performing custom actions - for example to create or modify a security + group or volume, and to update the template's options to use that. + +* Customize the `org.jclouds.compute.options.TemplateOptions` to be used when creating the machine. + The `TemplateOptions` could be cast to a cloud-specific sub-type (if this does not have to work + across different clouds). Where the use-case is to just set simple configuration on the + `TemplateOptions`, consider instead using the config key `templateOptions`, which takes a map + of type String to Object - the strings should match the method names in the `TemplateOptions`. + +* Customize the `org.apache.brooklyn.location.jclouds.JcloudsMachineLocation` that has been + created. For Linux-based VMs, if the config `waitForSshable` was not false, then this machine + is guaranteed to be ssh'able. Similarly for WinRM access to Windows machines, if + `waitForWinRmAvailable` was not false. + +* Pre-release of the machine. If the actions required are specific to jclouds (e.g. using jclouds + to make calls to the cloud provider) then this should be used; otherwise one should use the more + generic `MachineLocationCustomizer`. + +* Post-release of the machine (i.e. after asking jclouds to destroying the machine). + +To register a `JcloudsLocationCustomizer` in YAML, the config key `customizers` can be used to +provide a list of instances. Each instance can be defined using `$brooklyn:object` to indicate +the type and its configuration. For example: + + location: + jclouds:aws-ec2:us-east-1: + customizers: + - $brooklyn:object: + type: com.acme.brooklyn.MyJcloudsLocationCustomizer + +To register `JcloudsLocationCustomizer` instances programmatically, set the config key +`JcloudsLocationConfig.JCLOUDS_LOCATION_CUSTOMIZERS` on the location, or pass this +config option when calling `location.obtain(options)`. + + +## Machine Location Customizers + +*Warning: additional methods (i.e. customization hooks) may be added to the `MachineLocationCustomizer` +interface in future releases. Users are therefore strongly encouraged to sub-class +`BasicMachineLocationCustomizer`, rather than implementing `MachineLocationCustomizer` directly.* + +The `MachineLocationCustomizer` provides customization hooks for when a machine is obtained/released +from a `MachineProvisioningLocation`. The following hooks are supported: + +* After the machine has been provisioned/allocated, but before it has been returned. + +* When the machine is about to be released, but prior to actually destroying/unallocating the + machine. + +To register a `MachineLocationCustomizer` in YAML, the config key `machineCustomizers` can be used +to provide a list of instances. Each instance can be defined using `$brooklyn:object` to indicate +the type and its configuration. For example: + + location: + jclouds:aws-ec2:us-east-1: + machineCustomizers: + - $brooklyn:object: + type: com.acme.brooklyn.MyMachineLocationCustomizer + +To register `MachineLocationCustomizer` instances programmatically, set the config key +`CloudLocationConfig.MACHINE_LOCATION_CUSTOMIZERS` on the location, or pass this +config option when calling `location.obtain(options)`. http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/locations/more-locations.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/more-locations.md b/guide/ops/locations/more-locations.md new file mode 100644 index 0000000..0a00bf5 --- /dev/null +++ b/guide/ops/locations/more-locations.md @@ -0,0 +1,55 @@ +--- +title: More Locations +layout: website-normal +children: +- { section: Single Host } +- { section: The Multi Location } +- { section: The Server Pool } +--- + +Some additional location types are supported for specialized situations: + +### Single Host + +The spec `host`, taking a string argument (the address) or a map (`host`, `user`, `password`, etc.), +provides a convenient syntax when specifying a single host. +For example: + +{% highlight yaml %} +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + location: + host: 192.168.0.1 +{% endhighlight %} + +Or, in `brooklyn.properties`, set `brooklyn.location.named.host1=host:(192.168.0.1)`. + + +### The Multi Location + +The spec `multi` allows multiple locations, specified as `targets`, +to be combined and treated as one location. +When the first target is full, the next is tried, and so on: + +{% highlight yaml %} +location: + multi: + targets: + - byon:(hosts=192.168.0.1) + - jclouds:aws-ec2: + identity: acct1 + - jclouds:aws-ec2: + identity: acct2 +{% endhighlight %} + +The example above provisions the first node to `192.168.0.1`, +then it provisions into `acct1` at Amazon if possible, +and then to `acct2`. + + + +### The Server Pool + +The {% include java_link.html class_name="ServerPool" package_path="org/apache/brooklyn/entity/machine/pool" project_subpath="software/base" %} +entity type allows defining an entity which becomes available as a location. + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/locations/ssh-keys.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/ssh-keys.md b/guide/ops/locations/ssh-keys.md new file mode 100644 index 0000000..f805253 --- /dev/null +++ b/guide/ops/locations/ssh-keys.md @@ -0,0 +1,85 @@ +--- +title: SSH Keys +layout: website-normal +--- + +SSH keys are one of the simplest and most secure ways to access remote servers. +They consist of two parts: + +* A private key (e.g. `id_rsa`) which is known only to one party or group + +* A public key (e.g. `id_rsa.pub`) which can be given to anyone and everyone, + and which can be used to confirm that a party has a private key + (or has signed a communication with the private key) + +In this way, someone -- such as you -- can have a private key, +and can install a public key on a remote machine (in an `authorized_keys` file) +for secure automated access. +Commands such as `ssh` (and Brooklyn) can log in without +revealing the private key to the remote machine, +the remote machine can confirm it is you accessing it (if no one else has the private key), +and no one snooping on the network can decrypt of any of the traffic. + + +### Creating an SSH Key + +If you don't have an SSH key, create one with: + +{% highlight bash %} +$ ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa +{% endhighlight %} + + +### Localhost Setup + +If you want to deploy to `localhost`, ensure that you have a public and private key, +and that your key is authorized for ssh access: + +{% highlight bash %} +# _Appends_ id_rsa.pub to authorized_keys. Other keys are unaffected. +$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys +{% endhighlight %} + +Now verify that your setup by running the command: `ssh localhost echo hello world` + +If your setup is correct, you should see `hello world` printed back at you. + +On the first connection, you may see a message similar to this: + +<pre> +The authenticity of host 'localhost (::1)' can't be established. +RSA key fingerprint is 7b:e3:8e:c6:5b:2a:05:a1:7c:8a:cf:d1:6a:83:c2:ad. +Are you sure you want to continue connecting (yes/no)? +</pre> + +Simply answer 'yes' and then repeat the command again. + +If this isn't the case, see below. + + + + +### Potential Problems + +* **MacOS user?** In addition to the above, enable "Remote Login" in "System Preferences > Sharing". + +* **Got a passphrase?** Set `brooklyn.location.localhost.privateKeyPassphrase` + as described [here](index.html#os-setup). + If you're not sure, or you don't know what a passphrase is, you can test this by executing `ssh-keygen -y`. + If it does *not* ask for a passphrase, then your key has no passphrase. + If your key does have a passphrase, you can remove it by running `ssh-keygen -p`. + +* Check that you have an `~/.ssh/id_rsa` file (or `id_dsa`) and a corresponding public key with a `.pub` extension; + if not, create one as described above + +* `~/.ssh/` or files in that directory may have permissions they shouldn't: + they should be visible only to the user (apart from public keys), + both on the source machine and the target machine. + You can verify this with `ls -l ~/.ssh/`: lines should start with `-rw-------` or `-r--------` (or `-rwx------` for directories). + If it does not, execute `chmod go-rwx ~/.ssh ~/.ssh/*`. + +* Sometimes machines are configured with different sets of support SSL/TLS versions and ciphers; + if command-line `ssh` and `scp` work, but Brooklyn/java does not, check the versions enabled in Java and on both servers. + +* Missing entropy: creating and using ssh keys requires randomness available on the servers, + usually in `/dev/random`; see [here]({{ site.path.website }}/documentation/increase-entropy.html) for more information http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/locations/vpc-issues.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/vpc-issues.md b/guide/ops/locations/vpc-issues.md new file mode 100644 index 0000000..b2b362b --- /dev/null +++ b/guide/ops/locations/vpc-issues.md @@ -0,0 +1,32 @@ + +### AWS VPC issues which may affect users with older AWS accounts + +AWS now has different default behaviour depending on the age of your AWS account and whether you used the target region before, or during, 2013. +In this case VM provisioning may fail with an error like: + +{% highlight text %} + +Detected that your EC2 account is a legacy 'classic' account, but the recommended instance type requires VPC. +You can specify the 'eu-central-1' region to avoid this problem, or you can specify a classic-compatible instance type, +or you can specify a subnet to use with 'networkName' +taking care that the subnet auto-assigns public IP's and allows ingress on all ports, +as Brooklyn does not currently configure security groups for non-default VPC's; +or setting up Brooklyn to be in the subnet or have a jump host or other subnet access configuration). +For more information on VPC vs classic see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html. + +{% endhighlight %} + +Specifically, there are issues with the certain AMIs and instance types. If these are specified or a recommended +by brooklyn then you may see the above error. There are a few options for fixing this: + +- specify a different region which does not support EC2-classic. + You can check this on the AWS console under "Supported Platforms. + Frankfurt (eu-central-1) is guaranteed to be VPC only. + +- specify an instance type that is compatible with ec2-classic. + Instance types C4, M4, T2 are only supported in VPC so should not be used. + This is described [here](index.html#vm-creation) + +- create a subnet to use with the instance. Ensure that the subnet is set to auto-assign public IPs + and allows ingress on all ports. Brooklyn cannot currently do this for you. + Use the networkName parameter to specify this value in your blueprint. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/logging.md ---------------------------------------------------------------------- diff --git a/guide/ops/logging.md b/guide/ops/logging.md new file mode 100644 index 0000000..81d55e9 --- /dev/null +++ b/guide/ops/logging.md @@ -0,0 +1,72 @@ +--- +title: Logging +layout: website-normal +--- + +Brooklyn uses the SLF4J logging facade, which allows use of many popular frameworks including `logback`, +`java.util.logging` and `log4j`. + +The convention for log levels is as follows: + +* `ERROR` and above: exceptional situations which indicate that something has unexpectedly failed or +some other problem has occured which the user is expected to attend to +* `WARN`: exceptional situations which the user may which to know about but which do not necessarily indicate failure or require a response +* `INFO`: a synopsis of activity, but which should not generate large volumes of events nor overwhelm a human observer +* `DEBUG` and lower: detail of activity which is not normally of interest, but which might merit closer inspection under certain circumstances. + +Loggers follow the ``package.ClassName`` naming standard. + + +## Standard Configuration + +A `logback.xml` file is included in the `conf/` directly of the Brooklyn distro; +this is read by `brooklyn` at launch time. Changes to the logging configuration, +such as new appenders or different log levels, can be made directly in this file +or in a new file included from this. + + +## Advanced Configuration + +The default `logback.xml` file references a collection of other log configuration files +included in the Brooklyn jars. It is necessary to understand the source structure +in the [logback-includes]({{ site.brooklyn.url.git }}/usage/logback-includes) project. + +For example, to change the debug log inclusions, create a folder `brooklyn` under `conf` +and create a file `logback-debug.xml` based on the +[brooklyn/logback-debug.xml]({{ site.brooklyn.url.git }}/usage/logback-includes/src/main/resources/brooklyn/logback-debug.xml) +from that project. + + +## Log File Backup + +*This sub-section is a work in progress; feedback from the community is extremely welcome.* + +The default rolling log files can be backed up periodically, e.g. using a CRON job. + +Note however that the rolling log file naming scheme will rename the historic zipped log files +such that `brooklyn.debug-1.log.zip` is the most recent zipped log file. When the current +`brooklyn.debug.log` is to be zipped, the previous zip file will be renamed +`brooklyn.debug-2.log.zip`. This renaming of files can make RSYNC or backups tricky. + +An option is to covert/move the file to a name that includes the last-modified timestamp. +For example (on mac): + + LOG_FILE=brooklyn.debug-1.log.zip + TIMESTAMP=`stat -f '%Um' $LOG_FILE` + mv $LOG_FILE /path/to/archive/brooklyn.debug-$TIMESTAMP.log.zip + + +## Logging aggregators + +Integration with systems like Logstash and Splunk is possible using standard logback configuration. +Logback can be configured to [write to the syslog](http://logback.qos.ch/manual/appenders.html#SyslogAppender), +which can then [feed its logs to Logstash](http://www.logstash.net/docs/1.4.2/inputs/syslog). + + +## For More Information + +The following resources may be useful when configuring logging: + +* The [logback-includes]({{ site.brooklyn.url.git }}/usage/logback-includes) project +* [Brooklyn Developer Guide]({{ site.path.guide }}/dev/tips/logging.html) logging tips +* The [Logback Project](http://logback.qos.ch/) home page http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/persistence/index.md ---------------------------------------------------------------------- diff --git a/guide/ops/persistence/index.md b/guide/ops/persistence/index.md new file mode 100644 index 0000000..02c05ff --- /dev/null +++ b/guide/ops/persistence/index.md @@ -0,0 +1,379 @@ +--- +title: Persistence +layout: website-normal +children: +- { section: Command Line Options } +- { section: File-based Persistence } +- { section: Object Store Persistence } +- { section: Rebinding to State } +- { section: Writing Persistable Code } +- { section: Persisted State Backup } +--- + +Brooklyn can be configured to persist its state so that the Brooklyn server can be restarted, +or so that a high availability standby server can take over. + +Brooklyn can persist its state to one of two places: the file system, or to an Object Store +of your choice. + +Command Line Options +-------------------- + +To configure brooklyn, the relevant command line options for the `launch` commands are: + +* `--persist` <persistence mode> + The persistence mode. +* `--persistenceDir` <persistence dir> + The directory to read/write persisted state (or container name if using an object store). +* `--persistenceLocation` <persistence location> + The location spec for an object store to read/write persisted state. + +For the persistence mode, the possible values are: + +* `disabled` means that no state will be persisted or read; when Brooklyn stops all state is lost. +* `rebind` means that it will read existing state, and recreate entities, locations and policies + from that. If there is no existing state, startup will fail. +* `clean` means that any existing state will be deleted, and Brooklyn will be started afresh. +* `auto` means Brooklyn will rebind if there is any existing state, or will start afresh if + there is no state. + +The persistence directory and location can instead be specified from `brooklyn.properties` using +the following config keys: + +* `brooklyn.persistence.dir` +* `brooklyn.persistence.location.spec` + + +File-based Persistence +---------------------- + +To persist to the file system, start brooklyn with: + +{% highlight bash %} +brooklyn launch --persist auto --persistenceDir /path/to/myPersistenceDir +{% endhighlight %} + +If there is already data at `/path/to/myPersistenceDir`, then a backup of the directory will +be made. This will have a name like `/path/to/myPersistenceDir.20140701-142101345.bak`. + +The state is written to the given path. The file structure under that path is: + +* `./entities/` +* `./locations/` +* `./policies/` +* `./enrichers/` + +In each of those directories, an XML file will be created per item - for example a file per +entity in `./entities/`. This file will capture all of the state - for example, an +entity's: id; display name; type; config; attributes; tags; relationships to locations, child +entities, group membership, policies and enrichers; and dynamically added effectors and sensors. + +If using the default persistence dir (i.e. no `--persistenceDir` was specified), then Brooklyn will +write its state to `~/.brooklyn/brooklyn-persisted-state/data`. Copies of this directory +will be automatically created in `~/.brooklyn/brooklyn-persisted-state/backups/` each time Brooklyn +is restarted (or if a standby Brooklyn instances takes over as master). + +A custom directory for Brooklyn state can also be configured in `brooklyn.properties` using: + + # For all Brooklyn files + brooklyn.base.dir=/path/to/base/dir + + # Sub-directory of base.dir for writing persisted state (if relative). If directory + # starts with "/" (or "~/", or something like "c:\") then assumed to be absolute. + brooklyn.persistence.dir=data + + # Sub-directory of base.dir for creating backup directories (if relative). If directory + # starts with "/" (or "~/", or something like "c:\") then assumed to be absolute. + brooklyn.persistence.backups.dir=backups + +This `base.dir` will also include temporary files such as the OSGi cache. + +If `persistence.dir` is not specified then it will use the sub-directory +`brooklyn-persisted-state/data` of the `base.dir`. If the `backups.dir` is not specified +the backup directories will be created in the sub-directory `backups` of the persistence dir. + + +Object Store Persistence +------------------------ + +Brooklyn can persist its state to any Object Store API that jclouds supports including +S3, Swift and Azure. This gives access to any compatible Object Store product or cloud provider +including AWS-S3, SoftLayer, Rackspace, HP and Microsoft Azure. For a complete list of supported +providers, see [jclouds](http://jclouds.apache.org/reference/providers/#blobstore). + +To configure the Object Store, add the credentials to `~/.brooklyn/brooklyn.properties` such as: + +{% highlight properties %} +brooklyn.location.named.aws-s3-eu-west-1=aws-s3:eu-west-1 +brooklyn.location.named.aws-s3-eu-west-1.identity=ABCDEFGHIJKLMNOPQRSTU +brooklyn.location.named.aws-s3-eu-west-1.credential=abcdefghijklmnopqrstuvwxyz1234567890ab/c +{% endhighlight %} + +or: + +{% highlight properties %} +brooklyn.location.named.softlayer-swift-ams01=jclouds:swift:https://ams01.objectstorage.softlayer.net/auth/v1.0 +brooklyn.location.named.softlayer-swift-ams01.identity=ABCDEFGHIJKLM:myname +brooklyn.location.named.softlayer-swift-ams01.credential=abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopqrstuvwxyz12 +{% endhighlight %} + +Start Brooklyn pointing at this target object store, e.g.: + +{% highlight bash %} +nohup brooklyn launch --persist auto --persistenceDir myContainerName --persistenceLocation named:softlayer-swift-ams01 & +{% endhighlight %} + + +The following `brooklyn.properties` options can also be used: + + # Location spec string for an object store (e.g. jclouds:swift:URL) where persisted state + # should be kept; if blank or not supplied, the file system is used. + brooklyn.persistence.location.spec=<location> + + # Container name for writing persisted state + brooklyn.persistence.dir=/path/to/dataContainer + + # Location spec string for an object store (e.g. jclouds:swift:URL) where backups of persisted + # state should be kept; defaults to the local file system. + brooklyn.persistence.backups.location.spec=<location> + + # Container name for writing backups of persisted state; + # defaults to 'backups' inside the default persistence container. + brooklyn.persistence.backups.dir=/path/to/backupContainer + + +Rebinding to State +------------------ + +When Brooklyn starts up pointing at existing state, it will recreate the entities, locations +and policies based on that persisted state. + +Once all have been created, Brooklyn will "manage" the entities. This will bind to the +underlying entities under management to update the each entity's sensors (e.g. to poll over +HTTP or JMX). This new state will be reported in the web-console and can also trigger +any registered policies. + + +## CLI Commands for Copying State + +Brooklyn includes a command to copy persistence state easily between two locations. +The `copy-state` CLI command takes the following arguments: + +* `--persistenceDir` <source persistence dir> + The directory to read persisted state (or container name if using an object store). +* `--persistenceLocation` <source persistence location> + The location spec for an object store to read persisted state. +* `--destinationDir` <target persistence dir> + The directory to copy persistence data to (or container name if using an object store). +* `--destinationLocation` <target persistence location> + The location spec for an object store to copy data to. +* `--transformations` <transformations> + The local transformations file to be applied to the copy of the data before uploading it. + + +## Handling Rebind Failures + +If rebind fails fail for any reason, details of the underlying failures will be reported +in the `brooklyn.debug.log`. There are several approaches to resolving problems. + + +### Determine Underlying Cause + +The problems reported in brooklyn.debug.log will indicate where the problem lies - which +entities, locations or policies, and in what way it failed. + +### Ignore Errors + +The `~/.brooklyn/brooklyn.properties` has several configuration options: + +{% highlight properties %} +rebind.failureMode.danglingRef=continue +rebind.failureMode.loadPolicy=continue +rebind.failureMode.addPolicy=continue +rebind.failureMode.rebind=fail_at_end +rebind.failureMode.addConfig=fail_at_end +{% endhighlight %} + +For each of these configuration options, the possible values are: + +* `fail_fast`: stop rebind immediately upon errors; do not try to rebind other entities +* `fail_at_end`: continue rebinding all entities, but then fail so that all errors + encountered are reported +* `continue`: log a warning, but ignore the error to continue rebinding. Depending on the + type of error, this can cause serious problems later (e.g. if the state of an entity + was entirely missing, then all its children would be orphaned). + +The meaning of the configuration options is: + +* `rebind.failureMode.dangingRef`: if there is a reference to an entity, location or policy + that is missing... whether to continue (discarding the reference) or fail. +* `rebind.failureMode.loadPolicy`: if there is an error instantiate or reconstituting the + state of a policy or enricher... whether to continue (discarding the policy or enricher) + or fail. +* `rebind.failureMode.addPolicy`: if there is an error re-adding the policy or enricher to + its associated entity... whether to continue (discarding the policy or enricher) + or fail. +* `rebind.failureMode.addConfig`: if there is invalid config value, or some other error occurs when adding a config. +* `rebind.failureMode.rebind`: any errors on rebind not covered by the more specific error cases described above. + + +### Seek Help + +Help can be found at `[email protected]`, where folk will be able to investigate +issues and suggest work-arounds. + +By sharing the persisted state (with credentials removed), Brooklyn developers will be able to +reproduce and debug the problem. + + +### Fix-up the State + +The state of each entity, location, policy and enricher is persisted in XML. +It is thus human readable and editable. + +After first taking a backup of the state, it is possible to modify the state. For example, +an offending entity could be removed, or references to that entity removed, or its XML +could be fixed to remove the problem. + + +### Fixing with Groovy Scripts + +The final (powerful and dangerous!) tool is to execute Groovy code on the running Brooklyn +instance. If authorized, the REST api allows arbitrary Groovy scripts to be passed in and +executed. This allows the state of entities to be modified (and thus fixed) at runtime. + +If used, it is strongly recommended that Groovy scripts are run against a disconnected Brooklyn +instance. After fixing the entities, locations and/or policies, the Brooklyn instance's +new persisted state can be copied and used to fix the production instance. + + +Writing Persistable Code +------------------------ +The most common problem on rebind is that custom entity code has not been written in a way +that can be persisted and/or rebound. + +The rule of thumb when implementing new entities, locations, policies and enrichers is that +all state must be persistable. All state must be stored as config or as attributes, and must be +serializable. For making backwards compatibility simpler, the persisted state should be clean. + +Below are tips and best practices for when implementing an entity in Java (or any other +JVM language). + +How to store entity state: + +* Config keys and values are persisted. +* Store an entity's runtime state as attributes. +* Don't store state in arbitrary fields - the field will not be persisted (this is a design + decision, because Brooklyn cannot intercept the field being written to, so cannot know + when to persist). +* Don't just modify the retrieved attribute value (e.g. `getAttribute(MY_LIST).add("a")` is bad). + The value may not be persisted unless setAttribute() is called. +* For special cases, it is possible to call `entity.requestPerist()` which will trigger + asynchronous persistence of the entity. +* Overriding (and customizing) of `getRebindSupport()` is discouraged - this will change + in a future version. + + +How to store policy/enricher/location state: + +* Store values as config keys where applicable. +* Unfortunately these (currently) do not have attributes. Normally the state of a policy + or enricher is transient - on rebind it starts afresh, for example with monitoring the + performance or health metrics rather than relying on the persisted values. +* For special cases, you can annotate a field with `@SetFromFlag` for it be persisted. + When you call `requestPersist()` then values of these fields will be scheduled to be + persisted. *Warning: the `@SetFromFlag` functionality may change in future versions.* + +Persistable state: + +* Ensure values can be serialized. This (currently) uses xstream, which means it does not + need to implement `Serializable`. +* Always use static (or top-level) classes. Otherwise it will try to also persist the outer + instance! +* Any reference to an entity or location will be automatically swapped out for marker, and + re-injected with the new entity/location instance on rebind. The same applies for policies, + enrichers, feeds, catalog items and `ManagementContext`. + +Behaviour on rebind: + +* By extending `SoftwareProcess`, entities get a lot of the rebind logic for free. For + example, the default `rebind()` method will call `connectSensors()`. + See [`SoftwareProcess` Lifecycle]({{site.path.guide}}/java/entities.html#SoftwareProcess-lifecycle) + for more details. +* If necessary, implement rebind. The `entity.rebind()` is called automatically by the + Brooklyn framework on rebind, after configuring the entity's config/attributes but before + the entity is managed. + Note that `init()` will not be called on rebind. +* Feeds will be persisted if and only if `entity.addFeed(...)` was called. Otherwise the + feed needs to be re-registered on rebind. *Warning: this behaviour may change in future version.* +* All functions/predicates used with persisted feeds must themselves be persistable - + use of anonymous inner classes is strongly discouraged. +* Subscriptions (e.g. from calls to `subscribe(...)` for sensor events) are not persisted. + They must be re-registered on rebind. *Warning: this behaviour may change in future version.* + +Below are tips to make backwards-compatibility easier for persisted state: + +* Never use anonymous inner classes - even in static contexts. The auto-generated class names + are brittle, making backwards compatibility harder. +* Always use sensible field names (and use `transient` whenever you don't want it persisted). + The field names are part of the persisted state. +* Consider using Value Objects for persisted values. This can give clearer separation of + responsibilities in your code, and clearer control of what fields are being persisted. +* Consider writing transformers to handle backwards-incompatible code changes. + Brooklyn supports applying transformations to the persisted state, which can be done as + part of an upgrade process. + + +Persisted State Backup +---------------------- + +### File system backup + +When using the file system it is important to ensure it is backed up regularly. + +One could use `rsync` to regularly backup the contents to another server. + +It is also recommended to periodically create a complete archive of the state. +A simple mechanism is to run a CRON job periodically (e.g. every 30 minutes) that creates an +archive of the persistence directory, and uploads that to a backup +facility (e.g. to S3). + +Optionally, to avoid excessive load on the Brooklyn server, the archive-generation could be done +on another "data" server. This could get a copy of the data via an `rsync` job. + +An example script to be invoked by CRON is shown below: + + DATE=`date "+%Y%m%d.%H%M.%S"` + BACKUP_FILENAME=/path/to/archives/back-${DATE}.tar.gz + DATA_DIR=/path/to/base/dir/data + + tar --exclude '*/backups/*' -czvf $BACKUP_FILENAME $DATA_DIR + # For s3cmd installation see http://s3tools.org/repositories + s3cmd put $BACKUP_FILENAME s3://mybackupbucket + rm $BACKUP_FILENAME + + +### Object store backup + +Object Stores will normally handle replication. However, many such object stores do not handle +versioning (i.e. to allow access to an old version, if an object has been incorrectly changed or +deleted). + +The state can be downloaded periodically from the object store, archived and backed up. + +An example script to be invoked by CRON is shown below: + + DATE=`date "+%Y%m%d.%H%M.%S"` + BACKUP_FILENAME=/path/to/archives/back-${DATE}.tar.gz + TEMP_DATA_DIR=/path/to/tempdir + + amp copy-state \ + --persistenceLocation named:my-persistence-location \ + --persistenceDir /path/to/bucket \ + --destinationDir $TEMP_DATA_DIR + + tar --exclude '*/backups/*' -czvf $BACKUP_FILENAME $TEMP_DATA_DIR + # For s3cmd installation see http://s3tools.org/repositories + s3cmd put $BACKUP_FILENAME s3://mybackupbucket + rm $BACKUP_FILENAME + rm -r $TEMP_DATA_DIR http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/production-installation.md ---------------------------------------------------------------------- diff --git a/guide/ops/production-installation.md b/guide/ops/production-installation.md new file mode 100644 index 0000000..b0110d5 --- /dev/null +++ b/guide/ops/production-installation.md @@ -0,0 +1,103 @@ +--- +layout: website-normal +title: Production Installation +--- + +{% include fields.md %} + +To install Apache Brooklyn on a production server: + +1. [Set up the prerequisites](#prerequisites) +1. [Download Apache Brooklyn](#download) +1. [Configuring brooklyn.properties](#configuring-properties) +1. [Configuring default.catalog.bom](#configuring-catalog) +1. [Test the installation](#confirm) + +This guide covers the basics. You may also wish to configure: + +* [Logging](logging.md) +* [Persistence](persistence/) +* [High availability](high-availability.md) + + +### <a id="prerequisites"></a>Set up the Prerequisites + +Check that the server meets the [requirements](requirements.md). +Then configure the server as follows: + +* install Java JRE or JDK (version 7 or later) +* install an [SSH key]({{ site.path.guide }}/ops/locations/ssh-keys.html), if not available +* enable [passwordless ssh login]({{ site.path.guide }}/ops/locations/ssh-keys.html) +* create a `~/.brooklyn` directory on the host with `$ mkdir ~/.brooklyn` +* check your `iptables` or other firewall service, making sure that incoming connections on port 8443 is not blocked +* check that the [linux kernel entropy]({{ site.path.website }}/documentation/increase-entropy.html) is sufficient + + +### <a id="download"></a>Download Apache Brooklyn + +Download Brooklyn and obtain a binary build as described on [the download page]({{site.path.website}}/download/). + +{% if brooklyn_version contains 'SNAPSHOT' %} +Expand the `tar.gz` archive (note: as this is a -SNAPSHOT version, your filename will be slightly different): +{% else %} +Expand the `tar.gz` archive: +{% endif %} + +{% if brooklyn_version contains 'SNAPSHOT' %} +{% highlight bash %} +% tar -zxf apache-brooklyn-dist-{{ site.brooklyn-stable-version }}-timestamp-dist.tar.gz +{% endhighlight %} +{% else %} +{% highlight bash %} +% tar -zxf apache-brooklyn-{{ site.brooklyn-stable-version }}-dist.tar.gz +{% endhighlight %} +{% endif %} + +This will create a `apache-brooklyn-{{ site.brooklyn-stable-version }}` folder. + +Let's setup some paths for easy commands. + +{% highlight bash %} +% cd apache-brooklyn-{{ site.brooklyn-stable-version }} +% BROOKLYN_DIR="$(pwd)" +% export PATH=$PATH:$BROOKLYN_DIR/bin/ +{% endhighlight %} + + +### <a id="configuring-properties"></a>Configuring brooklyn.properties + +Set up `brooklyn.properties` as described [here](brooklyn_properties.md): + +* Configure the users who should have access +* Turn on HTTPS +* Supply credentials for any pre-defined clouds + +It may be useful to use the following script to install an initial `brooklyn.properties`: + +{% highlight bash %} +% mkdir -p ~/.brooklyn +% wget -O ~/.brooklyn/brooklyn.properties {{brooklyn_properties_url_live}} +% chmod 600 ~/.brooklyn/brooklyn.properties +{% endhighlight %} + + +### <a id="configuring-catalog"></a>Configuring the Catalog + +By default Brooklyn loads the catalog of available application components and services from +`default.catalog.bom` on the classpath. The initial catalog is in `conf/brooklyn/` in the dist. +If you have a preferred catalog, simply replace that file. + +[More information on the catalog is available here.](catalog/) + + +### <a id="confirm"></a>Confirm Installation + +Launch Brooklyn in a disconnected session so it will remain running after you have logged out: + +{% highlight bash %} +% nohup bin/brooklyn launch > /dev/null 2&>1 & +{% endhighlight %} + +Apache Brooklyn should now be running on port 8081 (or other port if so specified). + + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/6e86cb02/guide/ops/requirements.md ---------------------------------------------------------------------- diff --git a/guide/ops/requirements.md b/guide/ops/requirements.md new file mode 100644 index 0000000..634e4e6 --- /dev/null +++ b/guide/ops/requirements.md @@ -0,0 +1,70 @@ +--- +title: Requirements +layout: website-normal +--- + +## Server Specification + +The size of server required by Brooklyn depends on the amount of activity. This includes: + +* the number of entities/VMs being managed +* the number of VMs being deployed concurrently +* the amount of management and monitoring required per entity + +For dev/test or when there are only a handful of VMs being managed, a small VM is sufficient. +For example, an AWS m3.medium with one vCPU, 3.75GiB RAM and 4GB disk. + +For larger production uses, a more appropriate machine spec would be two or more cores, +at least 8GB RAM and 100GB disk. The disk is just for logs, a small amount of persisted state, and +any binaries for custom blueprints/integrations. + + +## Supported Operating Systems + +The recommended operating system is CentOS 6.x or RedHat 6.x. + +Brooklyn has also been tested on Ubuntu 12.04 and OS X. + + +## Software Requirements + +Brooklyn requires Java (JRE or JDK) minimum version 1.7. +OpenJDK is recommended. Brooklyn has also been tested on IBM J9 and Oracle's JVM. + + +## Configuration Requirements + +### Ports + +The ports used by Brooklyn are: + +* 8443 for https, to expose the web-console and REST api. +* 8081 for http, to expose the web-console and REST api. + +Whether to use https rather than http is configurable using the CLI option `--https`; +the port to use is configurable using the CLI option `--port <port>`. + +To enable remote Brooklyn access, ensure these ports are open in the firewall. +For example, to open port 8443 in iptables, ues the command: + + /sbin/iptables -I INPUT -p TCP --dport 8443 -j ACCEPT + + +### Locale + +Brooklyn expects a sensible set of locale information and time zones to be available; +without this, some time-and-date handling may be surprising. + +Brooklyn parses and reports times according to the time zone set at the server. +If Brooklyn is targetting geographically distributed users, +it is normally recommended that the server's time zone be set to UTC. + + +### User Setup + +It is normally recommended that Brooklyn run as a non-root user with keys installed to `~/.ssh/id_rsa{,.pub}`. + + +### Linux Kernel Entropy + +Check that the [linux kernel entropy]({{ site.path.website }}/documentation/increase-entropy.html) is sufficient.
