Merged conflicts from #19 & order bug
Project: http://git-wip-us.apache.org/repos/asf/brooklyn-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/brooklyn-docs/commit/5ee36ffd Tree: http://git-wip-us.apache.org/repos/asf/brooklyn-docs/tree/5ee36ffd Diff: http://git-wip-us.apache.org/repos/asf/brooklyn-docs/diff/5ee36ffd Branch: refs/heads/master Commit: 5ee36ffd47cbfd57433559322ae273092d42d6b3 Parents: f000a31 Author: Duncan Godwin <duncan.god...@cloudsoftcorp.com> Authored: Tue Mar 8 10:14:16 2016 +0000 Committer: Duncan Godwin <duncan.god...@cloudsoftcorp.com> Committed: Tue Mar 8 10:14:16 2016 +0000 ---------------------------------------------------------------------- _plugins/page_structure.rb | 20 ++++- guide/ops/locations/cloud-specific-setup.md | 88 -------------------- guide/ops/locations/clouds.md | 21 +++-- guide/ops/locations/index.md | 2 + .../inheritance-and-named-locations.md | 2 +- guide/ops/locations/localhost.md | 2 +- guide/ops/locations/more-clouds.md | 8 +- guide/ops/locations/more-locations.md | 54 ------------ guide/ops/locations/special-locations.md | 54 ++++++++++++ guide/ops/locations/ssh-keys.md | 2 +- guide/ops/locations/vpc-issues.md | 32 ------- 11 files changed, 95 insertions(+), 190 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/_plugins/page_structure.rb ---------------------------------------------------------------------- diff --git a/_plugins/page_structure.rb b/_plugins/page_structure.rb index 6b3a2a1..d2e419b 100644 --- a/_plugins/page_structure.rb +++ b/_plugins/page_structure.rb @@ -45,6 +45,14 @@ module PageStructureUtils yaml.sort{ |x,y| Gem::Version.new(x.yaml['section_position'].to_s) <=> Gem::Version.new(y.yaml['section_position'].to_s) } end ## + # This goes through the hash looking for the keys for the different types of children + # + def self.getDefiningParameterFromHash(hash) + param_name = hash['path'] + param_name = (param_name == nil ? hash['link'] : param_name) + (param_name == nil ? hash['section'] : param_name) + end + ## # Sorts a list of yaml children, if there's no numbering, use the YAML order to create a numbering # NOTE: doesn't alter the returned object as that seemed to break things downstream # @@ -57,12 +65,14 @@ module PageStructureUtils if i.instance_of? String position[i] = $major+"."+$minor.to_s else + # get the key for this type of child + defining_param = getDefiningParameterFromHash(i) if i['section_position'] == nil - position[i['path']] = $major+"."+$minor.to_s + position[defining_param] = $major+"."+$minor.to_s $minor += 1 else # Store any major, start incrementing minor - position[i['path']] = i['section_position'].to_s + position[defining_param] = i['section_position'].to_s $major = i['section_position'].to_s $minor = 1 end @@ -75,12 +85,14 @@ module PageStructureUtils if x.instance_of? String $pos_x = position[x] else - $pos_x = position[x['path']] + defining_param = getDefiningParameterFromHash(x) + $pos_x = position[defining_param] end if y.instance_of? String $pos_y = position[y] else - $pos_y = position[y['path']] + defining_param = getDefiningParameterFromHash(y) + $pos_y = position[defining_param] end Gem::Version.new($pos_x.to_s) <=> Gem::Version.new($pos_y.to_s) } http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/cloud-specific-setup.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/cloud-specific-setup.md b/guide/ops/locations/cloud-specific-setup.md deleted file mode 100644 index 56a701a..0000000 --- a/guide/ops/locations/cloud-specific-setup.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -section: Cloud Specific Setup -section_type: inline -section_position: 1.1 ---- - -### Cloud Specific Setup - -To connect to a Cloud, Brooklyn requires appropriate credentials. These comprise the "identity" and -"credential" in Brooklyn terminology. - -For private clouds (and for some clouds being targeted using a standard API), the "endpoint" -must also be specified, which is the cloud's URL. - -The [jclouds guides](https://jclouds.apache.org/guides) includes documentation on configuring -different clouds. - - -#### AWS - -##### Credentials - -AWS has an "access key" and a "secret key", which correspond to Brooklyn's identity and credential -respectively. - -These keys are the way for any programmatic mechanism to access the AWS API. - -To generate an access key and a secret key, see [jclouds instructions](http://jclouds.apache.org/guides/aws) -and [AWS IAM instructions](http://docs.aws.amazon.com/IAM/latest/UserGuide/ManagingCredentials.html). - -An example of the expected format is shown below: - - brooklyn.location.jclouds.aws-ec2.identity=ABCDEFGHIJKLMNOPQRST - brooklyn.location.jclouds.aws-ec2.credential=abcdefghijklmnopqrstu+vwxyzabcdefghijklm - - -##### Tidying up after jclouds - -Security groups are not always deleted by jclouds. This is due to a limitation in AWS (see -https://issues.apache.org/jira/browse/JCLOUDS-207). In brief, AWS prevents the security group -being deleted until there are no VMs using it. However, there is eventual consistency for -recording which VMs still reference those security groups. After deleting the VM, it can sometimes -take several minutes before the security group can be deleted. jclouds retries for 3 seconds, but -does not block for longer. - -There is utility written by Cloudsoft for deleting these unused resources: -http://www.cloudsoftcorp.com/blog/2013/03/tidying-up-after-jclouds. - - -#### Google Compute Engine - -##### Credentials - -Google Compute Engine (GCE) uses a service account e-mail address for the identity, and a private key -as the credential. - -To obtain these from GCE, see the [jclouds instructions](https://jclouds.apache.org/guides/google). - -An example of the expected format is shown below (note the credential is one long line, -with `\n` to represent the new line characters): - - brooklyn.location.jclouds.google-compute-engine.identity=123456789...@developer.gserviceaccount.com - brooklyn.location.jclouds.google-compute-engine.credential=-----BEGIN RSA PRIVATE KEY-----\nabcdefghijklmnopqrstuvwxyznabcdefghijk/lmnopqrstuvwxyzabcdefghij\nabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghij+lm\nnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklm\nnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxy\nzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijk\nlmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvw\nxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghi\njklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstu\nvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefg\nhijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrs\ntuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcde\nfghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvw\n-----END RSA PRIVATE KEY----- - - -##### Quotas - -GCE accounts can have low default [quotas](https://cloud.google.com/compute/docs/resource-quotas). - -It is easy to requesta quota increase by submitting a [quota increase form](https://support.google.com/cloud/answer/6075746?hl=en). - - -##### Networks - -GCE accounts often have a limit to the number of networks that can be created. One work around -is to manually create a network with the required open ports, and to refer to that named network -in Brooklyn's location configuration. - -To create a network, see [GCE network instructions](https://cloud.google.com/compute/docs/networking#networks_1). - -For example, for dev/demo purposes an "everything" network could be created that opens all ports. - -| Name | everything | -| Description | opens all tcp ports | -| Source IP Ranges | 0.0.0.0/0 | -| Allowed protocols and ports | tcp:0-65535 and udp:0-65535 | - - http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/clouds.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/clouds.md b/guide/ops/locations/clouds.md index 90f9eb4..cac9083 100644 --- a/guide/ops/locations/clouds.md +++ b/guide/ops/locations/clouds.md @@ -1,7 +1,7 @@ --- section: Clouds section_type: inline -section_position: 1.0 +section_position: 1 --- ### Clouds @@ -37,8 +37,12 @@ brooklyn.location.jclouds.aws-ec2.credential=s3cr3tsq1rr3ls3cr3tsq1rr3ls3cr3tsq1 And in this case you can reference the location in YAML with `location: jclouds:aws-ec2`. -The Getting Started [template brooklyn.properties]({{ site.path.guide }}/start/brooklyn.properties) contains more examples -of configuring cloud endpoints, including information on credential types used in different clouds. +Brooklyn irons out many of the differences between clouds so that blueprints run similarly +in a wide range of locations, including setting up access and configuring images and machine specs. +The configuration options are described in more detail below. + +In some cases, cloud providers have special features or unusual requirements. +These are outlined in **[More Details for Specific Clouds](more-clouds.html)**. #### OS Initial Login and Setup @@ -65,7 +69,8 @@ Following a successful logon, Brooklyn performs the following steps to configure 1. install the local user's `~/.ssh/id_rsa.pub` as an `authorized_keys` on the new machine, to make it easy for the operator to `ssh` in - (override with `privateKeyFile`; or if there is no `id_{r,d}sa{,.pub}` an ad hoc keypair will be generated; + (override with `privateKeyFile`; or if there is no `id_{r,d}sa{,.pub}` an ad hoc keypair will be generated + for the regular Brooklyn user; if there is a passphrase on the key, this must be supplied) 1. give `sudo` access to the newly created user (override with `grantUserSudo: false`) @@ -96,8 +101,10 @@ For more keys and more detail on the keys below, see - Specific Security Groups can be specified using `securityGroups`, as a list of strings (the existing security group names), or `inboundPorts` can be set, as a list of numeric ports (selected clouds only) -- A specific existing key pair known at the cloud to use can be specified with `keyPair` - (selected clouds only) +- Where a key pair is registered with a target cloud for logging in to machines, + Brooklyn can be configured to request this when provisioning VMs by setting `keyPair` (selected clouds only). + Note that if this `keyPair` does not correspond your default `~/.ssh/id_rsa`, you must typically + also specify the corresponding `loginUser.privateKeyFile` as a file or URL accessible from Brooklyn. - A specific VM name (often the hostname) base to be used can be specified by setting `groupId`. By default, this name is constructed based on the entity which is creating it, @@ -187,7 +194,7 @@ For more keys and more detail on the keys below, see This setting prevents scripts executed on the VMs from being deleted on completion. Note that some scripts run periodically so this can eventually fill a disk; it should only be used for dev/test. -###### Custom template options +###### Custom Template Options jclouds supports many additional options for configuring how a virtual machine is created and deployed, many of which are for cloud-specific features and enhancements. Brooklyn supports some of these, but if what you are looking for is http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/index.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/index.md b/guide/ops/locations/index.md index dc0c1dc..29a80e4 100644 --- a/guide/ops/locations/index.md +++ b/guide/ops/locations/index.md @@ -1,6 +1,8 @@ --- title: Locations layout: website-normal +children: +- { path: location-customizers.md, section_position: 8 } check_directory_for_children: true --- http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/inheritance-and-named-locations.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/inheritance-and-named-locations.md b/guide/ops/locations/inheritance-and-named-locations.md index 9854fea..53fe904 100644 --- a/guide/ops/locations/inheritance-and-named-locations.md +++ b/guide/ops/locations/inheritance-and-named-locations.md @@ -2,7 +2,7 @@ section: Inheritance and Named Locations title: Named Locations section_type: inline -section_position: 2 +section_position: 3 --- ### Inheritance and Named Locations http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/localhost.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/localhost.md b/guide/ops/locations/localhost.md index 86ef5b5..27d19bf 100644 --- a/guide/ops/locations/localhost.md +++ b/guide/ops/locations/localhost.md @@ -1,6 +1,6 @@ --- section: Localhost -section_position: 3 +section_position: 6 section_type: inline --- http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/more-clouds.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/more-clouds.md b/guide/ops/locations/more-clouds.md index 5b776df..8e95625 100644 --- a/guide/ops/locations/more-clouds.md +++ b/guide/ops/locations/more-clouds.md @@ -1,8 +1,12 @@ --- -title: More Details on Specific Clouds -layout: website-normal +section: More Details on Specific Clouds +title: More on Clouds +section_type: inline +section_position: 2 --- +### More Details on Specific Clouds + To connect to a Cloud, Brooklyn requires appropriate credentials. These comprise the `identity` and `credential` in Brooklyn terminology. http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/more-locations.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/more-locations.md b/guide/ops/locations/more-locations.md deleted file mode 100644 index 2ed2c83..0000000 --- a/guide/ops/locations/more-locations.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -section: Specialized Locations -section_position: 5 -section_type: inline ---- - -### Specialized Locations - -Some additional location types are supported for specialized situations: - -#### Single Host - -The spec `host`, taking a string argument (the address) or a map (`host`, `user`, `password`, etc.), -provides a convenient syntax when specifying a single host. -For example: - -{% highlight yaml %} -services: -- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server - location: - host: 192.168.0.1 -{% endhighlight %} - -Or, in `brooklyn.properties`, set `brooklyn.location.named.host1=host:(192.168.0.1)`. - - -#### The Multi Location - -The spec `multi` allows multiple locations, specified as `targets`, -to be combined and treated as one location. -When the first target is full, the next is tried, and so on: - -{% highlight yaml %} -location: - multi: - targets: - - byon:(hosts=192.168.0.1) - - jclouds:aws-ec2: - identity: acct1 - - jclouds:aws-ec2: - identity: acct2 -{% endhighlight %} - -The example above provisions the first node to `192.168.0.1`, -then it provisions into `acct1` at Amazon if possible, -and then to `acct2`. - - - -#### The Server Pool - -The {% include java_link.html class_name="ServerPool" package_path="org/apache/brooklyn/entity/machine/pool" project_subpath="software/base" %} -entity type allows defining an entity which becomes available as a location. - http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/special-locations.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/special-locations.md b/guide/ops/locations/special-locations.md new file mode 100644 index 0000000..8f19fa4 --- /dev/null +++ b/guide/ops/locations/special-locations.md @@ -0,0 +1,54 @@ +--- +section: Specialized Locations +section_position: 7 +section_type: inline +--- + +### Specialized Locations + +Some additional location types are supported for specialized situations: + +#### Single Host + +The spec `host`, taking a string argument (the address) or a map (`host`, `user`, `password`, etc.), +provides a convenient syntax when specifying a single host. +For example: + +{% highlight yaml %} +services: +- type: org.apache.brooklyn.entity.webapp.jboss.JBoss7Server + location: + host: 192.168.0.1 +{% endhighlight %} + +Or, in `brooklyn.properties`, set `brooklyn.location.named.host1=host:(192.168.0.1)`. + + +#### The Multi Location + +The spec `multi` allows multiple locations, specified as `targets`, +to be combined and treated as one location. +When the first target is full, the next is tried, and so on: + +{% highlight yaml %} +location: + multi: + targets: + - byon:(hosts=192.168.0.1) + - jclouds:aws-ec2: + identity: acct1 + - jclouds:aws-ec2: + identity: acct2 +{% endhighlight %} + +The example above provisions the first node to `192.168.0.1`, +then it provisions into `acct1` at Amazon if possible, +and then to `acct2`. + + + +#### The Server Pool + +The {% include java_link.html class_name="ServerPool" package_path="org/apache/brooklyn/entity/machine/pool" project_subpath="software/base" %} +entity type allows defining an entity which becomes available as a location. + http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/ssh-keys.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/ssh-keys.md b/guide/ops/locations/ssh-keys.md index e49d031..03ee84e 100644 --- a/guide/ops/locations/ssh-keys.md +++ b/guide/ops/locations/ssh-keys.md @@ -1,6 +1,6 @@ --- section: SSH Keys -section_position: 3.1 +section_position: 5 section_type: inline --- http://git-wip-us.apache.org/repos/asf/brooklyn-docs/blob/5ee36ffd/guide/ops/locations/vpc-issues.md ---------------------------------------------------------------------- diff --git a/guide/ops/locations/vpc-issues.md b/guide/ops/locations/vpc-issues.md deleted file mode 100644 index b2b362b..0000000 --- a/guide/ops/locations/vpc-issues.md +++ /dev/null @@ -1,32 +0,0 @@ - -### AWS VPC issues which may affect users with older AWS accounts - -AWS now has different default behaviour depending on the age of your AWS account and whether you used the target region before, or during, 2013. -In this case VM provisioning may fail with an error like: - -{% highlight text %} - -Detected that your EC2 account is a legacy 'classic' account, but the recommended instance type requires VPC. -You can specify the 'eu-central-1' region to avoid this problem, or you can specify a classic-compatible instance type, -or you can specify a subnet to use with 'networkName' -taking care that the subnet auto-assigns public IP's and allows ingress on all ports, -as Brooklyn does not currently configure security groups for non-default VPC's; -or setting up Brooklyn to be in the subnet or have a jump host or other subnet access configuration). -For more information on VPC vs classic see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-vpc.html. - -{% endhighlight %} - -Specifically, there are issues with the certain AMIs and instance types. If these are specified or a recommended -by brooklyn then you may see the above error. There are a few options for fixing this: - -- specify a different region which does not support EC2-classic. - You can check this on the AWS console under "Supported Platforms. - Frankfurt (eu-central-1) is guaranteed to be VPC only. - -- specify an instance type that is compatible with ec2-classic. - Instance types C4, M4, T2 are only supported in VPC so should not be used. - This is described [here](index.html#vm-creation) - -- create a subnet to use with the instance. Ensure that the subnet is set to auto-assign public IPs - and allows ingress on all ports. Brooklyn cannot currently do this for you. - Use the networkName parameter to specify this value in your blueprint. \ No newline at end of file
