Re: [openstack-dev] [heat] Sofware Config progress
We work with images provided by vendors over which we do not always have
control. So we are considering the cases where vendor image does not come
installed with cloud-init. Is there a way to support heat software config
in such scenarios?
Thanks
Susaant
On Mon, Jan 6, 2014 at 4:47 PM, Steve Baker sba...@redhat.com wrote:
On 07/01/14 06:25, Susaant Kondapaneni wrote:
Hi Steve,
I am trying to understand the software config implementation. Can you
clarify the following:
i. To use Software config and deploy in a template, instance resource
MUST always be accompanied by user_data. User_data should specify how to
bootstrap CM tool and signal it. Is that correct?
Yes, currently the user_data contains cfn-init formatted metadata which
tells os-collect-config how to poll for config changes. What happens when
new config is fetched depends on the os-apply-config templates and
os-refresh-config scripts which are already on that image (or set up with
cloud-init).
ii. Supposing we were to use images which do not have cloud-init
packaged in them, (and a custom CM tool that won't require bootstrapping on
the instance itself), can we still use software config and deploy resources
to deploy software on such instances?
Currently os-collect-config is more of a requirement than cloud-init,
but as Clint said cloud-init does a good job of boot config so you'll need
to elaborate on why you don't want to use it.
iii. If ii. were possible who would signal the deployment resource to
indicate that the instance is ready for the deployment?
os-collect-config polls for the deployment data, and triggers the
resulting deployment/config changes. One day this may be performed by a
different agent like the unified agent that has been discussed. Currently
os-collect-collect polls via a heat-api-cfn metadata call. This too may be
done in any number of ways in the future such as messaging or long-polling.
So you *could* consume the supplied user_data to know what to poll for
subsequent config changes without cloud-init or os-collect-config, but you
would have to describe what you're doing in detail for us to know if that
sounds like a good idea.
Thanks
Susaant
On Fri, Dec 13, 2013 at 3:46 PM, Steve Baker sba...@redhat.com wrote:
I've been working on a POC in heat for resources which perform software
configuration, with the aim of implementing this spec
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec
The code to date is here:
https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z
What would be helpful now is reviews which give the architectural
approach enough of a blessing to justify fleshing this POC out into a ready
to merge changeset.
Currently it is possible to:
- create templates containing OS::Heat::SoftwareConfig and
OS::Heat::SoftwareDeployment resources
- deploy configs to OS::Nova::Server, where the deployment resource
remains in an IN_PROGRESS state until it is signalled with the output values
- write configs which execute shell scripts and report back with output
values that other resources can have access to.
What follows is an overview of the architecture and implementation to
help with your reviews.
REST API
Like many heat resources, OS::Heat::SoftwareConfig and
OS::Heat::SoftwareDeployment are backed by real resources that are
invoked via a REST API. However in this case, the API that is called is
heat itself.
The REST API for these resources really just act as structured storage
for config and deployments, and the entities are managed via the REST paths
/{tenant_id}/software_configs and /{tenant_id}/software_deployments:
https://review.openstack.org/#/c/58878/7/heat/api/openstack/v1/__init__.py
https://review.openstack.org/#/c/58878/
RPC layer of REST API:
https://review.openstack.org/#/c/58877/
DB layer of REST API:
https://review.openstack.org/#/c/58876
heatclient lib access to REST API:
https://review.openstack.org/#/c/58885
This data could be stored in a less structured datastore like swift, but
this API has a couple of important implementation details which I think
justify it existing:
- SoftwareConfig resources are immutable once created. There is no update
API to modify an existing config. This gives confidence that a config can
have a long lifecycle without changing, and a certainty of what exactly is
deployed on a server with a given config.
- Fetching all the deployments and configs for a given server is an
operation done repeatedly throughout the lifecycle of the stack, so is
optimized to be able to do in a single operation. This is called by using
the deployments index API call,
/{tenant_id}/software_deployments?server_id=server_id. The resulting list
of deployments include the their associated config data[1].
OS::Heat::SoftwareConfig resource
=
OS::Heat::SoftwareConfig can be used directly in a template, but it may
Re: [openstack-dev] [heat] Sofware Config progress
Hi Steve,
I am trying to understand the software config implementation. Can you
clarify the following:
i. To use Software config and deploy in a template, instance resource MUST
always be accompanied by user_data. User_data should specify how to
bootstrap CM tool and signal it. Is that correct?
ii. Supposing we were to use images which do not have cloud-init packaged
in them, (and a custom CM tool that won't require bootstrapping on the
instance itself), can we still use software config and deploy resources to
deploy software on such instances?
iii. If ii. were possible who would signal the deployment resource to
indicate that the instance is ready for the deployment?
Thanks
Susaant
On Fri, Dec 13, 2013 at 3:46 PM, Steve Baker sba...@redhat.com wrote:
I've been working on a POC in heat for resources which perform software
configuration, with the aim of implementing this spec
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec
The code to date is here:
https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z
What would be helpful now is reviews which give the architectural approach
enough of a blessing to justify fleshing this POC out into a ready to merge
changeset.
Currently it is possible to:
- create templates containing OS::Heat::SoftwareConfig and
OS::Heat::SoftwareDeployment resources
- deploy configs to OS::Nova::Server, where the deployment resource
remains in an IN_PROGRESS state until it is signalled with the output values
- write configs which execute shell scripts and report back with output
values that other resources can have access to.
What follows is an overview of the architecture and implementation to help
with your reviews.
REST API
Like many heat resources, OS::Heat::SoftwareConfig and
OS::Heat::SoftwareDeployment are backed by real resources that are
invoked via a REST API. However in this case, the API that is called is
heat itself.
The REST API for these resources really just act as structured storage for
config and deployments, and the entities are managed via the REST paths
/{tenant_id}/software_configs and /{tenant_id}/software_deployments:
https://review.openstack.org/#/c/58878/7/heat/api/openstack/v1/__init__.py
https://review.openstack.org/#/c/58878/
RPC layer of REST API:
https://review.openstack.org/#/c/58877/
DB layer of REST API:
https://review.openstack.org/#/c/58876
heatclient lib access to REST API:
https://review.openstack.org/#/c/58885
This data could be stored in a less structured datastore like swift, but
this API has a couple of important implementation details which I think
justify it existing:
- SoftwareConfig resources are immutable once created. There is no update
API to modify an existing config. This gives confidence that a config can
have a long lifecycle without changing, and a certainty of what exactly is
deployed on a server with a given config.
- Fetching all the deployments and configs for a given server is an
operation done repeatedly throughout the lifecycle of the stack, so is
optimized to be able to do in a single operation. This is called by using
the deployments index API call,
/{tenant_id}/software_deployments?server_id=server_id. The resulting list
of deployments include the their associated config data[1].
OS::Heat::SoftwareConfig resource
=
OS::Heat::SoftwareConfig can be used directly in a template, but it may
end be more frequently used in a resource provider template which provides
a resource aimed at a particular configuration management tool.
http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareConfig
The contents of the config property will depend on the CM tool being used,
but at least one value in the config map will be the actual script that the
CM tool invokes. An inputs and outputs schema is also defined here. The
group property is used when the deployments data is actually delivered to
the server (more on that later).
Since a config is immutable, any changes to a OS::Heat::SoftwareConfig on
stack update result in replacement.
OS::Heat::SoftwareDeployment resource
=
OS::Heat::SoftwareDeployment joins a OS::Heat::SoftwareConfig resource
with a OS::Nova::Server resource. It allows server-specific input values to
be specified that map to the OS::Heat::SoftwareConfig inputs schema. Output
values that are signaled to the deployment resource are exposed as resource
attributes, using the names specified in the outputs schema. The
OS::Heat::SoftwareDeployment resource remains in an IN_PROGRESS state until
it receives a signal (containing any outputs) from the server.
http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareDeployment
A deployment has its own actions and statuses that
