Joshua Powers has proposed merging ~powersj/cloud-init:docs/reorganize into cloud-init:master.
Commit message: docs: reorganize TOC and clean up topics After seeing others come ask for help in IRC I decided to start reorganizing the project documentations. It has grown into useful docs for developers and users alike, however it lacks organization and focus. The first step was to clean up the table of contents (TOC) to provide sections and to take a first attempt at organizing the ordering and renaming topics. This is to help point users asking for help. Next some pages were serving multiple purposes or were not clear, so capabilities.rst, which was largely CLI focused, and moreinfo.rst, were done away with and the cli and faq pages created with their content and filled in. The remaining pages got additional examples and visual cleanups to aid in viewing them. Here is a summary page by page: - analyze.rst: added output examples - availability.rst: more complete list of clouds & distros - boot.rst: clean up formatting, organize sections - cli.rst: formerly capabilities.rst, focused on CLI - datasource.rst: cleanup outupt - faq.rst: formerly moreinfo.rst, focused on help and media links - index.rst: organize TOC and add better description of project Requested reviews: cloud-init commiters (cloud-init-dev) For more details, see: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/372105 -- Your team cloud-init commiters is requested to review the proposed merge of ~powersj/cloud-init:docs/reorganize into cloud-init:master.
diff --git a/doc/rtd/index.rst b/doc/rtd/index.rst index 20a99a3..233c086 100644 --- a/doc/rtd/index.rst +++ b/doc/rtd/index.rst @@ -1,50 +1,77 @@ .. _index: -.. http://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html -.. As suggested at link above for headings use: -.. # with overline, for parts -.. * with overline, for chapters -.. =, for sections -.. -, for subsections -.. ^, for subsubsections -.. “, for paragraphs +cloud-init Documentation +######################## -############# -Documentation -############# +Cloud-init is the *industry standard* multi-distribution method for +cross-platform cloud instance initialization. It is supported across all +major public cloud providers, provisioning systems for private cloud +infrastructure, and bare-metal installations. -.. rubric:: Everything about cloud-init, a set of **python** scripts and - utilities to make your cloud images be all they can be! +Cloud instances are initialized from a disk image and instance data: -******* -Summary -******* +- Cloud metadata +- User data (optional) +- Vendor data (optional) -`Cloud-init`_ is the *defacto* multi-distribution package that handles early -initialization of a cloud instance. +Cloud-init will identify the cloud it is running on during boot, read any +provided metadata from the cloud and initialize the system accordingly. This +may involve setting up the network and storage devices to configuring SSH +access key and many other aspects of a system. Later on the cloud-init will +also parse and any optional user or vendor data that was passed to the +instance. ----- +Getting help +************ + +Having trouble? We would like to help! + +- Try the :ref:`FAQ` – its got answers to some common questions +- Ask a question in the ``#cloud-init`` IRC channel on Freenode +- Join and ask questions on the `cloud-init mailing list <https://launchpad.net/~cloud-init>`_ +- Find a bug? `Report bugs on Launchpad <https://bugs.launchpad.net/cloud-init/+filebug>`_ .. toctree:: - :maxdepth: 2 + :hidden: + :titlesonly: + :caption: Getting Started - topics/capabilities.rst topics/availability.rst + topics/boot.rst + topics/cli.rst + topics/faq.rst + +.. toctree:: + :hidden: + :titlesonly: + :caption: User Data + topics/format.rst - topics/instancedata.rst - topics/dir_layout.rst topics/examples.rst - topics/boot.rst - topics/datasources.rst - topics/logging.rst topics/modules.rst topics/merging.rst - topics/network-config.rst + +.. toctree:: + :hidden: + :titlesonly: + :caption: Instance Data + + topics/instancedata.rst + topics/datasources.rst topics/vendordata.rst - topics/debugging.rst - topics/moreinfo.rst + topics/network-config.rst + +.. toctree:: + :hidden: + :titlesonly: + :caption: Development + topics/hacking.rst + topics/debugging.rst + topics/logging.rst + topics/dir_layout.rst + topics/analyze.rst + topics/docs.rst topics/tests.rst -.. _Cloud-init: https://launchpad.net/cloud-init -.. vi: textwidth=78 +.. vi: textwidth=80 diff --git a/doc/rtd/topics/analyze.rst b/doc/rtd/topics/analyze.rst index 5cf38bd..711152e 100644 --- a/doc/rtd/topics/analyze.rst +++ b/doc/rtd/topics/analyze.rst @@ -1,84 +1,318 @@ -************************* -Cloud-init Analyze Module -************************* - -Overview -======== -The analyze module was added to cloud-init in order to help analyze cloud-init boot time -performance. It is loosely based on systemd-analyze where there are 4 main actions: -show, blame, dump, and boot. - -The 'show' action is similar to 'systemd-analyze critical-chain' which prints a list of units, the -time they started and how long they took. For cloud-init, we have four stages, and within each stage -a number of modules may run depending on configuration. ‘cloudinit-analyze show’ will, for each -boot, print this information and a summary total time, per boot. - -The 'blame' action matches 'systemd-analyze blame' where it prints, in descending order, -the units that took the longest to run. This output is highly useful for examining where cloud-init -is spending its time during execution. - -The 'dump' action simply dumps the cloud-init logs that the analyze module is performing -the analysis on and returns a list of dictionaries that can be consumed for other reporting needs. - -The 'boot' action prints out kernel related timestamps that are not included in any of the -cloud-init logs. There are three different timestamps that are presented to the user: -kernel start, kernel finish boot, and cloud-init start. This was added for additional -clarity into the boot process that cloud-init does not have control over, to aid in debugging of -performance issues related to cloudinit startup or tracking regression. +.. _analyze: + +Analyze +******* + +The analyze subcommand was added to cloud-init in order to help analyze +cloud-init boot time performance. It is loosely based on systemd-analyze where +there are four subcommands: + +- blame +- show +- dump +- boot Usage ===== -Using each of the printing formats is as easy as running one of the following bash commands: + +The analyze command requires one of the four subcommands: .. code-block:: shell-session - cloud-init analyze show - cloud-init analyze blame - cloud-init analyze dump - cloud-init analyze boot + $ cloud-init analyze blame + $ cloud-init analyze show + $ cloud-init analyze dump + $ cloud-init analyze boot + +Availability +============ + +The analyze subcommand is generally available across all distributions with the +exception of Gentoo and FreeBSD. + +Subcommands +=========== + +Blame +----- + +The ``blame`` action matches ``systemd-analyze blame`` where it prints, in +descending order, the units that took the longest to run. This output is +highly useful for examining where cloud-init is spending its time during +execution. + +.. code-block:: shell-session + + $ cloud-init analyze blame + -- Boot Record 01 -- + 00.80300s (init-network/config-growpart) + 00.64300s (init-network/config-resizefs) + 00.62100s (init-network/config-ssh) + 00.57300s (modules-config/config-grub-dpkg) + 00.40300s (init-local/search-NoCloud) + 00.38200s (init-network/config-users-groups) + 00.19800s (modules-config/config-apt-configure) + 00.03700s (modules-final/config-keys-to-console) + 00.02100s (init-network/config-update_etc_hosts) + 00.02100s (init-network/check-cache) + 00.00800s (modules-final/config-ssh-authkey-fingerprints) + 00.00800s (init-network/consume-vendor-data) + 00.00600s (modules-config/config-timezone) + 00.00500s (modules-final/config-final-message) + 00.00400s (init-network/consume-user-data) + 00.00400s (init-network/config-mounts) + 00.00400s (init-network/config-disk_setup) + 00.00400s (init-network/config-bootcmd) + 00.00400s (init-network/activate-datasource) + 00.00300s (init-network/config-update_hostname) + 00.00300s (init-network/config-set_hostname) + 00.00200s (modules-final/config-snappy) + 00.00200s (init-network/config-rsyslog) + 00.00200s (init-network/config-ca-certs) + 00.00200s (init-local/check-cache) + 00.00100s (modules-final/config-scripts-vendor) + 00.00100s (modules-final/config-scripts-per-once) + 00.00100s (modules-final/config-salt-minion) + 00.00100s (modules-final/config-rightscale_userdata) + 00.00100s (modules-final/config-phone-home) + 00.00100s (modules-final/config-package-update-upgrade-install) + 00.00100s (modules-final/config-fan) + 00.00100s (modules-config/config-ubuntu-advantage) + 00.00100s (modules-config/config-ssh-import-id) + 00.00100s (modules-config/config-snap) + 00.00100s (modules-config/config-set-passwords) + 00.00100s (modules-config/config-runcmd) + 00.00100s (modules-config/config-locale) + 00.00100s (modules-config/config-byobu) + 00.00100s (modules-config/config-apt-pipelining) + 00.00100s (init-network/config-write-files) + 00.00100s (init-network/config-seed_random) + 00.00100s (init-network/config-migrator) + 00.00000s (modules-final/config-ubuntu-drivers) + 00.00000s (modules-final/config-scripts-user) + 00.00000s (modules-final/config-scripts-per-instance) + 00.00000s (modules-final/config-scripts-per-boot) + 00.00000s (modules-final/config-puppet) + 00.00000s (modules-final/config-power-state-change) + 00.00000s (modules-final/config-mcollective) + 00.00000s (modules-final/config-lxd) + 00.00000s (modules-final/config-landscape) + 00.00000s (modules-final/config-chef) + 00.00000s (modules-config/config-snap_config) + 00.00000s (modules-config/config-ntp) + 00.00000s (modules-config/config-emit_upstart) + 00.00000s (modules-config/config-disable-ec2-metadata) + 00.00000s (init-network/setup-datasource) + + 1 boot records analyzed + +Show +---- + +The ``show`` action is similar to ``systemd-analyze critical-chain`` which +prints a list of units, the time they started and how long they took. +Cloud-init has four stages and within each stage a number of modules may run +depending on configuration. ``cloudinit-analyze show`` will, for each boot, +print this information and a summary total time, per boot. + +The following is an abbreviated example of the show output: + +.. code-block:: shell-session + + $ cloud-init analyze show + -- Boot Record 01 -- + The total time elapsed since completing an event is printed after the "@" character. + The time the event takes is printed after the "+" character. + + Starting stage: init-local + |``->no cache found @00.01700s +00.00200s + |`->found local data from DataSourceNoCloud @00.11000s +00.40300s + Finished stage: (init-local) 00.94200 seconds + + Starting stage: init-network + |`->restored from cache with run check: DataSourceNoCloud [seed=/dev/sr0][dsmode=net] @04.79500s +00.02100s + |`->setting up datasource @04.88900s +00.00000s + |`->reading and applying user-data @04.90100s +00.00400s + |`->reading and applying vendor-data @04.90500s +00.00800s + |`->activating datasource @04.95200s +00.00400s + Finished stage: (init-network) 02.72100 seconds + + Starting stage: modules-config + |`->config-emit_upstart ran successfully @15.43100s +00.00000s + |`->config-snap ran successfully @15.43100s +00.00100s + ... + |`->config-runcmd ran successfully @16.22300s +00.00100s + |`->config-byobu ran successfully @16.23400s +00.00100s + Finished stage: (modules-config) 00.83500 seconds + + Starting stage: modules-final + |`->config-snappy ran successfully @16.87400s +00.00200s + |`->config-package-update-upgrade-install ran successfully @16.87600s +00.00100s + ... + |`->config-final-message ran successfully @16.93700s +00.00500s + |`->config-power-state-change ran successfully @16.94300s +00.00000s + Finished stage: (modules-final) 00.10300 seconds + + Total Time: 4.60100 seconds + + 1 boot records analyzed + +If additional boot records are detected then they are printed out from oldest +to newest. + +Dump +---- + +The ``dump`` action simply dumps the cloud-init logs that the analyze module +is performing the analysis on and returns a list of dictionaries that can be +consumed for other reporting needs. Each element in the list is a boot entry. + +.. code-block:: shell-session + + $ cloud-init analyze dump + [ + { + "description": "starting search for local datasources", + "event_type": "start", + "name": "init-local", + "origin": "cloudinit", + "timestamp": 1567057578.037 + }, + { + "description": "attempting to read from cache [check]", + "event_type": "start", + "name": "init-local/check-cache", + "origin": "cloudinit", + "timestamp": 1567057578.054 + }, + { + "description": "no cache found", + "event_type": "finish", + "name": "init-local/check-cache", + "origin": "cloudinit", + "result": "SUCCESS", + "timestamp": 1567057578.056 + }, + { + "description": "searching for local data from DataSourceNoCloud", + "event_type": "start", + "name": "init-local/search-NoCloud", + "origin": "cloudinit", + "timestamp": 1567057578.147 + }, + { + "description": "found local data from DataSourceNoCloud", + "event_type": "finish", + "name": "init-local/search-NoCloud", + "origin": "cloudinit", + "result": "SUCCESS", + "timestamp": 1567057578.55 + }, + { + "description": "searching for local datasources", + "event_type": "finish", + "name": "init-local", + "origin": "cloudinit", + "result": "SUCCESS", + "timestamp": 1567057578.979 + }, + { + "description": "searching for network datasources", + "event_type": "start", + "name": "init-network", + "origin": "cloudinit", + "timestamp": 1567057582.814 + }, + { + "description": "attempting to read from cache [trust]", + "event_type": "start", + "name": "init-network/check-cache", + "origin": "cloudinit", + "timestamp": 1567057582.832 + }, + ... + { + "description": "config-power-state-change ran successfully", + "event_type": "finish", + "name": "modules-final/config-power-state-change", + "origin": "cloudinit", + "result": "SUCCESS", + "timestamp": 1567057594.98 + }, + { + "description": "running modules for final", + "event_type": "finish", + "name": "modules-final", + "origin": "cloudinit", + "result": "SUCCESS", + "timestamp": 1567057594.982 + } + ] + + +Boot +---- + +The ``boot`` action prints out kernel related timestamps that are not included +in any of the cloud-init logs. There are three different timestamps that are +presented to the user: + +- kernel start +- kernel finish boot +- cloud-init start + +This was added for additional clarity into the boot process that cloud-init +does not have control over, to aid in debugging of performance issues related +to cloud-init startup, and tracking regression. + +.. code-block:: shell-session + + $ cloud-init analyze boot + -- Most Recent Boot Record -- + Kernel Started at: 2019-08-29 01:35:37.753790 + Kernel ended boot at: 2019-08-29 01:35:38.807407 + Kernel time to boot (seconds): 1.053617000579834 + Cloud-init activated by systemd at: 2019-08-29 01:35:43.992460 + Time between Kernel end boot and Cloud-init activation (seconds): 5.185053110122681 + Cloud-init start: 2019-08-29 08:35:45.867000 + successful + +Timestamp Gathering +^^^^^^^^^^^^^^^^^^^ + +The following boot related timestamps are gathered on demand when cloud-init +analyze boot runs: -Cloud-init analyze boot Timestamp Gathering -=========================================== -The following boot related timestamps are gathered on demand when cloud-init analyze boot runs: -- Kernel Startup, which is inferred from system uptime -- Kernel Finishes Initialization, which is inferred from systemd UserSpaceMonotonicTimestamp property -- Cloud-init activation, which is inferred from the property InactiveExitTimestamp of the cloud-init -local systemd unit. +- Kernel startup gathered from system uptime +- Kernel finishes initialization from systemd + UserSpaceMonotonicTimestamp property +- Cloud-init activation from the property InactiveExitTimestamp of the + cloud-init local systemd unit -In order to gather the necessary timestamps using systemd, running the commands +In order to gather the necessary timestamps using systemd, running the +commands below will gather the UserspaceTimestamp and InactiveExitTimestamp: .. code-block:: shell-session - systemctl show -p UserspaceTimestampMonotonic - systemctl show cloud-init-local -p InactiveExitTimestampMonotonic + $ systemctl show -p UserspaceTimestampMonotonic + UserspaceTimestampMonotonic=989279 + $ systemctl show cloud-init-local -p InactiveExitTimestampMonotonic + InactiveExitTimestampMonotonic=4493126 -will gather the UserspaceTimestamp and InactiveExitTimestamp. -The UserspaceTimestamp tracks when the init system starts, which is used as an indicator of kernel -finishing initialization. The InactiveExitTimestamp tracks when a particular systemd unit transitions -from the Inactive to Active state, which can be used to mark the beginning of systemd's activation -of cloud-init. +The UserspaceTimestamp tracks when the init system starts, which is used as +an indicator of kernel finishing initialization. The InactiveExitTimestamp +tracks when a particular systemd unit transitions from the Inactive to Active +state, which can be used to mark the beginning of systemd's activation of +cloud-init. -Currently this only works for distros that use systemd as the init process. We will be expanding -support for other distros in the future and this document will be updated accordingly. +Currently this only works for distros that use systemd as the init process. +We will be expanding support for other distros in the future and this document +will be updated accordingly. -If systemd is not present on the system, dmesg is used to attempt to find an event that logs the -beginning of the init system. However, with this method only the first two timestamps are able to be found; -dmesg does not monitor userspace processes, so no cloud-init start timestamps are emitted like when +If systemd is not present on the system, dmesg is used to attempt to find an +event that logs the beginning of the init system. However, with this method +only the first two timestamps are able to be found; dmesg does not monitor +userspace processes, so no cloud-init start timestamps are emitted like when using systemd. -List of Cloud-init analyze boot supported distros -================================================= -- Arch -- CentOS -- Debian -- Fedora -- OpenSuSE -- Red Hat Enterprise Linux -- Ubuntu -- SUSE Linux Enterprise Server -- CoreOS - -List of Cloud-init analyze boot unsupported distros -=================================================== -- FreeBSD -- Gentoo \ No newline at end of file +.. vi: textwidth=80 diff --git a/doc/rtd/topics/availability.rst b/doc/rtd/topics/availability.rst index ef5ae7b..1c775c2 100644 --- a/doc/rtd/topics/availability.rst +++ b/doc/rtd/topics/availability.rst @@ -1,21 +1,58 @@ -************ +.. _availability: + Availability ************ -It is currently installed in the `Ubuntu Cloud Images`_ and also in the official `Ubuntu`_ images available on EC2, Azure, GCE and many other clouds. +Below outlines the current availability of cloud-init across +distributions and clouds, both public and private. + +.. note:: + + If a distribution or cloud does not show up in the list below contact + them and ask for images to be generated using cloud-init! -Versions for other systems can be (or have been) created for the following distributions: +Distributions +============= + +Cloud-init has support across all major Linux distributions and +FreeBSD: - Ubuntu +- SLES/openSUSE +- RHEL/CentOS - Fedora +- Gentoo Linux - Debian -- RHEL -- CentOS -- *and more...* +- ArchLinux +- FreeBSD + +Clouds +====== + +Cloud-init provides support across a wide ranging list of execution +environments in the public cloud: + +- Amazon Web Services +- Microsoft Azure +- Google Cloud Platform +- Oracle Cloud Infrastructure +- Softlayer +- Rackspace Public Cloud +- IBM Cloud +- Digital Ocean +- Bigstep +- Hetzner +- Joyent +- CloudSigma +- Alibaba Cloud +- OVH -So ask your distribution provider where you can obtain an image with it built-in if one is not already available ☺ +Additionally, cloud-init is supported on these private clouds: +- Bare metal installs +- OpenStack +- LXD +- KVM +- Metal-as-a-Service (MAAS) -.. _Ubuntu Cloud Images: http://cloud-images.ubuntu.com/ -.. _Ubuntu: http://www.ubuntu.com/ -.. vi: textwidth=78 +.. vi: textwidth=80 diff --git a/doc/rtd/topics/boot.rst b/doc/rtd/topics/boot.rst index f2976fd..43f8362 100644 --- a/doc/rtd/topics/boot.rst +++ b/doc/rtd/topics/boot.rst @@ -1,43 +1,51 @@ .. _boot_stages: -*********** Boot Stages *********** -In order to be able to provide the functionality that it does, cloud-init -must be integrated into the boot in fairly controlled way. -There are 5 stages. +In order to be able to provide the functionality that it does, cloud-init +must be integrated into the boot in fairly controlled way. There are five +stages to boot: -1. **Generator** -2. **Local** -3. **Network** -4. **Config** -5. **Final** +1. Generator +2. Local +3. Network +4. Config +5. Final Generator ========= + When booting under systemd, a `generator <https://www.freedesktop.org/software/systemd/man/systemd.generator.html>`_ -will run that determines if cloud-init.target should be included in the -boot goals. By default, this generator will enable cloud-init. It will -not enable cloud-init if either: +will run that determines if cloud-init.target should be included in the boot +goals. By default, this generator will enable cloud-init. It will not enable +cloud-init if either: - * A file exists: ``/etc/cloud/cloud-init.disabled`` - * The kernel command line as found in /proc/cmdline contains ``cloud-init=disabled``. - When running in a container, the kernel command line is not honored, but - cloud-init will read an environment variable named ``KERNEL_CMDLINE`` in - its place. + * The file ``/etc/cloud/cloud-init.disabled`` exists + * The kernel command line as found in ``/proc/cmdline`` contains + ``cloud-init=disabled``. When running in a container, the kernel command + line is not honored, but cloud-init will read an environment variable named + ``KERNEL_CMDLINE`` in its place. -This mechanism for disabling at runtime currently only exists in systemd. +Again, these mechanisms for disabling cloud-init at runtime currently only +exist in systemd. Local ===== - * **systemd service**: ``cloud-init-local.service`` - * **runs**: As soon as possible with / mounted read-write. - * **blocks**: as much of boot as possible, *must* block network bringup. - * **modules**: none -The purpose of the local stage is: ++------------------+----------------------------------------------------------+ +| systemd service | ``cloud-init-local.service`` | ++---------+--------+----------------------------------------------------------+ +| runs | as soon as possible with ``/`` mounted read-write | ++---------+--------+----------------------------------------------------------+ +| blocks | as much of boot as possible, *must* block network | ++---------+--------+----------------------------------------------------------+ +| modules | none | ++---------+--------+----------------------------------------------------------+ + +The purpose of the local stage is to: + * locate "local" data sources. * apply networking configuration to the system (including "Fallback") @@ -45,13 +53,13 @@ In most cases, this stage does not do much more than that. It finds the datasource and determines the network configuration to be used. That network configuration can come from: - * the datasource - * fallback: Cloud-init's fallback networking consists of rendering the + * **datasource**: cloud provided network configuration via metadata + * **fallback**: cloud-init's fallback networking consists of rendering the equivalent to "dhcp on eth0", which was historically the most popular - mechanism for network configuration of a guest. - * none. network configuration can be disabled entirely with - config like the following in /etc/cloud/cloud.cfg: - '``network: {config: disabled}``'. + mechanism for network configuration of a guest + * **none**: network configuration can be disabled by writing the file + ``/etc/cloud/cloud.cfg`` with the content: + ``network: {config: disabled}`` If this is an instance's first boot, then the selected network configuration is rendered. This includes clearing of all previous (stale) configuration @@ -73,20 +81,26 @@ that require a network can operate at this stage. Network ======= - * **systemd service**: ``cloud-init.service`` - * **runs**: After local stage and configured networking is up. - * **blocks**: As much of remaining boot as possible. - * **modules**: ``cloud_init_modules`` in **/etc/cloud/cloud.cfg** + ++------------------+----------------------------------------------------------+ +| systemd service | ``cloud-init.service`` | ++---------+--------+----------------------------------------------------------+ +| runs | after local stage and configured networking is up | ++---------+--------+----------------------------------------------------------+ +| blocks | as much of remaining boot as possible | ++---------+--------+----------------------------------------------------------+ +| modules | *cloud_init_modules* in ``/etc/cloud/cloud.cfg`` | ++---------+--------+----------------------------------------------------------+ This stage requires all configured networking to be online, as it will fully process any user-data that is found. Here, processing means: - * retrive any ``#include`` or ``#include-once`` (recursively) including http - * uncompress any compressed content + * retrieve any ``#include`` or ``#include-once`` (recursively) including http + * decompress any compressed content * run any part-handler found. This stage runs the ``disk_setup`` and ``mounts`` modules which may partition -and format disks and configure mount points (such as in /etc/fstab). +and format disks and configure mount points (such as in ``/etc/fstab``). Those modules cannot run earlier as they may receive configuration input from sources only available via network. For example, a user may have provided user-data in a network resource that describes how local mounts @@ -94,30 +108,41 @@ should be done. On some clouds such as Azure, this stage will create filesystems to be mounted, including ones that have stale (previous instance) references in -/etc/fstab. As such, entries /etc/fstab other than those necessary for +``/etc/fstab``. As such, entries ``/etc/fstab`` other than those necessary for cloud-init to run should not be done until after this stage. -A part-handler will run at this stage, as will boothooks including +A part-handler will run at this stage, as will boot-hooks including cloud-config ``bootcmd``. The user of this functionality has to be aware that the system is in the process of booting when their code runs. Config ====== - * **systemd service**: ``cloud-config.service`` - * **runs**: After network stage. - * **blocks**: None. - * **modules**: ``cloud_config_modules`` in **/etc/cloud/cloud.cfg** + ++------------------+----------------------------------------------------------+ +| systemd service | ``cloud-config.service`` | ++---------+--------+----------------------------------------------------------+ +| runs | after network | ++---------+--------+----------------------------------------------------------+ +| blocks | nothing | ++---------+--------+----------------------------------------------------------+ +| modules | *cloud_config_modules* in ``/etc/cloud/cloud.cfg`` | ++---------+--------+----------------------------------------------------------+ This stage runs config modules only. Modules that do not really have an effect on other stages of boot are run here. - Final ===== - * **systemd service**: ``cloud-final.service`` - * **runs**: As final part of boot (traditional "rc.local") - * **blocks**: None. - * **modules**: ``cloud_final_modules`` in **/etc/cloud/cloud.cfg** + ++------------------+----------------------------------------------------------+ +| systemd service | ``cloud-final.service`` | ++---------+--------+----------------------------------------------------------+ +| runs | as final part of boot (traditional "rc.local") | ++---------+--------+----------------------------------------------------------+ +| blocks | nothing | ++---------+--------+----------------------------------------------------------+ +| modules | *cloud_final_modules* in ``/etc/cloud/cloud.cfg`` | ++---------+--------+----------------------------------------------------------+ This stage runs as late in boot as possible. Any scripts that a user is accustomed to running after logging into a system should run correctly here. @@ -127,9 +152,9 @@ Things that run here include * configuration management plugins (puppet, chef, salt-minion) * user-scripts (including ``runcmd``). -For scripts external to cloud-init looking to wait until cloud-init +For scripts external to cloud-init looking to wait until cloud-init is finished, the ``cloud-init status`` subcommand can help block external scripts until cloud-init is done without having to write your own systemd units dependency chains. See :ref:`cli_status` for more info. -.. vi: textwidth=78 +.. vi: textwidth=80 diff --git a/doc/rtd/topics/capabilities.rst b/doc/rtd/topics/capabilities.rst deleted file mode 100644 index 6d85a99..0000000 --- a/doc/rtd/topics/capabilities.rst +++ /dev/null @@ -1,300 +0,0 @@ -.. _capabilities: - -************ -Capabilities -************ - -- Setting a default locale -- Setting an instance hostname -- Generating instance SSH private keys -- Adding SSH keys to a user's ``.ssh/authorized_keys`` so they can log in -- Setting up ephemeral mount points -- Configuring network devices - -User configurability -==================== - -`Cloud-init`_ 's behavior can be configured via user-data. - - User-data can be given by the user at instance launch time. See - :ref:`user_data_formats` for acceptable user-data content. - - -This is done via the ``--user-data`` or ``--user-data-file`` argument to -ec2-run-instances for example. - -* Check your local client's documentation for how to provide a `user-data` - string or `user-data` file to cloud-init on instance creation. - - -Feature detection -================= - -Newer versions of cloud-init may have a list of additional features that they -support. This allows other applications to detect what features the installed -cloud-init supports without having to parse its version number. If present, -this list of features will be located at ``cloudinit.version.FEATURES``. - -Currently defined feature names include: - - - ``NETWORK_CONFIG_V1`` support for v1 networking configuration, - see :ref:`network_config_v1` documentation for examples. - - ``NETWORK_CONFIG_V2`` support for v2 networking configuration, - see :ref:`network_config_v2` documentation for examples. - - -CLI Interface -============= - -The command line documentation is accessible on any cloud-init installed -system: - -.. code-block:: shell-session - - % cloud-init --help - usage: cloud-init [-h] [--version] [--file FILES] - [--debug] [--force] - {init,modules,single,query,dhclient-hook,features,analyze,devel,collect-logs,clean,status} - ... - - optional arguments: - -h, --help show this help message and exit - --version, -v show program's version number and exit - --file FILES, -f FILES - additional yaml configuration files to use - --debug, -d show additional pre-action logging (default: False) - --force force running even if no datasource is found (use at - your own risk) - - Subcommands: - {init,modules,single,query,dhclient-hook,features,analyze,devel,collect-logs,clean,status} - init initializes cloud-init and performs initial modules - modules activates modules using a given configuration key - single run a single module - query Query instance metadata from the command line - dhclient-hook run the dhclient hookto record network info - features list defined features - analyze Devel tool: Analyze cloud-init logs and data - devel Run development tools - collect-logs Collect and tar all cloud-init debug info - clean Remove logs and artifacts so cloud-init can re-run - status Report cloud-init status or wait on completion - - -CLI Subcommand details -====================== - -.. _cli_features: - -cloud-init features -------------------- -Print out each feature supported. If cloud-init does not have the -features subcommand, it also does not support any features described in -this document. - -.. code-block:: shell-session - - % cloud-init features - NETWORK_CONFIG_V1 - NETWORK_CONFIG_V2 - -.. _cli_status: - -cloud-init status ------------------ -Report whether cloud-init is running, done, disabled or errored. Exits -non-zero if an error is detected in cloud-init. - -* **--long**: Detailed status information. -* **--wait**: Block until cloud-init completes. - -.. code-block:: shell-session - - % cloud-init status --long - status: done - time: Wed, 17 Jan 2018 20:41:59 +0000 - detail: - DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net] - - # Cloud-init running still short versus long options - % cloud-init status - status: running - % cloud-init status --long - status: running - time: Fri, 26 Jan 2018 21:39:43 +0000 - detail: - Running in stage: init-local - -.. _cli_collect_logs: - -cloud-init collect-logs ------------------------ -Collect and tar cloud-init generated logs, data files and system -information for triage. This subcommand is integrated with apport. - -**Note**: Ubuntu users can file bugs with `ubuntu-bug cloud-init` to -automaticaly attach these logs to a bug report. - -Logs collected are: - - * /var/log/cloud-init*log - * /run/cloud-init - * cloud-init package version - * dmesg output - * journalctl output - * /var/lib/cloud/instance/user-data.txt - -.. _cli_query: - -cloud-init query ------------------- -Query standardized cloud instance metadata crawled by cloud-init and stored -in ``/run/cloud-init/instance-data.json``. This is a convenience command-line -interface to reference any cached configuration metadata that cloud-init -crawls when booting the instance. See :ref:`instance_metadata` for more info. - -* **--all**: Dump all available instance data as json which can be queried. -* **--instance-data**: Optional path to a different instance-data.json file to - source for queries. -* **--list-keys**: List available query keys from cached instance data. - -.. code-block:: shell-session - - # List all top-level query keys available (includes standardized aliases) - % cloud-init query --list-keys - availability_zone - base64_encoded_keys - cloud_name - ds - instance_id - local_hostname - region - v1 - -* **<varname>**: A dot-delimited variable path into the instance-data.json - object. - -.. code-block:: shell-session - - # Query cloud-init standardized metadata on any cloud - % cloud-init query v1.cloud_name - aws # or openstack, azure, gce etc. - - # Any standardized instance-data under a <v#> key is aliased as a top-level - # key for convenience. - % cloud-init query cloud_name - aws # or openstack, azure, gce etc. - - # Query datasource-specific metadata on EC2 - % cloud-init query ds.meta_data.public_ipv4 - -* **--format** A string that will use jinja-template syntax to render a string - replacing - -.. code-block:: shell-session - - # Generate a custom hostname fqdn based on instance-id, cloud and region - % cloud-init query --format 'custom-{{instance_id}}.{{region}}.{{v1.cloud_name}}.com' - custom-i-0e91f69987f37ec74.us-east-2.aws.com - - -.. note:: - The standardized instance data keys under **v#** are guaranteed not to change - behavior or format. If using top-level convenience aliases for any - standardized instance data keys, the most value (highest **v#**) of that key - name is what is reported as the top-level value. So these aliases act as a - 'latest'. - - -.. _cli_analyze: - -cloud-init analyze ------------------- -Get detailed reports of where cloud-init spends most of its time. See -:ref:`boot_time_analysis` for more info. - -* **blame** Report ordered by most costly operations. -* **dump** Machine-readable JSON dump of all cloud-init tracked events. -* **show** show time-ordered report of the cost of operations during each - boot stage. -* **boot** show timestamps from kernel initialization, kernel finish initialization, and cloud-init start. - -.. _cli_devel: - -cloud-init devel ----------------- -Collection of development tools under active development. These tools will -likely be promoted to top-level subcommands when stable. - - * ``cloud-init devel schema``: A **#cloud-config** format and schema - validator. It accepts a cloud-config yaml file and annotates potential - schema errors locally without the need for deployment. Schema - validation is work in progress and supports a subset of cloud-config - modules. - - * ``cloud-init devel render``: Use cloud-init's jinja template render to - process **#cloud-config** or **custom-scripts**, injecting any variables - from ``/run/cloud-init/instance-data.json``. It accepts a user-data file - containing the jinja template header ``## template: jinja`` and renders - that content with any instance-data.json variables present. - - -.. _cli_clean: - -cloud-init clean ----------------- -Remove cloud-init artifacts from /var/lib/cloud and optionally reboot the -machine to so cloud-init re-runs all stages as it did on first boot. - -* **--logs**: Optionally remove /var/log/cloud-init*log files. -* **--reboot**: Reboot the system after removing artifacts. - -.. _cli_init: - -cloud-init init ---------------- -Generally run by OS init systems to execute cloud-init's stages -*init* and *init-local*. See :ref:`boot_stages` for more info. -Can be run on the commandline, but is generally gated to run only once -due to semaphores in **/var/lib/cloud/instance/sem/** and -**/var/lib/cloud/sem**. - -* **--local**: Run *init-local* stage instead of *init*. - -.. _cli_modules: - -cloud-init modules ------------------- -Generally run by OS init systems to execute *modules:config* and -*modules:final* boot stages. This executes cloud config :ref:`modules` -configured to run in the init, config and final stages. The modules are -declared to run in various boot stages in the file -**/etc/cloud/cloud.cfg** under keys **cloud_init_modules**, -**cloud_init_modules** and **cloud_init_modules**. Can be run on the -commandline, but each module is gated to run only once due to semaphores -in ``/var/lib/cloud/``. - -* **--mode (init|config|final)**: Run *modules:init*, *modules:config* or - *modules:final* cloud-init stages. See :ref:`boot_stages` for more info. - -.. _cli_single: - -cloud-init single ------------------ -Attempt to run a single named cloud config module. The following example -re-runs the cc_set_hostname module ignoring the module default frequency -of once-per-instance: - -* **--name**: The cloud-config module name to run -* **--frequency**: Optionally override the declared module frequency - with one of (always|once-per-instance|once) - -.. code-block:: shell-session - - % cloud-init single --name set_hostname --frequency always - -**Note**: Mileage may vary trying to re-run each cloud-config module, as -some are not idempotent. - -.. _Cloud-init: https://launchpad.net/cloud-init -.. vi: textwidth=78 diff --git a/doc/rtd/topics/cli.rst b/doc/rtd/topics/cli.rst new file mode 100644 index 0000000..e57dc11 --- /dev/null +++ b/doc/rtd/topics/cli.rst @@ -0,0 +1,304 @@ +.. _cli: + +CLI Interface +************* + +For the latest list of subcommands and arguments use cloud-init's ``--help`` +option. This can be used against cloud-init itself or any of its subcommands. + +.. code-block:: shell-session + + $ cloud-init --help + usage: /usr/bin/cloud-init [-h] [--version] [--file FILES] [--debug] [--force] + {init,modules,single,query,dhclient-hook,features,analyze,devel,collect-logs,clean,status} + ... + + optional arguments: + -h, --help show this help message and exit + --version, -v show program's version number and exit + --file FILES, -f FILES + additional yaml configuration files to use + --debug, -d show additional pre-action logging (default: False) + --force force running even if no datasource is found (use at + your own risk) + + Subcommands: + {init,modules,single,query,dhclient-hook,features,analyze,devel,collect-logs,clean,status} + init initializes cloud-init and performs initial modules + modules activates modules using a given configuration key + single run a single module + query Query standardized instance metadata from the command + line. + dhclient-hook Run the dhclient hook to record network info. + features list defined features + analyze Devel tool: Analyze cloud-init logs and data + devel Run development tools + collect-logs Collect and tar all cloud-init debug info + clean Remove logs and artifacts so cloud-init can re-run. + status Report cloud-init status or wait on completion. + +The rest of this document will give an overview of each of the subcommands. + + +.. _cli_analyze: + +analyze +======= + +Get detailed reports of where cloud-init spends its time during the boot +process. For more complete reference see :ref:`analyze`. + +Possible subcommands include: + +* *blame*: report ordered by most costly operations +* *dump*: machine-readable JSON dump of all cloud-init tracked events +* *show*: show time-ordered report of the cost of operations during each + boot stage +* *boot*: show timestamps from kernel initialization, kernel finish + initialization, and cloud-init start + + +.. _cli_clean: + +clean +===== + +Remove cloud-init artifacts from ``/var/lib/cloud`` to simulate a clean +instance. On reboot, cloud-init will re-run all stages as it did on first boot. + +* *\\-\\-logs*: optionally remove all cloud-init log files in ``/var/log/`` +* *\\-\\-reboot*: reboot the system after removing artifacts + + +.. _cli_collect_logs: + +collect-logs +============ + +Collect and tar cloud-init generated logs, data files, and system +information for triage. This subcommand is integrated with apport. + +Logs collected include: + + * ``/var/log/cloud-init.log`` + * ``/var/log/cloud-init-output.log`` + * ``/run/cloud-init`` + * ``/var/lib/cloud/instance/user-data.txt`` + * cloud-init package version + * ``dmesg`` output + * journalctl output + +.. note:: + + Ubuntu users can file bugs with ``ubuntu-bug cloud-init`` to + automatically attach these logs to a bug report + + +.. _cli_devel: + +devel +===== + +Collection of development tools under active development. These tools will +likely be promoted to top-level subcommands when stable. + +Do **NOT** rely on the output of these commands as they can and will change. + +Current subcommands: + + * ``schema``: a **#cloud-config** format and schema + validator. It accepts a cloud-config yaml file and annotates potential + schema errors locally without the need for deployment. Schema + validation is work in progress and supports a subset of cloud-config + modules. + + * ``render``: use cloud-init's jinja template render to + process **#cloud-config** or **custom-scripts**, injecting any variables + from ``/run/cloud-init/instance-data.json``. It accepts a user-data file + containing the jinja template header ``## template: jinja`` and renders + that content with any instance-data.json variables present. + + +.. _cli_features: + +features +======== + +Print out each feature supported. If cloud-init does not have the +features subcommand, it also does not support any features described in +this document. + +.. code-block:: shell-session + + $ cloud-init features + NETWORK_CONFIG_V1 + NETWORK_CONFIG_V2 + + +.. _cli_init: + +init +==== + +Generally run by OS init systems to execute cloud-init's stages +*init* and *init-local*. See :ref:`boot_stages` for more info. +Can be run on the commandline, but is generally gated to run only once +due to semaphores in ``/var/lib/cloud/instance/sem/`` and +``/var/lib/cloud/sem``. + +* *\\-\\-local*: run *init-local* stage instead of *init* + + +.. _cli_modules: + +modules +======= + +Generally run by OS init systems to execute *modules:config* and +*modules:final* boot stages. This executes cloud config :ref:`modules` +configured to run in the init, config and final stages. The modules are +declared to run in various boot stages in the file +``/etc/cloud/cloud.cfg`` under keys: + +* *cloud_init_modules* +* *cloud_config_modules* +* *cloud_init_modules* + +Can be run on the command line, but each module is gated to run only once due +to semaphores in ``/var/lib/cloud/``. + +* *\\-\\-mode [init|config|final]*: run *modules:init*, *modules:config* or + *modules:final* cloud-init stages. See :ref:`boot_stages` for more info. + + +.. _cli_query: + +query +===== + +Query standardized cloud instance metadata crawled by cloud-init and stored +in ``/run/cloud-init/instance-data.json``. This is a convenience command-line +interface to reference any cached configuration metadata that cloud-init +crawls when booting the instance. See :ref:`instance_metadata` for more info. + +* *\\-\\-all*: dump all available instance data as json which can be queried +* *\\-\\-instance-data*: optional path to a different instance-data.json file + to source for queries +* *\\-\\-list-keys*: list available query keys from cached instance data +* *\\-\\-format*: a string that will use jinja-template syntax to render a + string replacing +* *<varname>*: a dot-delimited variable path into the instance-data.json + object + +Below demonstrates how to list all top-level query keys that are standardized +aliases: + +.. code-block:: shell-session + + $ cloud-init query --list-keys + _beta_keys + availability_zone + base64_encoded_keys + cloud_name + ds + instance_id + local_hostname + platform + public_ssh_keys + region + sensitive_keys + subplatform + userdata + v1 + vendordata + +Below demonstrates how to query standardized metadata from clouds: + +.. code-block:: shell-session + + % cloud-init query v1.cloud_name + aws # or openstack, azure, gce etc. + + # Any standardized instance-data under a <v#> key is aliased as a top-level key for convenience. + % cloud-init query cloud_name + aws # or openstack, azure, gce etc. + + # Query datasource-specific metadata on EC2 + % cloud-init query ds.meta_data.public_ipv4 + +.. note:: + + The standardized instance data keys under **v#** are guaranteed not to change + behavior or format. If using top-level convenience aliases for any + standardized instance data keys, the most value (highest **v#**) of that key + name is what is reported as the top-level value. So these aliases act as a + 'latest'. + +This data can then be formatted to generate custom strings or data: + +.. code-block:: shell-session + + # Generate a custom hostname fqdn based on instance-id, cloud and region + % cloud-init query --format 'custom-{{instance_id}}.{{region}}.{{v1.cloud_name}}.com' + custom-i-0e91f69987f37ec74.us-east-2.aws.com + + +.. _cli_single: + +single +====== + +Attempt to run a single named cloud config module. + +* *\\-\\-name*: the cloud-config module name to run +* *\\-\\-frequency*: optionally override the declared module frequency + with one of (always|once-per-instance|once) + +The following example re-runs the cc_set_hostname module ignoring the module +default frequency of once-per-instance: + +.. code-block:: shell-session + + $ cloud-init single --name set_hostname --frequency always + +.. note:: + + Mileage may vary trying to re-run each cloud-config module, as + some are not idempotent. + + +.. _cli_status: + +status +====== + +Report whether cloud-init is running, done, disabled or errored. Exits +non-zero if an error is detected in cloud-init. + +* *\\-\\-long*: detailed status information +* *\\-\\-wait*: block until cloud-init completes + +Below are examples of output when cloud-init is running, showing status and +the currently running modules, as well as when it is done. + +.. code-block:: shell-session + + $ cloud-init status + status: running + + $ cloud-init status --long + status: running + time: Fri, 26 Jan 2018 21:39:43 +0000 + detail: + Running in stage: init-local + + $ cloud-init status + status: done + + $ cloud-init status --long + status: done + time: Wed, 17 Jan 2018 20:41:59 +0000 + detail: + DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net] + +.. vi: textwidth=80 diff --git a/doc/rtd/topics/datasources.rst b/doc/rtd/topics/datasources.rst index 2148cd5..001a969 100644 --- a/doc/rtd/topics/datasources.rst +++ b/doc/rtd/topics/datasources.rst @@ -1,89 +1,57 @@ .. _datasources: -*********** Datasources *********** -What is a datasource? -===================== - Datasources are sources of configuration data for cloud-init that typically -come from the user (aka userdata) or come from the stack that created the -configuration drive (aka metadata). Typical userdata would include files, +come from the user (e.g. userdata) or come from the cloud that created the +configuration drive (e.g. metadata). Typical userdata would include files, yaml, and shell scripts while typical metadata would include server name, -instance id, display name and other cloud specific details. Since there are -multiple ways to provide this data (each cloud solution seems to prefer its -own way) internally a datasource abstract class was created to allow for a -single way to access the different cloud systems methods to provide this data -through the typical usage of subclasses. - -Any metadata processed by cloud-init's datasources is persisted as -``/run/cloud-init/instance-data.json``. Cloud-init provides tooling -to quickly introspect some of that data. See :ref:`instance_metadata` for -more information. - - -Datasource API --------------- -The current interface that a datasource object must provide is the following: - -.. sourcecode:: python +instance id, display name and other cloud specific details. - # returns a mime multipart message that contains - # all the various fully-expanded components that - # were found from processing the raw userdata string - # - when filtering only the mime messages targeting - # this instance id will be returned (or messages with - # no instance id) - def get_userdata(self, apply_filter=False) - - # returns the raw userdata string (or none) - def get_userdata_raw(self) +Since there are multiple ways to provide this data (each cloud solution seems +to prefer its own way) internally a datasource abstract class was created to +allow for a single way to access the different cloud systems methods to provide +this data through the typical usage of subclasses. - # returns a integer (or none) which can be used to identify - # this instance in a group of instances which are typically - # created from a single command, thus allowing programmatic - # filtering on this launch index (or other selective actions) - @property - def launch_index(self) - - # the data sources' config_obj is a cloud-config formatted - # object that came to it from ways other than cloud-config - # because cloud-config content would be handled elsewhere - def get_config_obj(self) - - #returns a list of public ssh keys - def get_public_ssh_keys(self) - - # translates a device 'short' name into the actual physical device - # fully qualified name (or none if said physical device is not attached - # or does not exist) - def device_name_to_device(self, name) +Any metadata processed by cloud-init's datasources is persisted as +``/run/cloud-init/instance-data.json``. Cloud-init provides tooling to quickly +introspect some of that data. See :ref:`instance_metadata` for more +information. - # gets the locale string this instance should be applying - # which typically used to adjust the instances locale settings files - def get_locale(self) +Known Sources +============= - @property - def availability_zone(self) +The following is a list of documents for each supported datasource: - # gets the instance id that was assigned to this instance by the - # cloud provider or when said instance id does not exist in the backing - # metadata this will return 'iid-datasource' - def get_instance_id(self) +.. toctree:: + :titlesonly: - # gets the fully qualified domain name that this host should be using - # when configuring network or hostname releated settings, typically - # assigned either by the cloud provider or the user creating the vm - def get_hostname(self, fqdn=False) + datasources/aliyun.rst + datasources/altcloud.rst + datasources/ec2.rst + datasources/azure.rst + datasources/cloudsigma.rst + datasources/cloudstack.rst + datasources/configdrive.rst + datasources/digitalocean.rst + datasources/exoscale.rst + datasources/fallback.rst + datasources/gce.rst + datasources/maas.rst + datasources/nocloud.rst + datasources/opennebula.rst + datasources/openstack.rst + datasources/oracle.rst + datasources/ovf.rst + datasources/smartos.rst - def get_package_mirror_info(self) +Creation +======== -Adding a new Datasource ------------------------ The datasource objects have a few touch points with cloud-init. If you -are interested in adding a new datasource for your cloud platform you'll +are interested in adding a new datasource for your cloud platform you will need to take care of the following items: * **Identify a mechanism for positive identification of the platform**: @@ -139,31 +107,61 @@ need to take care of the following items: file in ``doc/datasources/<cloudplatform>.rst`` -Datasource Documentation -======================== -The following is a list of the implemented datasources. -Follow for more information. +API +=== -.. toctree:: - :maxdepth: 2 +The current interface that a datasource object must provide is the following: - datasources/aliyun.rst - datasources/altcloud.rst - datasources/azure.rst - datasources/cloudsigma.rst - datasources/cloudstack.rst - datasources/configdrive.rst - datasources/digitalocean.rst - datasources/ec2.rst - datasources/exoscale.rst - datasources/maas.rst - datasources/nocloud.rst - datasources/opennebula.rst - datasources/openstack.rst - datasources/oracle.rst - datasources/ovf.rst - datasources/smartos.rst - datasources/fallback.rst - datasources/gce.rst +.. sourcecode:: python + + # returns a mime multipart message that contains + # all the various fully-expanded components that + # were found from processing the raw user data string + # - when filtering only the mime messages targeting + # this instance id will be returned (or messages with + # no instance id) + def get_userdata(self, apply_filter=False) + + # returns the raw userdata string (or none) + def get_userdata_raw(self) + + # returns a integer (or none) which can be used to identify + # this instance in a group of instances which are typically + # created from a single command, thus allowing programmatic + # filtering on this launch index (or other selective actions) + @property + def launch_index(self) + + # the data sources' config_obj is a cloud-config formatted + # object that came to it from ways other than cloud-config + # because cloud-config content would be handled elsewhere + def get_config_obj(self) + + #returns a list of public ssh keys + def get_public_ssh_keys(self) + + # translates a device 'short' name into the actual physical device + # fully qualified name (or none if said physical device is not attached + # or does not exist) + def device_name_to_device(self, name) + + # gets the locale string this instance should be applying + # which typically used to adjust the instances locale settings files + def get_locale(self) + + @property + def availability_zone(self) + + # gets the instance id that was assigned to this instance by the + # cloud provider or when said instance id does not exist in the backing + # metadata this will return 'iid-datasource' + def get_instance_id(self) + + # gets the fully qualified domain name that this host should be using + # when configuring network or hostname related settings, typically + # assigned either by the cloud provider or the user creating the vm + def get_hostname(self, fqdn=False) + + def get_package_mirror_info(self) -.. vi: textwidth=78 +.. vi: textwidth=80 diff --git a/doc/rtd/topics/faq.rst b/doc/rtd/topics/faq.rst new file mode 100644 index 0000000..ab18ebd --- /dev/null +++ b/doc/rtd/topics/faq.rst @@ -0,0 +1,43 @@ +.. _faq: + +FAQ +*** + +Getting help +============ + +Having trouble? We would like to help! + +- Use the search bar at the upper left to search these docs +- Ask a question in the ``#cloud-init`` IRC channel on Freenode +- Join and ask questions on the `cloud-init mailing list <https://launchpad.net/~cloud-init>`_ +- Find a bug? `Report bugs on Launchpad <https://bugs.launchpad.net/cloud-init/+filebug>`_ + + +Media +===== + +Below are some videos, blog posts, and white papers about cloud-init from a +variety of sources. + +- `Cloud Instance Initialization with cloud-init (Whitepaper)`_ +- `cloud-init Summit 2018`_ +- `cloud-init - The cross-cloud Magic Sauce (PDF)`_ +- `cloud-init Summit 2017`_ +- `cloud-init - Building clouds one Linux box at a time (Video)`_ +- `cloud-init - Building clouds one Linux box at a time (PDF)`_ +- `Metadata and cloud-init`_ +- `The beauty of cloud-init`_ +- `Introduction to cloud-init`_ + +.. _Cloud Instance Initialization with cloud-init (Whitepaper): https://ubuntu.com/blog/cloud-instance-initialisation-with-cloud-init +.. _cloud-init Summit 2018: https://powersj.io/post/cloud-init-summit18/ +.. _cloud-init - The cross-cloud Magic Sauce (PDF): https://events.linuxfoundation.org/wp-content/uploads/2017/12/cloud-init-The-cross-cloud-Magic-Sauce-Scott-Moser-Chad-Smith-Canonical.pdf +.. _cloud-init Summit 2017: https://powersj.io/post/cloud-init-summit17/ +.. _cloud-init - Building clouds one Linux box at a time (Video): https://www.youtube.com/watch?v=1joQfUZQcPg +.. _cloud-init - Building clouds one Linux box at a time (PDF): https://annex.debconf.org/debconf-share/debconf17/slides/164-cloud-init_Building_clouds_one_Linux_box_at_a_time.pdf +.. _Metadata and cloud-init: https://www.youtube.com/watch?v=RHVhIWifVqU +.. _The beauty of cloud-init: http://brandon.fuller.name/archives/2011/05/02/06.40.57/ +.. _Introduction to cloud-init: http://www.youtube.com/watch?v=-zL3BdbKyGY + +.. vi: textwidth=80 diff --git a/doc/rtd/topics/instancedata.rst b/doc/rtd/topics/instancedata.rst index 231a008..5f0eca4 100644 --- a/doc/rtd/topics/instancedata.rst +++ b/doc/rtd/topics/instancedata.rst @@ -90,47 +90,107 @@ There are three basic top-level keys: The standardized keys present: -+----------------------+-----------------------------------------------+-----------------------------------+ -| Key path | Description | Examples | -+======================+===============================================+===================================+ -| v1._beta_keys | List of standardized keys still in 'beta'. | [subplatform] | -| | The format, intent or presence of these keys | | -| | can change. Do not consider them | | -| | production-ready. | | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.cloud_name | Where possible this will indicate the 'name' | aws, openstack, azure, | -| | of the cloud this system is running on. This | configdrive, nocloud, | -| | is specifically different than the 'platform' | ovf, etc. | -| | below. As an example, the name of Amazon Web | | -| | Services is 'aws' while the platform is 'ec2'.| | -| | | | -| | If no specific name is determinable or | | -| | provided in meta-data, then this field may | | -| | contain the same content as 'platform'. | | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.instance_id | Unique instance_id allocated by the cloud | i-<somehash> | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.local_hostname | The internal or local hostname of the system | ip-10-41-41-70, | -| | | <user-provided-hostname> | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.platform | An attempt to identify the cloud platform | ec2, openstack, lxd, gce | -| | instance that the system is running on. | nocloud, ovf | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.subplatform | Additional platform details describing the | metadata (http://168.254.169.254),| -| | specific source or type of metadata used. | seed-dir (/path/to/seed-dir/), | -| | The format of subplatform will be: | config-disk (/dev/cd0), | -| | <subplatform_type> (<url_file_or_dev_path>) | configdrive (/dev/sr0) | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.public_ssh_keys | A list of ssh keys provided to the instance | ['ssh-rsa AA...', ...] | -| | by the datasource metadata. | | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.region | The physical region/datacenter in which the | us-east-2 | -| | instance is deployed | | -+----------------------+-----------------------------------------------+-----------------------------------+ -| v1.availability_zone | The physical availability zone in which the | us-east-2b, nova, null | -| | instance is deployed | | -+----------------------+-----------------------------------------------+-----------------------------------+ +v1._beta_keys +------------- +List of standardized keys still in 'beta'. The format, intent or presence of +these keys can change. Do not consider them production-ready. +Example output: + +- [subplatform] + +v1.cloud_name +------------- +Where possible this will indicate the 'name' of the cloud the system is running +on. This is different than the 'platform' item. For example, the cloud name of +Amazone Web Services is 'aws', while the platform is 'ec2'. + +If determining a specific name is not possible or provided in meta-data, then +this filed may contain the same content as 'platform'. + +Example output: + +- aws +- openstack +- azure +- configdrive +- nocloud +- ovf + + +v1.instance_id +-------------- +Unique instance_id allocated by the cloud. + +Examples output: + +- i-<hash> + +v1.local_hostname +----------------- +The internal or local hostname of the system. + +Examples output: + +- ip-10-41-41-70 +- <user-provided-hostname> + +v1.platform +------------- +An attempt to identify the cloud platfrom instance that the system is running +on. + +Examples output: + +- ec2 +- openstack +- lxd +- gce +- nocloud +- ovf + +v1.subplatform +-------------- +Additional platform details describing the specific source or type of metadata +used. The format of subplatform will be: + +``<subplatform_type> (<url_file_or_dev_path>`` + +Examples output: + +- metadata (http://168.254.169.254) +- seed-dir (/path/to/seed-dir/) +- config-disk (/dev/cd0) +- configdrive (/dev/sr0) + +v1.public_ssh_keys +------------------ +A list of ssh keys provided to the instance by the datasource metadata. + +Examples output: + +- ['ssh-rsa AA...', ...] + +v1.region +--------- +The physical region/data center in which the instance is deployed. + +Examples output: + +- us-east-2 + +v1.availability_zone +-------------------- +The physical availability zone in which the instance is deployed. + +Examples output: + +- us-east-2b +- nova +- null + +Example Output +-------------- Below is an example of ``/run/cloud-init/instance_data.json`` on an EC2 instance: @@ -229,28 +289,28 @@ instance: "network": { "interfaces": { "macs": { - "06:74:8f:39:cd:a6": { - "device-number": "0", - "interface-id": "eni-052058bbd7831eaae", - "ipv4-associations": { - "18.218.221.122": "10.41.41.95" - }, - "local-hostname": "ip-10-41-41-95.us-east-2.compute.internal", - "local-ipv4s": "10.41.41.95", - "mac": "06:74:8f:39:cd:a6", - "owner-id": "437526006925", - "public-hostname": "ec2-18-218-221-122.us-east-2.compute.amazonaws.com", - "public-ipv4s": "18.218.221.122", - "security-group-ids": "sg-828247e9", - "security-groups": "Cloud-init integration test secgroup", - "subnet-id": "subnet-282f3053", - "subnet-ipv4-cidr-block": "10.41.41.0/24", - "subnet-ipv6-cidr-blocks": "2600:1f16:b80:ad00::/64", - "vpc-id": "vpc-252ef24d", - "vpc-ipv4-cidr-block": "10.41.0.0/16", - "vpc-ipv4-cidr-blocks": "10.41.0.0/16", - "vpc-ipv6-cidr-blocks": "2600:1f16:b80:ad00::/56" - } + "06:74:8f:39:cd:a6": { + "device-number": "0", + "interface-id": "eni-052058bbd7831eaae", + "ipv4-associations": { + "18.218.221.122": "10.41.41.95" + }, + "local-hostname": "ip-10-41-41-95.us-east-2.compute.internal", + "local-ipv4s": "10.41.41.95", + "mac": "06:74:8f:39:cd:a6", + "owner-id": "437526006925", + "public-hostname": "ec2-18-218-221-122.us-east-2.compute.amazonaws.com", + "public-ipv4s": "18.218.221.122", + "security-group-ids": "sg-828247e9", + "security-groups": "Cloud-init integration test secgroup", + "subnet-id": "subnet-282f3053", + "subnet-ipv4-cidr-block": "10.41.41.0/24", + "subnet-ipv6-cidr-blocks": "2600:1f16:b80:ad00::/64", + "vpc-id": "vpc-252ef24d", + "vpc-ipv4-cidr-block": "10.41.0.0/16", + "vpc-ipv4-cidr-blocks": "10.41.0.0/16", + "vpc-ipv6-cidr-blocks": "2600:1f16:b80:ad00::/56" + } } } }, @@ -319,7 +379,7 @@ user. Below are some examples of providing these types of user-data: -* Cloud config calling home with the ec2 public hostname and avaliability-zone +* Cloud config calling home with the ec2 public hostname and availability-zone .. code-block:: shell-session @@ -328,7 +388,7 @@ Below are some examples of providing these types of user-data: runcmd: - echo 'EC2 public hostname allocated to instance: {{ ds.meta_data.public_hostname }}' > /tmp/instance_metadata - - echo 'EC2 avaiability zone: {{ v1.availability_zone }}' >> + - echo 'EC2 availability zone: {{ v1.availability_zone }}' >> /tmp/instance_metadata - curl -X POST -d '{"hostname": "{{ds.meta_data.public_hostname }}", "availability-zone": "{{ v1.availability_zone }}"}' @@ -352,7 +412,7 @@ Below are some examples of providing these types of user-data: and the following string in your rendered user-data: ``CI_MISSING_JINJA_VAR/<your_varname>``. -Cloud-init also surfaces a commandline tool **cloud-init query** which can +Cloud-init also surfaces a command line tool **cloud-init query** which can assist developers or scripts with obtaining instance metadata easily. See :ref:`cli_query` for more information. diff --git a/doc/rtd/topics/moreinfo.rst b/doc/rtd/topics/moreinfo.rst deleted file mode 100644 index 9c3b7fb..0000000 --- a/doc/rtd/topics/moreinfo.rst +++ /dev/null @@ -1,13 +0,0 @@ -**************** -More information -**************** - -Useful external references -========================== - -- `The beauty of cloudinit`_ -- `Introduction to cloud-init`_ (video) - -.. _Introduction to cloud-init: http://www.youtube.com/watch?v=-zL3BdbKyGY -.. _The beauty of cloudinit: http://brandon.fuller.name/archives/2011/05/02/06.40.57/ -.. vi: textwidth=78
_______________________________________________ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
