Re: [Cloud-init-dev] [Merge] lp:~utlemming/cloud-init/digitalocean into lp:cloud-init

[Ben Howard]

I'm unable to see the logs in the failed CI run. "make check" tests pass 
locally. I'm happy to make the changes if someone can post the logs. 
-- 
https://code.launchpad.net/~utlemming/cloud-init/digitalocean/+merge/301123
Your team cloud init development team is requested to review the proposed merge 
of lp:~utlemming/cloud-init/digitalocean into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~utlemming/cloud-init/digitalocean into lp:cloud-init

[Server Team CI bot]

Review: Needs Fixing continuous-integration

FAILED: Continuous integration, rev:1260
https://server-team-jenkins.canonical.com/job/cloud-init-ci/2/
Executed test runs:
FAILURE: 
https://server-team-jenkins.canonical.com/job/cloud-init-ci/nodes=amd64/2/console
FAILURE: 
https://server-team-jenkins.canonical.com/job/cloud-init-ci/nodes=ppc64el/2/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/cloud-init-ci/2/rebuild

-- 
https://code.launchpad.net/~utlemming/cloud-init/digitalocean/+merge/301123
Your team cloud init development team is requested to review the proposed merge 
of lp:~utlemming/cloud-init/digitalocean into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~utlemming/cloud-init/digitalocean into lp:cloud-init

[Ben Howard]

Ben Howard has proposed merging lp:~utlemming/cloud-init/digitalocean into 
lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~utlemming/cloud-init/digitalocean/+merge/301123

This request comprises two commits to the DigitalOcean provider. 

Commit 1259: Update to the JSON meta-data endpoint
- Per the API documentation, the JSON endpoint is the preferred method of 
getting the metadata.
- Use the SMBIOS information to determine if the droplet ID has changed. In 
certain circumstances a droplet that is either migrated or snapshotted may not 
have metadata via the metadata URL. The droplets identity is authoritatively 
embedded in the SMBIOS. 
- Only hit the Metadata URL if the droplet identifies as being on DigitalOcean

Commit 1260: Update the tests cases.

-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~utlemming/cloud-init/digitalocean into lp:cloud-init.
=== modified file 'cloudinit/sources/DataSourceDigitalOcean.py'
--- cloudinit/sources/DataSourceDigitalOcean.py	2016-05-12 17:56:26 +
+++ cloudinit/sources/DataSourceDigitalOcean.py	2016-07-25 23:38:22 +
@@ -1,6 +1,7 @@
 # vi: ts=4 expandtab
 #
 #Author: Neal Shrader 
+#Author: Ben Howard  
 #
 #This program is free software: you can redistribute it and/or modify
 #it under the terms of the GNU General Public License version 3, as
@@ -14,22 +15,27 @@
 #You should have received a copy of the GNU General Public License
 #along with this program.  If not, see .
 
-from cloudinit import ec2_utils
+# DigitalOcean Droplet API:
+# https://developers.digitalocean.com/documentation/metadata/
+
 from cloudinit import log as logging
+from cloudinit import util
 from cloudinit import sources
-from cloudinit import util
-
-import functools
-
+from cloudinit import url_helper
+
+import json
+import os
 
 LOG = logging.getLogger(__name__)
 
 BUILTIN_DS_CONFIG = {
-'metadata_url': 'http://169.254.169.254/metadata/v1/',
-'mirrors_url': 'http://mirrors.digitalocean.com/'
+'metadata_url': 'http://169.254.169.254/metadata/v1.json',
 }
-MD_RETRIES = 0
-MD_TIMEOUT = 1
+
+# Wait for a up to a minute, retrying the meta-data server
+# every 2 seconds.
+MD_RETRIES = 30
+MD_TIMEOUT = 2
 
 
 class DataSourceDigitalOcean(sources.DataSource):
@@ -40,43 +46,67 @@
 util.get_cfg_by_path(sys_cfg, ["datasource", "DigitalOcean"], {}),
 BUILTIN_DS_CONFIG])
 self.metadata_address = self.ds_cfg['metadata_url']
-
-if self.ds_cfg.get('retries'):
-self.retries = self.ds_cfg['retries']
-else:
-self.retries = MD_RETRIES
-
-if self.ds_cfg.get('timeout'):
-self.timeout = self.ds_cfg['timeout']
-else:
-self.timeout = MD_TIMEOUT
-
-def get_data(self):
-caller = functools.partial(util.read_file_or_url,
-   timeout=self.timeout, retries=self.retries)
-
-def mcaller(url):
-return caller(url).contents
-
-md = ec2_utils.MetadataMaterializer(mcaller(self.metadata_address),
-base_url=self.metadata_address,
-caller=mcaller)
-
-self.metadata = md.materialize()
-
-if self.metadata.get('id'):
-return True
-else:
+self.retries = self.ds_cfg.get('retries', MD_RETRIES)
+self.timeout = self.ds_cfg.get('timeout', MD_TIMEOUT)
+
+def _get_sysinfo(self):
+# DigitalOcean embeds vendor ID and instance/droplet_id in the
+# SMBIOS information
+
+# DigitalOcean only supports i386/x86_64 hardware
+uname_arch = os.uname()[4]
+if uname_arch not in ('i386', 'i686', 'x86_64'):
+return (None, None)
+LOG.debug("checking if instance is a DigitalOcean droplet")
+
+is_do = False
+droplet_id = None
+
+# Detect if we are on DigitalOcean and return the Droplet's ID
+vendor_name = util.read_dmi_data("system-manufacturer")
+if vendor_name == "DigitalOcean":
+is_do = True
+LOG.info("running on DigitalOcean")
+
+droplet_id = util.read_dmi_data("system-serial-number")
+
+if droplet_id:
+LOG.debug(("system identified via SMBIOS as DigitalOcean "
+   "Droplet {}").format(droplet_id))
+else:
+LOG.critical(("system identified via SMBIOS as a DigitalOcean "
+  "Droplet, but not provide an ID. "
+  "Please file a support ticket at: "
+  "https://cloud.digitalocean.com/support/tickets";
+  "/new"))
+
+return (is_do, droplet_id)
+
+def get_data(self, apply_filter=False):
+(is_do, droplet_

Re: [Cloud-init-dev] [Merge] ~larsks/cloud-init:feature/move-to-git into cloud-init:master

[Joshua Harlow]

Diff comments:

> diff --git a/README.rst b/README.rst
> new file mode 100644
> index 000..e56e663
> --- /dev/null
> +++ b/README.rst
> @@ -0,0 +1,15 @@
> +Cloud-init
> +==
> +
> +Cloud-init is the defacto multi-distribution package that handles
> +early initialization of a cloud instance.
> +
> +The package is hosted at .

Is this updated, seems pretty old. Whose repo is this :-/

> +
> +Additional documentation can be found in the `doc/` subdirectory.
> +
> +Contributing
> +
> +
> +For information about how to contribute to cloud-init development, see
> +the file ``HACKING.rst`` included in this distribution.
> diff --git a/packages/redhat/cloud-init.spec.in 
> b/packages/redhat/cloud-init.spec.in
> index 254d209..71c71be 100644
> --- a/packages/redhat/cloud-init.spec.in
> +++ b/packages/redhat/cloud-init.spec.in
> @@ -5,13 +5,9 @@
>  # Or: http://fedoraproject.org/wiki/Packaging:ScriptletSnippets
>  # Or: http://www.rpm.org/max-rpm/ch-rpm-inside.html
>  
> -#for $d in $defines
> -%define ${d}
> -#end for
> -
>  Name:   cloud-init
>  Version:${version}
> -Release:${release}${subrelease}%{?dist}
> +Release:1${subrelease}%{?dist}

The 1 is the epoch?

>  Summary:Cloud instance init scripts
>  
>  Group:  System Environment/Base
> diff --git a/packages/suse/cloud-init.spec.in 
> b/packages/suse/cloud-init.spec.in
> index 53e6ad1..74b6f12 100644
> --- a/packages/suse/cloud-init.spec.in
> +++ b/packages/suse/cloud-init.spec.in
> @@ -157,7 +153,3 @@ mkdir -p %{buildroot}/var/lib/cloud
>  %{python_sitelib}/*
>  
>  /var/lib/cloud
> -
> -%changelog
> -
> -${changelog}

Damn, didn't like my changelog ;)

Yes I know redhat folks like to manually curate the changelog (and really its 
the spec file changelog, not the package/code changelog, which always confused 
me, but whatever, ha).



-- 
https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/300953
Your team cloud init development team is requested to review the proposed merge 
of ~larsks/cloud-init:feature/move-to-git into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:skip-things into cloud-init:master

[Joshua Harlow]

Joshua Harlow has proposed merging ~harlowja/cloud-init:skip-things into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301110
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:skip-things into cloud-init:master.
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
index 47deac6..72c0b94 100644
--- a/cloudinit/stages.py
+++ b/cloudinit/stages.py
@@ -808,7 +808,7 @@ class Modules(object):
 def run_section(self, section_name):
 raw_mods = self._read_modules(section_name)
 mostly_mods = self._fixup_modules(raw_mods)
-d_name = self.init.distro.name
+distro_name = self.init.distro.name
 
 skipped = []
 forced = []
@@ -819,7 +819,7 @@ class Modules(object):
 distros.Distro.expand_osfamily(mod.osfamilies))
 
 # module does not declare 'distros' or lists this distro
-if not worked_distros or d_name in worked_distros:
+if not worked_distros or distro_name in worked_distros:
 continue
 
 if name in overridden:
@@ -827,14 +827,20 @@ class Modules(object):
 else:
 skipped.append(name)
 
+cleaned_mostly_mods = []
+for (mod, name, freq, args) in mostly_mods:
+if name in skipped:
+continue
+cleaned_mostly_mods.append((mod, name, freq, args))
+
 if skipped:
 LOG.info("Skipping modules %s because they are not verified "
  "on distro '%s'.  To run anyway, add them to "
- "'unverified_modules' in config.", skipped, d_name)
+ "'unverified_modules' in config.", skipped, distro_name)
 if forced:
-LOG.info("running unverified_modules: %s", forced)
+LOG.info("Running unverified modules: %s", forced)
 
-return self._run_modules(mostly_mods)
+return self._run_modules(cleaned_mostly_mods)
 
 
 def fetch_base_config():
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:update-configobj into cloud-init:master

[Joshua Harlow]

Joshua Harlow has proposed merging ~harlowja/cloud-init:update-configobj into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301109
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:update-configobj into cloud-init:master.
diff --git a/requirements.txt b/requirements.txt
index cc1dc05..0c4951f 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -22,7 +22,7 @@ oauthlib
 # that the built-in config parser is not sufficent (ie
 # when we need to preserve comments, or do not have a top-level
 # section)...
-configobj
+configobj>=5.0.2
 
 # All new style configurations are in the yaml format
 pyyaml
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:distro-tags into cloud-init:master

[Joshua Harlow]

Joshua Harlow has proposed merging ~harlowja/cloud-init:distro-tags into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301108
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:distro-tags into cloud-init:master.
diff --git a/cloudinit/config/cc_lxd.py b/cloudinit/config/cc_lxd.py
index 70d4e7c..0086840 100644
--- a/cloudinit/config/cc_lxd.py
+++ b/cloudinit/config/cc_lxd.py
@@ -47,6 +47,8 @@ Example config:
 
 from cloudinit import util
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 # Get config
diff --git a/cloudinit/config/cc_rh_subscription.py b/cloudinit/config/cc_rh_subscription.py
index 3a113ae..d4ad724 100644
--- a/cloudinit/config/cc_rh_subscription.py
+++ b/cloudinit/config/cc_rh_subscription.py
@@ -18,6 +18,8 @@
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def handle(name, cfg, _cloud, log, _args):
 sm = SubscriptionManager(cfg)
diff --git a/cloudinit/config/cc_snappy.py b/cloudinit/config/cc_snappy.py
index 1a485ee..6bcd838 100644
--- a/cloudinit/config/cc_snappy.py
+++ b/cloudinit/config/cc_snappy.py
@@ -68,6 +68,8 @@ BUILTIN_CFG = {
 'config': {},
 }
 
+distros = ['ubuntu']
+
 
 def parse_filename(fname):
 fname = os.path.basename(fname)
diff --git a/cloudinit/config/cc_ubuntu_init_switch.py b/cloudinit/config/cc_ubuntu_init_switch.py
index 884d79f..bffb438 100644
--- a/cloudinit/config/cc_ubuntu_init_switch.py
+++ b/cloudinit/config/cc_ubuntu_init_switch.py
@@ -86,6 +86,8 @@ else
 fi
 """
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 """Handler method activated by cloud-init."""
diff --git a/cloudinit/config/cc_yum_add_repo.py b/cloudinit/config/cc_yum_add_repo.py
index 64fba86..22549e6 100644
--- a/cloudinit/config/cc_yum_add_repo.py
+++ b/cloudinit/config/cc_yum_add_repo.py
@@ -23,6 +23,8 @@ import six
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def _canonicalize_id(repo_id):
 repo_id = repo_id.lower().replace("-", "_")
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:file-logging into cloud-init:master

[Joshua Harlow]

Joshua Harlow has proposed merging ~harlowja/cloud-init:file-logging into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301107
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:file-logging into cloud-init:master.
diff --git a/config/cloud.cfg.d/05_logging.cfg b/config/cloud.cfg.d/05_logging.cfg
index 2e18073..aeecc42 100644
--- a/config/cloud.cfg.d/05_logging.cfg
+++ b/config/cloud.cfg.d/05_logging.cfg
@@ -54,13 +54,21 @@ _log:
 
 log_cfgs:
 # These will be joined into a string that defines the configuration
- - [ *log_base, *log_syslog ]
+#
+# If your desire syslog, just uncomment the following line.
+# - [ *log_base, *log_syslog ]
+#
 # These will be joined into a string that defines the configuration
+#
+# Otherwise by default we just use the most basic (known to work
+# mechanism that does not depend on a system service to operate; which
+# due to cloud-init execution lifecycle we may not always be able to
+# depend on working correctly).
  - [ *log_base, *log_file ]
-# A file path can also be used
+# A file path can also be used.
 # - /etc/log.conf
 
-# this tells cloud-init to redirect its stdout and stderr to
+# This tells cloud-init to redirect its stdout and stderr to
 # 'tee -a /var/log/cloud-init-output.log' so the user can see output
 # there without needing to look on the console.
 output: {all: '| tee -a /var/log/cloud-init-output.log'}
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~smoser/cloud-init/trunk.smartos-fabric into lp:cloud-init

[Server Team CI bot]

Review: Approve continuous-integration

PASSED: Continuous integration, rev:1259
https://server-team-jenkins.canonical.com/job/cloud-init-ci/1/
Executed test runs:
SUCCESS: 
https://server-team-jenkins.canonical.com/job/cloud-init-ci/nodes=amd64/1
SUCCESS: 
https://server-team-jenkins.canonical.com/job/cloud-init-ci/nodes=ppc64el/1

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/cloud-init-ci/1/rebuild

-- 
https://code.launchpad.net/~smoser/cloud-init/trunk.smartos-fabric/+merge/300115
Your team cloud init development team is requested to review the proposed merge 
of lp:~smoser/cloud-init/trunk.smartos-fabric into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp