from:"Joshua Harlow"

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:no-cheetah into cloud-init:master

2017-01-11 Thread Joshua Harlow

that could be a solution; i'd be ok with that, other option is to release 
cloud-init 1.0 and switch it then.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/308304
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:no-cheetah into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:chop-bitly into cloud-init:master

2016-12-02 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:chop-bitly into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/312390
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:chop-bitly into cloud-init:master.
diff --git a/cloudinit/ec2_utils.py b/cloudinit/ec2_utils.py
index 76dda04..5eb58d3 100644
--- a/cloudinit/ec2_utils.py
+++ b/cloudinit/ec2_utils.py
@@ -57,8 +57,6 @@ class MetadataLeafDecoder(object):
 return blob
 
 
-# See: http://bit.ly/TyoUQs
-#
 class MetadataMaterializer(object):
 def __init__(self, blob, base_url, caller, leaf_decoder=None):
 self._blob = blob
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:sys-io-errors into cloud-init:master

2016-10-26 Thread Joshua Harlow

*thought
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:sys-io-errors into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:sys-io-errors into cloud-init:master

2016-10-26 Thread Joshua Harlow

Will double check, though I fixed that recently.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:sys-io-errors into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:no-cheetah into cloud-init:master

2016-10-23 Thread Joshua Harlow

Ya, this changes the default away from cheetah; might be smart to bump some 
version number to denote this (just incase).
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/308304
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:no-cheetah into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:ug_util into cloud-init:master

2016-10-19 Thread Joshua Harlow

Yup, just moving.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307373
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:ug_util into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~wesley-wiedenmeier/cloud-init:error-output into cloud-init:master

2016-10-13 Thread Joshua Harlow



Diff comments:

> diff --git a/tests/unittests/test_util.py b/tests/unittests/test_util.py
> index fc6b9d4..f8d416c 100644
> --- a/tests/unittests/test_util.py
> +++ b/tests/unittests/test_util.py
> @@ -604,11 +604,70 @@ class TestSubp(helpers.TestCase):
>   util.target_path("/target/", "///my/path/"))
>  
>  
> +<<< tests/unittests/test_util.py

Probably need to do a rebase/conflict fix.

>  class TestEncode(helpers.TestCase):
>  """Test the encoding functions"""
>  def test_decode_binary_plain_text_with_hex(self):
>  blob = 'BOOTABLE_FLAG=\x80init=/bin/systemd'
>  text = util.decode_binary(blob)
>  self.assertEqual(text, blob)
> +===
> +class TestProcessExecutionError(helpers.TestCase):
> +
> +template = ('{description}\n'
> +'Command: {cmd}\n'
> +'Exit code: {exit_code}\n'
> +'Reason: {reason}\n'
> +'Stdout: {stdout}\n'
> +'Stderr: {stderr}')
> +empty_attr = '-'
> +empty_description = 'Unexpected error while running command.'
> +
> +def test_pexec_error_type(self):
> +self.assertIsInstance(util.ProcessExecutionError(), IOError)
> +
> +def test_pexec_error_empty_msgs(self):
> +error = util.ProcessExecutionError()
> +self.assertTrue(all(attr == self.empty_attr for attr in
> +(error.stderr, error.stdout, error.reason)))
> +self.assertEqual(error.description, self.empty_description)
> +self.assertEqual(error.message, self.template.format(
> +description=self.empty_description, exit_code=self.empty_attr,
> +reason=self.empty_attr, stdout=self.empty_attr,
> +stderr=self.empty_attr, cmd=self.empty_attr))
> +
> +def test_pexec_error_single_line_msgs(self):
> +stdout_msg = 'out out'
> +stderr_msg = 'error error'
> +cmd = 'test command'
> +exit_code = 3
> +error = util.ProcessExecutionError(
> +stdout=stdout_msg, stderr=stderr_msg, exit_code=3, cmd=cmd)
> +self.assertEqual(error.message, self.template.format(
> +description=self.empty_description, stdout=stdout_msg,
> +stderr=stderr_msg, exit_code=str(exit_code),
> +reason=self.empty_attr, cmd=cmd))
> +
> +def test_pexec_error_multi_line_msgs(self):
> +stdout_msg = 'multi\nline\noutput message'
> +stderr_msg = 'multi\nline\nerror message\n\n\n'
> +error = util.ProcessExecutionError(
> +stdout=stdout_msg, stderr=stderr_msg)
> +self.assertEqual(
> +error.message,
> +'\n'.join((
> +'{description}',
> +'Command: {empty_attr}',
> +'Exit code: {empty_attr}',
> +'Reason: {empty_attr}',
> +'Stdout: multi',
> +'line',
> +'output message',
> +'Stderr: multi',
> +'line',
> +'error message',
> +)).format(description=self.empty_description,
> +  empty_attr=self.empty_attr))
> +>>> tests/unittests/test_util.py
>  
>  # vi: ts=4 expandtab


-- 
https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/306731
Your team cloud init development team is requested to review the proposed merge 
of ~wesley-wiedenmeier/cloud-init:error-output into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~msaikia/cloud-init:topic-msaikia-vmware into cloud-init:master

2016-10-13 Thread Joshua Harlow



Diff comments:

> diff --git a/cloudinit/sources/helpers/vmware/imc/config.py 
> b/cloudinit/sources/helpers/vmware/imc/config.py
> index d645c49..a86233f 100644
> --- a/cloudinit/sources/helpers/vmware/imc/config.py
> +++ b/cloudinit/sources/helpers/vmware/imc/config.py
> @@ -93,3 +94,34 @@ class Config(object):
>  res.append(Nic(nic, self._configFile))
>  
>  return res
> +
> +@property
> +def admin_password(self):
> +"""Return the root password to be set."""
> +return self._configFile.get(Config.PASS, None)
> +
> +@property
> +def reset_password(self):
> +"""Retreives if the root password needs to be reset."""
> +resetPass = self._configFile.get(Config.RESETPASS, None)
> +if resetPass and not re.match('yes$|no$', resetPass.lower()):
> +raise ValueError("ResetPassword value should be yes/no")
> +return resetPass
> +
> +@property
> +def marker_id(self):
> +"""Returns marker id."""
> +return self._configFile.get(Config.MARKERID, None)
> +
> +@property
> +def GetPostGcStatus(self):

Let's not have properties start with 'get' and be in a mix of camel case and 
not. 

https://www.python.org/dev/peps/pep-0008/#prescriptive-naming-conventions

> +"""Retreives if customization status needs to be posted."""
> +postGcStatus = self._configFile.get(Config.POST_GC, None)
> +if postGcStatus and not re.match('yes$|no$', postGcStatus.lower()):
> +raise ValueError("GC status value should be yes/no")
> +return postGcStatus
> +
> +@property
> +def GetCustomScriptName(self):
> +"""Return the name of custom (pre/post) script."""
> +return self._configFile.get(Config.CUSTOM_SCRIPT, None)


-- 
https://code.launchpad.net/~msaikia/cloud-init/+git/cloud-init/+merge/305427
Your team cloud init development team is requested to review the proposed merge 
of ~msaikia/cloud-init:topic-msaikia-vmware into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~nrezinorn/cloud-init:fix_rhel_ssh_service into cloud-init:master

2016-10-13 Thread Joshua Harlow

So I think I got this one handled in a slightly nicer manner @ 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307485
-- 
https://code.launchpad.net/~nrezinorn/cloud-init/+git/cloud-init/+merge/306653
Your team cloud init development team is requested to review the proposed merge 
of ~nrezinorn/cloud-init:fix_rhel_ssh_service into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:no-cheetah into cloud-init:master

2016-10-12 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:no-cheetah into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/308304
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:no-cheetah into cloud-init:master.
diff --git a/setup.py b/setup.py
index 8ff667d..d190696 100755
--- a/setup.py
+++ b/setup.py
@@ -197,9 +197,6 @@ else:
 
 
 requirements = read_requires()
-if sys.version_info < (3,):
-requirements.append('cheetah')
-
 setuptools.setup(
 name='cloud-init',
 version=get_version(),
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:yaml-check-python into cloud-init:master

2016-10-07 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:yaml-check-python into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307996
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:yaml-check-python into cloud-init:master.
diff --git a/Makefile b/Makefile
old mode 100644
new mode 100755
index 5d35dcc..8f9a520
--- a/Makefile
+++ b/Makefile
@@ -30,7 +30,6 @@ endif
 READ_VERSION=$(shell $(PYVER) $(CWD)/tools/read-version)
 CODE_VERSION=$(shell $(PYVER) -c "from cloudinit import version; print(version.version_string())")
 
-
 all: check
 
 check: check_version pep8 $(pyflakes) test $(yaml)
diff --git a/tools/validate-yaml.py b/tools/validate-yaml.py
index ed9037d..6e16459 100755
--- a/tools/validate-yaml.py
+++ b/tools/validate-yaml.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 """Try to read a YAML file and report any errors.
 """
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:kill-brpm into cloud-init:master

2016-10-07 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:kill-brpm into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307991

Suse may need additional work btw.
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:kill-brpm into cloud-init:master.
diff --git a/packages/brpm b/packages/brpm
deleted file mode 100755
index 89696ab..000
--- a/packages/brpm
+++ /dev/null
@@ -1,218 +0,0 @@
-#!/usr/bin/env python
-
-import argparse
-import glob
-import json
-import os
-import shutil
-import sys
-import tempfile
-
-
-def find_root():
-# expected path is in /packages/
-top_dir = os.environ.get("CLOUD_INIT_TOP_D", None)
-if top_dir is None:
-top_dir = os.path.dirname(
-os.path.dirname(os.path.abspath(sys.argv[0])))
-if os.path.isfile(os.path.join(top_dir, 'setup.py')):
-return os.path.abspath(top_dir)
-raise OSError(("Unable to determine where your cloud-init topdir is."
-   " set CLOUD_INIT_TOP_D?"))
-
-
-if "avoid-pep8-E402-import-not-top-of-file":
-# Use the util functions from cloudinit
-sys.path.insert(0, find_root())
-from cloudinit import templater
-from cloudinit import util
-
-# Map python requirements to package names.  If a match isn't found
-# here, we assume 'python-'.
-PACKAGE_MAP = {
-'redhat': {
-'pyserial': 'pyserial',
-'pyyaml': 'PyYAML',
-},
-'suse': {
-'pyyaml': 'python-yaml',
-}
-}
-
-# Subdirectories of the ~/rpmbuild dir
-RPM_BUILD_SUBDIRS = ['BUILD', 'RPMS', 'SOURCES', 'SPECS', 'SRPMS']
-
-
-def run_helper(helper, args=None, strip=True):
-if args is None:
-args = []
-cmd = [util.abs_join(find_root(), 'tools', helper)] + args
-(stdout, _stderr) = util.subp(cmd)
-if strip:
-stdout = stdout.strip()
-return stdout
-
-
-def read_dependencies():
-'''Returns the Python depedencies from requirements.txt.  This explicitly
-removes 'argparse' from the list of requirements for python >= 2.7,
-because with 2.7 argparse became part of the standard library.'''
-stdout = run_helper('read-dependencies')
-return [p.lower().strip() for p in stdout.splitlines()
-if p != 'argparse' or (p == 'argparse' and
-   sys.version_info[0:2] < (2, 7))]
-
-
-def translate_dependencies(deps, distro):
-'''Maps python requirements into package names.  We assume
-python- for packages not listed explicitly in
-PACKAGE_MAP.'''
-return [PACKAGE_MAP[distro][req]
-if req in PACKAGE_MAP[distro] else 'python-%s' % req
-for req in deps]
-
-
-def read_version():
-return json.loads(run_helper('read-version', ['--json']))
-
-
-def generate_spec_contents(args, version_data, tmpl_fn, top_dir, arc_fn):
-
-# Tmpl params
-subs = {}
-
-if args.sub_release is not None:
-subs['subrelease'] = str(args.sub_release)
-else:
-subs['subrelease'] = ""
-
-subs['archive_name'] = arc_fn
-subs['source_name'] = os.path.basename(arc_fn).replace('.tar.gz', '')
-subs.update(version_data)
-
-# rpm does not like '-' in the Version, so change
-#   X.Y.Z-N-gHASH to X.Y.Z+N.gHASH
-if "-" in version_data.get('version'):
-ver, commits, ghash = version_data['version'].split("-")
-rpm_upstream_version = "%s+%s.%s" % (ver, commits, ghash)
-else:
-rpm_upstream_version = version_data['version']
-subs['rpm_upstream_version'] = rpm_upstream_version
-
-# Map to known packages
-python_deps = read_dependencies()
-package_deps = translate_dependencies(python_deps, args.distro)
-subs['requires'] = package_deps
-
-if args.boot == 'sysvinit':
-subs['sysvinit'] = True
-else:
-subs['sysvinit'] = False
-
-if args.boot == 'systemd':
-subs['systemd'] = True
-else:
-subs['systemd'] = False
-
-subs['init_sys'] = args.boot
-subs['patches'] = [os.path.basename(p) for p in args.patches]
-return templater.render_from_file(tmpl_fn, params=subs)
-
-
-def main():
-
-parser = argparse.ArgumentParser()
-parser.add_argument("-d", "--distro", dest="distro",
-

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:tpl-cloud-cfg into cloud-init:master

2016-10-03 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:tpl-cloud-cfg into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307485
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:tpl-cloud-cfg into cloud-init:master.
diff --git a/cloudinit/helpers.py b/cloudinit/helpers.py
index fb95bab..12c8ef3 100644
--- a/cloudinit/helpers.py
+++ b/cloudinit/helpers.py
@@ -33,6 +33,7 @@ from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
 CFG_ENV_NAME)
 
 from cloudinit import log as logging
+from cloudinit import templater
 from cloudinit import type_utils
 from cloudinit import util
 
@@ -200,18 +201,47 @@ class Runners(object):
 return (True, results)
 
 
+class ConfigError(Exception):
+pass
+
+
 class ConfigMerger(object):
 def __init__(self, paths=None, datasource=None,
  additional_fns=None, base_cfg=None,
  include_vendor=True):
 self._paths = paths
 self._ds = datasource
-self._fns = additional_fns
+self._fns = additional_fns or []
 self._base_cfg = base_cfg
 self._include_vendor = include_vendor
 # Created on first use
 self._cfg = None
 
+@staticmethod
+def _read_conf(fn, kind="config", tpl_params=None):
+try:
+c = util.load_file(fn, quiet=False)
+except (IOError, OSError) as e:
+raise ConfigError("Failed loading of %s from"
+  " %s due to: %s" % (kind, fn, e))
+else:
+if fn.endswith(".tpl"):
+if tpl_params is None:
+tpl_params = {}
+tpl_params.setdefault('system', util.system_info())
+try:
+c = templater.render_string(c, tpl_params)
+except Exception as e:
+raise ConfigError(
+"Failed rendering %s from %s"
+" using params %s due to: %s" % (kind, fn,
+ tpl_params, e))
+try:
+return util.load_yaml(c, default={})
+except TypeError as e:
+raise ConfigError("Failed parsing (into yaml) %s"
+  " from %s due to: %s" % (kind, fn, e))
+
 def _get_datasource_configs(self):
 d_cfgs = []
 if self._ds:
@@ -229,10 +259,9 @@ class ConfigMerger(object):
 if CFG_ENV_NAME in os.environ:
 e_fn = os.environ[CFG_ENV_NAME]
 try:
-e_cfgs.append(util.read_conf(e_fn))
-except Exception:
-util.logexc(LOG, 'Failed loading of env. config from %s',
-e_fn)
+e_cfgs.append(self._read_conf(e_fn, "environment config"))
+except ConfigError as e:
+util.logexc(LOG, str(e))
 return e_cfgs
 
 def _get_instance_configs(self):
@@ -241,19 +270,16 @@ class ConfigMerger(object):
 # a configuration file to use when running...
 if not self._paths:
 return i_cfgs
-
-cc_paths = ['cloud_config']
+cc_paths = [("cloud config", 'cloud_config')]
 if self._include_vendor:
-cc_paths.append('vendor_cloud_config')
-
-for cc_p in cc_paths:
+cc_paths.append(("vendor cloud config", 'vendor_cloud_config'))
+for cc_kind, cc_p in cc_paths:
 cc_fn = self._paths.get_ipath_cur(cc_p)
 if cc_fn and os.path.isfile(cc_fn):
 try:
-i_cfgs.append(util.read_conf(cc_fn))
-except Exception:
-util.logexc(LOG, 'Failed loading of cloud-config from %s',
-cc_fn)
+i_cfgs.append(self._read_conf(cc_fn, cc_kind))
+except ConfigError as e:
+util.logexc(LOG, str(e))
 return i_cfgs
 
 def _read_cfg(self):
@@ -264,14 +290,11 @@ class ConfigMerger(object):
 # configs which override
 # base configuration
 cfgs = []
-if self._fns:
-for c_fn in self._fns:
-try:
-cfgs.append(util.read_conf(c_fn))
-except Exception:
-util.logexc(LOG, "Failed loading of configuration from %s",
-c_fn)
-
+for c_fn in self._fns:
+try:
+cfgs.append(self._read_conf(c_fn, "config"))
+except ConfigError as e:
+

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:ug_util into cloud-init:master

2016-09-30 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:ug_util into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307373
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:ug_util into cloud-init:master.
diff --git a/cloudinit/config/cc_byobu.py b/cloudinit/config/cc_byobu.py
old mode 100644
new mode 100755
index 1f00dd9..4a616e2
--- a/cloudinit/config/cc_byobu.py
+++ b/cloudinit/config/cc_byobu.py
@@ -50,12 +50,7 @@ Valid configuration options for this module are:
 byobu_by_default: 
 """
 
-
-# Ensure this is aliased to a name not 'distros'
-# since the module attribute 'distros'
-# is a list of distros that are supported, not a sub-module
-from cloudinit import distros as ds
-
+from cloudinit.distros import ug_util
 from cloudinit import util
 
 distros = ['ubuntu', 'debian']
@@ -94,8 +89,8 @@ def handle(name, cfg, cloud, log, args):
 
 shcmd = ""
 if mod_user:
-(users, _groups) = ds.normalize_users_groups(cfg, cloud.distro)
-(user, _user_config) = ds.extract_default(users)
+(users, _groups) = ug_util.normalize_users_groups(cfg, cloud.distro)
+(user, _user_config) = ug_util.extract_default(users)
 if not user:
 log.warn(("No default byobu user provided, "
   "can not launch %s for the default user"), bl_inst)
diff --git a/cloudinit/config/cc_set_passwords.py b/cloudinit/config/cc_set_passwords.py
old mode 100644
new mode 100755
index 9471601..6fc0051
--- a/cloudinit/config/cc_set_passwords.py
+++ b/cloudinit/config/cc_set_passwords.py
@@ -66,11 +66,7 @@ enabled, disabled, or left to system defaults using ``ssh_pwauth``.
 
 import sys
 
-# Ensure this is aliased to a name not 'distros'
-# since the module attribute 'distros'
-# is a list of distros that are supported, not a sub-module
-from cloudinit import distros as ds
-
+from cloudinit.distros import ug_util
 from cloudinit import ssh_util
 from cloudinit import util
 
@@ -99,8 +95,8 @@ def handle(_name, cfg, cloud, log, args):
 expire = util.get_cfg_option_bool(chfg, 'expire', expire)
 
 if not plist and password:
-(users, _groups) = ds.normalize_users_groups(cfg, cloud.distro)
-(user, _user_config) = ds.extract_default(users)
+(users, _groups) = ug_util.normalize_users_groups(cfg, cloud.distro)
+(user, _user_config) = ug_util.extract_default(users)
 if user:
 plist = "%s:%s" % (user, password)
 else:
diff --git a/cloudinit/config/cc_ssh.py b/cloudinit/config/cc_ssh.py
old mode 100644
new mode 100755
index 6138fb5..576fa58
--- a/cloudinit/config/cc_ssh.py
+++ b/cloudinit/config/cc_ssh.py
@@ -109,11 +109,7 @@ import glob
 import os
 import sys
 
-# Ensure this is aliased to a name not 'distros'
-# since the module attribute 'distros'
-# is a list of distros that are supported, not a sub-module
-from cloudinit import distros as ds
-
+from cloudinit.distros import ug_util
 from cloudinit import ssh_util
 from cloudinit import util
 
@@ -197,8 +193,8 @@ def handle(_name, cfg, cloud, log, _args):
 "file %s", keytype, keyfile)
 
 try:
-(users, _groups) = ds.normalize_users_groups(cfg, cloud.distro)
-(user, _user_config) = ds.extract_default(users)
+(users, _groups) = ug_util.normalize_users_groups(cfg, cloud.distro)
+(user, _user_config) = ug_util.extract_default(users)
 disable_root = util.get_cfg_option_bool(cfg, "disable_root", True)
 disable_root_opts = util.get_cfg_option_str(cfg, "disable_root_opts",
 DISABLE_ROOT_OPTS)
diff --git a/cloudinit/config/cc_ssh_authkey_fingerprints.py b/cloudinit/config/cc_ssh_authkey_fingerprints.py
old mode 100644
new mode 100755
index 6f3d0ee..7eeb0f8
--- a/cloudinit/config/cc_ssh_authkey_fingerprints.py
+++ b/cloudinit/config/cc_ssh_authkey_fingerprints.py
@@ -42,11 +42,7 @@ import hashlib
 
 from prettytable import PrettyTable
 
-# Ensure this is aliased to a name not 'distros'
-# since the module attribute 'distros'
-# is a list of distros that are supported, not a sub-module
-from cloudinit import distros as ds
-
+from cloudinit.distros import ug_util
 from cloudinit import ssh_util
 from cloudinit import util
 
@@ -119,7 +115,7 @@ def handle(name, cfg, cloud, log, _args):
 return
 
 hash_meth = util.get_cfg_option_str(cfg, "authkey_hash", "md5")
-(users, _groups) = ds.normalize_users_groups(cfg, cloud.distro)
+(users, _groups) = ug_util.normalize_users_groups(cfg, cloud.distro)
 for (user_name, _c

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:sys-io-errors into cloud-init:master

2016-09-30 Thread Joshua Harlow

Oh, lol.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:sys-io-errors into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:fix-py26 into cloud-init:master

2016-09-30 Thread Joshua Harlow

Looks fine to me, thanks scott :)
-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/307363
Your team cloud init development team is requested to review the proposed merge 
of ~smoser/cloud-init:fix-py26 into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:split-cloud-cfg into cloud-init:master

2016-09-29 Thread Joshua Harlow

So let me see how templating goes. You are right, we do have it. With 
templating I think we don't need setup.py to do the selection then.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307100
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:split-cloud-cfg into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:split-cloud-cfg into cloud-init:master

2016-09-28 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:split-cloud-cfg into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307100

Packages for redhat building we also fixed (other packagers were updated, but 
not sure if they were done right).
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:split-cloud-cfg into cloud-init:master.
diff --git a/config/cloud.cfg b/config/cloud.cfg
deleted file mode 100644
index d608dc8..000
--- a/config/cloud.cfg
+++ /dev/null
@@ -1,116 +0,0 @@
-# The top level settings are used as module
-# and system configuration.
-
-# A set of users which may be applied and/or used by various modules
-# when a 'default' entry is found it will reference the 'default_user'
-# from the distro configuration specified below
-users:
-   - default
-
-# If this is set, 'root' will not be able to ssh in and they 
-# will get a message to login instead as the above $user (ubuntu)
-disable_root: true
-
-# This will cause the set+update hostname module to not operate (if true)
-preserve_hostname: false
-
-# Example datasource config
-# datasource: 
-#Ec2: 
-#  metadata_urls: [ 'blah.com' ]
-#  timeout: 5 # (defaults to 50 seconds)
-#  max_wait: 10 # (defaults to 120 seconds)
-
-# The modules that run in the 'init' stage
-cloud_init_modules:
- - migrator
- - ubuntu-init-switch
- - seed_random
- - bootcmd
- - write-files
- - growpart
- - resizefs
- - disk_setup
- - mounts
- - set_hostname
- - update_hostname
- - update_etc_hosts
- - ca-certs
- - rsyslog
- - users-groups
- - ssh
-
-# The modules that run in the 'config' stage
-cloud_config_modules:
-# Emit the cloud config ready event
-# this can be used by upstart jobs for 'start on cloud-config'.
- - emit_upstart
- - ssh-import-id
- - locale
- - set-passwords
- - grub-dpkg
- - apt-pipelining
- - apt-configure
- - ntp
- - timezone
- - disable-ec2-metadata
- - runcmd
- - byobu
-
-# The modules that run in the 'final' stage
-cloud_final_modules:
- - snappy
- - package-update-upgrade-install
- - fan
- - landscape
- - lxd
- - puppet
- - chef
- - salt-minion
- - mcollective
- - rightscale_userdata
- - scripts-vendor
- - scripts-per-once
- - scripts-per-boot
- - scripts-per-instance
- - scripts-user
- - ssh-authkey-fingerprints
- - keys-to-console
- - phone-home
- - final-message
- - power-state-change
-
-# System and/or distro specific settings
-# (not accessible to handlers/transforms)
-system_info:
-   # This will affect which distro class gets used
-   distro: ubuntu
-   # Default user name + that default users groups (if added/used)
-   default_user:
- name: ubuntu
- lock_passwd: True
- gecos: Ubuntu
- groups: [adm, audio, cdrom, dialout, dip, floppy, lxd, netdev, plugdev, sudo, video]
- sudo: ["ALL=(ALL) NOPASSWD:ALL"]
- shell: /bin/bash
-   # Other config here will be given to the distro class and/or path classes
-   paths:
-  cloud_dir: /var/lib/cloud/
-  templates_dir: /etc/cloud/templates/
-  upstart_dir: /etc/init/
-   package_mirrors:
- - arches: [i386, amd64]
-   failsafe:
- primary: http://archive.ubuntu.com/ubuntu
- security: http://security.ubuntu.com/ubuntu
-   search:
- primary:
-   - http://%(ec2_region)s.ec2.archive.ubuntu.com/ubuntu/
-   - http://%(availability_zone)s.clouds.archive.ubuntu.com/ubuntu/
-   - http://%(region)s.clouds.archive.ubuntu.com/ubuntu/
- security: []
- - arches: [armhf, armel, default]
-   failsafe:
- primary: http://ports.ubuntu.com/ubuntu-ports
- security: http://ports.ubuntu.com/ubuntu-ports
-   ssh_svcname: ssh
diff --git a/config/cloud.cfg-fedora b/config/cloud.cfg-fedora
new file mode 100644
index 000..0c18a1c
--- /dev/null
+++ b/config/cloud.cfg-fedora
@@ -0,0 +1,103 @@
+# The top level settings are used as module
+# and system configuration.
+
+# A set of users which may be applied and/or used by various modules
+# when a 'default' entry is found it will reference the 'default_user'
+# from the distro configuration specified below
+users:
+   - default
+
+# If this is set, 'root' will not be able to ssh in and they 
+# will get a message to login instead as the default user (fedora in this
+# case).
+disable_root: true
+
+# This will cause the set+update hostname module to not operate (if true)
+preserve_hostname: false
+
+# Example datasource config
+# datasource: 
+#Ec2: 
+#  metadata_urls: [ 'blah.com' ]
+#  timeout: 5 # (defaults to 50 seconds)
+#  max_wait: 10 # (defaults to 120 seconds)
+
+# The modules that run in the 'init' stage
+cloud_init_modules:
+ - migrator
+ - seed_random
+ - bootcmd
+ # S

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:just-file-logs into cloud-init:master

2016-09-28 Thread Joshua Harlow

Ya, so I'm on the fence a little on this. If this change merges people that 
were using syslog will no longer (by default) see the syslog stuff, that's bad. 
But the was it working in a systemd world anymore, perhaps not, so that's good. 
Is there any middle area for this where the old and the new can work (and not 
be crazy complicated?)
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301729
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:just-file-logs into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:sys-io-errors into cloud-init:master

2016-09-21 Thread Joshua Harlow

Ya, that makes sense to me, I can adjust this one.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:sys-io-errors into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:local-before-nm into cloud-init:master

2016-09-20 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:local-before-nm into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/306283
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:local-before-nm into cloud-init:master.
diff --git a/systemd/cloud-init-local.service b/systemd/cloud-init-local.service
index bc2db60..4f8dd67 100644
--- a/systemd/cloud-init-local.service
+++ b/systemd/cloud-init-local.service
@@ -7,7 +7,11 @@ After=local-fs.target
 Before=basic.target
 Before=network-pre.target
 Before=shutdown.target
+<<<<<<< systemd/cloud-init-local.service
 Conflicts=shutdown.target
+===
+Before=NetworkManager.service
+>>>>>>> systemd/cloud-init-local.service
 
 [Service]
 Type=oneshot
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:sys-io-errors into cloud-init:master

2016-09-15 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:sys-io-errors into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:sys-io-errors into cloud-init:master.
diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
index 7e58bfe..b48a234 100644
--- a/cloudinit/net/__init__.py
+++ b/cloudinit/net/__init__.py
@@ -147,7 +147,7 @@ def generate_fallback_config():
 if carrier:
 connected.append(interface)
 continue
-except OSError:
+except (IOError, OSError):
 pass
 # check if nic is dormant or down, as this may make a nick appear to
 # not have a carrier even though it could acquire one when brought
@@ -157,14 +157,14 @@ def generate_fallback_config():
 if dormant:
 possibly_connected.append(interface)
 continue
-except OSError:
+except (IOError, OSError):
 pass
 try:
 operstate = sys_netdev_info(interface, 'operstate')
 if operstate in ['dormant', 'down', 'lowerlayerdown', 'unknown']:
 possibly_connected.append(interface)
 continue
-except OSError:
+except (IOError, OSError):
 pass
 
 # don't bother with interfaces that might not be connected if there are
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:fix-26 into cloud-init:master

2016-08-25 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:fix-26 into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/304001
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:fix-26 into cloud-init:master.
diff --git a/cloudinit/util.py b/cloudinit/util.py
index 9c89de6..7c37eb8 100644
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -113,7 +113,7 @@ def _lsb_release(target=None):
 
 except ProcessExecutionError as err:
 LOG.warn("Unable to get lsb_release --all: %s", err)
-data = {v: "UNAVAILABLE" for v in fmap.values()}
+data = dict((v, "UNAVAILABLE") for v in fmap.values())
 
 return data
 
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:newer-configobj into cloud-init:master

2016-08-11 Thread Joshua Harlow

Shouldn't be a problem anymore.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301728
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:newer-configobj into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:newer-configobj into cloud-init:master

2016-08-10 Thread Joshua Harlow

Sure.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301728
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:newer-configobj into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:config-entrypoints into cloud-init:master

2016-08-10 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:config-entrypoints into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/302609
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:config-entrypoints into cloud-init:master.
diff --git a/cloudinit/config/__init__.py b/cloudinit/config/__init__.py
index d57453b..948ce5f 100644
--- a/cloudinit/config/__init__.py
+++ b/cloudinit/config/__init__.py
@@ -39,8 +39,6 @@ def form_module_name(name):
 canon_name = canon_name.strip()
 if not canon_name:
 return None
-if not canon_name.startswith(MOD_PREFIX):
-canon_name = '%s%s' % (MOD_PREFIX, canon_name)
 return canon_name
 
 
diff --git a/cloudinit/importer.py b/cloudinit/importer.py
index fb57253..fb68d99 100644
--- a/cloudinit/importer.py
+++ b/cloudinit/importer.py
@@ -20,6 +20,7 @@
 #You should have received a copy of the GNU General Public License
 #along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import pkg_resources
 import sys
 
 
@@ -56,3 +57,13 @@ def find_module(base_name, search_paths, required_attrs=None):
 if found_attrs == len(required_attrs):
 found_paths.append(full_path)
 return (found_paths, lookup_paths)
+
+
+def iter_entry_points(namespace, loader=None, name=None):
+for e in pkg_resources.iter_entry_points(namespace, name=name):
+if loader is None:
+thing = e.load()
+else:
+thing = loader(e)
+if thing is not None:
+yield (e.name, thing)
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
index 47deac6..beb3cba 100644
--- a/cloudinit/stages.py
+++ b/cloudinit/stages.py
@@ -54,6 +54,7 @@ LOG = logging.getLogger(__name__)
 
 NULL_DATA_SOURCE = None
 NO_PREVIOUS_INSTANCE_ID = "NO_PREVIOUS_INSTANCE_ID"
+EP_CONFIG_NAMESPACE = 'cloudinit.config'
 
 
 class Init(object):
@@ -731,21 +732,39 @@ class Modules(object):
 raw_name = raw_mod['mod']
 freq = raw_mod.get('freq')
 run_args = raw_mod.get('args') or []
-mod_name = config.form_module_name(raw_name)
-if not mod_name:
+canon_name = config.form_module_name(raw_name)
+if not canon_name:
 continue
+ep_locs = list(
+importer.iter_entry_points(EP_CONFIG_NAMESPACE,
+   name=canon_name))
+if not ep_locs:
+LOG.warn("Could not find entrypoint matching name '%s'"
+ " (which is the canonicalized form of '%s')"
+ " registered under namespace '%s'",
+ canon_name, raw_name, EP_CONFIG_NAMESPACE)
+continue
+ep_names = [mod_name for (mod_name, _mod) in ep_locs]
+if len(ep_names) > 1:
+LOG.warn("Selecting '%s' entrypoint for configuration"
+ " module '%s' (which is the canonicalized form"
+ " of '%s') even though %s other entrypoints were"
+ " found", ep_names[0], canon_name, raw_name,
+ ep_names)
+else:
+LOG.debug("Selecting '%s' entrypoint for configuration"
+  " module '%s' (which is the canonicalized"
+  " form of '%s')", ep_names[0], canon_name, raw_name)
+# Entry points are yielded from the active distributions in
+# the order that the distributions appear on sys.path
+ep_name, mod = ep_locs[0]
+mod = config.fixup_module(mod)
 if freq and freq not in FREQUENCIES:
-LOG.warn(("Config specified module %s"
-  " has an unknown frequency %s"), raw_name, freq)
+LOG.warn("Config specified module '%s' (found at entrypoint"
+ " '%s') has an unknown frequency %s",
+ raw_name, ep_name, freq)
 # Reset it so when ran it will get set to a known value
 freq = None
-mod_locs, looked_locs = importer.find_module(
-mod_name, ['', type_utils.obj_name(config)], ['handle'])
-if not mod_locs:
-LOG.warn("Could not find module named %s (searched %s)",
- mod_name, looked_locs)
-continue
-mod = config.fix

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:just-a-tar into cloud-init:master

2016-08-09 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:just-a-tar into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/302479
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:just-a-tar into cloud-init:master.
diff --git a/tools/make-tarball b/tools/make-tarball
index d870589..7a2a1ee 100755
--- a/tools/make-tarball
+++ b/tools/make-tarball
@@ -55,5 +55,11 @@ if [ "$rev" = HEAD ] && ! git diff-index --quiet HEAD --; then
 fi
 fi
 
-git archive --format=tar.gz --prefix="$archive_base/" "$rev" > "$output"
+TEMP_D=$(mktemp -d)
+working_dir=$PWD
+# this just uses tar because older versions of git are feature lacking...
+git archive --format=tar --prefix="$archive_base/" "$rev" > "$TEMP_D/$archive_base.tar"
+cd $TEMP_D/
+gzip "$archive_base.tar"
+mv "$archive_base.tar.gz" $working_dir/$output
 echo "$output"
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:fix-gce into cloud-init:master

2016-08-08 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:fix-gce into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/302350
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:fix-gce into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceGCE.py b/cloudinit/sources/DataSourceGCE.py
index c660a35..6c12d70 100644
--- a/cloudinit/sources/DataSourceGCE.py
+++ b/cloudinit/sources/DataSourceGCE.py
@@ -31,7 +31,7 @@ REQUIRED_FIELDS = ('instance-id', 'availability-zone', 'local-hostname')
 
 
 class GoogleMetadataFetcher(object):
-headers = {'X-Google-Metadata-Request': True}
+headers = {'X-Google-Metadata-Request': 'True'}
 
 def __init__(self, metadata_address):
 self.metadata_address = metadata_address
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:scm-version into cloud-init:master

2016-08-03 Thread Joshua Harlow

So https://review.openstack.org/#/c/350861 is the addition to pbr for the 'deb 
version' so this imho makes me lean towards using pbr vs this package, and 
using either over a custom hand rolled version calculation scheme. Because with 
a hand rolled scheme, I'll still need to be translating the output of that hand 
rolled scheme into a RPM compatible form, which means, at that point a 
automatic version derived from git isn't offering that much value (because a 
human still needs to translate it) :(
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301727
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:scm-version into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~harlowja/cloud-init:scm-version into cloud-init:master

2016-08-03 Thread Joshua Harlow

I guess the only thing that I can think about that makes using these tools 
useful is that they can provide a consistent way of getting that version 
(instead of yet another one-off way of getting it); pbr also has the nice 
ability to generate a debian version string and a redhat version string, which 
are quite useful when actually creating a package.

For example:

$ python setup.py rpm_version
running rpm_version
[pbr] Extracting rpm version
1.32.0.dev6

Not quite sure where the pbr output for deb_version is (thought it had this, 
ha).

That's usually the next question, ok, now that we have a version string crap 
here, well now I need a debian version of that string and I need a redhat 
version of that string.
-- 
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301727
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:scm-version into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:space into cloud-init:master

2016-08-01 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:space into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301731
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:space into cloud-init:master.
diff --git a/cloudinit/config/cc_spacewalk.py b/cloudinit/config/cc_spacewalk.py
new file mode 100644
index 000..d82cfbf
--- /dev/null
+++ b/cloudinit/config/cc_spacewalk.py
@@ -0,0 +1,67 @@
+# vi: ts=4 expandtab
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+**Summary:** helper to setup https://fedorahosted.org/spacewalk/
+
+**Description:** This module will enable for configuring the needed
+actions to setup spacewalk on redhat based systems.
+
+It can be configured with the following option structure::
+
+spacewalk:
+   proxy: spacewalk api proxy (required)
+   activation_key: spacewalk api activation key to use (required)
+"""
+
+from cloudinit import util
+
+
+distros = ['redhat', 'fedora']
+
+
+def handle(name, cfg, cloud, log, _args):
+if 'spacewalk' not in cfg:
+log.debug(("Skipping module named %s,"
+   " no 'spacewalk' key in configuration"), name)
+return
+cfg = cfg['spacewalk']
+spacewalk_proxy = cfg.get('proxy')
+spacewalk_activation_key = cfg.get('activation_key')
+if spacewalk_proxy and spacewalk_activation_key:
+distro = cloud.distro
+# Need to have this installed before further things will work.
+distro.install_packages(['rhn-setup'])
+# Check to see if already registered and don't bother; this is
+# apparently done by trying to sync and if that fails then we
+# assume we aren't registered; which is sorta ghetto...
+already_registered = False
+try:
+util.subp(['rhn-profile-sync', '--verbose'], capture=False)
+already_registered = True
+except util.ProcessExecutionError as e:
+if e.exit_code != 1:
+raise
+if not already_registered:
+util.subp([
+'rhnreg_ks',
+'--serverUrl=https://%s/XMLRPC' % spacewalk_proxy,
+'--sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT',
+'--activationkey=%s' % spacewalk_activation_key,
+'--profilename=%s' % cloud.datasource.get_hostname(fqdn=True),
+], capture=False)
+else:
+log.debug(("Skipping module named %s,"
+   " 'spacewalk/proxy' or 'spacewalk/activation_key' key(s)"
+   " were not found in configured data"), name)
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:newer-configobj into cloud-init:master

2016-08-01 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:newer-configobj into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301728
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:newer-configobj into cloud-init:master.
diff --git a/requirements.txt b/requirements.txt
index cc1dc05..0c4951f 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -22,7 +22,7 @@ oauthlib
 # that the built-in config parser is not sufficent (ie
 # when we need to preserve comments, or do not have a top-level
 # section)...
-configobj
+configobj>=5.0.2
 
 # All new style configurations are in the yaml format
 pyyaml
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:just-file-logs into cloud-init:master

2016-08-01 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:just-file-logs into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301729
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:just-file-logs into cloud-init:master.
diff --git a/config/cloud.cfg.d/05_logging.cfg b/config/cloud.cfg.d/05_logging.cfg
index 2e18073..aeecc42 100644
--- a/config/cloud.cfg.d/05_logging.cfg
+++ b/config/cloud.cfg.d/05_logging.cfg
@@ -54,13 +54,21 @@ _log:
 
 log_cfgs:
 # These will be joined into a string that defines the configuration
- - [ *log_base, *log_syslog ]
+#
+# If your desire syslog, just uncomment the following line.
+# - [ *log_base, *log_syslog ]
+#
 # These will be joined into a string that defines the configuration
+#
+# Otherwise by default we just use the most basic (known to work
+# mechanism that does not depend on a system service to operate; which
+# due to cloud-init execution lifecycle we may not always be able to
+# depend on working correctly).
  - [ *log_base, *log_file ]
-# A file path can also be used
+# A file path can also be used.
 # - /etc/log.conf
 
-# this tells cloud-init to redirect its stdout and stderr to
+# This tells cloud-init to redirect its stdout and stderr to
 # 'tee -a /var/log/cloud-init-output.log' so the user can see output
 # there without needing to look on the console.
 output: {all: '| tee -a /var/log/cloud-init-output.log'}
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:fix-distro-tags into cloud-init:master

2016-08-01 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:fix-distro-tags into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301730



-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:fix-distro-tags into cloud-init:master.
diff --git a/cloudinit/config/cc_lxd.py b/cloudinit/config/cc_lxd.py
index 70d4e7c..0086840 100644
--- a/cloudinit/config/cc_lxd.py
+++ b/cloudinit/config/cc_lxd.py
@@ -47,6 +47,8 @@ Example config:
 
 from cloudinit import util
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 # Get config
diff --git a/cloudinit/config/cc_rh_subscription.py b/cloudinit/config/cc_rh_subscription.py
index 3a113ae..d4ad724 100644
--- a/cloudinit/config/cc_rh_subscription.py
+++ b/cloudinit/config/cc_rh_subscription.py
@@ -18,6 +18,8 @@
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def handle(name, cfg, _cloud, log, _args):
 sm = SubscriptionManager(cfg)
diff --git a/cloudinit/config/cc_snappy.py b/cloudinit/config/cc_snappy.py
index 1a485ee..6bcd838 100644
--- a/cloudinit/config/cc_snappy.py
+++ b/cloudinit/config/cc_snappy.py
@@ -68,6 +68,8 @@ BUILTIN_CFG = {
 'config': {},
 }
 
+distros = ['ubuntu']
+
 
 def parse_filename(fname):
 fname = os.path.basename(fname)
diff --git a/cloudinit/config/cc_ubuntu_init_switch.py b/cloudinit/config/cc_ubuntu_init_switch.py
index 884d79f..bffb438 100644
--- a/cloudinit/config/cc_ubuntu_init_switch.py
+++ b/cloudinit/config/cc_ubuntu_init_switch.py
@@ -86,6 +86,8 @@ else
 fi
 """
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 """Handler method activated by cloud-init."""
diff --git a/cloudinit/config/cc_yum_add_repo.py b/cloudinit/config/cc_yum_add_repo.py
index 64fba86..22549e6 100644
--- a/cloudinit/config/cc_yum_add_repo.py
+++ b/cloudinit/config/cc_yum_add_repo.py
@@ -23,6 +23,8 @@ import six
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def _canonicalize_id(repo_id):
 repo_id = repo_id.lower().replace("-", "_")
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:scm-version into cloud-init:master

2016-08-01 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:scm-version into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301727
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:scm-version into cloud-init:master.
diff --git a/cloudinit/version.py b/cloudinit/version.py
index 3d1d1d2..4fc958c 100644
--- a/cloudinit/version.py
+++ b/cloudinit/version.py
@@ -16,12 +16,4 @@
 #You should have received a copy of the GNU General Public License
 #along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-from distutils import version as vr
-
-
-def version():
-return vr.StrictVersion("0.7.7")
-
-
-def version_string():
-return str(version())
+from setuptools_scm import get_version as version_string
diff --git a/requirements.txt b/requirements.txt
index cc1dc05..b5a4ef2 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,4 +1,5 @@
 # Pypi requirements for cloud-init to work
+setuptools_scm
 
 # Used for untemplating any files or strings with parameters.
 jinja2
diff --git a/setup.py b/setup.py
index 0af576a..b07d0d2 100755
--- a/setup.py
+++ b/setup.py
@@ -209,6 +209,8 @@ setuptools.setup(
 data_files=data_files,
 install_requires=requirements,
 cmdclass=cmdclass,
+use_scm_version=True,
+setup_requires=['setuptools_scm'],
 entry_points={
 'console_scripts': [
 'cloud-init = cloudinit.cmd.main:main'
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:update-configobj into cloud-init:master

2016-07-29 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:update-configobj into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301541
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:update-configobj into cloud-init:master.
diff --git a/requirements.txt b/requirements.txt
index cc1dc05..ebadda2 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -22,7 +22,11 @@ oauthlib
 # that the built-in config parser is not sufficent (ie
 # when we need to preserve comments, or do not have a top-level
 # section)...
+<<<<<<< requirements.txt
 configobj
+===
+configobj>=5.0.2
+>>>>>>> requirements.txt
 
 # All new style configurations are in the yaml format
 pyyaml
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:spacewalk into cloud-init:master

2016-07-26 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:spacewalk into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301224
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:spacewalk into cloud-init:master.
diff --git a/cloudinit/config/cc_spacewalk.py b/cloudinit/config/cc_spacewalk.py
new file mode 100644
index 000..d82cfbf
--- /dev/null
+++ b/cloudinit/config/cc_spacewalk.py
@@ -0,0 +1,67 @@
+# vi: ts=4 expandtab
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+**Summary:** helper to setup https://fedorahosted.org/spacewalk/
+
+**Description:** This module will enable for configuring the needed
+actions to setup spacewalk on redhat based systems.
+
+It can be configured with the following option structure::
+
+spacewalk:
+   proxy: spacewalk api proxy (required)
+   activation_key: spacewalk api activation key to use (required)
+"""
+
+from cloudinit import util
+
+
+distros = ['redhat', 'fedora']
+
+
+def handle(name, cfg, cloud, log, _args):
+if 'spacewalk' not in cfg:
+log.debug(("Skipping module named %s,"
+   " no 'spacewalk' key in configuration"), name)
+return
+cfg = cfg['spacewalk']
+spacewalk_proxy = cfg.get('proxy')
+spacewalk_activation_key = cfg.get('activation_key')
+if spacewalk_proxy and spacewalk_activation_key:
+distro = cloud.distro
+# Need to have this installed before further things will work.
+distro.install_packages(['rhn-setup'])
+# Check to see if already registered and don't bother; this is
+# apparently done by trying to sync and if that fails then we
+# assume we aren't registered; which is sorta ghetto...
+already_registered = False
+try:
+util.subp(['rhn-profile-sync', '--verbose'], capture=False)
+already_registered = True
+except util.ProcessExecutionError as e:
+if e.exit_code != 1:
+raise
+if not already_registered:
+util.subp([
+'rhnreg_ks',
+'--serverUrl=https://%s/XMLRPC' % spacewalk_proxy,
+'--sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT',
+'--activationkey=%s' % spacewalk_activation_key,
+'--profilename=%s' % cloud.datasource.get_hostname(fqdn=True),
+], capture=False)
+else:
+log.debug(("Skipping module named %s,"
+   " 'spacewalk/proxy' or 'spacewalk/activation_key' key(s)"
+   " were not found in configured data"), name)
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~larsks/cloud-init:feature/move-to-git into cloud-init:master

2016-07-25 Thread Joshua Harlow



Diff comments:

> diff --git a/README.rst b/README.rst
> new file mode 100644
> index 000..e56e663
> --- /dev/null
> +++ b/README.rst
> @@ -0,0 +1,15 @@
> +Cloud-init
> +==
> +
> +Cloud-init is the defacto multi-distribution package that handles
> +early initialization of a cloud instance.
> +
> +The package is hosted at .

Is this updated, seems pretty old. Whose repo is this :-/

> +
> +Additional documentation can be found in the `doc/` subdirectory.
> +
> +Contributing
> +
> +
> +For information about how to contribute to cloud-init development, see
> +the file ``HACKING.rst`` included in this distribution.
> diff --git a/packages/redhat/cloud-init.spec.in 
> b/packages/redhat/cloud-init.spec.in
> index 254d209..71c71be 100644
> --- a/packages/redhat/cloud-init.spec.in
> +++ b/packages/redhat/cloud-init.spec.in
> @@ -5,13 +5,9 @@
>  # Or: http://fedoraproject.org/wiki/Packaging:ScriptletSnippets
>  # Or: http://www.rpm.org/max-rpm/ch-rpm-inside.html
>  
> -#for $d in $defines
> -%define ${d}
> -#end for
> -
>  Name:   cloud-init
>  Version:${version}
> -Release:${release}${subrelease}%{?dist}
> +Release:1${subrelease}%{?dist}

The 1 is the epoch?

>  Summary:Cloud instance init scripts
>  
>  Group:  System Environment/Base
> diff --git a/packages/suse/cloud-init.spec.in 
> b/packages/suse/cloud-init.spec.in
> index 53e6ad1..74b6f12 100644
> --- a/packages/suse/cloud-init.spec.in
> +++ b/packages/suse/cloud-init.spec.in
> @@ -157,7 +153,3 @@ mkdir -p %{buildroot}/var/lib/cloud
>  %{python_sitelib}/*
>  
>  /var/lib/cloud
> -
> -%changelog
> -
> -${changelog}

Damn, didn't like my changelog ;)

Yes I know redhat folks like to manually curate the changelog (and really its 
the spec file changelog, not the package/code changelog, which always confused 
me, but whatever, ha).



-- 
https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/300953
Your team cloud init development team is requested to review the proposed merge 
of ~larsks/cloud-init:feature/move-to-git into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:skip-things into cloud-init:master

2016-07-25 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:skip-things into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301110
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:skip-things into cloud-init:master.
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
index 47deac6..72c0b94 100644
--- a/cloudinit/stages.py
+++ b/cloudinit/stages.py
@@ -808,7 +808,7 @@ class Modules(object):
 def run_section(self, section_name):
 raw_mods = self._read_modules(section_name)
 mostly_mods = self._fixup_modules(raw_mods)
-d_name = self.init.distro.name
+distro_name = self.init.distro.name
 
 skipped = []
 forced = []
@@ -819,7 +819,7 @@ class Modules(object):
 distros.Distro.expand_osfamily(mod.osfamilies))
 
 # module does not declare 'distros' or lists this distro
-if not worked_distros or d_name in worked_distros:
+if not worked_distros or distro_name in worked_distros:
 continue
 
 if name in overridden:
@@ -827,14 +827,20 @@ class Modules(object):
 else:
 skipped.append(name)
 
+cleaned_mostly_mods = []
+for (mod, name, freq, args) in mostly_mods:
+if name in skipped:
+continue
+cleaned_mostly_mods.append((mod, name, freq, args))
+
 if skipped:
 LOG.info("Skipping modules %s because they are not verified "
  "on distro '%s'.  To run anyway, add them to "
- "'unverified_modules' in config.", skipped, d_name)
+ "'unverified_modules' in config.", skipped, distro_name)
 if forced:
-LOG.info("running unverified_modules: %s", forced)
+LOG.info("Running unverified modules: %s", forced)
 
-return self._run_modules(mostly_mods)
+return self._run_modules(cleaned_mostly_mods)
 
 
 def fetch_base_config():
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:update-configobj into cloud-init:master

2016-07-25 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:update-configobj into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301109
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:update-configobj into cloud-init:master.
diff --git a/requirements.txt b/requirements.txt
index cc1dc05..0c4951f 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -22,7 +22,7 @@ oauthlib
 # that the built-in config parser is not sufficent (ie
 # when we need to preserve comments, or do not have a top-level
 # section)...
-configobj
+configobj>=5.0.2
 
 # All new style configurations are in the yaml format
 pyyaml
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:distro-tags into cloud-init:master

2016-07-25 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:distro-tags into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301108
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:distro-tags into cloud-init:master.
diff --git a/cloudinit/config/cc_lxd.py b/cloudinit/config/cc_lxd.py
index 70d4e7c..0086840 100644
--- a/cloudinit/config/cc_lxd.py
+++ b/cloudinit/config/cc_lxd.py
@@ -47,6 +47,8 @@ Example config:
 
 from cloudinit import util
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 # Get config
diff --git a/cloudinit/config/cc_rh_subscription.py b/cloudinit/config/cc_rh_subscription.py
index 3a113ae..d4ad724 100644
--- a/cloudinit/config/cc_rh_subscription.py
+++ b/cloudinit/config/cc_rh_subscription.py
@@ -18,6 +18,8 @@
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def handle(name, cfg, _cloud, log, _args):
 sm = SubscriptionManager(cfg)
diff --git a/cloudinit/config/cc_snappy.py b/cloudinit/config/cc_snappy.py
index 1a485ee..6bcd838 100644
--- a/cloudinit/config/cc_snappy.py
+++ b/cloudinit/config/cc_snappy.py
@@ -68,6 +68,8 @@ BUILTIN_CFG = {
 'config': {},
 }
 
+distros = ['ubuntu']
+
 
 def parse_filename(fname):
 fname = os.path.basename(fname)
diff --git a/cloudinit/config/cc_ubuntu_init_switch.py b/cloudinit/config/cc_ubuntu_init_switch.py
index 884d79f..bffb438 100644
--- a/cloudinit/config/cc_ubuntu_init_switch.py
+++ b/cloudinit/config/cc_ubuntu_init_switch.py
@@ -86,6 +86,8 @@ else
 fi
 """
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 """Handler method activated by cloud-init."""
diff --git a/cloudinit/config/cc_yum_add_repo.py b/cloudinit/config/cc_yum_add_repo.py
index 64fba86..22549e6 100644
--- a/cloudinit/config/cc_yum_add_repo.py
+++ b/cloudinit/config/cc_yum_add_repo.py
@@ -23,6 +23,8 @@ import six
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def _canonicalize_id(repo_id):
 repo_id = repo_id.lower().replace("-", "_")
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~harlowja/cloud-init:file-logging into cloud-init:master

2016-07-25 Thread Joshua Harlow

Joshua Harlow has proposed merging ~harlowja/cloud-init:file-logging into 
cloud-init:master.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301107
-- 
Your team cloud init development team is requested to review the proposed merge 
of ~harlowja/cloud-init:file-logging into cloud-init:master.
diff --git a/config/cloud.cfg.d/05_logging.cfg b/config/cloud.cfg.d/05_logging.cfg
index 2e18073..aeecc42 100644
--- a/config/cloud.cfg.d/05_logging.cfg
+++ b/config/cloud.cfg.d/05_logging.cfg
@@ -54,13 +54,21 @@ _log:
 
 log_cfgs:
 # These will be joined into a string that defines the configuration
- - [ *log_base, *log_syslog ]
+#
+# If your desire syslog, just uncomment the following line.
+# - [ *log_base, *log_syslog ]
+#
 # These will be joined into a string that defines the configuration
+#
+# Otherwise by default we just use the most basic (known to work
+# mechanism that does not depend on a system service to operate; which
+# due to cloud-init execution lifecycle we may not always be able to
+# depend on working correctly).
  - [ *log_base, *log_file ]
-# A file path can also be used
+# A file path can also be used.
 # - /etc/log.conf
 
-# this tells cloud-init to redirect its stdout and stderr to
+# This tells cloud-init to redirect its stdout and stderr to
 # 'tee -a /var/log/cloud-init-output.log' so the user can see output
 # there without needing to look on the console.
 output: {all: '| tee -a /var/log/cloud-init-output.log'}
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-tag-distros into lp:cloud-init

2016-07-22 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-tag-distros into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-tag-distros/+merge/300840
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-tag-distros into lp:cloud-init.
=== modified file 'cloudinit/config/cc_lxd.py'
--- cloudinit/config/cc_lxd.py	2016-06-03 04:42:22 +
+++ cloudinit/config/cc_lxd.py	2016-07-22 07:10:01 +
@@ -47,6 +47,8 @@
 
 from cloudinit import util
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 # Get config

=== modified file 'cloudinit/config/cc_rh_subscription.py'
--- cloudinit/config/cc_rh_subscription.py	2016-04-06 17:57:00 +
+++ cloudinit/config/cc_rh_subscription.py	2016-07-22 07:10:01 +
@@ -18,6 +18,8 @@
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def handle(name, cfg, _cloud, log, _args):
 sm = SubscriptionManager(cfg)

=== modified file 'cloudinit/config/cc_snappy.py'
--- cloudinit/config/cc_snappy.py	2016-05-12 17:56:26 +
+++ cloudinit/config/cc_snappy.py	2016-07-22 07:10:01 +
@@ -68,6 +68,8 @@
 'config': {},
 }
 
+distros = ['ubuntu']
+
 
 def parse_filename(fname):
 fname = os.path.basename(fname)

=== modified file 'cloudinit/config/cc_ubuntu_init_switch.py'
--- cloudinit/config/cc_ubuntu_init_switch.py	2016-05-12 17:56:26 +
+++ cloudinit/config/cc_ubuntu_init_switch.py	2016-07-22 07:10:01 +
@@ -86,6 +86,8 @@
 fi
 """
 
+distros = ['ubuntu']
+
 
 def handle(name, cfg, cloud, log, args):
 """Handler method activated by cloud-init."""

=== modified file 'cloudinit/config/cc_yum_add_repo.py'
--- cloudinit/config/cc_yum_add_repo.py	2016-03-03 22:20:10 +
+++ cloudinit/config/cc_yum_add_repo.py	2016-07-22 07:10:01 +
@@ -23,6 +23,8 @@
 
 from cloudinit import util
 
+distros = ['fedora', 'rhel']
+
 
 def _canonicalize_id(repo_id):
 repo_id = repo_id.lower().replace("-", "_")

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~kaihuan-pkh/cloud-init/cloud-init into lp:cloud-init

2016-07-21 Thread Joshua Harlow



Diff comments:

> 
> === added file 'cloudinit/sources/DataSourceAliYun.py'
> --- cloudinit/sources/DataSourceAliYun.py 1970-01-01 00:00:00 +
> +++ cloudinit/sources/DataSourceAliYun.py 2016-07-19 05:45:39 +
> @@ -0,0 +1,115 @@
> +# vi: ts=4 expandtab
> +#
> +#Copyright (C) 2009-2010 Canonical Ltd.
> +#Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
> +#Copyright (C) 2012 Yahoo! Inc.
> +#
> +#Author: Scott Moser 
> +#    Author: Juerg Hafliger 
> +#Author: Joshua Harlow 
> +#
> +#This program is free software: you can redistribute it and/or modify
> +#it under the terms of the GNU General Public License version 3, as
> +#published by the Free Software Foundation.
> +#
> +#This program is distributed in the hope that it will be useful,
> +#but WITHOUT ANY WARRANTY; without even the implied warranty of
> +#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +#GNU General Public License for more details.
> +#
> +#You should have received a copy of the GNU General Public License
> +#along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +
> +import os
> +import time
> +from copy import deepcopy
> +
> +from cloudinit import ec2_utils as ec2
> +from cloudinit import log as logging
> +from cloudinit import sources
> +from cloudinit import url_helper as uhelp
> +from cloudinit import util
> +from cloudinit.sources import DataSourceEc2 as EC2
> +
> +LOG = logging.getLogger(__name__)
> +
> +DEF_MD_URL = "http://100.100.100.200";
> +
> +DEF_MD_VERSION = 'latest'
> +
> +# Default metadata urls that will be used if none are provided
> +# They will be checked for 'resolveability' and some of the
> +# following may be discarded if they do not resolve
> +DEF_MD_URLS = [DEF_MD_URL,]
> +
> +
> +class DataSourceAliYun(EC2.DataSourceEc2):
> +EC2.DEF_MD_URL = DEF_MD_URL
> +EC2.DEF_MD_URLS = DEF_MD_URLS
> +
> +def __init__(self, sys_cfg, distro, paths):
> +super(DataSourceAliYun, self).__init__(sys_cfg, distro, paths)
> +self.metadata_address = DEF_MD_URL
> +self.seed_dir = os.path.join(paths.seed_dir, "AliYun")
> +self.api_ver = DEF_MD_VERSION
> +
> +def get_instance_id(self):
> +if not self.metadata:
> +url = uhelp.combine_url(self.metadata_address, self.api_ver, 
> 'meta-data/', 'instance-id')
> +try:
> +out = util.read_file_or_url(url=url, ssl_details=None, 
> timeout=5, retries=5)
> +except Exception as e:
> +out = ''
> +finally:
> +return str(out).strip()
> +else:
> +return self.metadata.get('instance-id', '')
> +
> +def get_hostname(self, fqdn=False, _resolve_ip=False):
> +return self.metadata.get('hostname', 'localhost.localdomain')
> +
> +def get_public_ssh_keys(self):
> +try:
> +public_keys = self.metadata.get('public-keys')
> +if not public_keys: return []
> +except:

How does this except case get triggered?

> +return []
> +
> +keys = []
> +for key_id, key_body in public_keys.items():
> +if isinstance(key_body, (str, basestring)):
> +keys.append( key_body.strip() )
> +elif isinstance(key_body, list):
> +keys.extend( key_body )
> +elif isinstance(key_body, dict):
> +for key_type in ('openssh-key', 'ssh-key'):
> +key = key_body.get(key_type)
> +if isinstance(key, str):
> +keys.append( key.strip() )
> +elif isinstance(key, list):
> +keys.extend(key)
> +return keys
> +
> +def get_ntp_conf(self):
> +if not self.metadata:
> +return {}
> +return self.metadata.get('ntp-conf')
> +
> +def get_source_address(self):
> +if not self.metadata or not self.metadata.get('source-address'):
> +return []
> +source_address = self.metadata.get('source-address')
> +if isinstance(source_address, str):
> +source_address = [source_address]
> +return source_address
> +
> +
> +# Used to match classes to dependencies
> +datasources = [
> +  (DataSourceAliYun, (sources.DEP_FILESYSTEM, sources.DEP_NETWORK)),
> +]
> +
> +
> +# Return a list of data source

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-configobj-bound into lp:cloud-init

2016-07-21 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-fix-configobj-bound into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)
Related bugs:
  Bug #1603238 in cloud-init: "BOM error updating hostname on centos6.x"
  https://bugs.launchpad.net/cloud-init/+bug/1603238

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-configobj-bound/+merge/300826
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-configobj-bound into lp:cloud-init.
=== modified file 'requirements.txt'
--- requirements.txt	2016-05-11 21:18:02 +
+++ requirements.txt	2016-07-22 01:30:00 +
@@ -22,7 +22,7 @@
 # that the built-in config parser is not sufficent (ie
 # when we need to preserve comments, or do not have a top-level
 # section)...
-configobj
+configobj>=5.0.2
 
 # All new style configurations are in the yaml format
 pyyaml

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check into lp:cloud-init

2016-07-21 Thread Joshua Harlow

Yup, I updated it to try that approach instead; seems like its better IMHO.
-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-dynamic-distro-check/+merge/300805
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check into lp:cloud-init

2016-07-21 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-dynamic-distro-check/+merge/300805
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-dynamic-distro-check into lp:cloud-init.
=== modified file 'cloudinit/stages.py'
--- cloudinit/stages.py	2016-06-10 21:22:17 +
+++ cloudinit/stages.py	2016-07-21 18:53:48 +
@@ -808,31 +808,43 @@
 def run_section(self, section_name):
 raw_mods = self._read_modules(section_name)
 mostly_mods = self._fixup_modules(raw_mods)
-d_name = self.init.distro.name
+distro = self.init.distro
+
+def is_usable_here(mod):
+# Ok how this works is the following, if the module provides
+# a distro set and its empty, then its accepted to work everywhere
+# if it is not empty and the current distro is in that set
+# then its accepted to work, otherwise if the module provides
+# a 'is_usable_on' function that takes one argument (the distro
+# being loaded) and returns true then it is expected to work,
+# otherwise it is not expected to work in the current system
+# and it will be skipped (unless the module name has itself been
+# included via a 'unverified_modules' configuration set/list).
+worked_distros = set(mod.distros)
+worked_distros.update(
+distros.Distro.expand_osfamily(mod.osfamilies))
+if len(worked_distros) == 0 or distro.name in worked_distros:
+return True
+is_usable_func = getattr(mod, 'is_usable_on', None)
+if is_usable_func is not None and six.callable(is_usable_func):
+return is_usable_func(distro)
+return False
 
 skipped = []
 forced = []
 overridden = self.cfg.get('unverified_modules', [])
 for (mod, name, _freq, _args) in mostly_mods:
-worked_distros = set(mod.distros)
-worked_distros.update(
-distros.Distro.expand_osfamily(mod.osfamilies))
-
-# module does not declare 'distros' or lists this distro
-if not worked_distros or d_name in worked_distros:
-continue
-
 if name in overridden:
 forced.append(name)
 else:
-skipped.append(name)
-
+if not is_usable_here(mod):
+skipped.append(name)
 if skipped:
 LOG.info("Skipping modules %s because they are not verified "
  "on distro '%s'.  To run anyway, add them to "
- "'unverified_modules' in config.", skipped, d_name)
+ "'unverified_modules' in config.", skipped, distro.name)
 if forced:
-LOG.info("running unverified_modules: %s", forced)
+LOG.info("Running unverified modules: %s", forced)
 
 return self._run_modules(mostly_mods)
 

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-really-skip-baddies into lp:cloud-init

2016-07-21 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-really-skip-baddies into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-really-skip-baddies/+merge/300802
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-really-skip-baddies into lp:cloud-init.
=== modified file 'cloudinit/stages.py'
--- cloudinit/stages.py	2016-06-10 21:22:17 +
+++ cloudinit/stages.py	2016-07-21 18:17:26 +
@@ -808,7 +808,7 @@
 def run_section(self, section_name):
 raw_mods = self._read_modules(section_name)
 mostly_mods = self._fixup_modules(raw_mods)
-d_name = self.init.distro.name
+distro_name = self.init.distro.name
 
 skipped = []
 forced = []
@@ -819,7 +819,7 @@
 distros.Distro.expand_osfamily(mod.osfamilies))
 
 # module does not declare 'distros' or lists this distro
-if not worked_distros or d_name in worked_distros:
+if not worked_distros or distro_name in worked_distros:
 continue
 
 if name in overridden:
@@ -827,14 +827,20 @@
 else:
 skipped.append(name)
 
+cleaned_mostly_mods = []
+for (mod, name, freq, args) in mostly_mods:
+if name in skipped:
+continue
+cleaned_mostly_mods.append((mod, name, freq, args))
+
 if skipped:
 LOG.info("Skipping modules %s because they are not verified "
  "on distro '%s'.  To run anyway, add them to "
- "'unverified_modules' in config.", skipped, d_name)
+ "'unverified_modules' in config.", skipped, distro_name)
 if forced:
-LOG.info("running unverified_modules: %s", forced)
+LOG.info("Running unverified modules: %s", forced)
 
-return self._run_modules(mostly_mods)
+return self._run_modules(cleaned_mostly_mods)
 
 
 def fetch_base_config():

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-file-logging into lp:cloud-init

2016-07-21 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-file-logging into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-file-logging/+merge/300798

It's less problematic to just depend on basic file logging to avoid syslog 
patterns, syslog being active, syslog being useable as a core dependency of 
getting debug logs and other from cloudinit.
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-file-logging into lp:cloud-init.
=== modified file 'config/cloud.cfg.d/05_logging.cfg'
--- config/cloud.cfg.d/05_logging.cfg	2014-01-14 18:18:14 +
+++ config/cloud.cfg.d/05_logging.cfg	2016-07-21 18:01:22 +
@@ -54,13 +54,21 @@
 
 log_cfgs:
 # These will be joined into a string that defines the configuration
- - [ *log_base, *log_syslog ]
+#
+# If your desire syslog, just uncomment the following line.
+# - [ *log_base, *log_syslog ]
+#
 # These will be joined into a string that defines the configuration
+#
+# Otherwise by default we just use the most basic (known to work
+# mechanism that does not depend on a system service to operate; which
+# due to cloud-init execution lifecycle we may not always be able to
+# depend on working correctly).
  - [ *log_base, *log_file ]
-# A file path can also be used
+# A file path can also be used.
 # - /etc/log.conf
 
-# this tells cloud-init to redirect its stdout and stderr to
+# This tells cloud-init to redirect its stdout and stderr to
 # 'tee -a /var/log/cloud-init-output.log' so the user can see output
 # there without needing to look on the console.
 output: {all: '| tee -a /var/log/cloud-init-output.log'}

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-rpm-argparse into lp:cloud-init

2016-07-15 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-fix-rpm-argparse into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)
Related bugs:
  Bug #1603533 in cloud-init: "Can't build correctly using brpm on cent7"
  https://bugs.launchpad.net/cloud-init/+bug/1603533

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-rpm-argparse/+merge/300235
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-rpm-argparse into lp:cloud-init.
=== modified file 'packages/brpm'
--- packages/brpm	2015-05-19 12:16:46 +
+++ packages/brpm	2016-07-15 18:49:45 +
@@ -137,6 +137,9 @@
 # Map to known packages
 requires = []
 for p in pkgs:
+if p == 'argparse' and sys.version_info[0:2] >= (2, 7):
+# Not needed on anything but 2.6 or older.
+continue
 tgt_pkg = PKG_MP[args.distro].get(p)
 if not tgt_pkg:
 raise RuntimeError(("Do not know how to translate pypi dependency"

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-26 into lp:cloud-init

2016-07-12 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/cloud-init-fix-26 
into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-26/+merge/299891

Fixes a stray dict comprehension that snuck in/was left over.
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-26 into lp:cloud-init.
=== modified file 'cloudinit/net/__init__.py'
--- cloudinit/net/__init__.py	2016-06-20 18:54:12 +
+++ cloudinit/net/__init__.py	2016-07-13 00:19:07 +
@@ -252,7 +252,8 @@
 cur_bymac[mac] = cur
 
 def update_byname(bymac):
-return {data['name']: data for data in bymac.values()}
+return dict((data['name'], data)
+for data in bymac.values())
 
 def rename(cur, new):
 util.subp(["ip", "link", "set", cur, "name", new], capture=True)

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-dnf into lp:cloud-init

2016-07-06 Thread Joshua Harlow



Diff comments:

> === modified file 'cloudinit/distros/rhel.py'
> --- cloudinit/distros/rhel.py 2016-06-15 23:11:24 +
> +++ cloudinit/distros/rhel.py 2016-07-01 00:07:56 +
> @@ -201,14 +203,22 @@
>  if pkgs is None:
>  pkgs = []
>  
> -cmd = ['yum']
> -# If enabled, then yum will be tolerant of errors on the command line
> -# with regard to packages.
> -# For example: if you request to install foo, bar and baz and baz is
> -# installed; yum won't error out complaining that baz is already
> -# installed.
> -cmd.append("-t")
> -# Determines whether or not yum prompts for confirmation
> +if (os.path.isfile('/usr/bin/dnf') or

Switching to that, forgot about that...

> +self._cfg.get('force_dnf', False)):
> +LOG.debug('Using DNF for package management')
> +cmd = ['dnf']
> +else:

Ya, let's not bother with this and just use dnf.

> +LOG.debug('Using YUM for package management')
> +cmd = ['yum']
> +# If enabled, then yum will be tolerant of errors on the command
> +# line with regard to packages.
> +#
> +# For example: if you request to install foo, bar and baz and
> +# baz is installed; yum won't error out complaining that baz
> +# is already installed.
> +cmd.append("-t")
> +
> +# Determines whether or not yum/dnf prompts for confirmation
>  # of critical actions. We don't want to prompt...
>  cmd.append("-y")
>  


-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-dnf/+merge/298851
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-dnf into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-dnf into lp:cloud-init

2016-06-30 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/cloud-init-fix-dnf 
into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-dnf/+merge/298851



Enable usage of DNF for package management

DNF is the new YUM replacement so if it exists
or if its usage is forced try to use it instead
to install packages.

-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-dnf into lp:cloud-init.
=== modified file 'cloudinit/distros/rhel.py'
--- cloudinit/distros/rhel.py	2016-06-15 23:11:24 +
+++ cloudinit/distros/rhel.py	2016-07-01 00:07:56 +
@@ -20,6 +20,8 @@
 #You should have received a copy of the GNU General Public License
 #along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import os
+
 from cloudinit import distros
 from cloudinit import helpers
 from cloudinit import log as logging
@@ -201,14 +203,22 @@
 if pkgs is None:
 pkgs = []
 
-cmd = ['yum']
-# If enabled, then yum will be tolerant of errors on the command line
-# with regard to packages.
-# For example: if you request to install foo, bar and baz and baz is
-# installed; yum won't error out complaining that baz is already
-# installed.
-cmd.append("-t")
-# Determines whether or not yum prompts for confirmation
+if (os.path.isfile('/usr/bin/dnf') or
+self._cfg.get('force_dnf', False)):
+LOG.debug('Using DNF for package management')
+cmd = ['dnf']
+else:
+LOG.debug('Using YUM for package management')
+cmd = ['yum']
+# If enabled, then yum will be tolerant of errors on the command
+# line with regard to packages.
+#
+# For example: if you request to install foo, bar and baz and
+# baz is installed; yum won't error out complaining that baz
+# is already installed.
+cmd.append("-t")
+
+# Determines whether or not yum/dnf prompts for confirmation
 # of critical actions. We don't want to prompt...
 cmd.append("-y")
 

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-rpm-build into lp:cloud-init

2016-06-30 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-fix-rpm-build into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-rpm-build/+merge/298849



Fixes missing/unpacked rpm files

These new files were not getting picked up during
packaging (and they need to, otherwise rpm building
fails).

- 66-azure-ephemeral.rules
- cloud-init-generator

-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-rpm-build into lp:cloud-init.
=== modified file 'packages/redhat/cloud-init.spec.in'
--- packages/redhat/cloud-init.spec.in	2014-11-25 19:46:10 +
+++ packages/redhat/cloud-init.spec.in	2016-06-30 23:01:18 +
@@ -160,6 +160,8 @@
 
 %files
 
+/lib/udev/rules.d/66-azure-ephemeral.rules
+
 #if $sysvinit
 %attr(0755, root, root) %{_initddir}/cloud-config
 %attr(0755, root, root) %{_initddir}/cloud-final
@@ -168,6 +170,8 @@
 #end if
 
 #if $systemd
+/usr/lib/systemd/system-generators/cloud-init-generator
+%{_unitdir}/cloud-*
 %{_unitdir}/cloud-*
 #end if
 

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-rpm-build into lp:cloud-init

2016-06-30 Thread Joshua Harlow

The proposal to merge lp:~harlowja/cloud-init/cloud-init-fix-rpm-build into 
lp:cloud-init has been updated.

Description changed to:



Fixes missing/unpacked rpm files

These new files were not getting picked up during
packaging (and they need to, otherwise rpm building
fails).

- 66-azure-ephemeral.rules
- cloud-init-generator


For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-rpm-build/+merge/298849
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-rpm-build into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-enhanced-bootcmd into lp:cloud-init

2016-06-23 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-enhanced-bootcmd into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-enhanced-bootcmd/+merge/298254

Some small enhancements to bootcmd module.
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-enhanced-bootcmd into lp:cloud-init.
=== modified file 'cloudinit/config/cc_bootcmd.py'
--- cloudinit/config/cc_bootcmd.py	2016-05-12 17:56:26 +
+++ cloudinit/config/cc_bootcmd.py	2016-06-23 20:04:24 +
@@ -26,6 +26,25 @@
 frequency = PER_ALWAYS
 
 
+def create_env(cloud):
+env = os.environ.copy()
+iid = cloud.get_instance_id()
+if iid:
+env['INSTANCE_ID'] = str(iid)
+launch_index = cloud.launch_index()
+if launch_index is not None:
+env['LAUNCH_INDEX'] = str(launch_index)
+hostname = cloud.get_hostname()
+if hostname:
+env['HOSTNAME'] = str(hostname)
+for k, v in [('AVAILABILITY_ZONE', cloud.datasource.availability_zone),
+ ('REGION', cloud.datasource.region),
+ ('LOCALE', cloud.get_locale())]:
+if v:
+env[k] = str(v)
+return env
+
+
 def handle(name, cfg, cloud, log, _args):
 
 if "bootcmd" not in cfg:
@@ -43,12 +62,8 @@
 raise
 
 try:
-env = os.environ.copy()
-iid = cloud.get_instance_id()
-if iid:
-env['INSTANCE_ID'] = str(iid)
 cmd = ['/bin/sh', tmpf.name]
-util.subp(cmd, env=env, capture=False)
+util.subp(cmd, env=create_env(cloud), capture=False)
 except Exception:
 util.logexc(log, "Failed to run bootcmd module %s", name)
 raise

=== modified file 'cloudinit/util.py'
--- cloudinit/util.py	2016-06-16 03:32:22 +
+++ cloudinit/util.py	2016-06-23 20:04:24 +
@@ -1746,16 +1746,14 @@
 #  for each entry in the list
 #if it is an array, shell protect it (with single ticks)
 #if it is a string, do nothing
-def shellify(cmdlist, add_header=True):
+def shellify(cmdlist, add_header=True, header="#!/bin/sh\n"):
 content = ''
-if add_header:
-content += "#!/bin/sh\n"
 escaped = "%s%s%s%s" % ("'", '\\', "'", "'")
 cmds_made = 0
 for args in cmdlist:
 # If the item is a list, wrap all items in single tick.
 # If its not, then just write it directly.
-if isinstance(args, list):
+if isinstance(args, (list, tuple)):
 fixed = []
 for f in args:
 fixed.append("'%s'" % (six.text_type(f).replace("'", escaped)))
@@ -1769,6 +1767,8 @@
 " which is not a list or string")
% (type_utils.obj_name(args)))
 LOG.debug("Shellified %s commands.", cmds_made)
+if add_header and header and not content.startswith(header):
+content = header + content
 return content
 
 

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~smoser/cloud-init/trunk.1532072 into lp:cloud-init

2016-06-21 Thread Joshua Harlow

Seems ok to me!
-- 
https://code.launchpad.net/~smoser/cloud-init/trunk.1532072/+merge/298066
Your team cloud init development team is requested to review the proposed merge 
of lp:~smoser/cloud-init/trunk.1532072 into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init

2016-06-20 Thread Joshua Harlow



Diff comments:

> === modified file 'cloudinit/net/sysconfig.py'
> --- cloudinit/net/sysconfig.py2016-06-15 23:15:28 +
> +++ cloudinit/net/sysconfig.py2016-06-18 01:07:35 +
> @@ -238,6 +247,13 @@
>iface_cfg.name))
>  if 'netmask' in subnet:
>  iface_cfg['NETMASK'] = subnet['netmask']
> +if 'dns_nameservers' in subnet:
> +for nameserver in subnet['dns_nameservers']:
> +ns_key = "DNS%s" % iface_cfg.last_dns_idx.value

Ya, so this is the special case where u have multiple networks configured for a 
interface; in that case we need to share the DNS idx across all the files that 
belong to that interface.

http://ithelpblog.com/os/linux/redhat/howto-create-sub-interface-or-secondary-ip-address-on-redhat-rhel/

These things; networking is crazy, lol.

> +iface_cfg.last_dns_idx.value += 1
> +iface_cfg[ns_key] = nameserver
> +if 'dns_search' in subnet:
> +iface_cfg['SEARCH'] = " ".join(subnet['dns_search'])
>  for route in subnet.get('routes', []):
>  if _is_default_route(route):
>  if route_cfg.has_set_default:


-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-dns-sysconfig/+merge/297817
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init

2016-06-20 Thread Joshua Harlow

Ya, I had a hard time finding out how to make it so that each interface could 
have its own dns server and such configuration; it really seems like not many 
systems allow for doing this very well...
-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-dns-sysconfig/+merge/297817
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init

2016-06-17 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-dns-sysconfig/+merge/297817

It appears that 'dns_nameservers' and 'dns_search' can be a per subnet/iface 
specification and the eni renderer actually handles this so make an attempt to 
in the sysconfig renderer as well.
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-dns-sysconfig into lp:cloud-init.
=== modified file 'cloudinit/net/sysconfig.py'
--- cloudinit/net/sysconfig.py	2016-06-15 23:15:28 +
+++ cloudinit/net/sysconfig.py	2016-06-18 01:03:27 +
@@ -131,6 +131,7 @@
 super(NetInterface, self).__init__()
 self.children = []
 self.routes = Route(iface_name, base_sysconf_dir)
+self.last_dns_idx = 0
 self._kind = kind
 self._iface_name = iface_name
 self._conf['DEVICE'] = iface_name
@@ -238,6 +239,13 @@
   iface_cfg.name))
 if 'netmask' in subnet:
 iface_cfg['NETMASK'] = subnet['netmask']
+if 'dns_nameservers' in subnet:
+for nameserver in subnet['dns_nameservers']:
+ns_key = "DNS%s" % iface_cfg.last_dns_idx
+iface_cfg.last_dns_idx += 1
+iface_cfg[ns_key] = nameserver
+if 'dns_search' in subnet:
+iface_cfg['SEARCH'] = " ".join(subnet['dns_search'])
 for route in subnet.get('routes', []):
 if _is_default_route(route):
 if route_cfg.has_set_default:
@@ -332,13 +340,18 @@
 @staticmethod
 def _render_dns(network_state, existing_dns_path=None):
 content = resolv_conf.ResolvConf("")
+is_existing = False
 if existing_dns_path and os.path.isfile(existing_dns_path):
 content = resolv_conf.ResolvConf(util.load_file(existing_dns_path))
+is_existing = True
 for nameserver in network_state.dns_nameservers:
 content.add_nameserver(nameserver)
 for searchdomain in network_state.dns_searchdomains:
 content.add_search_domain(searchdomain)
-return "\n".join([_make_header(';'), str(content)])
+if is_existing:
+return str(content)
+else:
+return "\n".join([_make_header(';'), str(content)])
 
 @classmethod
 def _render_bridge_interfaces(cls, network_state, iface_contents):

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-render-tool into lp:cloud-init

2016-06-17 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-render-tool into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-render-tool/+merge/297816
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-render-tool into lp:cloud-init.
=== modified file 'setup.py'
--- setup.py	2016-06-14 21:56:51 +
+++ setup.py	2016-06-18 00:17:45 +
@@ -211,7 +211,7 @@
 cmdclass=cmdclass,
 entry_points={
 'console_scripts': [
-'cloud-init = cloudinit.cmd.main:main'
+'cloud-init = cloudinit.cmd.main:main',
 ],
 }
 )

=== added file 'tools/net-convert.py'
--- tools/net-convert.py	1970-01-01 00:00:00 +
+++ tools/net-convert.py	2016-06-18 00:17:45 +
@@ -0,0 +1,89 @@
+#!/usr/bin/python
+# vi: ts=4 expandtab
+#
+#Copyright (C) 2012 Canonical Ltd.
+#Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
+#Copyright (C) 2012 Yahoo! Inc.
+#
+#Author: Scott Moser 
+#Author: Juerg Haefliger 
+#Author: Joshua Harlow 
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import argparse
+import json
+import os
+
+from cloudinit.sources.helpers import openstack
+
+from cloudinit.net import eni
+from cloudinit.net import network_state
+from cloudinit.net import sysconfig
+
+
+def main():
+parser = argparse.ArgumentParser()
+parser.add_argument("--network-data", "-p", type=open,
+metavar="PATH", required=True)
+parser.add_argument("--kind", "-k",
+choices=['eni', 'network_data.json'],
+required=True)
+parser.add_argument("-d", "--directory",
+metavar="PATH",
+help="directory to place output in",
+required=True)
+parser.add_argument("-m", "--mac",
+metavar="name,mac",
+action='append',
+help="interface name to mac mapping")
+parser.add_argument("--output-kind", "-ok",
+choices=['eni', 'sysconfig'],
+required=True)
+args = parser.parse_args()
+
+if not os.path.isdir(args.directory):
+os.makedirs(args.directory)
+
+if args.mac:
+known_macs = {}
+for item in args.mac:
+iface_name, iface_mac = item.split(",", 1)
+known_macs[iface_mac] = iface_name
+else:
+known_macs = None
+
+net_data = args.network_data.read()
+if args.kind == "eni":
+pre_ns = eni.convert_eni_data(net_data)
+ns = network_state.parse_net_config_data(pre_ns)
+else:
+pre_ns = openstack.convert_net_json(
+json.loads(net_data), known_macs=known_macs)
+ns = network_state.parse_net_config_data(pre_ns)
+
+if not ns:
+raise RuntimeError("No valid network_state object created from"
+   "input data")
+
+if args.output_kind == "eni":
+r_cls = eni.Renderer
+else:
+r_cls = sysconfig.Renderer
+
+r = r_cls()
+r.render_network_state(args.directory, ns)
+
+
+if __name__ == '__main__':
+main()

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init

2016-06-15 Thread Joshua Harlow

done.
-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-sysconfig/+merge/297115
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-up-cli into lp:cloud-init

2016-06-14 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-fix-up-cli into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-up-cli/+merge/297409
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-up-cli into lp:cloud-init.
=== removed directory 'bin'
=== added directory 'cloudinit/cmd'
=== added file 'cloudinit/cmd/__init__.py'
--- cloudinit/cmd/__init__.py	1970-01-01 00:00:00 +
+++ cloudinit/cmd/__init__.py	2016-06-14 21:58:15 +
@@ -0,0 +1,21 @@
+# vi: ts=4 expandtab
+#
+#Copyright (C) 2012 Canonical Ltd.
+#Copyright (C) 2012 Hewlett-Packard Development Company, L.P.
+#Copyright (C) 2012 Yahoo! Inc.
+#
+#Author: Scott Moser 
+#Author: Juerg Haefliger 
+#Author: Joshua Harlow 
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== renamed file 'bin/cloud-init' => 'cloudinit/cmd/main.py'
--- bin/cloud-init	2016-05-31 21:17:39 +
+++ cloudinit/cmd/main.py	2016-06-14 21:58:15 +
@@ -25,17 +25,10 @@
 import json
 import os
 import sys
+import tempfile
 import time
-import tempfile
 import traceback
 
-# This is more just for running from the bin folder so that
-# cloud-init binary can find the cloudinit module
-possible_topdir = os.path.normpath(os.path.join(os.path.abspath(
-sys.argv[0]), os.pardir, os.pardir))
-if os.path.exists(os.path.join(possible_topdir, "cloudinit", "__init__.py")):
-sys.path.insert(0, possible_topdir)
-
 from cloudinit import patcher
 patcher.patch()
 
@@ -46,9 +39,10 @@
 from cloudinit import stages
 from cloudinit import templater
 from cloudinit import util
+from cloudinit import version
+
 from cloudinit import reporting
 from cloudinit.reporting import events
-from cloudinit import version
 
 from cloudinit.settings import (PER_INSTANCE, PER_ALWAYS, PER_ONCE,
 CLOUD_CONFIG)
@@ -188,7 +182,7 @@
 LOG.debug("Closing stdin")
 util.close_stdin()
 (outfmt, errfmt) = util.fixup_output(init.cfg, name)
-except:
+except Exception:
 util.logexc(LOG, "Failed to setup output redirection!")
 print_exc("Failed to setup output redirection!")
 if args.debug:
@@ -325,7 +319,7 @@
 if outfmt_orig != outfmt or errfmt_orig != errfmt:
 LOG.warn("Stdout, stderr changing to (%s, %s)", outfmt, errfmt)
 (outfmt, errfmt) = util.fixup_output(mods.cfg, name)
-except:
+except Exception:
 util.logexc(LOG, "Failed to re-adjust output redirection!")
 logging.setupLogging(mods.cfg)
 
@@ -367,7 +361,7 @@
 LOG.debug("Closing stdin")
 util.close_stdin()
 util.fixup_output(mods.cfg, name)
-except:
+except Exception:
 util.logexc(LOG, "Failed to setup output redirection!")
 if args.debug:
 # Reset so that all the debug handlers are closed out
@@ -430,7 +424,7 @@
 LOG.debug("Closing stdin")
 util.close_stdin()
 util.fixup_output(mods.cfg, None)
-except:
+except Exception:
 util.logexc(LOG, "Failed to setup output redirection!")
 if args.debug:
 # Reset so that all the debug handlers are closed out
@@ -510,7 +504,7 @@
 else:
 try:
 status = json.loads(util.load_file(status_path))
-except:
+except Exception:
 pass
 
 if status is None:
@@ -569,8 +563,12 @@
 return len(v1[mode]['errors'])
 
 
-def main():
-parser = argparse.ArgumentParser()
+def main(sysv_args=None):
+if sysv_args is not None:
+parser = argparse.ArgumentParser(prog=sysv_args[0])
+sysv_args = sysv_args[1:]
+else:
+parser = argparse.ArgumentParser()
 
 # Top level args
 parser.add_argument('--version', '-v', action='version',
@@ -646,7 +644,7 @@
  ' pass to this module'))
 parser_single.set_defaults(action=('single', main_single))
 
-args = parser.parse_args()
+args = parser.parse_args(args=sysv_args)
 
 # Setup basic logging to start (u

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init

2016-06-14 Thread Joshua Harlow



Diff comments:

> === modified file 'cloudinit/distros/rhel.py'
> --- cloudinit/distros/rhel.py 2015-06-02 20:27:57 +
> +++ cloudinit/distros/rhel.py 2016-06-14 17:59:09 +
> @@ -59,10 +61,16 @@
>  # should only happen say once per instance...)
>  self._runner = helpers.Runners(paths)
>  self.osfamily = 'redhat'
> +self._net_renderer = sysconfig.Renderer()
>  
>  def install_packages(self, pkglist):
>  self.package_command('install', pkgs=pkglist)
>  
> +def _write_network_config(self, netconfig):
> +self._net_renderer.render_network_state(
> +target="/", parse_net_config_data(netconfig))

Yup, there are differences; I can move those differences to be arguments 
options passed to the Renderer() __init__ method, or do them here but they will 
likely always be needed, thoughts?

> +return []
> +
>  def _write_network(self, settings):
>  # TODO(harlowja) fix this... since this is the ubuntu format
>  entries = net_util.translate_network(settings)


-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-sysconfig/+merge/297115
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init

2016-06-10 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-sysconfig/+merge/297115
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-net-sysconfig into lp:cloud-init.
=== modified file 'cloudinit/distros/rhel.py'
--- cloudinit/distros/rhel.py	2015-06-02 20:27:57 +
+++ cloudinit/distros/rhel.py	2016-06-10 21:38:18 +
@@ -23,6 +23,8 @@
 from cloudinit import distros
 from cloudinit import helpers
 from cloudinit import log as logging
+from cloudinit.net.network_state import parse_net_config_data
+from cloudinit.net import sysconfig
 from cloudinit import util
 
 from cloudinit.distros import net_util
@@ -59,10 +61,16 @@
 # should only happen say once per instance...)
 self._runner = helpers.Runners(paths)
 self.osfamily = 'redhat'
+self._net_renderer = sysconfig.Renderer()
 
 def install_packages(self, pkglist):
 self.package_command('install', pkgs=pkglist)
 
+def _write_network_config(self, netconfig):
+self._net_renderer.render_network_state(
+target="/", parse_net_config_data(netconfig))
+return []
+
 def _write_network(self, settings):
 # TODO(harlowja) fix this... since this is the ubuntu format
 entries = net_util.translate_network(settings)

=== added file 'cloudinit/net/sysconfig.py'
--- cloudinit/net/sysconfig.py	1970-01-01 00:00:00 +
+++ cloudinit/net/sysconfig.py	2016-06-10 21:38:18 +
@@ -0,0 +1,476 @@
+# vi: ts=4 expandtab
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import re
+
+import six
+
+from cloudinit.distros.parsers import resolv_conf
+from cloudinit import util
+
+from . import network_state
+from .udev import generate_udev_rule
+
+
+def _filter_by_type(match_type):
+return lambda iface: match_type == iface['type']
+
+
+def _filter_by_name(match_name):
+return lambda iface: match_name == iface['name']
+
+
+_filter_by_physical = _filter_by_type('physical')
+
+
+def _is_default_route(route):
+if route['network'] == '::' and route['netmask'] == 0:
+return True
+if route['network'] == '0.0.0.0' and route['netmask'] == '0.0.0.0':
+return True
+return False
+
+
+def _quote_value(value):
+if re.search(r"\s", value):
+# This doesn't handle complex cases...
+if value.startswith('"') and value.endswith('"'):
+return value
+else:
+return '"%s"' % value
+else:
+return value
+
+
+class NetworkStateHelper(object):
+def __init__(self, network_state):
+self._network_state = network_state.copy()
+
+@property
+def dns_nameservers(self):
+return self._network_state['dns']['nameservers']
+
+@property
+def dns_searchdomains(self):
+return self._network_state['dns']['search']
+
+def iter_interfaces(self, filter_func=None):
+ifaces = self._network_state.get('interfaces')
+if ifaces:
+for iface in ifaces.values():
+if filter_func is None:
+yield iface
+else:
+if filter_func(iface):
+yield iface
+
+
+class ConfigMap(object):
+"""Sysconfig like dictionary object."""
+
+default_header = ('# Created by cloud-init on instance'
+  ' boot automatically, do not edit.\n#')
+
+# Why does redhat prefer yes/no to true/false??
+_bool_map = {
+True: 'yes',
+False: 'no',
+}
+
+def __init__(self):
+self._conf = {}
+
+def __setitem__(self, key, value):
+self._conf[key] = value
+
+def drop(self, key):
+self._conf.pop(key, None)
+
+def __len__(self):
+return len(self._conf)
+
+def to_string(self):
+buf = compat.StringIO()
+buf.write(self.default_header)
+

Re: [Cloud-init-dev] [Merge] lp:~smoser/cloud-init/trunk.fix-networking into lp:cloud-init

2016-06-03 Thread Joshua Harlow

Can u add a part of the description around 'def rename_interfaces' so that 
people know why this is needed, be much appreciated :)
-- 
https://code.launchpad.net/~smoser/cloud-init/trunk.fix-networking/+merge/296272
Your team cloud init development team is requested to review the proposed merge 
of lp:~smoser/cloud-init/trunk.fix-networking into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init

2016-05-24 Thread Joshua Harlow

The proposal to merge lp:~harlowja/cloud-init/cloud-init-fix-test-times into 
lp:cloud-init has been updated.

Commit Message changed to:

Fixes up tests taking forever due to retries and timeouts.

Timing info:

Fixed timing via `time nosetests`:

real0m12.376s
user0m6.797s
sys 0m0.616s

Not fixed timing via `time nosetests`:

real1m12.886s
user0m7.202s
sys 0m0.666s

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-test-times/+merge/294854
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-net-refactor into lp:cloud-init

2016-05-23 Thread Joshua Harlow

The proposal to merge lp:~harlowja/cloud-init/cloud-init-net-refactor into 
lp:cloud-init has been updated.

Commit Message changed to:

Refactors a large part of the networking code.

Splits off distro specific code into specific files so that
other kinds of networking configuration can be written by the
various distro(s) that cloud-init supports.

It also isolates some of the cloudinit.net code so that it can
be more easily used on its own (and incorporated into other
projects such as curtin).

During this process it adds tests so that the net process can
be tested (to some level) so that the format conversion processes
can be tested going forward.

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/+merge/293957
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-net-refactor into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~gfidente/cloud-init/try_fqdn_from_hosts into lp:cloud-init

2016-05-18 Thread Joshua Harlow



Diff comments:

> 
> === modified file 'tests/unittests/test_util.py'
> --- tests/unittests/test_util.py  2016-03-14 13:45:11 +
> +++ tests/unittests/test_util.py  2016-05-14 00:02:24 +
> @@ -69,6 +83,85 @@
>  self.assertEqual([], result)
>  
>  
> +class TestGetHostnameFqdn(helpers.TestCase):
> +def setUp(self):
> +super(TestGetHostnameFqdn, self).setUp()
> +tmp_h, self.tmp_path = tempfile.mkstemp()
> +self.f = os.fdopen(tmp_h, "w")

Can u change this to self.addCleanup(os.remove, self.tmp_path)

> +self.f.write("9.10.11.12 cloudname.clouddomain cloudname\n")
> +self.f.close()
> +self.cloud = FakeCloudWithDomain()
> +self.cloud_no_domain = FakeCloudWithoutDomain()
> +
> +def tearDown(self):
> +super(TestGetHostnameFqdn, self).tearDown()
> +os.remove(self.tmp_path)
> +
> +def test_with_fqdn(self):
> +"""Results based only on fqdn config key"""
> +cfg = {'fqdn': 'thisname.thisdomain'}
> +hostname, fqdn = util.get_hostname_fqdn(cfg, self.cloud, 
> self.tmp_path)
> +self.assertEqual("thisname", hostname)
> +self.assertEqual("thisname.thisdomain", fqdn)
> +
> +def test_with_dotted_hostname(self):
> +"""Results based only on hostname config key"""
> +cfg = {'hostname': 'thisname.thisdomain'}
> +hostname, fqdn = util.get_hostname_fqdn(cfg, self.cloud, 
> self.tmp_path)
> +self.assertEqual("thisname", hostname)
> +self.assertEqual("thisname.thisdomain", fqdn)
> +
> +def test_with_short_hostname(self):
> +"""The hostname is from config key, the fqdn from cloud"""
> +cfg = {'hostname': 'thisname'}
> +hostname, fqdn = util.get_hostname_fqdn(cfg, self.cloud, 
> self.tmp_path)
> +self.assertEqual("thisname", hostname)
> +self.assertEqual("cloudname.clouddomain", fqdn)
> +
> +def test_from_cloud(self):
> +"""Results based only on cloud"""
> +cfg = {}
> +hostname, fqdn = util.get_hostname_fqdn(cfg, self.cloud, 
> self.tmp_path)
> +self.assertEqual("cloudname", hostname)
> +self.assertEqual("cloudname.clouddomain", fqdn)
> +
> +def test_from_cloud_without_domain(self):
> +"""The hostname is from cloud, the fqdn mapped from hosts file"""
> +cfg = {}
> +hostname, fqdn = util.get_hostname_fqdn(cfg,
> +self.cloud_no_domain,
> +self.tmp_path)
> +self.assertEqual("cloudname", hostname)
> +self.assertEqual("cloudname.clouddomain", fqdn)
> +
> +
> +class TestGetFqdnFromHosts(helpers.TestCase):
> +def setUp(self):
> +super(TestGetFqdnFromHosts, self).setUp()
> +tmp_h, self.tmp_path = tempfile.mkstemp()
> +self.f = os.fdopen(tmp_h, "w")
> +self.f.write("# comment\n")
> +self.f.write("1.2.3.4 name.domain name\n")
> +self.f.write("5.6.7.8 myname.mydomain myname # comment\n")
> +self.f.close()
> +
> +def tearDown(self):
> +super(TestGetFqdnFromHosts, self).tearDown()
> +os.remove(self.tmp_path)

Same as above.

https://docs.python.org/2.7/library/unittest.html#unittest.TestCase.addCleanup

> +
> +def test_fqdn_found(self):
> +"""Test fqdn is returned when hostname is found"""
> +hostname = "myname"
> +fqdn = util.get_fqdn_from_hosts(hostname, self.tmp_path)
> +self.assertEqual("myname.mydomain", fqdn)
> +
> +def test_fqdn_not_found(self):
> +"""Test None is returned when hostname is not found"""
> +hostname = "unnamed"
> +fqdn = util.get_fqdn_from_hosts(hostname, self.tmp_path)
> +self.assertEqual(None, fqdn)
> +
> +
>  class TestWriteFile(helpers.TestCase):
>  def setUp(self):
>  super(TestWriteFile, self).setUp()


-- 
https://code.launchpad.net/~gfidente/cloud-init/try_fqdn_from_hosts/+merge/294680
Your team cloud init development team is requested to review the proposed merge 
of lp:~gfidente/cloud-init/try_fqdn_from_hosts into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init

2016-05-17 Thread Joshua Harlow

Right right, maybe let me exclude that. The timer part is really only for 
detecting the times, not for fixing them. We can leave the inclusion of that to 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-nose-timer/+merge/294569
 instead, this at least has the fixes for thinks taking to much time (due to 
retrying and such).
-- 
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-test-times/+merge/294854
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init

2016-05-16 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-fix-test-times/+merge/294854
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-fix-test-times into lp:cloud-init.
=== modified file 'cloudinit/sources/DataSourceOpenStack.py'
--- cloudinit/sources/DataSourceOpenStack.py	2016-04-04 16:31:28 +
+++ cloudinit/sources/DataSourceOpenStack.py	2016-05-16 23:19:21 +
@@ -103,7 +103,7 @@
 self.metadata_address = url2base.get(avail_url)
 return bool(avail_url)
 
-def get_data(self):
+def get_data(self, retries=5, timeout=5):
 try:
 if not self.wait_for_metadata_service():
 return False
@@ -115,7 +115,9 @@
 'Crawl of openstack metadata service',
 read_metadata_service,
 args=[self.metadata_address],
-kwargs={'ssl_details': self.ssl_details})
+kwargs={'ssl_details': self.ssl_details,
+'retries': retries,
+'timeout': timeout})
 except openstack.NonReadable:
 return False
 except (openstack.BrokenMetadata, IOError):
@@ -153,8 +155,10 @@
 return sources.instance_id_matches_system_uuid(self.get_instance_id())
 
 
-def read_metadata_service(base_url, ssl_details=None):
-reader = openstack.MetadataReader(base_url, ssl_details=ssl_details)
+def read_metadata_service(base_url, ssl_details=None,
+  timeout=5, retries=5):
+reader = openstack.MetadataReader(base_url, ssl_details=ssl_details,
+  timeout=timeout, retries=retries)
 return reader.read_v2()
 
 

=== modified file 'test-requirements.txt'
--- test-requirements.txt	2016-05-12 18:35:18 +
+++ test-requirements.txt	2016-05-16 23:19:21 +
@@ -2,6 +2,7 @@
 httpretty>=0.7.1
 mock
 nose
+nose-timer
 
 # Only really needed on older versions of python
 contextlib2

=== modified file 'tests/unittests/test_datasource/test_openstack.py'
--- tests/unittests/test_datasource/test_openstack.py	2016-05-12 20:43:11 +
+++ tests/unittests/test_datasource/test_openstack.py	2016-05-16 23:19:21 +
@@ -135,13 +135,17 @@
 body=get_request_callback)
 
 
+def _read_metadata_service():
+return ds.read_metadata_service(BASE_URL, retries=0, timeout=0.1)
+
+
 class TestOpenStackDataSource(test_helpers.HttprettyTestCase):
 VERSION = 'latest'
 
 @hp.activate
 def test_successful(self):
 _register_uris(self.VERSION, EC2_FILES, EC2_META, OS_FILES)
-f = ds.read_metadata_service(BASE_URL)
+f = _read_metadata_service()
 self.assertEqual(VENDOR_DATA, f.get('vendordata'))
 self.assertEqual(CONTENT_0, f['files']['/etc/foo.cfg'])
 self.assertEqual(CONTENT_1, f['files']['/etc/bar/bar.cfg'])
@@ -163,7 +167,7 @@
 @hp.activate
 def test_no_ec2(self):
 _register_uris(self.VERSION, {}, {}, OS_FILES)
-f = ds.read_metadata_service(BASE_URL)
+f = _read_metadata_service()
 self.assertEqual(VENDOR_DATA, f.get('vendordata'))
 self.assertEqual(CONTENT_0, f['files']['/etc/foo.cfg'])
 self.assertEqual(CONTENT_1, f['files']['/etc/bar/bar.cfg'])
@@ -178,8 +182,7 @@
 if k.endswith('meta_data.json'):
 os_files.pop(k, None)
 _register_uris(self.VERSION, {}, {}, os_files)
-self.assertRaises(openstack.NonReadable, ds.read_metadata_service,
-  BASE_URL)
+self.assertRaises(openstack.NonReadable, _read_metadata_service)
 
 @hp.activate
 def test_bad_uuid(self):
@@ -190,8 +193,7 @@
 if k.endswith('meta_data.json'):
 os_files[k] = json.dumps(os_meta)
 _register_uris(self.VERSION, {}, {}, os_files)
-self.assertRaises(openstack.BrokenMetadata, ds.read_metadata_service,
-  BASE_URL)
+self.assertRaises(openstack.BrokenMetadata, _read_metadata_service)
 
 @hp.activate
 def test_userdata_empty(self):
@@ -200,7 +202,7 @@
 if k.endswith('user_data'):
 os_files.pop(k, None)
 _register_uris(self.VERSION, {}, {}, os_files)
-f = ds.read_metadata_service(BASE_URL)
+f = _read_metadata_service()
 self.assertEqual(VENDOR_DA

Re: [Cloud-init-dev] [Merge] lp:~gfidente/cloud-init/try_fqdn_from_hosts into lp:cloud-init

2016-05-13 Thread Joshua Harlow

Anyway to add some unittests for this?
-- 
https://code.launchpad.net/~gfidente/cloud-init/try_fqdn_from_hosts/+merge/294680
Your team cloud init development team is requested to review the proposed merge 
of lp:~gfidente/cloud-init/try_fqdn_from_hosts into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-nose-timer into lp:cloud-init

2016-05-12 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-nose-timer into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-nose-timer/+merge/294569

Add top 10 slow tests and use nose to figure it out.
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-nose-timer into lp:cloud-init.
=== modified file 'test-requirements.txt'
--- test-requirements.txt	2016-05-12 18:35:18 +
+++ test-requirements.txt	2016-05-12 21:31:10 +
@@ -2,6 +2,7 @@
 httpretty>=0.7.1
 mock
 nose
+nose-timer
 
 # Only really needed on older versions of python
 contextlib2

=== modified file 'tox.ini'
--- tox.ini	2016-05-12 20:43:11 +
+++ tox.ini	2016-05-12 21:31:10 +
@@ -3,7 +3,7 @@
 recreate = True
 
 [testenv]
-commands = python -m nose {posargs:tests}
+commands = python -m nose --with-timer --timer-top-n 10 {posargs:tests}
 deps = -r{toxinidir}/test-requirements.txt
 -r{toxinidir}/requirements.txt
 
@@ -19,7 +19,7 @@
 LC_ALL = en_US.utf-8
 
 [testenv:py26]
-commands = nosetests {posargs:tests}
+commands = nosetests --with-timer --timer-top-n 10 {posargs:tests}
 deps =
  contextlib2
  httpretty>=0.7.1

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-flake8-fixups into lp:cloud-init

2016-05-12 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-flake8-fixups into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-flake8-fixups/+merge/294548
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-flake8-fixups into lp:cloud-init.
=== modified file 'cloudinit/config/cc_apt_configure.py'
--- cloudinit/config/cc_apt_configure.py	2016-03-03 22:20:10 +
+++ cloudinit/config/cc_apt_configure.py	2016-05-12 18:09:03 +
@@ -207,20 +207,20 @@
 ks = ent['keyserver']
 try:
 ent['key'] = getkeybyid(ent['keyid'], ks)
-except:
+except Exception:
 errorlist.append([source, "failed to get key from %s" % ks])
 continue
 
 if 'key' in ent:
 try:
 util.subp(('apt-key', 'add', '-'), ent['key'])
-except:
+except Exception:
 errorlist.append([source, "failed add key"])
 
 try:
 contents = "%s\n" % (source)
 util.write_file(ent['filename'], contents, omode="ab")
-except:
+except Exception:
 errorlist.append([source,
  "failed write to file %s" % ent['filename']])
 

=== modified file 'cloudinit/config/cc_bootcmd.py'
--- cloudinit/config/cc_bootcmd.py	2015-02-10 21:33:11 +
+++ cloudinit/config/cc_bootcmd.py	2016-05-12 18:09:03 +
@@ -38,7 +38,7 @@
 content = util.shellify(cfg["bootcmd"])
 tmpf.write(util.encode_text(content))
 tmpf.flush()
-except:
+except Exception:
 util.logexc(log, "Failed to shellify bootcmd")
 raise
 
@@ -49,6 +49,6 @@
 env['INSTANCE_ID'] = str(iid)
 cmd = ['/bin/sh', tmpf.name]
 util.subp(cmd, env=env, capture=False)
-except:
+except Exception:
 util.logexc(log, "Failed to run bootcmd module %s", name)
 raise

=== modified file 'cloudinit/config/cc_disk_setup.py'
--- cloudinit/config/cc_disk_setup.py	2016-04-04 19:02:00 +
+++ cloudinit/config/cc_disk_setup.py	2016-05-12 18:09:03 +
@@ -198,7 +198,7 @@
 d_type = ""
 try:
 d_type = device_type(name)
-except:
+except Exception:
 LOG.warn("Query against device %s failed" % name)
 return False
 

=== modified file 'cloudinit/config/cc_emit_upstart.py'
--- cloudinit/config/cc_emit_upstart.py	2015-03-16 17:20:26 +
+++ cloudinit/config/cc_emit_upstart.py	2016-05-12 18:09:03 +
@@ -20,8 +20,8 @@
 
 import os
 
+from cloudinit import log as logging
 from cloudinit.settings import PER_ALWAYS
-from cloudinit import log as logging
 from cloudinit import util
 
 frequency = PER_ALWAYS

=== modified file 'cloudinit/config/cc_fan.py'
--- cloudinit/config/cc_fan.py	2015-10-09 16:39:23 +
+++ cloudinit/config/cc_fan.py	2016-05-12 18:09:03 +
@@ -37,8 +37,8 @@
 """
 
 from cloudinit import log as logging
+from cloudinit.settings import PER_INSTANCE
 from cloudinit import util
-from cloudinit.settings import PER_INSTANCE
 
 LOG = logging.getLogger(__name__)
 

=== modified file 'cloudinit/config/cc_final_message.py'
--- cloudinit/config/cc_final_message.py	2016-03-04 06:45:58 +
+++ cloudinit/config/cc_final_message.py	2016-05-12 18:09:03 +
@@ -66,7 +66,7 @@
 try:
 contents = "%s - %s - v. %s\n" % (uptime, ts, cver)
 util.write_file(boot_fin_fn, contents)
-except:
+except Exception:
 util.logexc(log, "Failed to write boot finished file %s", boot_fin_fn)
 
 if cloud.datasource.is_disconnected:

=== modified file 'cloudinit/config/cc_grub_dpkg.py'
--- cloudinit/config/cc_grub_dpkg.py	2016-03-04 06:45:58 +
+++ cloudinit/config/cc_grub_dpkg.py	2016-05-12 18:09:03 +
@@ -69,5 +69,5 @@
 
 try:
 util.subp(['debconf-set-selections'], dconf_sel)
-except:
+except Exception:
 util.logexc(log, "Failed to run debconf-set-selections for grub-dpkg")

=== modified file 'cloudinit/config/cc_keys_to_console.py'
--- cloudinit/config/cc_keys_to_console.py	2016-03-03 22:20:10 +
+++ cloudinit/config/cc_keys_to_console.py	2016-05-12 18:09:03 +
@@ -57,6 +57,6 @@
 (stdout, _stderr) = util.subp(cmd)
 util.multi_log("%s\n" % (stdout.strip()),
stderr=False, console=True)
-except:
+except Exception:
 log.w

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-net-refactor into lp:cloud-init

2016-05-05 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-net-refactor into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/+merge/293957
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-net-refactor into lp:cloud-init.
=== modified file 'cloudinit/net/__init__.py'
--- cloudinit/net/__init__.py	2016-04-15 20:21:05 +
+++ cloudinit/net/__init__.py	2016-05-06 00:03:46 +
@@ -262,7 +262,7 @@
 
 
 def parse_net_config_data(net_config):
-"""Parses the config, returns NetworkState dictionary
+"""Parses the config, returns NetworkState object
 
 :param net_config: curtin network config dict
 """

=== modified file 'cloudinit/net/network_state.py'
--- cloudinit/net/network_state.py	2016-03-23 16:05:22 +
+++ cloudinit/net/network_state.py	2016-05-06 00:03:46 +
@@ -15,6 +15,10 @@
 #   You should have received a copy of the GNU Affero General Public License
 #   along with Curtin.  If not, see <http://www.gnu.org/licenses/>.
 
+import copy
+
+import six
+
 from cloudinit import log as logging
 from cloudinit import util
 from cloudinit.util import yaml_dumps as dump_config
@@ -32,34 +36,75 @@
 state = util.read_conf(state_file)
 network_state = NetworkState()
 network_state.load(state)
-
 return network_state
 
 
-class NetworkState:
+class InvalidCommand(Exception):
+pass
+
+
+def ensure_command_keys(required_keys):
+required_keys = frozenset(required_keys)
+
+def extract_missing(command):
+missing_keys = set()
+for key in required_keys:
+if key not in command:
+missing_keys.add(key)
+return missing_keys
+
+def wrapper(func):
+
+@six.wraps(func)
+def decorator(self, command, *args, **kwargs):
+if required_keys:
+missing_keys = extract_missing(command)
+if missing_keys:
+raise InvalidCommand("Command missing %s of required"
+ " keys %s" % (missing_keys,
+   required_keys))
+return func(self, command, *args, **kwargs)
+
+return decorator
+
+return wrapper
+
+
+class CommandHandlerMeta(type):
+"""Metaclass that dynamically creates a 'command_handlers' attribute.
+
+This will scan the to-be-created class for methods that start with
+'handle_' and on finding those will populate a class attribute mapping
+so that those methods can be quickly located and called.
+"""
+def __new__(cls, name, parents, dct):
+command_handlers = {}
+for attr_name, attr in six.iteritems(dct):
+if six.callable(attr) and attr_name.startswith('handle_'):
+handles_what = attr_name[len('handle_'):]
+if handles_what:
+command_handlers[handles_what] = attr
+dct['command_handlers'] = command_handlers
+return super(CommandHandlerMeta, cls).__new__(cls, name,
+  parents, dct)
+
+
+@six.add_metaclass(CommandHandlerMeta)
+class NetworkState(object):
+
+initial_network_state = {
+'interfaces': {},
+'routes': [],
+'dns': {
+'nameservers': [],
+'search': [],
+}
+}
+
 def __init__(self, version=NETWORK_STATE_VERSION, config=None):
 self.version = version
 self.config = config
-self.network_state = {
-'interfaces': {},
-'routes': [],
-'dns': {
-'nameservers': [],
-'search': [],
-}
-}
-self.command_handlers = self.get_command_handlers()
-
-def get_command_handlers(self):
-METHOD_PREFIX = 'handle_'
-methods = filter(lambda x: callable(getattr(self, x)) and
- x.startswith(METHOD_PREFIX),  dir(self))
-handlers = {}
-for m in methods:
-key = m.replace(METHOD_PREFIX, '')
-handlers[key] = getattr(self, m)
-
-return handlers
+self.network_state = copy.deepcopy(self.initial_network_state)
 
 def dump(self):
 state = {
@@ -83,24 +128,30 @@
 # v1 - direct attr mapping, except version
 for key in [k for k in required_keys if k not in ['version']]:
 setattr(self, key, state[key])
-self.command_handlers = self.get_command_handlers()
 
 def dump_network_st

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-net-moves into lp:cloud-init

2016-05-04 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/cloud-init-net-moves 
into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-moves/+merge/293807

Moves around the net_util file into the net module, removes some relative 
imports.

Tiny cleanups.
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-net-moves into lp:cloud-init.
=== modified file 'cloudinit/distros/arch.py'
--- cloudinit/distros/arch.py	2016-03-03 22:20:10 +
+++ cloudinit/distros/arch.py	2016-05-04 17:26:47 +
@@ -21,7 +21,7 @@
 from cloudinit import log as logging
 from cloudinit import util
 
-from cloudinit.distros import net_util
+from cloudinit.net import converter
 from cloudinit.distros.parsers.hostname import HostnameConf
 
 from cloudinit.settings import PER_INSTANCE
@@ -61,7 +61,7 @@
 self.package_command('', pkgs=pkglist)
 
 def _write_network(self, settings):
-entries = net_util.translate_network(settings)
+entries = converter.translate_network(settings)
 LOG.debug("Translated ubuntu style network settings %s into %s",
   settings, entries)
 dev_names = entries.keys()

=== modified file 'cloudinit/distros/freebsd.py'
--- cloudinit/distros/freebsd.py	2016-04-04 16:07:19 +
+++ cloudinit/distros/freebsd.py	2016-05-04 17:26:47 +
@@ -28,7 +28,7 @@
 from cloudinit import ssh_util
 from cloudinit import util
 
-from cloudinit.distros import net_util
+from cloudinit.net import converter
 from cloudinit.distros.parsers.resolv_conf import ResolvConf
 
 from cloudinit.settings import PER_INSTANCE
@@ -284,7 +284,7 @@
 ssh_util.setup_user_keys(keys, name, options=None)
 
 def _write_network(self, settings):
-entries = net_util.translate_network(settings)
+entries = converter.translate_network(settings)
 nameservers = []
 searchdomains = []
 dev_names = entries.keys()

=== removed file 'cloudinit/distros/net_util.py'
--- cloudinit/distros/net_util.py	2015-01-21 22:56:53 +
+++ cloudinit/distros/net_util.py	1970-01-01 00:00:00 +
@@ -1,182 +0,0 @@
-# vi: ts=4 expandtab
-#
-#Copyright (C) 2012 Canonical Ltd.
-#Copyright (C) 2012, 2013 Hewlett-Packard Development Company, L.P.
-#Copyright (C) 2012 Yahoo! Inc.
-#
-#Author: Scott Moser 
-#Author: Juerg Haefliger 
-#Author: Joshua Harlow 
-#
-#This program is free software: you can redistribute it and/or modify
-#it under the terms of the GNU General Public License version 3, as
-#published by the Free Software Foundation.
-#
-#This program is distributed in the hope that it will be useful,
-#but WITHOUT ANY WARRANTY; without even the implied warranty of
-#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#GNU General Public License for more details.
-#
-#You should have received a copy of the GNU General Public License
-#along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-
-# This is a util function to translate debian based distro interface blobs as
-# given in /etc/network/interfaces to an *somewhat* agnostic format for
-# distributions that use other formats.
-#
-# TODO(harlowja) remove when we have python-netcf active...
-#
-# The format is the following:
-# {
-#: {
-## All optional (if not existent in original format)
-#"netmask": ,
-#"broadcast": ,
-#"gateway": ,
-#"address": ,
-#"bootproto": "static"|"dhcp",
-#"dns-search": ,
-#"hwaddress": ,
-#"auto": True (or non-existent),
-#"dns-nameservers": [, ...],
-#}
-# }
-#
-# Things to note, comments are removed, if a ubuntu/debian interface is
-# marked as auto then only then first segment (?) is retained, ie
-# 'auto eth0 eth0:1' just marks eth0 as auto (not eth0:1).
-#
-# Example input:
-#
-# auto lo
-# iface lo inet loopback
-#
-# auto eth0
-# iface eth0 inet static
-# address 10.0.0.1
-# netmask 255.255.252.0
-# broadcast 10.0.0.255
-# gateway 10.0.0.2
-# dns-nameservers 98.0.0.1 98.0.0.2
-#
-# Example output:
-# {
-# "lo": {
-# "auto": true
-# },
-# "eth0": {
-# "auto": true,
-# "dns-nameservers": [
-# "98.0.0.1",
-# "98.0.0.2"
-# ],
-# "broadcast": "10.0.0.255",
-# "netmask": "255.255.252.0",
-# "bootproto": "static",
-# "address": "10.0.0.1",
-# "gateway

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/cloud-init-git-tarball-make into lp:cloud-init

2015-09-16 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/cloud-init-git-tarball-make into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/cloud-init-git-tarball-make/+merge/271394

Make tools/make-tarball work under existence of a .git directory (or a .bzr 
one).
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/cloud-init-git-tarball-make into lp:cloud-init.
=== modified file 'tools/make-tarball'
--- tools/make-tarball	2014-02-28 21:19:19 +
+++ tools/make-tarball	2015-09-16 22:03:26 +
@@ -1,4 +1,5 @@
 #!/bin/sh
+
 set -e
 
 find_root() {
@@ -18,22 +19,33 @@
 exit 1;
 fi
 
-REVNO=$(bzr revno "$ROOT_DIR")
+if [ -d "$ROOT_DIR/.git" ]; then
+VERSION=$(cd $ROOT_DIR && python setup.py --version)
+REVPOSTFIX=""
+else
+REVNO=$(bzr revno "$ROOT_DIR")
+REVPOSTFIX="~bzr$REVNO"
+VERSION=$("$ROOT_DIR/tools/read-version")
+fi
 
 if [ ! -z "$1" ]; then
 ARCHIVE_FN="$1"
 else
-VERSION=$("$ROOT_DIR/tools/read-version")
-ARCHIVE_FN="$PWD/cloud-init-$VERSION~bzr$REVNO.tar.gz"
-fi
-
-export_uncommitted=""
-if [ "${UNCOMMITTED:-0}" != "0" ]; then
-   export_uncommitted="--uncommitted"
-fi
-
-bzr export ${export_uncommitted} \
-   --format=tgz --root="cloud-init-$VERSION~bzr$REVNO" \
-"--revision=${REVNO}" "${ARCHIVE_FN}" "$ROOT_DIR"
+ARCHIVE_FN="$PWD/cloud-init-${VERSION}${REVPOSTFIX}.tar.gz"
+fi
+
+if [ -d "$ROOT_DIR/.git" ]; then
+git_hash=$(cd $ROOT_DIR &&  git log --pretty=format:'%h' -n 1)
+cd $ROOT_DIR && git archive -o "$ARCHIVE_FN" \
+--prefix="cloud-init-${VERSION}${REVPOSTFIX}/" "$git_hash"
+else
+export_uncommitted=""
+if [ "${UNCOMMITTED:-0}" != "0" ]; then
+   export_uncommitted="--uncommitted"
+fi
+bzr export ${export_uncommitted} \
+   --format=tgz --root="cloud-init-${VERSION}${REVPOSTFIX}" \
+"--revision=${REVNO}" "${ARCHIVE_FN}" "$ROOT_DIR"
+fi
 
 echo "$ARCHIVE_FN"

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/encode-resolve-fix into lp:cloud-init

2015-07-30 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/encode-resolve-fix 
into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)
Related bugs:
  Bug #1479988 in cloud-init: "AttributeError: 'ResolvConf' object has no 
attribute 'encode'"
  https://bugs.launchpad.net/cloud-init/+bug/1479988

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/encode-resolve-fix/+merge/266490
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/encode-resolve-fix into lp:cloud-init.
=== modified file 'cloudinit/distros/rhel_util.py'
--- cloudinit/distros/rhel_util.py	2015-01-27 19:24:22 +
+++ cloudinit/distros/rhel_util.py	2015-07-30 22:20:35 +
@@ -86,4 +86,4 @@
 r_conf.add_search_domain(s)
 except ValueError:
 util.logexc(LOG, "Failed at adding search domain %s", s)
-util.write_file(fn, r_conf, 0o644)
+util.write_file(fn, str(r_conf), 0o644)

=== modified file 'tests/unittests/test_distros/test_resolv.py'
--- tests/unittests/test_distros/test_resolv.py	2015-01-27 01:02:31 +
+++ tests/unittests/test_distros/test_resolv.py	2015-07-30 22:20:35 +
@@ -1,6 +1,8 @@
 from cloudinit.distros.parsers import resolv_conf
+from cloudinit.distros import rhel_util
 
 import re
+import tempfile
 from ..helpers import TestCase
 
 
@@ -19,6 +21,10 @@
 rp_r = str(rp).strip()
 self.assertEquals(BASE_RESOLVE, rp_r)
 
+def test_write_works(self):
+with tempfile.NamedTemporaryFile() as fh:
+rhel_util.update_resolve_conf_file(fh.name, [], [])
+
 def test_local_domain(self):
 rp = resolv_conf.ResolvConf(BASE_RESOLVE)
 self.assertEquals(None, rp.local_domain)

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~gerhard-1/cloud-init/fixate-uid into lp:cloud-init

2015-04-21 Thread Joshua Harlow

The proposal to merge lp:~gerhard-1/cloud-init/fixate-uid into lp:cloud-init 
has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~gerhard-1/cloud-init/fixate-uid/+merge/256140
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~gerhard-1/cloud-init/fixate-uid into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~brak/cloud-init/centos-7-fixes into lp:cloud-init

2015-04-21 Thread Joshua Harlow

The proposal to merge lp:~brak/cloud-init/centos-7-fixes into lp:cloud-init has 
been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~brak/cloud-init/centos-7-fixes/+merge/256340
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~brak/cloud-init/centos-7-fixes into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~daniel-thewatkins/cloud-init/lp1403617 into lp:cloud-init

2015-04-21 Thread Joshua Harlow

The proposal to merge lp:~daniel-thewatkins/cloud-init/lp1403617 into 
lp:cloud-init has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/lp1403617/+merge/256812
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~daniel-thewatkins/cloud-init/lp1403617 into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~brak/cloud-init/dont-overwrite-hostname into lp:cloud-init

2015-04-21 Thread Joshua Harlow

The proposal to merge lp:~brak/cloud-init/dont-overwrite-hostname into 
lp:cloud-init has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~brak/cloud-init/dont-overwrite-hostname/+merge/256348
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~brak/cloud-init/dont-overwrite-hostname into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~suro-patz/cloud-init/vm-clone-ip-reusage-dep-fix into lp:cloud-init

2015-04-09 Thread Joshua Harlow

Review: Approve


-- 
https://code.launchpad.net/~suro-patz/cloud-init/vm-clone-ip-reusage-dep-fix/+merge/255543
Your team cloud init development team is requested to review the proposed merge 
of lp:~suro-patz/cloud-init/vm-clone-ip-reusage-dep-fix into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/write-files-fetch-from-somewhere into lp:cloud-init

2015-03-31 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/write-files-fetch-from-somewhere into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/write-files-fetch-from-somewhere/+merge/254816
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/write-files-fetch-from-somewhere into lp:cloud-init.
=== modified file 'cloudinit/config/cc_write_files.py'
--- cloudinit/config/cc_write_files.py	2015-01-26 17:41:04 +
+++ cloudinit/config/cc_write_files.py	2015-03-31 19:11:17 +
@@ -25,6 +25,13 @@
 
 frequency = PER_INSTANCE
 
+# Content with these prefix(s) will be read/fetched externally to grab its
+# contents (which will replace any provided contents...)
+READ_EXTERNAL_PREFIXES = [
+"http://";,
+'file://',
+"https://";,
+]
 DEFAULT_OWNER = "root:root"
 DEFAULT_PERMS = 0o644
 UNKNOWN_ENC = 'text/plain'
@@ -70,8 +77,18 @@
  i + 1, name)
 continue
 path = os.path.abspath(path)
+pre_content = f_info.get('content', '')
+fetch_external = util.is_true(f_info.get('fetch_external', False))
+if not fetch_external:
+# See if we should anyway by looking for common prefixes...
+for p in READ_EXTERNAL_PREFIXES:
+if pre_content.startswith(p):
+fetch_external = True
+break
+if fetch_external:
+pre_content = util.read_file_or_url(pre_content)
 extractions = canonicalize_extraction(f_info.get('encoding'), log)
-contents = extract_contents(f_info.get('content', ''), extractions)
+contents = extract_contents(pre_content, extractions)
 (u, g) = util.extract_usergroup(f_info.get('owner', DEFAULT_OWNER))
 perms = decode_perms(f_info.get('permissions'), DEFAULT_PERMS, log)
 util.write_file(path, contents, mode=perms)

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~daniel-thewatkins/cloud-init/smartos-v2-metadata into lp:cloud-init

2015-03-12 Thread Joshua Harlow



Diff comments:

> === modified file 'cloudinit/sources/DataSourceSmartOS.py'
> --- cloudinit/sources/DataSourceSmartOS.py2015-01-27 20:03:52 +
> +++ cloudinit/sources/DataSourceSmartOS.py2015-03-04 16:40:58 +
> @@ -29,9 +29,10 @@
>  #   http://us-east.manta.joyent.com/jmc/public/mdata/datadict.html
>  #   Comments with "@datadictionary" are snippets of the definition
>  
> -import base64
>  import binascii
>  import os
> +import random
> +import re
>  import serial
>  
>  from cloudinit import log as logging
> @@ -301,6 +302,59 @@
>  return ser
>  
>  
> +class JoyentMetadataFetchException(Exception):
> +pass
> +
> +
> +class JoyentMetadataClient(object):
> +
> +def __init__(self, serial):
> +self.serial = serial
> +
> +def _checksum(self, body):
> +return '{0:08x}'.format(
> +binascii.crc32(body.encode('utf-8')) & 0x)
> +
> +def _get_value_from_frame(self, expected_request_id, frame):
> +regex = (
> +r'V2 (?P\d+) (?P[0-9a-f]+)'

Is this xml (sorta looks like it from your tag names)? Why not just use one of 
the built-in xml parsers that exist in python instead of something custom

> +r' (?P(?P[0-9a-f]+) 
> (?PSUCCESS|NOTFOUND)'
> +r'( (?P.+))?)')
> +frame_data = re.match(regex, frame).groupdict()
> +if int(frame_data['length']) != len(frame_data['body']):
> +raise JoyentMetadataFetchException(
> +'Incorrect frame length given ({0} != {1}).'.format(
> +frame_data['length'], len(frame_data['body'])))
> +expected_checksum = self._checksum(frame_data['body'])
> +if frame_data['checksum'] != expected_checksum:
> +raise JoyentMetadataFetchException(
> +'Invalid checksum (expected: {0}; got {1}).'.format(
> +expected_checksum, frame_data['checksum']))
> +if frame_data['request_id'] != expected_request_id:
> +raise JoyentMetadataFetchException(
> +'Request ID mismatch (expected: {0}; got {1}).'.format(
> +expected_request_id, frame_data['request_id']))
> +if not frame_data.get('payload', None):
> +LOG.info('No value found.')
> +return None
> +value = util.b64d(frame_data['payload'])
> +LOG.info('Value "%s" found.', value)
> +return value
> +
> +def get_metadata(self, metadata_key):
> +LOG.info('Fetching metadata key "%s"...', metadata_key)
> +request_id = '{0:08x}'.format(random.randint(0, 0x))
> +message_body = '{0} GET {1}'.format(request_id,
> +util.b64e(metadata_key))
> +msg = 'V2 {0} {1} {2}\n'.format(
> +len(message_body), self._checksum(message_body), message_body)
> +LOG.debug('Writing "%s" to serial port.', msg)
> +self.serial.write(msg)
> +response = self.serial.readline()
> +LOG.debug('Read "%s" from serial port.', response)
> +return self._get_value_from_frame(request_id, response)
> +
> +
>  def query_data(noun, seed_device, seed_timeout, strip=False, default=None,
> b64=None):
>  """Makes a request to via the serial console via "GET "
> @@ -314,33 +368,21 @@
>  encoded, so this method relies on being told if the data is base64 or
>  not.
>  """
> -
>  if not noun:
>  return False
>  
>  ser = get_serial(seed_device, seed_timeout)
> -ser.write("GET %s\n" % noun.rstrip())
> -status = str(ser.readline()).rstrip()
> -response = []
> -eom_found = False
>  
> -if 'SUCCESS' not in status:
> -ser.close()
> +client = JoyentMetadataClient(ser)

Seems like the above should just use contextlib.closing to ensure its safely 
closed

> +response = client.get_metadata(noun)
> +ser.close()
> +if response is None:
>  return default
>  
> -while not eom_found:
> -m = ser.readline()
> -if m.rstrip() == ".":
> -eom_found = True
> -else:
> -response.append(m)
> -
> -ser.close()
> -
>  if b64 is None:
>  b64 = query_data('b64-%s' % noun, seed_device=seed_device,
> -seed_timeout=seed_timeout, b64=False,
> -default=False, strip=True)
> + seed_timeout=seed_timeout, b64=False,
> + default=False, strip=True)
>  b64 = util.is_true(b64)
>  
>  resp = None
> 
> === modified file 'tests/unittests/test_datasource/test_smartos.py'
> --- tests/unittests/test_datasource/test_smartos.py   2015-01-27 20:03:52 
> +
> +++ tests/unittests/test_datasource/test_smartos.py   2015-03-04 16:40:58 
> +
> @@ -24,18 +24,27 @@
>  
>  from __future__ import print_function
>  
> -from cloudinit import helpers as c_helpers
> -from cloudinit.sources impo

Re: [Cloud-init-dev] [Merge] lp:~barry/cloud-init/py2-3 into lp:cloud-init

2015-01-27 Thread Joshua Harlow

> Thanks, I'll address each of the comments separately.
> 
> test-requirements.txt can probably go away.  If you use tox, it won't be used
> any more.  The tox.ini file specifies the test dependencies, but I'll leave
> removal of test-requirements.txt to you.  (I also am not using the Makefile;
> season to taste there too.)

Ok dokie :-)
-- 
https://code.launchpad.net/~barry/cloud-init/py2-3/+merge/247239
Your team cloud init development team is requested to review the proposed merge 
of lp:~barry/cloud-init/py2-3 into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~barry/cloud-init/py2-3 into lp:cloud-init

2015-01-23 Thread Joshua Harlow

Looks pretty good to me (seems like there is a new conflict against 
trunk/head); thanks for beating the tests into shape.
-- 
https://code.launchpad.net/~barry/cloud-init/py2-3/+merge/247239
Your team cloud init development team is requested to review the proposed merge 
of lp:~barry/cloud-init/py2-3 into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~barry/cloud-init/py2-3 into lp:cloud-init

2015-01-22 Thread Joshua Harlow

Thanks for chugging away on this :)
-- 
https://code.launchpad.net/~barry/cloud-init/py2-3/+merge/247239
Your team cloud init development team is requested to review the proposed merge 
of lp:~barry/cloud-init/py2-3 into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/py2-3 into lp:cloud-init

2015-01-21 Thread Joshua Harlow

Likely is behind (seeing that its not updated in a while); if you are working 
on a newer branch (derived from this one) that's cool and probably means we 
don't need this one at that point. Maybe time to slowly (or fastly) move to 
mock then...
-- 
https://code.launchpad.net/~harlowja/cloud-init/py2-3/+merge/225240
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/py2-3 into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/netinfo-cleanup into lp:cloud-init

2014-11-30 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/netinfo-cleanup into 
lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/netinfo-cleanup/+merge/243252

Cleanups for netinfo.py

- Handle ipv6 route information not existing gracefully (for systems that don't 
have it)
- Fix the getgateway function (broken due to ipv4/ipv6 keys now existing in 
route info)
- Separate the centering of the route info ipv4 information from the centering 
of the ipv6 information so that this looks prettier...
- Use try: except: else instead of settings value to None and then later 
checking for None (more pythonic this way)
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/netinfo-cleanup into lp:cloud-init.
=== modified file 'cloudinit/netinfo.py'
--- cloudinit/netinfo.py	2014-11-21 18:32:30 +
+++ cloudinit/netinfo.py	2014-12-01 05:39:04 +
@@ -97,7 +97,6 @@
 
 def route_info():
 (route_out, _err) = util.subp(["netstat", "-rn"])
-(route_out6, _err6) = util.subp(["netstat", "-A inet6", "-n"])
 
 routes = {}
 routes['ipv4'] = []
@@ -108,7 +107,6 @@
 if not line:
 continue
 toks = line.split()
-
 # FreeBSD shows 6 items in the routing table:
 #  Destination  GatewayFlags RefsUse  Netif Expire
 #  default  10.65.0.1  UGS  0  34920 vtnet0
@@ -120,13 +118,11 @@
 toks[0] == "Destination" or toks[0] == "Internet" or
 toks[0] == "Internet6" or toks[0] == "Routing"):
 continue
-
 if len(toks) < 8:
 toks.append("-")
 toks.append("-")
 toks[7] = toks[5]
 toks[5] = "-"
-
 entry = {
 'destination': toks[0],
 'gateway': toks[1],
@@ -137,39 +133,42 @@
 'use': toks[6],
 'iface': toks[7],
 }
-
 routes['ipv4'].append(entry)
 
-entries6 = route_out6.splitlines()[1:]
-for line in entries6:
-if not line:
-continue
-toks = line.split()
-
-if (len(toks) < 6 or toks[0] == "Kernel" or
-toks[0] == "Proto" or toks[0] == "Active"):
-continue
-entry = {
-'proto': toks[0],
-'recv-q': toks[1],
-'send-q': toks[2],
-'local address': toks[3],
-'foreign address': toks[4],
-'state': toks[5],
-}
-routes['ipv6'].append(entry)
+try:
+(route_out6, _err6) = util.subp(["netstat", "-A", "inet6", "-n"])
+except util.ProcessExecutionError:
+pass
+else:
+entries6 = route_out6.splitlines()[1:]
+for line in entries6:
+if not line:
+continue
+toks = line.split()
+if (len(toks) < 6 or toks[0] == "Kernel" or
+toks[0] == "Proto" or toks[0] == "Active"):
+continue
+entry = {
+'proto': toks[0],
+'recv-q': toks[1],
+'send-q': toks[2],
+'local address': toks[3],
+'foreign address': toks[4],
+'state': toks[5],
+}
+routes['ipv6'].append(entry)
 return routes
 
 
 def getgateway():
-routes = []
 try:
 routes = route_info()
 except:
 pass
-for r in routes:
-if r['flags'].find("G") >= 0:
-return "%s[%s]" % (r['gateway'], r['iface'])
+else:
+for r in routes.get('ipv4', []):
+if r['flags'].find("G") >= 0:
+return "%s[%s]" % (r['gateway'], r['iface'])
 return None
 
 
@@ -179,14 +178,14 @@
 netdev = netdev_info(empty=".")
 except Exception:
 lines.append(util.center("Net device info failed", '!', 80))
-netdev = None
-if netdev is not None:
+else:
 fields = ['Device', 'Up', 'Address', 'Mask', 'Scope', 'Hw-Address']
 tbl = PrettyTable(fields)
 for (dev, d) in netdev.iteritems():
 tbl.add_row([dev, d["up"], d["addr"], d["mask"], ".", d["hwaddr"]])
-if d["addr6"

[Cloud-init-dev] [Merge] lp:~shraddha-pandhe/cloud-init/cloud-init-ipv6-support into lp:cloud-init

2014-11-24 Thread Joshua Harlow

The proposal to merge lp:~shraddha-pandhe/cloud-init/cloud-init-ipv6-support 
into lp:cloud-init has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~shraddha-pandhe/cloud-init/cloud-init-ipv6-support/+merge/242547
-- 
Your team cloud init development team is requested to review the proposed merge 
of lp:~shraddha-pandhe/cloud-init/cloud-init-ipv6-support into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~shraddha-pandhe/cloud-init/cloud-init-ipv6-support into lp:cloud-init

2014-11-21 Thread Joshua Harlow



Diff comments:

> === modified file 'cloudinit/distros/net_util.py'
> --- cloudinit/distros/net_util.py 2014-01-24 21:20:54 +
> +++ cloudinit/distros/net_util.py 2014-11-21 22:24:21 +
> @@ -113,6 +113,10 @@
>  for info in ifaces:
>  if 'iface' not in info:
>  continue
> +use_ipv6 = False
> +# Check if current device has an ipv6 IP
> +if 'inet6' in info['iface']:
> +use_ipv6 = True
>  iface_details = info['iface'].split(None)
>  dev_name = None
>  if len(iface_details) >= 1:
> @@ -122,6 +126,7 @@
>  if not dev_name:
>  continue
>  iface_info = {}
> +iface_info['ipv6'] = {}
>  if len(iface_details) >= 3:
>  proto_type = iface_details[2].strip().lower()
>  # Seems like this can be 'loopback' which we don't
> @@ -129,26 +134,39 @@
>  if proto_type in ['dhcp', 'static']:
>  iface_info['bootproto'] = proto_type
>  # These can just be copied over
> -for k in ['netmask', 'address', 'gateway', 'broadcast']:
> -if k in info:
> -val = info[k].strip().lower()
> -if val:
> -iface_info[k] = val
> -# Name server info provided??
> -if 'dns-nameservers' in info:
> -iface_info['dns-nameservers'] = info['dns-nameservers'].split()
> -# Name server search info provided??
> -if 'dns-search' in info:
> -iface_info['dns-search'] = info['dns-search'].split()
> -# Is any mac address spoofing going on??
> -if 'hwaddress' in info:
> -hw_info = info['hwaddress'].lower().strip()
> -hw_split = hw_info.split(None, 1)
> -if len(hw_split) == 2 and hw_split[0].startswith('ether'):
> -hw_addr = hw_split[1]
> -if hw_addr:
> -iface_info['hwaddress'] = hw_addr
> -real_ifaces[dev_name] = iface_info
> +if use_ipv6:
> +for k in ['address', 'gateway']:
> +if k in info:
> +val = info[k].strip().lower()
> +if val:
> +iface_info['ipv6'][k] = val
> +else:
> +for k in ['netmask', 'address', 'gateway', 'broadcast']:
> +if k in info:
> +val = info[k].strip().lower()
> +if val:
> +iface_info[k] = val
> +# Name server info provided??
> +if 'dns-nameservers' in info:
> +iface_info['dns-nameservers'] = 
> info['dns-nameservers'].split()
> +# Name server search info provided??
> +if 'dns-search' in info:
> +iface_info['dns-search'] = info['dns-search'].split()
> +# Is any mac address spoofing going on??
> +if 'hwaddress' in info:
> +hw_info = info['hwaddress'].lower().strip()
> +hw_split = hw_info.split(None, 1)
> +if len(hw_split) == 2 and hw_split[0].startswith('ether'):
> +hw_addr = hw_split[1]
> +if hw_addr:
> +iface_info['hwaddress'] = hw_addr
> +
> +# If ipv6 is enabled, device will have multiple IPs.
> +# Update the dictionary instead of overwriting it
> +if dev_name in real_ifaces:
> +real_ifaces[dev_name].update(iface_info)
> +else:
> +real_ifaces[dev_name] = iface_info
>  # Check for those that should be started on boot via 'auto'
>  for (cmd, args) in entries:
>  if cmd == 'auto':
> @@ -160,4 +178,6 @@
>  dev_name = args[0].strip().lower()
>  if dev_name in real_ifaces:
>  real_ifaces[dev_name]['auto'] = True
> +if cmd == 'iface' and 'inet6' in args:
> +real_ifaces[dev_name]['inet6'] = True

indentation seems off here. to many spaces?

>  return real_ifaces
> 
> === modified file 'cloudinit/distros/rhel.py'
> --- cloudinit/distros/rhel.py 2014-10-17 19:32:41 +
> +++ cloudinit/distros/rhel.py 2014-11-21 22:24:21 +
> @@ -71,6 +71,7 @@
>  nameservers = []
>  searchservers = []
>  dev_names = entries.keys()
> +use_ipv6 = False
>  for (dev, info) in entries.iteritems():
>  net_fn = self.network_script_tpl % (dev)
>  net_cfg = {
> @@ -83,6 +84,13 @@
>  'MACADDR': info.get('hwaddress'),
>  'ONBOOT': _make_sysconfig_bool(info.get('auto')),
>  }
> +if info.get('inet6'):
> +use_ipv6 = True
> +net_cfg.update({
> +'IPV6INIT': _make_sysconfig_bool(True),
> +'IPV6ADDR': info.get('ipv6').get('address'),
> +'IPV6_DEFAULTGW': info.get('ipv6').get('gateway'),
> +

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/fix-digital-ocean-test-py26 into lp:cloud-init

2014-10-30 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/fix-digital-ocean-test-py26 into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/fix-digital-ocean-test-py26/+merge/240167

Fix the digital ocean test on py2.6

The digital ocean datasource test is using assertIs which
is only created/existent on py2.7, so for the older py2.6
we need to add similar logic so that the test works correctly
there.

-- 
https://code.launchpad.net/~harlowja/cloud-init/fix-digital-ocean-test-py26/+merge/240167
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/fix-digital-ocean-test-py26 into lp:cloud-init.
=== modified file 'tests/unittests/helpers.py'
--- tests/unittests/helpers.py	2014-07-23 16:56:39 +
+++ tests/unittests/helpers.py	2014-10-30 20:20:20 +
@@ -35,6 +35,11 @@
 if PY26:
 # For now add these on, taken from python 2.7 + slightly adjusted
 class TestCase(unittest.TestCase):
+def assertIs(self, expr1, expr2, msg=None):
+if expr1 is not expr2:
+standardMsg = '%r is not %r' % (expr1, expr2)
+self.fail(self._formatMessage(msg, standardMsg))
+
 def assertIn(self, member, container, msg=None):
 if member not in container:
 standardMsg = '%r not found in %r' % (member, container)

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/setup-virtualenv-installable into lp:cloud-init

2014-10-29 Thread Joshua Harlow

Joshua Harlow has proposed merging 
lp:~harlowja/cloud-init/setup-virtualenv-installable into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/setup-virtualenv-installable/+merge/240069

Only use datafiles and initsys addon outside virtualenvs

To make it so that cloud-init is installable in a virtualenv
where it can be tested in an isolated scenario we need to avoid
using and including datafiles (which won't be written into the
virtualenv) and also avoid using our initsys helper class which
also adds on its own files when we are being ran from a virtualenv.

-- 
https://code.launchpad.net/~harlowja/cloud-init/setup-virtualenv-installable/+merge/240069
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/setup-virtualenv-installable into lp:cloud-init.
=== modified file 'setup.py'
--- setup.py	2014-09-02 16:18:21 +
+++ setup.py	2014-10-30 00:24:52 +
@@ -23,6 +23,7 @@
 from glob import glob
 
 import os
+import sys
 
 import setuptools
 from setuptools.command.install import install
@@ -86,6 +87,17 @@
 ETC = "/usr/local/etc"
 
 
+# Avoid having datafiles installed in a virtualenv...
+def in_virtualenv():
+try:
+if sys.real_prefix == sys.prefix:
+return False
+else:
+return True
+except AttributeError:
+return False
+
+
 def get_version():
 cmd = ['tools/read-version']
 (ver, _e) = tiny_p(cmd)
@@ -135,6 +147,29 @@
 self.distribution.reinitialize_command('install_data', True)
 
 
+if in_virtualenv():
+data_files = []
+cmdclass = {}
+else:
+data_files = [
+(ETC + '/cloud', glob('config/*.cfg')),
+(ETC + '/cloud/cloud.cfg.d', glob('config/cloud.cfg.d/*')),
+(ETC + '/cloud/templates', glob('templates/*')),
+(USR + '/lib/cloud-init', ['tools/uncloud-init',
+   'tools/write-ssh-key-fingerprints']),
+(USR + '/share/doc/cloud-init', [f for f in glob('doc/*') if is_f(f)]),
+(USR + '/share/doc/cloud-init/examples',
+[f for f in glob('doc/examples/*') if is_f(f)]),
+(USR + '/share/doc/cloud-init/examples/seed', 
+[f for f in glob('doc/examples/seed/*') if is_f(f)]),
+]
+# Use a subclass for install that handles
+# adding on the right init system configuration files
+cmdclass = {
+'install': InitsysInstallData,
+}
+
+
 setuptools.setup(name='cloud-init',
   version=get_version(),
   description='EC2 initialisation magic',
@@ -146,23 +181,7 @@
'tools/cloud-init-per',
],
   license='GPLv3',
-  data_files=[(ETC + '/cloud', glob('config/*.cfg')),
-  (ETC + '/cloud/cloud.cfg.d', glob('config/cloud.cfg.d/*')),
-  (ETC + '/cloud/templates', glob('templates/*')),
-  (USR + '/lib/cloud-init',
-['tools/uncloud-init',
- 'tools/write-ssh-key-fingerprints']),
-  (USR + '/share/doc/cloud-init',
-   [f for f in glob('doc/*') if is_f(f)]),
-  (USR + '/share/doc/cloud-init/examples',
-   [f for f in glob('doc/examples/*') if is_f(f)]),
-  (USR + '/share/doc/cloud-init/examples/seed',
-   [f for f in glob('doc/examples/seed/*') if is_f(f)]),
- ],
+  data_files=data_files,
   install_requires=read_requires(),
-  cmdclass={
-  # Use a subclass for install that handles
-  # adding on the right init system configuration files
-  'install': InitsysInstallData,
-  },
+  cmdclass=cmdclass,
   )

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/start-module-docs into lp:cloud-init

2014-10-23 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/start-module-docs 
into lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/start-module-docs/+merge/239523

Start adding cloud config module docs.

Add the basics of docs that can be extracted from the code itself (also impose 
a initial format that will be useful for further modules to follow). In this 
initial addition modify the cc_debug.py and cc_ubuntu_init_switch.py to use 
this new style.
-- 
https://code.launchpad.net/~harlowja/cloud-init/start-module-docs/+merge/239523
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/start-module-docs into lp:cloud-init.
=== modified file 'cloudinit/config/cc_debug.py'
--- cloudinit/config/cc_debug.py	2014-01-23 19:28:59 +
+++ cloudinit/config/cc_debug.py	2014-10-24 00:49:09 +
@@ -14,6 +14,21 @@
 #You should have received a copy of the GNU General Public License
 #along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+"""
+**Summary:** helper to debug cloud-init *internal* datastructures.
+
+**Description:** This module will enable for outputting various internal
+information that cloud-init sources provide to either a file or to the output
+console/log location that this cloud-init has been configured with when
+running.
+
+It can be configured with the following option structure::
+
+debug:
+   verbose: (defaulting to true)
+   output: (location to write output, defaulting to console + log)
+"""
+
 from cloudinit import type_utils
 from cloudinit import util
 import copy

=== modified file 'cloudinit/config/cc_ubuntu_init_switch.py'
--- cloudinit/config/cc_ubuntu_init_switch.py	2014-07-31 20:03:41 +
+++ cloudinit/config/cc_ubuntu_init_switch.py	2014-10-24 00:49:09 +
@@ -17,30 +17,27 @@
 #along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 """
-ubuntu_init_switch: reboot system into another init
+**Summary:** reboot system into another init.
 
-This provides a way for the user to boot with systemd even if the
-image is set to boot with upstart.  It should be run as one of the first
-cloud_init_modules, and will switch the init system and then issue a reboot.
-The next boot will come up in the target init system and no action will
+**Description:** This module provides a way for the user to boot with systemd
+even if the image is set to boot with upstart.  It should be run as one of the
+first cloud_init_modules, and will switch the init system and then issue a
+reboot. The next boot will come up in the target init system and no action will
 be taken.
 
 This should be inert on non-ubuntu systems, and also exit quickly.
 
-config is comes under the top level 'init_switch' dictionary.
-
-#cloud-config
-init_switch:
- target: systemd
- reboot: true
-
-'target' can be 'systemd' or 'upstart'.  Best effort is made, but its possible
-this system will break, and probably won't interact well with any other
-mechanism you've used to switch the init system.
-
-'reboot': [default=true].
-   true: reboot if a change was made.
-   false: do not reboot.
+It can be configured with the following option structure::
+
+init_switch:
+  target: systemd (can be 'systemd' or 'upstart')
+  reboot: true (reboot if a change was made, or false to not reboot)
+
+.. note::
+
+Best effort is made, but it's possible
+this system will break, and probably won't interact well with any other
+mechanism you've used to switch the init system.
 """
 
 from cloudinit.settings import PER_INSTANCE

=== modified file 'doc/rtd/conf.py'
--- doc/rtd/conf.py	2014-01-24 19:47:28 +
+++ doc/rtd/conf.py	2014-10-24 00:49:09 +
@@ -27,6 +27,8 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 extensions = [
 'sphinx.ext.intersphinx',
+'sphinx.ext.autodoc',
+'sphinx.ext.viewcode',
 ]
 
 intersphinx_mapping = {

=== modified file 'doc/rtd/topics/modules.rst'
--- doc/rtd/topics/modules.rst	2013-01-25 02:41:26 +
+++ doc/rtd/topics/modules.rst	2014-10-24 00:49:09 +
@@ -1,3 +1,339 @@
-=
+===
 Modules
-=
+===
+
+Apt Configure
+-
+
+*Internal name:* ``cc_apt_configure``
+
+.. automodule:: cloudinit.config.cc_apt_configure
+
+Apt Pipelining
+--
+
+*Internal name:* ``cc_apt_pipelining``
+
+.. automodule:: cloudinit.config.cc_apt_pipelining
+
+Bootcmd
+---
+
+*Internal name:* ``cc_bootcmd``
+
+.. automodule:: cloudinit.config.cc_bootcmd
+
+Byobu
+-
+
+*Internal name:* ``cc_byobu``
+
+.. automodule:: cloudinit.config.cc_byobu
+
+Ca Certs
+-

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/ssh-key-types into lp:cloud-init

2014-10-21 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/ssh-key-types into 
lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/ssh-key-types/+merge/239121

Instead of only expected a list, tuple, or set type
allow for a string type and dict to be passed in for 'ssh_authorized_keys',
and add log message that occurs if some other type is used that
can not be correctly processed.
-- 
https://code.launchpad.net/~harlowja/cloud-init/ssh-key-types/+merge/239121
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/ssh-key-types into lp:cloud-init.
=== modified file 'cloudinit/distros/__init__.py'
--- cloudinit/distros/__init__.py	2014-09-10 18:32:37 +
+++ cloudinit/distros/__init__.py	2014-10-21 19:04:56 +
@@ -387,8 +387,19 @@
 
 # Import SSH keys
 if 'ssh_authorized_keys' in kwargs:
-keys = set(kwargs['ssh_authorized_keys']) or []
-ssh_util.setup_user_keys(keys, name, options=None)
+# Try to handle this in a smart manner.
+keys = kwargs['ssh_authorized_keys']
+if isinstance(keys, (basestring, str)):
+keys = [keys]
+if isinstance(keys, dict):
+keys = list(keys.values())
+if not isinstance(keys, (tuple, list, set)):
+util.multi_log("Invalid type detected for"
+   " 'ssh_authorized_keys', expected list, string"
+   " , dict, or set.")
+else:
+keys = set(keys) or []
+ssh_util.setup_user_keys(keys, name, options=None)
 
 return True
 

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/debug-pretty into lp:cloud-init

2014-10-20 Thread Joshua Harlow



Diff comments:

> === modified file 'cloudinit/config/cc_debug.py'
> --- cloudinit/config/cc_debug.py  2014-01-23 19:28:59 +
> +++ cloudinit/config/cc_debug.py  2014-10-18 16:32:48 +
> @@ -14,10 +14,13 @@
>  #You should have received a copy of the GNU General Public License
>  #along with this program.  If not, see .
>  
> +import copy
> +from StringIO import StringIO
> +
>  from cloudinit import type_utils
>  from cloudinit import util
> -import copy
> -from StringIO import StringIO
> +
> +SKIP_KEYS = frozenset(['log_cfgs'])
>  
>  
>  def _make_header(text):
> @@ -31,6 +34,11 @@
>  return header.getvalue()
>  
>  
> +def _dumps(obj):
> +text = util.yaml_dumps(obj, explicit_start=False, explicit_end=False)
> +return text.rstrip()
> +
> +
>  def handle(name, cfg, cloud, log, args):
>  verbose = util.get_cfg_by_path(cfg, ('debug', 'verbose'), default=True)
>  if args:
> @@ -46,7 +54,7 @@
>  return
>  # Clean out some keys that we just don't care about showing...
>  dump_cfg = copy.deepcopy(cfg)
> -for k in ['log_cfgs']:
> +for k in SKIP_KEYS:
>  dump_cfg.pop(k, None)
>  all_keys = list(dump_cfg.keys())
>  for k in all_keys:
> @@ -55,10 +63,10 @@
>  # Now dump it...
>  to_print = StringIO()
>  to_print.write(_make_header("Config"))
> -to_print.write(util.yaml_dumps(dump_cfg))
> +to_print.write(_dumps(dump_cfg))
>  to_print.write("\n")
>  to_print.write(_make_header("MetaData"))
> -to_print.write(util.yaml_dumps(cloud.datasource.metadata))
> +to_print.write(_dumps(cloud.datasource.metadata))
>  to_print.write("\n")
>  to_print.write(_make_header("Misc"))
>  to_print.write("Datasource: %s\n" %
> 
> === modified file 'cloudinit/util.py'
> --- cloudinit/util.py 2014-09-30 20:24:54 +
> +++ cloudinit/util.py 2014-10-18 16:32:48 +
> @@ -1270,14 +1270,14 @@
>  logexc(LOG, "Failed writing url content to %s", target_fn)
>  
>  
> -def yaml_dumps(obj):
> -formatted = yaml.dump(obj,
> -line_break="\n",
> -indent=4,
> -explicit_start=True,
> -explicit_end=True,
> -default_flow_style=False)
> -return formatted
> +def yaml_dumps(obj, explicit_start=True, explicit_end=True):
> +return yaml.safe_dump(obj,
> +  line_break="\n",
> +  indent=4,
> +  explicit_start=explicit_start,
> +  explicit_end=explicit_end,
> +  default_flow_style=False,
> +  allow_unicode=True)

I could, but since nobody was actually sending those I decided not to, if 
someone starts to desire to send those then we could further modify this to 
allow that, but currently nobody is desiring to override those so seems not so 
useful "yet" to do that.

>  
>  
>  def ensure_dir(path, mode=None):
> 
> === added file 'tests/unittests/test_handler/test_handler_debug.py'
> --- tests/unittests/test_handler/test_handler_debug.py1970-01-01 
> 00:00:00 +
> +++ tests/unittests/test_handler/test_handler_debug.py2014-10-18 
> 16:32:48 +
> @@ -0,0 +1,78 @@
> +# vi: ts=4 expandtab
> +#
> +#Copyright (C) 2014 Yahoo! Inc.
> +#
> +#This program is free software: you can redistribute it and/or modify
> +#it under the terms of the GNU General Public License version 3, as
> +#published by the Free Software Foundation.
> +#
> +#This program is distributed in the hope that it will be useful,
> +#but WITHOUT ANY WARRANTY; without even the implied warranty of
> +#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +#GNU General Public License for more details.
> +#
> +#You should have received a copy of the GNU General Public License
> +#along with this program.  If not, see .
> +
> +from cloudinit.config import cc_debug
> +
> +from cloudinit import cloud
> +from cloudinit import distros
> +from cloudinit import helpers
> +from cloudinit import util
> +
> +from cloudinit.sources import DataSourceNone
> +
> +from .. import helpers as t_help
> +
> +import logging
> +
> +LOG = logging.getLogger(__name__)
> +
> +
> +class TestDebug(t_help.FilesystemMockingTestCase):
> +def setUp(self):
> +super(TestDebug, self).setUp()
> +self.new_root = self.makeDir(prefix="unittest_")
> +
> +def _get_cloud(self, distro, metadata=None):
> +self.patchUtils(self.new_root)
> +paths = helpers.Paths({})
> +cls = distros.fetch(distro)
> +d = cls(distro, {}, paths)
> +ds = DataSourceNone.DataSourceNone({}, d, paths)
> +if metadata:
> +ds.metadata.update(metadata)
> +return cloud.Cloud(ds, paths, {}, d, None)
> +
> +def test_debug_write(self):
> +cfg = {
> +'ab

[Cloud-init-dev] [Merge] lp:~harlowja/cloud-init/debug-pretty into lp:cloud-init

2014-10-18 Thread Joshua Harlow

Joshua Harlow has proposed merging lp:~harlowja/cloud-init/debug-pretty into 
lp:cloud-init.

Requested reviews:
  cloud init development team (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~harlowja/cloud-init/debug-pretty/+merge/238799

Pretty up the debug module

Previously the usage of the yaml_dumps module was causing
any python unicode key and value to show up as:

'item': !!python/unicode "some string"

This was not very pretty...

Fix this by using safe_dumps (which is also a good thing to
use and allow_unicode=True). Also create a tiny helper function
in the cc_debug module that does not include the yaml start and
end footers (since this module has its own footers and headers).

Also includes a basic sanity test for this module.

-- 
https://code.launchpad.net/~harlowja/cloud-init/debug-pretty/+merge/238799
Your team cloud init development team is requested to review the proposed merge 
of lp:~harlowja/cloud-init/debug-pretty into lp:cloud-init.
=== modified file 'cloudinit/config/cc_debug.py'
--- cloudinit/config/cc_debug.py	2014-01-23 19:28:59 +
+++ cloudinit/config/cc_debug.py	2014-10-18 16:32:48 +
@@ -14,10 +14,13 @@
 #You should have received a copy of the GNU General Public License
 #along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import copy
+from StringIO import StringIO
+
 from cloudinit import type_utils
 from cloudinit import util
-import copy
-from StringIO import StringIO
+
+SKIP_KEYS = frozenset(['log_cfgs'])
 
 
 def _make_header(text):
@@ -31,6 +34,11 @@
 return header.getvalue()
 
 
+def _dumps(obj):
+text = util.yaml_dumps(obj, explicit_start=False, explicit_end=False)
+return text.rstrip()
+
+
 def handle(name, cfg, cloud, log, args):
 verbose = util.get_cfg_by_path(cfg, ('debug', 'verbose'), default=True)
 if args:
@@ -46,7 +54,7 @@
 return
 # Clean out some keys that we just don't care about showing...
 dump_cfg = copy.deepcopy(cfg)
-for k in ['log_cfgs']:
+for k in SKIP_KEYS:
 dump_cfg.pop(k, None)
 all_keys = list(dump_cfg.keys())
 for k in all_keys:
@@ -55,10 +63,10 @@
 # Now dump it...
 to_print = StringIO()
 to_print.write(_make_header("Config"))
-to_print.write(util.yaml_dumps(dump_cfg))
+to_print.write(_dumps(dump_cfg))
 to_print.write("\n")
 to_print.write(_make_header("MetaData"))
-to_print.write(util.yaml_dumps(cloud.datasource.metadata))
+to_print.write(_dumps(cloud.datasource.metadata))
 to_print.write("\n")
 to_print.write(_make_header("Misc"))
 to_print.write("Datasource: %s\n" %

=== modified file 'cloudinit/util.py'
--- cloudinit/util.py	2014-09-30 20:24:54 +
+++ cloudinit/util.py	2014-10-18 16:32:48 +
@@ -1270,14 +1270,14 @@
 logexc(LOG, "Failed writing url content to %s", target_fn)
 
 
-def yaml_dumps(obj):
-formatted = yaml.dump(obj,
-line_break="\n",
-indent=4,
-explicit_start=True,
-explicit_end=True,
-default_flow_style=False)
-return formatted
+def yaml_dumps(obj, explicit_start=True, explicit_end=True):
+return yaml.safe_dump(obj,
+  line_break="\n",
+  indent=4,
+  explicit_start=explicit_start,
+  explicit_end=explicit_end,
+  default_flow_style=False,
+  allow_unicode=True)
 
 
 def ensure_dir(path, mode=None):

=== added file 'tests/unittests/test_handler/test_handler_debug.py'
--- tests/unittests/test_handler/test_handler_debug.py	1970-01-01 00:00:00 +
+++ tests/unittests/test_handler/test_handler_debug.py	2014-10-18 16:32:48 +
@@ -0,0 +1,78 @@
+# vi: ts=4 expandtab
+#
+#Copyright (C) 2014 Yahoo! Inc.
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from cloudinit.config import cc_debug
+
+from cloudinit import cloud
+from cloudinit import distros
+from cloudinit import helpers
+from cloudinit import util
+
+from cloudinit.sources import DataSourceNone
+
+from .. import helpers as t_help
+
+import logging
+
+LOG = logging.getLogger(__name__)
+
+
+

Re: [Cloud-init-dev] [Merge] lp:~nshrader/cloud-init/digitalocean-datasource into lp:cloud-init

2014-10-17 Thread Joshua Harlow

Thanks, looks good to me. Glad the metadata materializer worked out :)
-- 
https://code.launchpad.net/~nshrader/cloud-init/digitalocean-datasource/+merge/238590
Your team cloud init development team is requested to review the proposed merge 
of lp:~nshrader/cloud-init/digitalocean-datasource into lp:cloud-init.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] lp:~nshrader/cloud-init/digitalocean-datasource into lp:cloud-init

2014-10-17 Thread Joshua Harlow



Diff comments:

> === added file 'cloudinit/sources/DataSourceDigitalOcean.py'
> --- cloudinit/sources/DataSourceDigitalOcean.py   1970-01-01 00:00:00 
> +
> +++ cloudinit/sources/DataSourceDigitalOcean.py   2014-10-17 22:04:47 
> +
> @@ -0,0 +1,105 @@
> +# vi: ts=4 expandtab
> +#
> +#Author: Neal Shrader 
> +#
> +#This program is free software: you can redistribute it and/or modify
> +#it under the terms of the GNU General Public License version 3, as
> +#published by the Free Software Foundation.
> +#
> +#This program is distributed in the hope that it will be useful,
> +#but WITHOUT ANY WARRANTY; without even the implied warranty of
> +#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +#GNU General Public License for more details.
> +#
> +#You should have received a copy of the GNU General Public License
> +#along with this program.  If not, see .
> +
> +from cloudinit import log as logging
> +from cloudinit import util
> +from cloudinit import sources
> +from cloudinit import url_helper
> +from cloudinit import ec2_utils
> +from types import *

Can u explicitly import what u need, this avoids namespace pollution. Thanks :)

> +import functools
> +
> +
> +LOG = logging.getLogger(__name__)
> +
> +BUILTIN_DS_CONFIG = {
> +'metadata_url': 'http://169.254.169.254/metadata/v1/',
> +'mirrors_url': 'http://mirrors.digitalocean.com/'
> +}
> +MD_RETRIES = 0
> +MD_TIMEOUT = 1
> +
> +class DataSourceDigitalOcean(sources.DataSource):
> +def __init__(self, sys_cfg, distro, paths):
> +sources.DataSource.__init__(self, sys_cfg, distro, paths)
> +self.metadata = dict()
> +self.ds_cfg = util.mergemanydict([
> +util.get_cfg_by_path(sys_cfg, ["datasource", "DigitalOcean"], 
> {}),
> +BUILTIN_DS_CONFIG])
> +self.metadata_address = self.ds_cfg['metadata_url']
> +
> +if self.ds_cfg.get('retries'):
> +self.retries = self.ds_cfg['retries']
> +else:
> +self.retries = MD_RETRIES
> +
> +if self.ds_cfg.get('timeout'):
> +self.timeout = self.ds_cfg['timeout']
> +else:
> +self.timeout = MD_TIMEOUT
> +
> +def get_data(self):
> +caller = functools.partial(util.read_file_or_url, 
> timeout=self.timeout, 
> +   retries=self.retries)
> +md = 
> ec2_utils.MetadataMaterializer(str(caller(self.metadata_address)),
> +base_url=self.metadata_address, 
> +caller=caller)
> +
> +self.metadata = md.materialize()
> +
> +if self.metadata.get('id'):
> +return True
> +else:
> +return False
> +
> +def get_userdata_raw(self):
> +return "\n".join(self.metadata['user-data'])
> +
> +def get_vendordata_raw(self):
> +return "\n".join(self.metadata['vendor-data'])
> +
> +def get_public_ssh_keys(self):
> +if type(self.metadata['public-keys']) is StringType:
> +   return [self.metadata['public-keys']]
> +else:
> +   return self.metadata['public-keys']
> +
> +@property
> +def availability_zone(self):
> +return self.metadata['region']
> +
> +def get_instance_id(self):
> +return self.metadata['id']
> +
> +def get_hostname(self, fqdn=False):
> +return self.metadata['hostname']
> +
> +def get_package_mirror_info(self):
> +return self.ds_cfg['mirrors_url']
> +
> +@property
> +def launch_index(self):
> +return None
> +
> +# Used to match classes to dependencies
> +datasources = [
> +  (DataSourceDigitalOcean, (sources.DEP_FILESYSTEM, sources.DEP_NETWORK)),
> +  ]
> +
> +
> +# Return a list of data sources that match this set of dependencies
> +def get_datasource_list(depends):
> +return sources.list_from_depends(depends, datasources)
> 
> === added directory 'doc/sources/digitalocean'
> === added file 'doc/sources/digitalocean/README.rst'
> --- doc/sources/digitalocean/README.rst   1970-01-01 00:00:00 +
> +++ doc/sources/digitalocean/README.rst   2014-10-17 22:04:47 +
> @@ -0,0 +1,21 @@
> + The `DigitalOcean`_ datasource consumes the content served from 
> DigitalOcean's `metadata service`_.  This
> +metadata service serves information about the running droplet via HTTP over 
> the link local address
> +169.254.169.254.  The metadata API endpoints are fully described at 
> +`https://developers.digitalocean.com/metadata/ 
> `_.
> +
> +Configuration
> +~
> +
> +DigitalOcean's datasource can be configured as follows:
> +
> +  datasource:
> +DigitalOcean:
> +  retries: 3
> +  timeout: 2
> +
> +- *retries*: Determines the number of times to attempt to connect to the 
> metadata service
> +- *timeout*: Determines the timeout in

1 2 3 >

1 - 100 of 278 matches

Mail list logo