Re: [Cloud-init-dev] [Merge] ~powersj/cloud-init:test-ci into cloud-init:master

2017-06-15 Thread Server Team CI bot 
Review: Approve continuous-integration

PASSED: Continuous integration, rev:ef13ce3def0728ed8c95c675c4552c1258f4d5d0
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1/
Executed test runs:
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/1
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/1
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/1
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/1
SUCCESS: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/1

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1/rebuild

-- 
https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/325777
Your team cloud-init commiters is requested to review the proposed merge of 
~powersj/cloud-init:test-ci into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~powersj/cloud-init:test-ci into cloud-init:master

2017-06-15 Thread Joshua Powers 
Joshua Powers has proposed merging ~powersj/cloud-init:test-ci into 
cloud-init:master.

Commit message:
doc: test

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/325777

DO NOT MERGE
This is a test for the new CI system only!
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~powersj/cloud-init:test-ci into cloud-init:master.
diff --git a/README.md b/README.md
new file mode 100644
index 000..4904846
--- /dev/null
+++ b/README.md
@@ -0,0 +1 @@
+# cloud-init
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~powersj/cloud-init:test-ci into cloud-init:master

2017-06-15 Thread Joshua Powers 
The proposal to merge ~powersj/cloud-init:test-ci into cloud-init:master has 
been updated.

Description changed to:

DO NOT MERGE
This is a test for the new CI system only!

For more details, see:
https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/325777
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~powersj/cloud-init:test-ci into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into cloud-init:master

2017-06-15 Thread Scott Moser 
The proposal to merge ~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into 
cloud-init:master has been updated.

Status: Approved => Merged

For more details, see:
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325713
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:fix/freebsd-test-failure into cloud-init:master

2017-06-15 Thread Chad Smith 


Diff comments:

> diff --git a/cloudinit/sources/DataSourceAzure.py 
> b/cloudinit/sources/DataSourceAzure.py
> index ebb53d0..e5514ad 100644
> --- a/cloudinit/sources/DataSourceAzure.py
> +++ b/cloudinit/sources/DataSourceAzure.py
> @@ -803,18 +803,22 @@ def encrypt_pass(password, salt_id="$6$"):
>  return crypt.crypt(password, salt_id + util.rand_str(strlen=16))
>  
>  
> +def _check_freebsd_cdrom(cdrom_dev):
> +"""Return boolean indicating path to cdrom device has content."""

Where's the return here? it'll return None in all cases right?

> +try:
> +with open(cdrom_dev) as fp:
> +fp.read(1024)
> +devlist.append(cdrom_dev)
> +except IOError:
> +LOG.debug("cdrom (%s) is not configured", cdrom_dev)
> +
> +
>  def list_possible_azure_ds_devs():
>  devlist = []
>  if util.is_FreeBSD():
> -# add '/dev/cd0' to devlist if it is configured
> -# here wants to test whether '/dev/cd0' is available
>  cdrom_dev = "/dev/cd0"
> -try:
> -with open(cdrom_dev) as fp:
> -fp.read(1024)
> -devlist.append(cdrom_dev)
> -except IOError:
> -LOG.debug("cdrom (%s) is not configured", cdrom_dev)
> +if _check_freebsd_cdrom(cdrom_dev):
> +return [cdrom_dev]
>  else:
>  for fstype in ("iso9660", "udf"):
>  devlist.extend(util.find_devs_with("TYPE=%s" % fstype))


-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325771
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:fix/freebsd-test-failure into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~smoser/cloud-init:fix/freebsd-test-failure into cloud-init:master

2017-06-15 Thread Scott Moser 
The proposal to merge ~smoser/cloud-init:fix/freebsd-test-failure into 
cloud-init:master has been updated.

Commit Message changed to:

FreeBSD: fix test failure

The previous commit caused test failure.
This separates out _check_freebsd_cdrom and mocks it in a test
rather than patching open.

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325771
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:fix/freebsd-test-failure into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~smoser/cloud-init:fix/freebsd-test-failure into cloud-init:master

2017-06-15 Thread Scott Moser 
Scott Moser has proposed merging ~smoser/cloud-init:fix/freebsd-test-failure 
into cloud-init:master.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325771
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:fix/freebsd-test-failure into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index ebb53d0..e5514ad 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -803,18 +803,22 @@ def encrypt_pass(password, salt_id="$6$"):
 return crypt.crypt(password, salt_id + util.rand_str(strlen=16))
 
 
+def _check_freebsd_cdrom(cdrom_dev):
+"""Return boolean indicating path to cdrom device has content."""
+try:
+with open(cdrom_dev) as fp:
+fp.read(1024)
+devlist.append(cdrom_dev)
+except IOError:
+LOG.debug("cdrom (%s) is not configured", cdrom_dev)
+
+
 def list_possible_azure_ds_devs():
 devlist = []
 if util.is_FreeBSD():
-# add '/dev/cd0' to devlist if it is configured
-# here wants to test whether '/dev/cd0' is available
 cdrom_dev = "/dev/cd0"
-try:
-with open(cdrom_dev) as fp:
-fp.read(1024)
-devlist.append(cdrom_dev)
-except IOError:
-LOG.debug("cdrom (%s) is not configured", cdrom_dev)
+if _check_freebsd_cdrom(cdrom_dev):
+return [cdrom_dev]
 else:
 for fstype in ("iso9660", "udf"):
 devlist.extend(util.find_devs_with("TYPE=%s" % fstype))
diff --git a/tests/unittests/test_datasource/test_azure.py b/tests/unittests/test_datasource/test_azure.py
index 114b1a5..8ee07d7 100644
--- a/tests/unittests/test_datasource/test_azure.py
+++ b/tests/unittests/test_datasource/test_azure.py
@@ -544,14 +544,16 @@ fdescfs/dev/fd  fdescfs rw  0 0
 ds.get_data()
 self.assertEqual(self.instance_id, ds.metadata['instance-id'])
 
-def test_list_possible_azure_ds_devs(self):
+@patch("cloudinit.sources.DataSourceAzure.util.is_FreeBSD")
+@patch("cloudinit.sources.DataSourceAzure._check_freebsd_cdrom")
+def test_list_possible_azure_ds_devs(self, m_check_fbsd_cdrom,
+ m_is_FreeBSD):
 devlist = []
-with patch('platform.platform',
-   mock.MagicMock(return_value="FreeBSD")):
-name = 'builtins.open' if PY3 else '__builtin__.open'
-with patch(name, mock_open(read_data="data")):
-devlist.extend(dsaz.list_possible_azure_ds_devs())
-self.assertEqual(devlist, ['/dev/cd0'])
+m_is_FreeBSD.return_value = True
+m_check_fbsd_cdrom.return_value = True
+self.assertEqual(dsaz.list_possible_azure_ds_devs(), ['/dev/cd0'])
+self.assertEqual(
+[mock.call("/dev/cd0")], m_check_fbsd_cdrom.call_args_list)
 
 
 class TestAzureBounce(TestCase):
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Chad Smith 
The proposal to merge ~smoser/cloud-init:feature/freebsd-variant into 
cloud-init:master has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Chad Smith 
+1 on that suggestion

-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Scott Moser 
actually, on further thought, lets leve suse out right now.
there is a 'sles' distro (cloudinit/distros/sles.py) but per 
 https://github.com/hpcugent/easybuild/wiki/OS_flavor_name_version
we would see 'suse' as the dist.

not sure what the right change is.

i didn't *change* any behavior here i dont think.
so i'd rather leave it is and just enable the freebsd.

and we can/should fix suse/sles later.

-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~b1sandmann/cloud-init:fix-debian-template into cloud-init:master

2017-06-15 Thread Scott Moser 
The proposal to merge ~b1sandmann/cloud-init:fix-debian-template into 
cloud-init:master has been updated.

Status: Approved => Merged

For more details, see:
https://code.launchpad.net/~b1sandmann/cloud-init/+git/cloud-init/+merge/325478
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~b1sandmann/cloud-init:fix-debian-template into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into cloud-init:master

2017-06-15 Thread Scott Moser 
The proposal to merge ~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into 
cloud-init:master has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325713
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Server Team CI bot 
Review: Approve continuous-integration

PASSED: Continuous integration, rev:c6af5b9a1648c208c82b3a2704668391abadd8ab
https://jenkins.ubuntu.com/server/job/cloud-init-ci/539/
Executed test runs:
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/539
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/539
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/539
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/539
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/539

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/539/rebuild

-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Scott Moser 
The proposal to merge ~smoser/cloud-init:feature/freebsd-variant into 
cloud-init:master has been updated.

Commit Message changed to:

freebsd: Make freebsd a variant, fix unittests and tools/build-on-freebsd.

 - Simplify the logic of 'variant' in util.system_info
   much of the data from
   https://github.com/hpcugent/easybuild/wiki/OS_flavor_name_version
 - fix get_resource_disk_on_freebsd when running on a system without
   an Azure resource disk.
 - fix tools/build-on-freebsd to replace oauth with oauthlib and add
   bash which is a dependency for tests.
 - update a fiew places that were checking for freebsd but not using
   the util.is_FreeBSD()

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Server Team CI bot 
Review: Needs Fixing continuous-integration

FAILED: Continuous integration, rev:2151902fa7a5cba7e420a14f51c851ee06d83d30
https://jenkins.ubuntu.com/server/job/cloud-init-ci/538/
Executed test runs:
FAILURE: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/538/console
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/538
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/538
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/538
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/538

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/538/rebuild

-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Scott Moser 
this replaces some of the changes from
 https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325718

-- 
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master

2017-06-15 Thread Scott Moser 
i've put up
 https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760

that covers most of these.

-- 
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325718
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~smoser/cloud-init:feature/freebsd-variant into cloud-init:master

2017-06-15 Thread Scott Moser 
Scott Moser has proposed merging ~smoser/cloud-init:feature/freebsd-variant 
into cloud-init:master.

Commit message:
fixes for freebsd

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325760
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~smoser/cloud-init:feature/freebsd-variant into cloud-init:master.
diff --git a/cloudinit/config/cc_growpart.py b/cloudinit/config/cc_growpart.py
index d2bc6e6..bafca9d 100644
--- a/cloudinit/config/cc_growpart.py
+++ b/cloudinit/config/cc_growpart.py
@@ -214,7 +214,7 @@ def device_part_info(devpath):
 
 # FreeBSD doesn't know of sysfs so just get everything we need from
 # the device, like /dev/vtbd0p2.
-if util.system_info()["platform"].startswith('FreeBSD'):
+if util.is_FreeBSD():
 m = re.search('^(/dev/.+)p([0-9])$', devpath)
 return (m.group(1), m.group(2))
 
diff --git a/cloudinit/config/cc_power_state_change.py b/cloudinit/config/cc_power_state_change.py
index c1c6fe7..eba58b0 100644
--- a/cloudinit/config/cc_power_state_change.py
+++ b/cloudinit/config/cc_power_state_change.py
@@ -71,7 +71,7 @@ def givecmdline(pid):
 # Example output from procstat -c 1
 #   PID COMM ARGS
 # 1 init /bin/init --
-if util.system_info()["platform"].startswith('FreeBSD'):
+if util.is_FreeBSD():
 (output, _err) = util.subp(['procstat', '-c', str(pid)])
 line = output.splitlines()[1]
 m = re.search('\d+ (\w|\.|-)+\s+(/\w.+)', line)
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index a0b9eae..9c5ce59 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -101,7 +101,7 @@ def get_dev_storvsc_sysctl():
 sysctl_out, err = util.subp(['sysctl', 'dev.storvsc'])
 except util.ProcessExecutionError:
 LOG.debug("Fail to execute sysctl dev.storvsc")
-return None
+sysctl_out = ""
 return sysctl_out
 
 
diff --git a/cloudinit/util.py b/cloudinit/util.py
index 415ca37..6f2b38c 100644
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -573,7 +573,7 @@ def is_ipv4(instr):
 
 
 def is_FreeBSD():
-return system_info()['platform'].startswith('FreeBSD')
+return system_info()['variant'] == "freebsd"
 
 
 def get_cfg_option_bool(yobj, key, default=False):
@@ -598,37 +598,29 @@ def get_cfg_option_int(yobj, key, default=0):
 def system_info():
 info = {
 'platform': platform.platform(),
+'system': platform.system(),
 'release': platform.release(),
 'python': platform.python_version(),
 'uname': platform.uname(),
-'dist': platform.linux_distribution(),  # pylint: disable=W1505
+'dist': platform.dist(),
 }
-plat = info['platform'].lower()
-# Try to get more info about what it actually is, in a format
-# that we can easily use across linux and variants...
-if plat.startswith('darwin'):
-info['variant'] = 'darwin'
-elif plat.endswith("bsd"):
-info['variant'] = 'bsd'
-elif plat.startswith('win'):
-info['variant'] = 'windows'
-elif 'linux' in plat:
-# Try to get a single string out of these...
-linux_dist, _version, _id = info['dist']
-linux_dist = linux_dist.lower()
-if linux_dist in ('ubuntu', 'linuxmint', 'mint'):
-info['variant'] = 'ubuntu'
+system = info['system'].lower()
+var = 'unknown'
+if system == "linux":
+linux_dist = info['dist'][0].lower()
+if linux_dist in ('centos', 'fedora', 'debian'):
+var = linux_dist
+elif linux_dist in ('ubuntu', 'linuxmint', 'mint'):
+var = 'ubuntu'
+elif linux_dist == 'redhat':
+var = 'rhel'
 else:
-for prefix, variant in [('redhat', 'rhel'),
-('centos', 'centos'),
-('fedora', 'fedora'),
-('debian', 'debian')]:
-if linux_dist.startswith(prefix):
-info['variant'] = variant
-if 'variant' not in info:
-info['variant'] = 'linux'
-if 'variant' not in info:
-info['variant'] = 'unknown'
+var = 'linux'
+elif system in ('windows', 'darwin', "freebsd"):
+var = system
+
+info['variant'] = var
+
 return info
 
 
diff --git a/config/cloud.cfg.tmpl b/config/cloud.cfg.tmpl
index 5af2a88..f4b9069 100644
--- a/config/cloud.cfg.tmpl
+++ b/config/cloud.cfg.tmpl
@@ -2,7 +2,7 @@
 # The top level settings are used as module
 # and system configuration.
 
-{% if variant in ["bsd"] %}
+{% if variant in ["freebsd"] %}
 syslog_fix_perms: root:wheel
 {% endif %}
 # A set of users which may be applied and/or used by various modules
@@ -13,7 +13,7

Re: [Cloud-init-dev] [Merge] ~jcastets/cloud-init:scaleway-datasource into cloud-init:master

2017-06-15 Thread Server Team CI bot 
Review: Approve continuous-integration

PASSED: Continuous integration, rev:a0748674f74e97e70a3274d312c9592b28a6dc08
https://jenkins.ubuntu.com/server/job/cloud-init-ci/537/
Executed test runs:
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/537
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/537
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/537
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/537
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/537

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/537/rebuild

-- 
https://code.launchpad.net/~jcastets/cloud-init/+git/cloud-init/+merge/325740
Your team cloud-init commiters is requested to review the proposed merge of 
~jcastets/cloud-init:scaleway-datasource into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master

2017-06-15 Thread Scott Moser 


Diff comments:

> diff --git a/tools/build-on-freebsd b/tools/build-on-freebsd
> index ccc10b4..f16d62e 100755
> --- a/tools/build-on-freebsd
> +++ b/tools/build-on-freebsd
> @@ -17,6 +17,7 @@ pkgs="
> py27-jsonpatch
> py27-jsonpointer
> py27-oauth

we can drop py27-oauth now. its replaced with oauthlib.

> +   py27-oauthlib
> py27-prettytable
> py27-requests
> py27-serial


-- 
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325718
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~jcastets/cloud-init:scaleway-datasource into cloud-init:master

2017-06-15 Thread Scott Moser 

Some things, and some content inline:
 * We'll need some unit tests for this.  Otherwise it is at increased risk of 
being inadvertently broken.
 * We will need to add knowledge of the datasource to tools/ds-identify 
(without that your datasource will only ever be considered if it is a single 
entry in the configured list)
 * we really, *REALLY* want a positive non-network identification.  Without 
such a thing, we can't enable the datasource by default, meaning Ubuntu or 
other images that would work elsewhere wont work on your platform.


Also, give a nicer commit message:
  Summary
  
  More information
  ...
  
  LP: #XXX


Diff comments:

> diff --git a/cloudinit/sources/DataSourceScaleway.py 
> b/cloudinit/sources/DataSourceScaleway.py
> new file mode 100644
> index 000..7ac1b1a
> --- /dev/null
> +++ b/cloudinit/sources/DataSourceScaleway.py
> @@ -0,0 +1,264 @@
> +# vi: ts=4 expandtab

this goes at the bottom.

> +#
> +#Author: Julien Castets 

please copy other headers.
If you want to put 'Author' thats fine, but then after that a blank line single 
line license info.

so, yours might look like:
# Author: Julien Castets 
#
# This file is part of cloud-init. See LICENSE file for license information.

> +#
> +#This program is free software: you can redistribute it and/or modify
> +#it under the terms of the GNU General Public License version 3, as
> +#published by the Free Software Foundation.
> +#
> +#This program is distributed in the hope that it will be useful,
> +#but WITHOUT ANY WARRANTY; without even the implied warranty of
> +#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +#GNU General Public License for more details.
> +#
> +#You should have received a copy of the GNU General Public License
> +#along with this program.  If not, see .
> +
> +import json
> +import socket
> +import time
> +
> +import requests
> +
> +# pylint fails to import the two modules below.
> +# pylint: disable=E0401
> +from requests.packages.urllib3.connection import HTTPConnection
> +from requests.packages.urllib3.poolmanager import PoolManager
> +
> +from cloudinit import log as logging
> +from cloudinit import sources
> +from cloudinit import url_helper
> +from cloudinit import util
> +
> +
> +LOG = logging.getLogger(__name__)
> +
> +BUILTIN_DS_CONFIG = {
> +'metadata_url': 'http://169.254.42.42/conf?format=json',
> +'userdata_url': 'http://169.254.42.42/user_data/cloud-init',
> +'vendordata_url': 'http://169.254.42.42/vendor_data/cloud-init'
> +}
> +
> +DEF_MD_RETRIES = 5
> +DEF_MD_TIMEOUT = 10
> +
> +
> +def on_scaleway(user_data_url, retries):
> +"""
> +Check if we are on Scaleway.
> +
> +The only backward-compatible way to check if you are on a Scaleway 
> instance
> +is to query the metadata API. For security reasons, the special endpoint
> +/user_data/ can only be accessed from a privileged TCP source port (ie.
> +below 1024), otherwise the API returns a HTTP/403.
> +
> +In other words, on Scaleway:
> +
> +#> curl http://169.254.42.42/user_data
> +[...] 403 error
> +
> +#> curl --local-port 1-1024 http://169.254.42.42/user_data
> +[...] 200 OK
> +
> +This function queries the endpoint /user_data/ and returns True if a
> +HTTP/403 is returned.
> +"""
> +tries = max(int(retries) + 1, 1)
> +for ntry in range(tries):
> +try:
> +code = requests.head(user_data_url).status_code
> +if code == 403:
> +return True
> +
> +# If we are rate limited or if there is a server error, we might
> +# not be talking to the Scaleway metadata API and we need to try
> +# again.
> +if code != 429 and code < 500:
> +return False
> +
> +# Couldn't query the API.
> +except (requests.exceptions.ConnectionError,
> +requests.exceptions.Timeout):
> +return False
> +
> +# Be nice, and wait a bit before retrying.
> +time.sleep(5)
> +
> +return False
> +
> +
> +class SourceAddressAdapter(requests.adapters.HTTPAdapter):
> +"""
> +Adapter for requests to choose the local address to bind to.
> +"""
> +
> +def __init__(self, source_address, **kwargs):
> +self.source_address = source_address
> +super(SourceAddressAdapter, self).__init__(**kwargs)
> +
> +def init_poolmanager(self, connections, maxsize, block=False):
> +socket_options = HTTPConnection.default_socket_options + [
> +(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
> +]
> +self.poolmanager = PoolManager(num_pools=connections,
> +   maxsize=maxsize,
> +   block=block,
> +   source_address=self.source_address,
> +

Re: [Cloud-init-dev] [Merge] ~jcastets/cloud-init:scaleway-datasource into cloud-init:master

2017-06-15 Thread Server Team CI bot 
Review: Approve continuous-integration

PASSED: Continuous integration, rev:adfca12743e2dd78ac1ad7f83253f41c06eb3795
https://jenkins.ubuntu.com/server/job/cloud-init-ci/536/
Executed test runs:
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/536
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/536
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/536
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/536
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/536

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/536/rebuild

-- 
https://code.launchpad.net/~jcastets/cloud-init/+git/cloud-init/+merge/325740
Your team cloud-init commiters is requested to review the proposed merge of 
~jcastets/cloud-init:scaleway-datasource into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~jcastets/cloud-init:scaleway-datasource into cloud-init:master

2017-06-15 Thread Julien Castets 
Julien Castets has proposed merging ~jcastets/cloud-init:scaleway-datasource 
into cloud-init:master.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~jcastets/cloud-init/+git/cloud-init/+merge/325740

Implements Scaleway datasource with user and vendor data.
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~jcastets/cloud-init:scaleway-datasource into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceScaleway.py b/cloudinit/sources/DataSourceScaleway.py
new file mode 100644
index 000..7ac1b1a
--- /dev/null
+++ b/cloudinit/sources/DataSourceScaleway.py
@@ -0,0 +1,264 @@
+# vi: ts=4 expandtab
+#
+#Author: Julien Castets 
+#
+#This program is free software: you can redistribute it and/or modify
+#it under the terms of the GNU General Public License version 3, as
+#published by the Free Software Foundation.
+#
+#This program is distributed in the hope that it will be useful,
+#but WITHOUT ANY WARRANTY; without even the implied warranty of
+#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#GNU General Public License for more details.
+#
+#You should have received a copy of the GNU General Public License
+#along with this program.  If not, see .
+
+import json
+import socket
+import time
+
+import requests
+
+# pylint fails to import the two modules below.
+# pylint: disable=E0401
+from requests.packages.urllib3.connection import HTTPConnection
+from requests.packages.urllib3.poolmanager import PoolManager
+
+from cloudinit import log as logging
+from cloudinit import sources
+from cloudinit import url_helper
+from cloudinit import util
+
+
+LOG = logging.getLogger(__name__)
+
+BUILTIN_DS_CONFIG = {
+'metadata_url': 'http://169.254.42.42/conf?format=json',
+'userdata_url': 'http://169.254.42.42/user_data/cloud-init',
+'vendordata_url': 'http://169.254.42.42/vendor_data/cloud-init'
+}
+
+DEF_MD_RETRIES = 5
+DEF_MD_TIMEOUT = 10
+
+
+def on_scaleway(user_data_url, retries):
+"""
+Check if we are on Scaleway.
+
+The only backward-compatible way to check if you are on a Scaleway instance
+is to query the metadata API. For security reasons, the special endpoint
+/user_data/ can only be accessed from a privileged TCP source port (ie.
+below 1024), otherwise the API returns a HTTP/403.
+
+In other words, on Scaleway:
+
+#> curl http://169.254.42.42/user_data
+[...] 403 error
+
+#> curl --local-port 1-1024 http://169.254.42.42/user_data
+[...] 200 OK
+
+This function queries the endpoint /user_data/ and returns True if a
+HTTP/403 is returned.
+"""
+tries = max(int(retries) + 1, 1)
+for ntry in range(tries):
+try:
+code = requests.head(user_data_url).status_code
+if code == 403:
+return True
+
+# If we are rate limited or if there is a server error, we might
+# not be talking to the Scaleway metadata API and we need to try
+# again.
+if code != 429 and code < 500:
+return False
+
+# Couldn't query the API.
+except (requests.exceptions.ConnectionError,
+requests.exceptions.Timeout):
+return False
+
+# Be nice, and wait a bit before retrying.
+time.sleep(5)
+
+return False
+
+
+class SourceAddressAdapter(requests.adapters.HTTPAdapter):
+"""
+Adapter for requests to choose the local address to bind to.
+"""
+
+def __init__(self, source_address, **kwargs):
+self.source_address = source_address
+super(SourceAddressAdapter, self).__init__(**kwargs)
+
+def init_poolmanager(self, connections, maxsize, block=False):
+socket_options = HTTPConnection.default_socket_options + [
+(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+]
+self.poolmanager = PoolManager(num_pools=connections,
+   maxsize=maxsize,
+   block=block,
+   source_address=self.source_address,
+   socket_options=socket_options)
+
+
+def _get_type_data(typedata_address, timeout, requests_session):
+"""
+Retrieve user data or vendor data.
+
+Scaleway userdata/vendordata API returns HTTP/404 if user/vendor data is
+not set.
+
+This function calls `url_helper.readurl` but instead of considering
+HTTP/404 as an error that requires a retry, it considers it as empty
+user/vendor data.
+
+Also, be aware the user data/vendor API requires the source port to be
+below 1024. If requests raises ConnectionError (EADDRINUSE), the caller
+should retry to call this function on an other port.
+"""
+try:
+resp = url_helper.readurl(
+typedata_address,

Re: [Cloud-init-dev] [Merge] ~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into cloud-init:master

2017-06-15 Thread Scott Moser 
this seems sane just looking at the diff, i'll look further today.

i think you'd opened a bug though... need to link to it.

-- 
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325713
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fix-ifdown-for-frbsd-on-Azure into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master

2017-06-15 Thread Scott Moser 
I'll look at this later, but i'm not sure we can just 
- 'platform': platform.platform(),
+ 'platform': platform.uname()[0],

without breaking other things.  I'll try this out today though and will find a 
freebsd to test on.


-- 
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325718
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Cloud-init-dev] [Merge] ~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master

2017-06-15 Thread Server Team CI bot 
Review: Approve continuous-integration

PASSED: Continuous integration, rev:3ac3e560626348e3efd88b30341e3e433b961a9e
https://jenkins.ubuntu.com/server/job/cloud-init-ci/535/
Executed test runs:
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-amd64/535
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/535
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/535
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/535
SUCCESS: 
https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/535

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/535/rebuild

-- 
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325718
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp

[Cloud-init-dev] [Merge] ~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master

2017-06-15 Thread Hongjiang Zhang 
The proposal to merge ~redriver/cloud-init:fail_to_install_on_frbsd into 
cloud-init:master has been updated.

Commit Message changed to:

tools/build-on-freebsd was blocked

fail to run tools/build-on-freebsd on FreeBSD platform.

LP: #1698077

For more details, see:
https://code.launchpad.net/~redriver/cloud-init/+git/cloud-init/+merge/325718
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~redriver/cloud-init:fail_to_install_on_frbsd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp