Wesley Wiedenmeier has proposed merging ~wesley-wiedenmeier/cloud-init:integration-testing-distro-features into cloud-init:master.
Requested reviews: cloud init development team (cloud-init-dev) For more details, see: https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/321029 Integration Testing: Improvments to testing on alternate distros - Allow images to override user data settings - image config option 'user_data_overrides' accepts dict of attributes to update a testcase's user data with before launching an image - workaround for (LP #1575779) - should only be used if this is the only way to get an image working - Add support for distro feature flags - add framework for feature flags to release config with feature groups and overrides allowed in any release conf override level - add support for feature flags in platform and config handling - during collect, skip testcases that require features not supported by the image with a warning message - Add required features to testcase configs - skip testcases with distro compatibility issues - allow test suite to be run in full on other distros - many testcases with compatibility issues can be updated to allow them to work across distros - Updated documentation - updated documentation for testsuite config - explain how config is merged - describe new image config format - explain how to configure an image's system_ready_script - add documentation on how feature flags work - add documentation on how error handling works - add documentation on setup_image options -- Your team cloud init development team is requested to review the proposed merge of ~wesley-wiedenmeier/cloud-init:integration-testing-distro-features into cloud-init:master.
diff --git a/doc/rtd/topics/tests.rst b/doc/rtd/topics/tests.rst index 0663811..de71a9e 100644 --- a/doc/rtd/topics/tests.rst +++ b/doc/rtd/topics/tests.rst @@ -47,7 +47,7 @@ The test configuration is a YAML file such as *ntp_server.yaml* below: cat /etc/ntp.conf | grep '^server' -There are two keys, 1 required and 1 optional, in the YAML file: +There are several keys, 1 required and some optional, in the YAML file: 1. The required key is ``cloud_config``. This should be a string of valid YAML that is exactly what would normally be placed in a cloud-config file, @@ -61,6 +61,11 @@ There are two keys, 1 required and 1 optional, in the YAML file: reported. The name of the sub-key is important. The sub-key is used by the verification script to recall the output of the commands ran. +3. The ``required_features`` key may be used to specify a list of features + flags that an image must have to be able to run the testcase. For example, + if a testcase relies on an image supporting apt, then the config for the + testcase should include ``required_features: [ apt ]``. + Default Collect Scripts ----------------------- @@ -154,6 +159,7 @@ Development Checklist * Optionally, commands to capture additional output * Valid YAML * Placed in the appropriate sub-folder in the configs directory + * Any image features required for the test are specified * Verification File * Named 'your_test_here.py' * Valid unit tests validating output collected @@ -252,6 +258,242 @@ configuration users can run the integration tests via tox: Users need to invoke the citest enviornment and then pass any additional arguments. +Setup Image +----------- +The ``run`` and ``collect`` commands have many options to setup the image +before running tests in addition to installing a deb in the target. Any +combination of the following can be used: + +* ``--deb``: install a deb into the image +* ``--rpm``: install a rpm into the image +* ``--repo``: enable a repository and update cloud-init afterwards +* ``--ppa``: enable a ppa and update cloud-init afterwards +* ``--upgrade``: upgrade cloud-init from repos +* ``--upgrade-full``: run a full system upgrade +* ``--script``: execute a script in the image. this can perform any setup + required that is not covered by the other options + +Configuring the Test Suite +========================== + +Most of the behavior of the test suite is configurable through several yaml +files. These control the behavior of the test suite's platforms, images, and +tests. The main config files for platforms, images and testcases are +``platforms.yaml``, ``releases.yaml`` and ``testcases.yaml``. + +Config handling +--------------- +All configurable parts of the test suite use a defaults + overrides system for +managing config entries. All base config items are dictionaries. + +Merging is done on a key by key basis, with all keys in the default and +overrides represented in the final result. If a key exists both in +the defaults and the overrides, then behavior depends on the type of data the +key refers to. If it is atomic data or a list, then the overrides will replace +the default. If the data is a dictionary then the value will be the result of +merging that dictionary from the default config and that dictionary from the +overrides. + +Merging is done using the function ``tests.cloud_tests.config.merge_config``, +which can be examined for more detail on config merging behavior. + +The following demonstrates merge behavior: + +.. code-block:: yaml + + defaults: + list_item: + - list_entry_1 + - list_entry_2 + int_item_1: 123 + int_item_2: 234 + dict_item: + subkey_1: 1 + subkey_2: 2 + subkey_dict: + subsubkey_1: a + subsubkey_2: b + + overrides: + list_item: + - overridden_list_entry + int_item_1: 0 + dict_item: + subkey_2: false + subkey_dict: + subsubkey_2: 'new value' + + result: + list_item: + - overridden_list_entry + int_item_1: 0 + int_item_2: 234 + dict_item: + subkey_1: 1 + subkey_2: false + subkey_dict: + subsubkey_1: a + subsubkey_2: 'new value' + + +Image Config Structure +---------------------- +Image configuration is handled in ``releases.yaml``. The image configuration +controls how platforms locate and acquire images, how the platforms should +interact with the images, how platforms should detect when an image has fully +booted, any options that are required to set the image up, and features that +the image supports. + +Since settings for locating an image and interacting with it differ from +platform to platform, there are 4 levels of settings available for images on +top of the default image settings. The structure of the image config file is: + +.. code-block:: yaml + + default_release_config: + default: + ... + <platform>: + ... + <platform>: + ... + + releases: + <release name>: + <default>: + ... + <platform>: + ... + <platform>: + ... + + +The base config is created from the overall defaults and the overrides for the +platform. The overrides are created from the default config for the image and +the platform specific overrides for the image. + +Image Config for System Boot +---------------------------- +The test suite must be able to test if a system has fully booted and if +cloud-init has finished running, so that running collect scripts does not race +against the target image booting. This is done using the +``system_ready_script`` and ``cloud_init_ready_script`` image config keys. + +Each of these keys accepts a small bash test statement as a string that must +return 0 or 1. Since this test statement will be added into a larger bash +statement it must be a single statement using the ``[`` test syntax. + +The default image config provides a system ready script that works for any +systemd based image. If the iamge is not systmed based, then a different test +statement must be provided. The default config also provides a test for whether +or not cloud-init has finished which checks for the file +``/run/cloud-init/result.json``. This should be sufficient for most systems, as +writing to this file is one of the last things cloud-init does. + +The setting ``boot_timeout`` controls how long, in seconds, the platform should +wait for an image to boot. If the system ready script has not indicated that +the system is fully booted within this time an error will be raised. + +Image Config Feature Flags +-------------------------- +Not all testcases can work on all images due to features the testcase requires +not being present on that image. If a testcase requires features in an image +that are not likely to be present across all distros and platforms that the +test suite supports, then the test can be skipped everywhere it is not +supported. + +This is done through feature flags, which are names for features supported on +some images but not all that may be required by testcases. Configuration for +feature flags is provided in ``releases.yaml`` under the ``features`` top level +key. The features config includes a list of all currently defined feature flags +and their meanings, and a list of feature groups. + +Feature groups are groups of features that many images have in common. For +example, the ``ubuntu_specific`` feature group includes features that should be +present across most ubuntu releases, but may or may not be for other distros. +Feature groups are specified for an image as a list under the key +``feature_groups``. + +An image's feature flags are derived from the features groups that that image +has and any feature overrides provided. Feature overrides can be specified +under the ``features`` key which accepts a dictionary of +``{<feature_name>: true/false}`` mappings. If a feature is omitted from an +image's feature flags or set to false in the overrides then the test suite will +skip any tests that require that feature when using that image. + +Image Config Setup Overrides +---------------------------- +If an image requires some of the options for image setup to be used, then it +may specify overrides for the command line arguments passed into setup image. +These may be specified as a dictionary under the ``setup_overrides`` key. When +an image is set up, the arguments that control how it is set up will be the +arguments from the command line, with any entries in ``setup_overrides`` used +to override these arguments. + +For example, images that do not come with cloud-init already installed should +have ``setup_overrides: {upgrade: true}`` specified so that in the event that +no additional setup options are given, cloud-init will be installed from the +image's repos before running tests. Note that if other options such as +``--deb`` are passed in on the command line, these will still work as expected, +since apt's policy for cloud-init would prefer the locally installed deb over +an older version from the repos. + +Image Config Platform Specific Options +-------------------------------------- +There are many platform specific options in image configuration that allow +platforms to locate images and that control additional setup that the platform +may have to do to make the image useable. For information on how these work, +please consult the documentation for that platform in the integration testing +suite and the ``releases.yaml`` file for examples. + +Error Handling Behavior +======================= + +The test suite makes an attempt to run as many tests as possible even in the +event of some failing so that automated runs collect as much data as possible. +In the event that something goes wrong while setting up for or running a test, +the test suite will attempt to continue running any tests which have not been +effected by the error. + +For example, if the test suite was told to run tests on one platform for two +releases and an error occured setting up the first image, all tests for that +image would be skipped, and the test suite would continue to set up the second +image and run tests on it. Or, if the system does not start properly for one +testcase out of many to run on that image, that testcase will be skipped and +the next one will be run. + +Note that if any errors at all occur, the test suite will record the failure +and where it occurred in the result data and write it out to the specified +result file. + +Exit Codes +---------- +The test suite counts how many errors occur throughout a run. The exit code +after a run is the number of errors that occured. If the exit code is non-zero +than something is wrong either with the test suite, the configuration for an +image, a testcase, or cloud-init itself. + +Note that the exit code does not always direclty correspond to the number +of failed testcases, since in some cases, a single error during image setup +can mean that several testcases are not run. If run is used, then the exit code +will be the sum of the number of errors in the collect and verify stages. + +Result Data +----------- +The test suite generates result data that includes how long each stage of the +test suite took and which parts were and were not successful. This data is +dumped to the log after the collect and verify stages, and may also be written +out in yaml format to a file. If part of the setup failed, the traceback for +the failure and the error message will be included in the result file. If a +test verifier finds a problem with the collected data from a test run, the +class, test function and test will be recorded in the result data. + +Data Dir +-------- +When using run, the collected data is written into a temporary directory. In +the even that all tests pass, this directory is deleted. In the even that a +test fails or an error occurs, this data will be left in place, and a message +will be written to the log giving the location of the data. Architecture ============ diff --git a/tests/cloud_tests/args.py b/tests/cloud_tests/args.py index b68cc98..a96714c 100644 --- a/tests/cloud_tests/args.py +++ b/tests/cloud_tests/args.py @@ -9,11 +9,11 @@ ARG_SETS = { 'COLLECT': ( (('-p', '--platform'), {'help': 'platform(s) to run tests on', 'metavar': 'PLATFORM', - 'action': 'append', 'choices': config.list_enabled_platforms(), + 'action': 'append', 'choices': config.ENABLED_PLATFORMS, 'default': []}), (('-n', '--os-name'), {'help': 'the name(s) of the OS(s) to test', 'metavar': 'NAME', - 'action': 'append', 'choices': config.list_enabled_distros(), + 'action': 'append', 'choices': config.ENABLED_DISTROS, 'default': []}), (('-t', '--test-config'), {'help': 'test config file(s) to use', 'metavar': 'FILE', @@ -61,8 +61,12 @@ ARG_SETS = { {'help': 'ppa to enable (implies -u)', 'metavar': 'NAME', 'action': 'store'}), (('-u', '--upgrade'), - {'help': 'upgrade before starting tests', 'action': 'store_true', - 'default': False}),), + {'help': 'upgrade or install cloud-init from repo', + 'action': 'store_true', 'default': False}), + (('--upgrade-full',), + {'help': 'do full system upgrade from repo (implies -u)', + 'action': 'store_true', 'default': False}),), + } SUBCMDS = { @@ -121,15 +125,15 @@ def normalize_collect_args(args): """ # platform should default to all supported if len(args.platform) == 0: - args.platform = config.list_enabled_platforms() + args.platform = config.ENABLED_PLATFORMS args.platform = util.sorted_unique(args.platform) # os name should default to all enabled # if os name is provided ensure that all provided are supported if len(args.os_name) == 0: - args.os_name = config.list_enabled_distros() + args.os_name = config.ENABLED_DISTROS else: - supported = config.list_enabled_distros() + supported = config.ENABLED_DISTROS invalid = [os_name for os_name in args.os_name if os_name not in supported] if len(invalid) != 0: diff --git a/tests/cloud_tests/collect.py b/tests/cloud_tests/collect.py index 68b47d7..3bb46cd 100644 --- a/tests/cloud_tests/collect.py +++ b/tests/cloud_tests/collect.py @@ -18,8 +18,10 @@ def collect_script(instance, base_dir, script, script_name): return_value: None, may raise errors """ LOG.debug('running collect script: %s', script_name) - util.write_file(os.path.join(base_dir, script_name), - instance.run_script(script)) + (out, err, exit) = instance.run_script( + script, rcs=range(0, 256), + description='collect: {}'.format(script_name)) + util.write_file(os.path.join(base_dir, script_name), out) def collect_test_data(args, snapshot, os_name, test_name): @@ -39,15 +41,27 @@ def collect_test_data(args, snapshot, os_name, test_name): test_scripts = test_config['collect_scripts'] test_output_dir = os.sep.join( (args.data_dir, snapshot.platform_name, os_name, test_name)) - boot_timeout = (test_config.get('boot_timeout') - if isinstance(test_config.get('boot_timeout'), int) else - snapshot.config.get('timeout')) # if test is not enabled, skip and return 0 failures if not test_config.get('enabled', False): LOG.warn('test config %s is not enabled, skipping', test_name) return ({}, 0) + # if testcase requires a feature flag that the image does not support, + # skip the testcase with a warning + req_features = test_config.get('required_features', []) + if any(feature not in snapshot.features for feature in req_features): + LOG.warn('test config %s requires features not supported by image, ' + 'skipping.\nrequired features: %s\nsupported features: %s', + test_name, req_features, snapshot.features) + return ({}, 0) + + # if there are user data overrides required for this test case, apply them + overrides = snapshot.config.get('user_data_overrides', {}) + if overrides: + LOG.debug('updating user data for collect with: %s', overrides) + user_data = util.update_user_data(user_data, overrides) + # create test instance component = PlatformComponent( partial(instances.get_instance, snapshot, user_data, @@ -56,7 +70,7 @@ def collect_test_data(args, snapshot, os_name, test_name): LOG.info('collecting test data for test: %s', test_name) with component as instance: start_call = partial(run_single, 'boot instance', partial( - instance.start, wait=True, wait_time=boot_timeout)) + instance.start, wait=True, wait_for_cloud_init=True)) collect_calls = [partial(run_single, 'script {}'.format(script_name), partial(collect_script, instance, test_output_dir, script, script_name)) @@ -100,10 +114,8 @@ def collect_image(args, platform, os_name): """ res = ({}, 1) - os_config = config.load_os_config(os_name) - if not os_config.get('enabled'): - raise ValueError('OS {} not enabled'.format(os_name)) - + os_config = config.load_os_config( + platform.platform_name, os_name, require_enabled=True) component = PlatformComponent( partial(images.get_image, platform, os_config)) @@ -126,10 +138,8 @@ def collect_platform(args, platform_name): """ res = ({}, 1) - platform_config = config.load_platform_config(platform_name) - if not platform_config.get('enabled'): - raise ValueError('Platform {} not enabled'.format(platform_name)) - + platform_config = config.load_platform_config( + platform_name, require_enabled=True) component = PlatformComponent( partial(platforms.get_platform, platform_name, platform_config)) diff --git a/tests/cloud_tests/config.py b/tests/cloud_tests/config.py index f3a13c9..e490e39 100644 --- a/tests/cloud_tests/config.py +++ b/tests/cloud_tests/config.py @@ -14,6 +14,20 @@ RELEASES_CONF = os.path.join(BASE_DIR, 'releases.yaml') TESTCASE_CONF = os.path.join(BASE_DIR, 'testcases.yaml') +def get(base, key): + """ + get config entry 'key' from base, ensuring is dictionary + """ + return base[key] if key in base and base[key] is not None else {} + + +def enabled(config): + """ + test if config item is enabled + """ + return isinstance(config, dict) and config.get('enabled', False) + + def path_to_name(path): """ convert abs or rel path to test config to path under configs/ @@ -61,22 +75,57 @@ def merge_config(base, override): return res -def load_platform_config(platform): +def merge_feature_groups(feature_conf, feature_groups, overrides): + """ + combine feature groups and overrides to construct a supported feature list + feature_conf: feature config from releases.yaml + feature_groups: feature groups the release is a member of + overrides: overrides specified by the release's config + return_value: dict of {feature: true/false} settings + """ + res = dict().fromkeys(feature_conf['all']) + for group in feature_groups: + res.update(feature_conf['groups'][group]) + res.update(overrides) + return res + + +def load_platform_config(platform_name, require_enabled=False): """ load configuration for platform + platform_name: name of platform to retrieve config for + require_enabled: if true, raise error if 'enabled' not True + return_value: config dict """ main_conf = c_util.read_conf(PLATFORM_CONF) - return merge_config(main_conf.get('default_platform_config'), - main_conf.get('platforms')[platform]) + conf = merge_config(main_conf['default_platform_config'], + main_conf['platforms'][platform_name]) + if require_enabled and not enabled(conf): + raise ValueError('Platform is not enabled') + return conf -def load_os_config(os_name): +def load_os_config(platform_name, os_name, require_enabled=False): """ load configuration for os + platform_name: platform name to load os config for + os_name: name of os to retrieve config for + require_enabled: if true, raise error if 'enabled' not True + return_value: config dict """ main_conf = c_util.read_conf(RELEASES_CONF) - return merge_config(main_conf.get('default_release_config'), - main_conf.get('releases')[os_name]) + default = main_conf['default_release_config'] + image = main_conf['releases'][os_name] + conf = merge_config(merge_config(get(default, 'default'), + get(default, platform_name)), + merge_config(get(image, 'default'), + get(image, platform_name))) + feature_conf = main_conf['features'] + conf['features'] = merge_feature_groups( + feature_conf, conf.get('feature_groups', []), conf.get('features', {})) + if require_enabled and not enabled(conf): + raise ValueError('OS is not enabled') + return conf def load_test_config(path): @@ -91,16 +140,22 @@ def list_enabled_platforms(): """ list all platforms enabled for testing """ - platforms = c_util.read_conf(PLATFORM_CONF).get('platforms') - return [k for k, v in platforms.items() if v.get('enabled')] + platforms = get(c_util.read_conf(PLATFORM_CONF), 'platforms') + return [k for k, v in platforms.items() if enabled(v)] -def list_enabled_distros(): +def list_enabled_distros(platforms): """ - list all distros enabled for testing + list all distros enabled for testing on specified platforms """ - releases = c_util.read_conf(RELEASES_CONF).get('releases') - return [k for k, v in releases.items() if v.get('enabled')] + + def platform_has_enabled(config): + return any(enabled(merge_config(get(config, 'default'), + get(config, platform))) + for platform in platforms) + + releases = get(c_util.read_conf(RELEASES_CONF), 'releases') + return [k for k, v in releases.items() if platform_has_enabled(v)] def list_test_configs(): @@ -110,4 +165,8 @@ def list_test_configs(): return [os.path.abspath(f) for f in glob.glob(os.sep.join((TEST_CONF_DIR, '*', '*.yaml')))] + +ENABLED_PLATFORMS = list_enabled_platforms() +ENABLED_DISTROS = list_enabled_distros(ENABLED_PLATFORMS) + # vi: ts=4 expandtab diff --git a/tests/cloud_tests/configs/bugs/lp1628337.yaml b/tests/cloud_tests/configs/bugs/lp1628337.yaml index 1d6bf48..e39b3cd 100644 --- a/tests/cloud_tests/configs/bugs/lp1628337.yaml +++ b/tests/cloud_tests/configs/bugs/lp1628337.yaml @@ -1,6 +1,9 @@ # # LP Bug 1628337: cloud-init tries to install NTP before even configuring the archives # +required_features: + - apt + - lsb_release cloud_config: | #cloud-config ntp: diff --git a/tests/cloud_tests/configs/examples/add_apt_repositories.yaml b/tests/cloud_tests/configs/examples/add_apt_repositories.yaml index b896435..4b8575f 100644 --- a/tests/cloud_tests/configs/examples/add_apt_repositories.yaml +++ b/tests/cloud_tests/configs/examples/add_apt_repositories.yaml @@ -4,6 +4,8 @@ # 2016-11-17: Disabled as covered by module based tests # enabled: False +required_features: + - apt cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_conf.yaml b/tests/cloud_tests/configs/modules/apt_configure_conf.yaml index 163ae3f..de45300 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_conf.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_conf.yaml @@ -1,6 +1,8 @@ # # Provide a configuration for APT # +required_features: + - apt cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml b/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml index 73e4a53..9880067 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_disable_suites.yaml @@ -1,6 +1,9 @@ # # Disables everything in sources.list # +required_features: + - apt + - lsb_release cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_primary.yaml b/tests/cloud_tests/configs/modules/apt_configure_primary.yaml index 2ec30ca..41bcf2f 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_primary.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_primary.yaml @@ -1,6 +1,9 @@ # # Setup a custome primary sources.list # +required_features: + - apt + - apt_src_cont cloud_config: | #cloud-config apt: @@ -16,4 +19,8 @@ collect_scripts: #!/bin/bash grep -v '^#' /etc/apt/sources.list | sed '/^\s*$/d' | grep -c gtlib.gatech.edu + sources.list: | + #!/bin/bash + cat /etc/apt/sources.list + # vi: ts=4 expandtab diff --git a/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml b/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml index e737130..be6c6f8 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_proxy.yaml @@ -1,6 +1,8 @@ # # Set apt proxy # +required_features: + - apt cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_security.yaml b/tests/cloud_tests/configs/modules/apt_configure_security.yaml index f6a2c82..83dd51d 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_security.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_security.yaml @@ -1,6 +1,9 @@ # # Add security to sources.list # +required_features: + - apt + - ubuntu_repos cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml index e7568a6..bde9398 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_sources_key.yaml @@ -1,6 +1,9 @@ # # Add a sources.list entry with a given key (Debian Jessie) # +required_features: + - apt + - lsb_release cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml index 1a4a238..11da61e 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_sources_keyserver.yaml @@ -1,6 +1,9 @@ # # Add a sources.list entry with a key from a keyserver # +required_features: + - apt + - lsb_release cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml index 057fc72..143cb08 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_sources_list.yaml @@ -1,6 +1,9 @@ # # Generate a sources.list # +required_features: + - apt + - lsb_release cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml b/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml index dee9dc7..9847588 100644 --- a/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml +++ b/tests/cloud_tests/configs/modules/apt_configure_sources_ppa.yaml @@ -1,6 +1,9 @@ # # Add a PPA to source.list # +required_features: + - apt + - ppa cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml b/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml index 5fa0cee..bd9b5d0 100644 --- a/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml +++ b/tests/cloud_tests/configs/modules/apt_pipelining_disable.yaml @@ -1,6 +1,8 @@ # # Disable apt pipelining value # +required_features: + - apt cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml b/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml index 87d183e..cbed3ba 100644 --- a/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml +++ b/tests/cloud_tests/configs/modules/apt_pipelining_os.yaml @@ -1,6 +1,8 @@ # # Set apt pipelining value to OS # +required_features: + - apt cloud_config: | #cloud-config apt: diff --git a/tests/cloud_tests/configs/modules/byobu.yaml b/tests/cloud_tests/configs/modules/byobu.yaml index fd648c7..a9aa1f3 100644 --- a/tests/cloud_tests/configs/modules/byobu.yaml +++ b/tests/cloud_tests/configs/modules/byobu.yaml @@ -1,6 +1,8 @@ # # Install and enable byobu system wide and default user # +required_features: + - byobu cloud_config: | #cloud-config byobu_by_default: enable diff --git a/tests/cloud_tests/configs/modules/keys_to_console.yaml b/tests/cloud_tests/configs/modules/keys_to_console.yaml index a90e42c..5d86e73 100644 --- a/tests/cloud_tests/configs/modules/keys_to_console.yaml +++ b/tests/cloud_tests/configs/modules/keys_to_console.yaml @@ -1,6 +1,8 @@ # # Hide printing of ssh key and fingerprints for specific keys # +required_features: + - syslog cloud_config: | #cloud-config ssh_fp_console_blacklist: [ssh-dss, ssh-dsa, ecdsa-sha2-nistp256] diff --git a/tests/cloud_tests/configs/modules/landscape.yaml b/tests/cloud_tests/configs/modules/landscape.yaml index e6f4955..ed2c37c 100644 --- a/tests/cloud_tests/configs/modules/landscape.yaml +++ b/tests/cloud_tests/configs/modules/landscape.yaml @@ -4,6 +4,8 @@ # 2016-11-17: Disabled due to this not working # enabled: false +required_features: + - landscape cloud_config: | #cloud-conifg landscape: diff --git a/tests/cloud_tests/configs/modules/locale.yaml b/tests/cloud_tests/configs/modules/locale.yaml index af5ad63..e3220af 100644 --- a/tests/cloud_tests/configs/modules/locale.yaml +++ b/tests/cloud_tests/configs/modules/locale.yaml @@ -1,6 +1,8 @@ # # Set locale to non-default option and verify # +required_features: + - engb_locale cloud_config: | #cloud-config locale: en_GB.UTF-8 diff --git a/tests/cloud_tests/configs/modules/lxd_bridge.yaml b/tests/cloud_tests/configs/modules/lxd_bridge.yaml index 568bb70..e6b7e76 100644 --- a/tests/cloud_tests/configs/modules/lxd_bridge.yaml +++ b/tests/cloud_tests/configs/modules/lxd_bridge.yaml @@ -1,6 +1,8 @@ # # LXD configured with directory backend and IPv4 bridge # +required_features: + - lxd cloud_config: | #cloud-config lxd: diff --git a/tests/cloud_tests/configs/modules/lxd_dir.yaml b/tests/cloud_tests/configs/modules/lxd_dir.yaml index 99b9219..f93a3fa 100644 --- a/tests/cloud_tests/configs/modules/lxd_dir.yaml +++ b/tests/cloud_tests/configs/modules/lxd_dir.yaml @@ -1,6 +1,8 @@ # # LXD configured with directory backend # +required_features: + - lxd cloud_config: | #cloud-config lxd: diff --git a/tests/cloud_tests/configs/modules/ntp.yaml b/tests/cloud_tests/configs/modules/ntp.yaml index d094157..babf84f 100644 --- a/tests/cloud_tests/configs/modules/ntp.yaml +++ b/tests/cloud_tests/configs/modules/ntp.yaml @@ -1,6 +1,11 @@ # # Emtpy NTP config to setup using defaults # +# NOTE: this should not require apt feature, use 'which' rather than 'dpkg -l' +# NOTE: the verifier should check for any ntp server not 'ubuntu.pool.ntp.org' +required_features: + - apt + - ubuntu_ntp cloud_config: | #cloud-config ntp: diff --git a/tests/cloud_tests/configs/modules/ntp_pools.yaml b/tests/cloud_tests/configs/modules/ntp_pools.yaml index bd0ac29..e6857ef 100644 --- a/tests/cloud_tests/configs/modules/ntp_pools.yaml +++ b/tests/cloud_tests/configs/modules/ntp_pools.yaml @@ -1,6 +1,13 @@ # # NTP config using specific pools # +# NOTE: this should not require apt feature, use 'which' rather than 'dpkg -l' +# NOTE: lsb_release listed here because with recent cloud-init deb with +# (LP: 1628337) resolved, cloud-init will attempt to configure archives. +# this fails without lsb_release as UNAVAILABLE is used for $RELEASE +required_features: + - apt + - lsb_release cloud_config: | #cloud-config ntp: diff --git a/tests/cloud_tests/configs/modules/ntp_servers.yaml b/tests/cloud_tests/configs/modules/ntp_servers.yaml index 934b9c5..8156001 100644 --- a/tests/cloud_tests/configs/modules/ntp_servers.yaml +++ b/tests/cloud_tests/configs/modules/ntp_servers.yaml @@ -1,6 +1,9 @@ # # NTP config using specific servers # +# NOTE: this should not require apt feature, use 'which' rather than 'dpkg -l' +required_features: + - apt cloud_config: | #cloud-config ntp: diff --git a/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml b/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml index d027d54..42a823b 100644 --- a/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml +++ b/tests/cloud_tests/configs/modules/package_update_upgrade_install.yaml @@ -1,6 +1,13 @@ # # Update/upgrade via apt and then install a pair of packages # +# NOTE: this should not require apt feature, use 'which' rather than 'dpkg -l' +# NOTE: the testcase for this looks for the command in history.log as +# /usr/bin/apt-get..., which is not how it always appears. it should +# instead look for just apt-get... +required_features: + - apt + - apt_hist_fmt cloud_config: | #cloud-config packages: diff --git a/tests/cloud_tests/configs/modules/set_hostname.yaml b/tests/cloud_tests/configs/modules/set_hostname.yaml index 5aae150..c96344c 100644 --- a/tests/cloud_tests/configs/modules/set_hostname.yaml +++ b/tests/cloud_tests/configs/modules/set_hostname.yaml @@ -1,6 +1,8 @@ # # Set the hostname and update /etc/hosts # +required_features: + - hostname cloud_config: | #cloud-config hostname: myhostname diff --git a/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml b/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml index 0014c19..daf7593 100644 --- a/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml +++ b/tests/cloud_tests/configs/modules/set_hostname_fqdn.yaml @@ -1,6 +1,8 @@ # # Set the hostname and update /etc/hosts # +required_features: + - hostname cloud_config: | #cloud-config manage_etc_hosts: true diff --git a/tests/cloud_tests/configs/modules/set_password.yaml b/tests/cloud_tests/configs/modules/set_password.yaml index 8fa46d9..04d7c58 100644 --- a/tests/cloud_tests/configs/modules/set_password.yaml +++ b/tests/cloud_tests/configs/modules/set_password.yaml @@ -1,6 +1,8 @@ # # Set password of default user # +required_features: + - ubuntu_user cloud_config: | #cloud-config password: password diff --git a/tests/cloud_tests/configs/modules/set_password_expire.yaml b/tests/cloud_tests/configs/modules/set_password_expire.yaml index 926731f..789604b 100644 --- a/tests/cloud_tests/configs/modules/set_password_expire.yaml +++ b/tests/cloud_tests/configs/modules/set_password_expire.yaml @@ -1,6 +1,8 @@ # # Expire password for all users # +required_features: + - sshd cloud_config: | #cloud-config chpasswd: { expire: True } diff --git a/tests/cloud_tests/configs/modules/snappy.yaml b/tests/cloud_tests/configs/modules/snappy.yaml index 923bfe1..030b790 100644 --- a/tests/cloud_tests/configs/modules/snappy.yaml +++ b/tests/cloud_tests/configs/modules/snappy.yaml @@ -1,6 +1,8 @@ # # Install snappy # +required_features: + - snap cloud_config: | #cloud-config snappy: diff --git a/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml index 33943bd..746653e 100644 --- a/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml +++ b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_disable.yaml @@ -1,6 +1,8 @@ # # Disable fingerprint printing # +required_features: + - syslog cloud_config: | #cloud-config ssh_genkeytypes: [] diff --git a/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml index 4c97077..bb401e7 100644 --- a/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml +++ b/tests/cloud_tests/configs/modules/ssh_auth_key_fingerprints_enable.yaml @@ -1,6 +1,8 @@ # # Print auth keys with different hash than md5 # +required_features: + - syslog cloud_config: | #cloud-config ssh_genkeytypes: diff --git a/tests/cloud_tests/configs/modules/ssh_import_id.yaml b/tests/cloud_tests/configs/modules/ssh_import_id.yaml index 6e5a163..b62d3f6 100644 --- a/tests/cloud_tests/configs/modules/ssh_import_id.yaml +++ b/tests/cloud_tests/configs/modules/ssh_import_id.yaml @@ -1,6 +1,9 @@ # # Import a user's ssh key via gh or lp # +required_features: + - ubuntu_user + - sudo cloud_config: | #cloud-config ssh_import_id: diff --git a/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml b/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml index 637d783..659fd93 100644 --- a/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml +++ b/tests/cloud_tests/configs/modules/ssh_keys_generate.yaml @@ -1,6 +1,8 @@ # # SSH keys generated using cloud-init # +required_features: + - ubuntu_user cloud_config: | #cloud-config ssh_genkeytypes: diff --git a/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml b/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml index 25df645..5ceb362 100644 --- a/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml +++ b/tests/cloud_tests/configs/modules/ssh_keys_provided.yaml @@ -2,6 +2,9 @@ # SSH keys provided via cloud config # enabled: False +required_features: + - ubuntu_user + - sudo cloud_config: | #cloud-config disable_root: false diff --git a/tests/cloud_tests/configs/modules/timezone.yaml b/tests/cloud_tests/configs/modules/timezone.yaml index 8c96ed4..5112aa9 100644 --- a/tests/cloud_tests/configs/modules/timezone.yaml +++ b/tests/cloud_tests/configs/modules/timezone.yaml @@ -1,6 +1,8 @@ # # Set system timezone # +required_features: + - daylight_time cloud_config: | #cloud-config timezone: US/Aleutian diff --git a/tests/cloud_tests/configs/modules/user_groups.yaml b/tests/cloud_tests/configs/modules/user_groups.yaml index 9265595..71cc9da 100644 --- a/tests/cloud_tests/configs/modules/user_groups.yaml +++ b/tests/cloud_tests/configs/modules/user_groups.yaml @@ -1,6 +1,8 @@ # # Create groups and users with various options # +required_features: + - ubuntu_user cloud_config: | #cloud-config # Add groups to the system diff --git a/tests/cloud_tests/images/base.py b/tests/cloud_tests/images/base.py index 394b11f..1f604cf 100644 --- a/tests/cloud_tests/images/base.py +++ b/tests/cloud_tests/images/base.py @@ -7,13 +7,14 @@ class Image(object): """ platform_name = None - def __init__(self, name, config, platform): + def __init__(self, platform, config): """ - setup + Set up image + platform: platform object + config: image configuration """ - self.name = name - self.config = config self.platform = platform + self.config = config def __str__(self): """ @@ -28,10 +29,24 @@ class Image(object): """ raise NotImplementedError - # FIXME: instead of having execute and push_file and other instance methods - # here which pass through to a hidden instance, it might be better - # to expose an instance that the image can be modified through - def execute(self, command, stdin=None, stdout=None, stderr=None, env={}): + @property + def features(self): + """ + feature flags supported by this image + return_value: list of feature names + """ + return [k for k, v in self.config.get('features', {}).items() if v] + + @property + def setup_overrides(self): + """ + setup options that need to be overridden for the image + return_value: dictionary to update args with + """ + # NOTE: more sophisticated options may be requied at some point + return self.config.get('setup_overrides', {}) + + def execute(self, *args, **kwargs): """ execute command in image, modifying image """ @@ -43,7 +58,7 @@ class Image(object): """ raise NotImplementedError - def run_script(self, script): + def run_script(self, *args, **kwargs): """ run script in image, modifying image return_value: script output diff --git a/tests/cloud_tests/images/lxd.py b/tests/cloud_tests/images/lxd.py index 7a41614..4d66fbc 100644 --- a/tests/cloud_tests/images/lxd.py +++ b/tests/cloud_tests/images/lxd.py @@ -2,6 +2,10 @@ from tests.cloud_tests.images import base from tests.cloud_tests.snapshots import lxd as lxd_snapshot +from tests.cloud_tests import util + +import os +import shutil class LXDImage(base.Image): @@ -10,27 +14,44 @@ class LXDImage(base.Image): """ platform_name = "lxd" - def __init__(self, name, config, platform, pylxd_image): + def __init__(self, platform, config, pylxd_image): """ - setup + Set up image + platform: platform object + config: image configuration """ - self.platform = platform - self._pylxd_image = pylxd_image + self.modified = False self._instance = None - super(LXDImage, self).__init__(name, config, platform) + self._pylxd_image = None + self.pylxd_image = pylxd_image + super(LXDImage, self).__init__(platform, config) @property def pylxd_image(self): - self._pylxd_image.sync() + if self._pylxd_image: + self._pylxd_image.sync() return self._pylxd_image + @pylxd_image.setter + def pylxd_image(self, pylxd_image): + if self._instance: + self._instance.destroy() + self._instance = None + if (self._pylxd_image and + (self._pylxd_image is not pylxd_image) and + (not self.config.get('cache_base_image') or self.modified)): + self._pylxd_image.delete(wait=True) + self.modified = False + self._pylxd_image = pylxd_image + @property def instance(self): if not self._instance: self._instance = self.platform.launch_container( - image=self.pylxd_image.fingerprint, - image_desc=str(self), use_desc='image-modification') - self._instance.start(wait=True, wait_time=self.config.get('timeout')) + self.properties, self.config, self.features, + use_desc='image-modification', image_desc=str(self), + image=self.pylxd_image.fingerprint) + self._instance.start() return self._instance @property @@ -46,6 +67,78 @@ class LXDImage(base.Image): 'release': properties.get('release'), } + def export_image(self, output_dir): + """ + export image from lxd image store to (split) tarball on disk + output_dir: dir to store tarballs in + return_value: tuple of path to metadata tarball and rootfs tarball + """ + # pylxd's image export feature doesn't do split exports, so use cmdline + util.subp(['lxc', 'image', 'export', self.pylxd_image.fingerprint, + output_dir], capture=True) + tarballs = [p for p in os.listdir(output_dir) if p.endswith('tar.xz')] + metadata = os.path.join( + output_dir, next(p for p in tarballs if p.startswith('meta-'))) + rootfs = os.path.join( + output_dir, next(p for p in tarballs if not p.startswith('meta-'))) + return (metadata, rootfs) + + def import_image(self, metadata, rootfs): + """ + import image to lxd image store from (split) tarball on disk + note, this will replace and delete the current pylxd_image + metadata: metadata tarball + rootfs: rootfs tarball + return_value: imported image fingerprint + """ + alias = util.gen_instance_name( + image_desc=str(self), use_desc='update-metadata') + util.subp(['lxc', 'image', 'import', metadata, rootfs, + '--alias', alias], capture=True) + self.pylxd_image = self.platform.query_image_by_alias(alias) + return self.pylxd_image.fingerprint + + def update_templates(self, template_config, template_data): + """ + update the image's template configuration + note, this will replace and delete the current pylxd_image + template_config: config overrides for template portion of metadata + template_data: template data to place into templates/ + """ + # set up tmp files + export_dir = util.tmpdir() + extract_dir = util.tmpdir() + new_metadata = os.path.join(export_dir, 'new-meta.tar.xz') + metadata_yaml = os.path.join(extract_dir, 'metadata.yaml') + template_dir = os.path.join(extract_dir, 'templates') + + try: + # extract old data + (metadata, rootfs) = self.export_image(export_dir) + shutil.unpack_archive(metadata, extract_dir) + + # update metadata + metadata = util.read_conf(metadata_yaml) + templates = metadata.get('templates', {}) + templates.update(template_config) + metadata['templates'] = templates + util.yaml_dump(metadata, metadata_yaml) + + # write out template files + for name, content in template_data.items(): + path = os.path.join(template_dir, name) + util.write_file(path, content) + + # store new data, mark new image as modified + util.flat_tar(new_metadata, extract_dir) + self.import_image(new_metadata, rootfs) + self.modified = True + + finally: + # remove tmpfiles + shutil.rmtree(export_dir) + shutil.rmtree(extract_dir) + def execute(self, *args, **kwargs): """ execute command in image, modifying image @@ -58,12 +151,12 @@ class LXDImage(base.Image): """ return self.instance.push_file(local_path, remote_path) - def run_script(self, script): + def run_script(self, *args, **kwargs): """ run script in image, modifying image return_value: script output """ - return self.instance.run_script(script) + return self.instance.run_script(*args, **kwargs) def snapshot(self): """ @@ -71,22 +164,22 @@ class LXDImage(base.Image): """ # clone current instance, start and freeze clone instance = self.platform.launch_container( + self.properties, self.config, self.features, container=self.instance.name, image_desc=str(self), use_desc='snapshot') - instance.start(wait=True, wait_time=self.config.get('timeout')) + instance.start() if self.config.get('boot_clean_script'): instance.run_script(self.config.get('boot_clean_script')) instance.freeze() return lxd_snapshot.LXDSnapshot( - self.properties, self.config, self.platform, instance) + self.platform, self.properties, self.config, + self.features, instance) def destroy(self): """ clean up data associated with image """ - if self._instance: - self._instance.destroy() - self.pylxd_image.delete(wait=True) + self.pylxd_image = None super(LXDImage, self).destroy() # vi: ts=4 expandtab diff --git a/tests/cloud_tests/instances/base.py b/tests/cloud_tests/instances/base.py index 9559d28..252c4c5 100644 --- a/tests/cloud_tests/instances/base.py +++ b/tests/cloud_tests/instances/base.py @@ -1,8 +1,5 @@ # This file is part of cloud-init. See LICENSE file for license information. -import os -import uuid - class Instance(object): """ @@ -10,26 +7,39 @@ class Instance(object): """ platform_name = None - def __init__(self, name): + def __init__(self, platform, name, properties, config, features): """ - setup + Set up instance + platform: platform object + name: hostname of instance + properties: image properties + config: image config + features: supported feature flags """ + self.platform = platform self.name = name + self.properties = properties + self.config = config + self.features = features - def execute(self, command, stdin=None, stdout=None, stderr=None, env={}): + def execute(self, command, stdout=None, stderr=None, env={}, + rcs=None, description=None): """ + Execute command in instance, recording output, error and exit code. + Assumes functional networking and execution as root with the + target filesystem being available at /. + command: the command to execute as root inside the image - stdin, stderr, stdout: file handles + stdout, stderr: file handles to write output and error to env: environment variables - - Execute assumes functional networking and execution as root with the - target filesystem being available at /. + rcs: allowed return codes from command + description: purpose of command return_value: tuple containing stdout data, stderr data, exit code """ raise NotImplementedError - def read_data(self, remote_path, encode=False): + def read_data(self, remote_path, decode=False): """ read_data from instance filesystem remote_path: path in instance @@ -49,6 +59,8 @@ class Instance(object): def pull_file(self, remote_path, local_path): """ copy file at 'remote_path', from instance to 'local_path' + remote_path: path on remote instance + local_path: path on local instance """ with open(local_path, 'wb') as fp: fp.write(self.read_data(remote_path), encode=True) @@ -56,18 +68,34 @@ class Instance(object): def push_file(self, local_path, remote_path): """ copy file at 'local_path' to instance at 'remote_path' + local_path: path on local instance + remote_path: path on remote instance """ with open(local_path, 'rb') as fp: self.write_data(remote_path, fp.read()) - def run_script(self, script): + def run_script(self, script, rcs=None, description=None): """ run script in target and return stdout + script: script contents + rcs: allowed return codes from script + description: purpose of script + return_value: stdout from script """ - script_path = os.path.join('/tmp', str(uuid.uuid1())) - self.write_data(script_path, script) - (out, err, exit_code) = self.execute(['/bin/bash', script_path]) - return out + script_path = self.tmpfile() + try: + self.write_data(script_path, script) + return self.execute( + ['/bin/bash', script_path], rcs=rcs, description=description) + finally: + self.execute(['rm', script_path], rcs=rcs) + + def tmpfile(self): + """ + get a tmp file in the target + return_value: path to new file in target + """ + return self.execute(['mktemp'])[0].strip() def console_log(self): """ @@ -87,7 +115,7 @@ class Instance(object): """ raise NotImplementedError - def start(self, wait=True): + def start(self, wait=True, wait_for_cloud_init=False): """ start instance """ @@ -99,22 +127,32 @@ class Instance(object): """ pass - def _wait_for_cloud_init(self, wait_time): + def _wait_for_system(self, wait_for_cloud_init): """ wait until system has fully booted and cloud-init has finished + wait_time: maximum time to wait + return_value: None, may raise OSError if wait_time exceeded """ - if not wait_time: - return - found_msg = 'found' - cmd = ('for ((i=0;i<{wait};i++)); do [ -f "{file}" ] && ' - '{{ echo "{msg}";break; }} || sleep 1; done').format( - file='/run/cloud-init/result.json', - wait=wait_time, msg=found_msg) + def clean_test(test): + """ + clean formatting for system ready test testcase + """ + return ' '.join(l for l in test.strip().splitlines() + if not l.lstrip().startswith('#')) + + time = self.config['boot_timeout'] + tests = [self.config['system_ready_script']] + if wait_for_cloud_init: + tests.append(self.config['cloud_init_ready_script']) + + formatted_tests = ' && '.join(clean_test(t) for t in tests) + test_cmd = ('for ((i=0;i<{time};i++)); do {test} && exit 0; sleep 1; ' + 'done; exit 1;').format(time=time, test=formatted_tests) + cmd = ['/bin/bash', '-c', test_cmd] + + if self.execute(cmd, rcs=(0, 1))[-1] != 0: + raise OSError('timeout: after {}s system not started'.format(time)) - (out, err, exit) = self.execute(['/bin/bash', '-c', cmd]) - if out.strip() != found_msg: - raise OSError('timeout: after {}s, cloud-init has not started' - .format(wait_time)) # vi: ts=4 expandtab diff --git a/tests/cloud_tests/instances/lxd.py b/tests/cloud_tests/instances/lxd.py index f0aa121..dfc8363 100644 --- a/tests/cloud_tests/instances/lxd.py +++ b/tests/cloud_tests/instances/lxd.py @@ -1,6 +1,7 @@ # This file is part of cloud-init. See LICENSE file for license information. from tests.cloud_tests.instances import base +from tests.cloud_tests import util class LXDInstance(base.Instance): @@ -9,41 +10,69 @@ class LXDInstance(base.Instance): """ platform_name = "lxd" - def __init__(self, name, platform, pylxd_container): + def __init__(self, platform, name, properties, config, features, + pylxd_container): """ - setup + Set up instance + platform: platform object + name: hostname of instance + properties: image properties + config: image config + features: supported feature flags """ - self.platform = platform self._pylxd_container = pylxd_container - super(LXDInstance, self).__init__(name) + super(LXDInstance, self).__init__( + platform, name, properties, config, features) @property def pylxd_container(self): self._pylxd_container.sync() return self._pylxd_container - def execute(self, command, stdin=None, stdout=None, stderr=None, env={}): + def execute(self, command, stdout=None, stderr=None, env={}, + rcs=None, description=None): """ + Execute command in instance, recording output, error and exit code. + Assumes functional networking and execution as root with the + target filesystem being available at /. + command: the command to execute as root inside the image - stdin, stderr, stdout: file handles + stdout, stderr: file handles to write output and error to env: environment variables - - Execute assumes functional networking and execution as root with the - target filesystem being available at /. + rcs: allowed return codes from command + description: purpose of command return_value: tuple containing stdout data, stderr data, exit code """ - # TODO: the pylxd api handler for container.execute needs to be - # extended to properly pass in stdin - # TODO: the pylxd api handler for container.execute needs to be - # extended to get the return code, for now just use 0 + # ensure instance is running and execute the command self.start() - if stdin: - raise NotImplementedError res = self.pylxd_container.execute(command, environment=env) - for (f, data) in (i for i in zip((stdout, stderr), res) if i[0]): - f.write(data) - return res + (0,) + + # get out, exit and err from pylxd return + if hasattr(res, 'exit_code'): + # pylxd 2.2 returns ContainerExecuteResult, named tuple of + # (exit_code, out, err) + (exit, out, err) = res + else: + # pylxd 2.1.3 and earlier only return out and err, no exit + # LOG.warning('using pylxd version < 2.2') + (out, err) = res + exit = 0 + + # write data to file descriptors if needed + if stdout: + stdout.write(out) + if stderr: + stderr.write(err) + + # if the command exited with a code not allowed in rcs, then fail + if exit not in (rcs if rcs else (0,)): + error_desc = ('Failed command to: {}'.format(description) + if description else None) + raise util.InTargetExecuteError( + out, err, exit, command, self.name, error_desc) + + return (out, err, exit) def read_data(self, remote_path, decode=False): """ @@ -83,14 +112,14 @@ class LXDInstance(base.Instance): if self.pylxd_container.status != 'Stopped': self.pylxd_container.stop(wait=wait) - def start(self, wait=True, wait_time=None): + def start(self, wait=True, wait_for_cloud_init=False): """ start instance """ if self.pylxd_container.status != 'Running': self.pylxd_container.start(wait=wait) - if wait and isinstance(wait_time, int): - self._wait_for_cloud_init(wait_time) + if wait: + self._wait_for_system(wait_for_cloud_init) def freeze(self): """ diff --git a/tests/cloud_tests/platforms.yaml b/tests/cloud_tests/platforms.yaml index 5972b32..b91834a 100644 --- a/tests/cloud_tests/platforms.yaml +++ b/tests/cloud_tests/platforms.yaml @@ -10,7 +10,55 @@ default_platform_config: platforms: lxd: enabled: true - get_image_timeout: 600 + # overrides for image templates + template_overrides: + /var/lib/cloud/seed/nocloud-net/meta-data: + when: + - create + - copy + template: cloud-init-meta.tpl + /var/lib/cloud/seed/nocloud-net/network-config: + when: + - create + - copy + template: cloud-init-network.tpl + /var/lib/cloud/seed/nocloud-net/user-data: + when: + - create + - copy + template: cloud-init-user.tpl + properties: + default: | + #cloud-config + {} + /var/lib/cloud/seed/nocloud-net/vendor-data: + when: + - create + - copy + template: cloud-init-vendor.tpl + properties: + default: | + #cloud-config + {} + # overrides image template files + template_files: + cloud-init-meta.tpl: | + #cloud-config + instance-id: {{ container.name }} + local-hostname: {{ container.name }} + {{ config_get("user.meta-data", "") }} + cloud-init-network.tpl: | + {% if config_get("user.network-config", "") == "" %}version: 1 + config: + - type: physical + name: eth0 + subnets: + - type: {% if config_get("user.network_mode", "") == "link-local" %}manual{% else %}dhcp{% endif %} + control: auto{% else %}{{ config_get("user.network-config", "") }}{% endif %} + cloud-init-user.tpl: | + {{ config_get("user.user-data", properties.default) }} + cloud-init-vendor.tpl: | + {{ config_get("user.vendor-data", properties.default) }} ec2: {} azure: {} diff --git a/tests/cloud_tests/platforms/base.py b/tests/cloud_tests/platforms/base.py index 615e2e0..2b6e514 100644 --- a/tests/cloud_tests/platforms/base.py +++ b/tests/cloud_tests/platforms/base.py @@ -15,17 +15,7 @@ class Platform(object): def get_image(self, img_conf): """ - Get image using 'img_conf', where img_conf is a dict containing all - image configuration parameters - - in this dict there must be a 'platform_ident' key containing - configuration for identifying each image on a per platform basis - - see implementations for get_image() for details about the contents - of the platform's config entry - - note: see 'releases' main_config.yaml for example entries - + get image using specified image configuration img_conf: configuration for image return_value: cloud_tests.images instance """ @@ -37,17 +27,4 @@ class Platform(object): """ pass - def _extract_img_platform_config(self, img_conf): - """ - extract platform configuration for current platform from img_conf - """ - platform_ident = img_conf.get('platform_ident') - if not platform_ident: - raise ValueError('invalid img_conf, missing \'platform_ident\'') - ident = platform_ident.get(self.platform_name) - if not ident: - raise ValueError('img_conf: {} missing config for platform {}' - .format(img_conf, self.platform_name)) - return ident - # vi: ts=4 expandtab diff --git a/tests/cloud_tests/platforms/lxd.py b/tests/cloud_tests/platforms/lxd.py index 847cc54..4d8b58c 100644 --- a/tests/cloud_tests/platforms/lxd.py +++ b/tests/cloud_tests/platforms/lxd.py @@ -27,28 +27,32 @@ class LXDPlatform(base.Platform): def get_image(self, img_conf): """ - Get image - img_conf: dict containing config for image. platform_ident must have: - alias: alias to use for simplestreams server - sstreams_server: simplestreams server to use, or None for default + get image using specified image configuration + img_conf: configuration for image return_value: cloud_tests.images instance """ - lxd_conf = self._extract_img_platform_config(img_conf) - image = self.client.images.create_from_simplestreams( - lxd_conf.get('sstreams_server', DEFAULT_SSTREAMS_SERVER), - lxd_conf['alias']) - return lxd_image.LXDImage( - image.properties['description'], img_conf, self, image) + pylxd_image = self.client.images.create_from_simplestreams( + img_conf.get('sstreams_server', DEFAULT_SSTREAMS_SERVER), + img_conf['alias']) + image = lxd_image.LXDImage(self, img_conf, pylxd_image) + if img_conf.get('override_templates', False): + image.update_templates(self.config.get('template_overrides', {}), + self.config.get('template_files', {})) + return image - def launch_container(self, image=None, container=None, ephemeral=False, - config=None, block=True, - image_desc=None, use_desc=None): + def launch_container(self, properties, config, features, + image=None, container=None, ephemeral=False, + container_config=None, block=True, image_desc=None, + use_desc=None): """ launch a container + properties: image properties + config: image configuration + features: image features image: image fingerprint to launch from container: container to copy ephemeral: delete image after first shutdown - config: config options for instance as dict + container_config: config options for instance as dict block: wait until container created image_desc: description of image being launched use_desc: description of container's use @@ -61,11 +65,13 @@ class LXDPlatform(base.Platform): use_desc=use_desc, used_list=self.list_containers()), 'ephemeral': bool(ephemeral), - 'config': config if isinstance(config, dict) else {}, + 'config': (container_config + if isinstance(container_config, dict) else {}), 'source': ({'type': 'image', 'fingerprint': image} if image else {'type': 'copy', 'source': container}) }, wait=block) - return lxd_instance.LXDInstance(container.name, self, container) + return lxd_instance.LXDInstance(self, container.name, properties, + config, features, container) def container_exists(self, container_name): """ @@ -88,6 +94,14 @@ class LXDPlatform(base.Platform): """ return [container.name for container in self.client.containers.all()] + def query_image_by_alias(self, alias): + """ + get image by alias in local image store + alias: alias of image + return_value: pylxd image (not cloud_tests.images instance) + """ + return self.client.images.get_by_alias(alias) + def destroy(self): """ Clean up platform data diff --git a/tests/cloud_tests/releases.yaml b/tests/cloud_tests/releases.yaml index 3ffa68f..cb7bc84 100644 --- a/tests/cloud_tests/releases.yaml +++ b/tests/cloud_tests/releases.yaml @@ -1,79 +1,258 @@ # ============================= Release Config ================================ default_release_config: - # all are disabled by default - enabled: false - # timeout for booting image and running cloud init - timeout: 120 - # platform_ident values for the image, with data to identify the image - # on that platform. see platforms.base for more information - platform_ident: {} - # a script to run after a boot that is used to modify an image, before - # making a snapshot of the image. may be useful for removing data left - # behind from cloud-init booting, such as logs, to ensure that data from - # snapshot.launch() will not include a cloud-init.log from a boot used to - # create the snapshot, if cloud-init has not run - boot_clean_script: | - #!/bin/bash - rm -rf /var/log/cloud-init.log /var/log/cloud-init-output.log \ - /var/lib/cloud/ /run/cloud-init/ /var/log/syslog + # global default configuration options + default: + # all are disabled by default + enabled: false + # timeout for booting image and running cloud init + boot_timeout: 120 + # a script to run after a boot that is used to modify an image, before + # making a snapshot of the image. may be useful for removing data left + # behind from cloud-init booting, such as logs, to ensure that data + # from snapshot.launch() will not include a cloud-init.log from a boot + # used to create the snapshot, if cloud-init has not run + boot_clean_script: | + #!/bin/bash + rm -rf /var/log/cloud-init.log /var/log/cloud-init-output.log \ + /var/lib/cloud/ /run/cloud-init/ /var/log/syslog + # test script to determine if system is booted fully + system_ready_script: | + # permit running or degraded state as both indicate complete boot + [ $(systemctl is-system-running) = 'running' -o + $(systemctl is-system-running) = 'degraded' ] + # test script to determine if cloud-init has finished + cloud_init_ready_script: | + [ -f '/run/cloud-init/result.json' ] + # currently used features and their uses are: + # features groups and additional feature settings + feature_groups: [] + features: {} + + # lxd specific default configuration options + lxd: + # default sstreams server to use for lxd image retrieval + sstreams_server: https://us.images.linuxcontainers.org:8443 + # keep base image, avoids downloading again next run + cache_base_image: true + # lxd images from linuxcontainers.org do not have the nocloud seed + # templates in place, so the image metadata must be modified + override_templates: true + # arg overrides to set image up + setup_overrides: + # lxd images from linuxcontainers.org do not come with + # cloud-init, so must pull cloud-init in from repo using + # setup_image.upgrade + upgrade: true + +features: + # all currently supported feature flags + all: + - apt # image supports apt package manager + - byobu # byobu is available in repositories + - landscape # landscape-client available in repos + - lxd # lxd is available in the image + - ppa # image supports ppas + - rpm # image supports rpms + - snap # supports snapd + # NOTE: the following feature flags are to work around bugs in the + # images, and can be removed when no longer needed + - hostname # setting system hostname works + # NOTE: the following feature flags are to work around issues in the + # testcases, and can be removed when no longer needed + - apt_src_cont # default contents and format of sources.list matches + # ubuntu sources.list + - apt_hist_fmt # apt command history entries use full paths to apt + # executable rather than relative paths + - daylight_time # timezones are daylight not standard time + - engb_locale # locale en_GB.UTF-8 is available + - sshd # requires ssh server to be installed by default + - syslog # test case requires syslog to be written by default + - ubuntu_ntp # expect ubuntu.pool.ntp.org to be used as ntp server + - ubuntu_repos # test case requres ubuntu repositories to be used + - ubuntu_user # test case needs user with the name 'ubuntu' to exist + # NOTE: the following feature flags are to work around issues that may + # be considered bugs in cloud-init + - lsb_release # image has lsb_release installed, maybe should install + # if missing by default + - sudo # image has sudo installed, should not be required + # feature flag groups + groups: + base: + hostname: true + ubuntu_specific: + apt_src_cont: true + apt_hist_fmt: true + byobu: true + daylight_time: true + engb_locale: true + landscape: true + lsb_release: true + lxd: true + ppa: true + snap: true + sshd: true + sudo: true + syslog: true + ubuntu_ntp: true + ubuntu_repos: true + ubuntu_user: true + debian_base: + apt: true + rhel_base: + rpm: true releases: - trusty: - enabled: true - platform_ident: - lxd: - # if sstreams_server is omitted, default is used, defined in - # tests.cloud_tests.platforms.lxd.DEFAULT_SSTREAMS_SERVER as: - # sstreams_server: https://us.images.linuxcontainers.org:8443 - #alias: ubuntu/trusty/default - alias: t - sstreams_server: https://cloud-images.ubuntu.com/daily - xenial: - enabled: true - platform_ident: - lxd: - #alias: ubuntu/xenial/default - alias: x - sstreams_server: https://cloud-images.ubuntu.com/daily - yakkety: - enabled: true - platform_ident: - lxd: - #alias: ubuntu/yakkety/default - alias: y - sstreams_server: https://cloud-images.ubuntu.com/daily + # UBUNTU ================================================================= zesty: - enabled: true - platform_ident: - lxd: - #alias: ubuntu/zesty/default - alias: z - sstreams_server: https://cloud-images.ubuntu.com/daily - jessie: - platform_ident: - lxd: - alias: debian/jessie/default + # EOL: Jan 2018 + default: + enabled: true + feature_groups: + - base + - debian_base + - ubuntu_specific + lxd: + sstreams_server: https://cloud-images.ubuntu.com/daily + alias: zesty + setup_overrides: null + override_templates: false + yakkety: + # EOL: Jul 2017 + default: + enabled: true + feature_groups: + - base + - debian_base + - ubuntu_specific + lxd: + sstreams_server: https://cloud-images.ubuntu.com/daily + alias: yakkety + setup_overrides: null + override_templates: false + xenial: + # EOL: Apr 2021 + default: + enabled: true + feature_groups: + - base + - debian_base + - ubuntu_specific + lxd: + sstreams_server: https://cloud-images.ubuntu.com/daily + alias: xenial + setup_overrides: null + override_templates: false + trusty: + # EOL: Apr 2019 + default: + enabled: true + feature_groups: + - base + - debian_base + - ubuntu_specific + system_ready_script: | + #!/bin/bash + # upstart based, so use old style runlevels + [ $(runlevel | awk '{print $2}') = '2' ] + lxd: + sstreams_server: https://cloud-images.ubuntu.com/daily + alias: trusty + setup_overrides: null + override_templates: false + precise: + # EOL: Apr 2017 + default: + # still supported but not relevant for development, not enabled + # tests should still work though unless they use newer features + enabled: false + feature_groups: + - base + - debian_base + - ubuntu_specific + features: + lxd: false + system_ready_script: | + #!/bin/bash + # upstart based, so use old style runlevels + [ $(runlevel | awk '{print $2}') = '2' ] + lxd: + sstreams_server: https://cloud-images.ubuntu.com/daily + alias: precise + setup_overrides: null + override_templates: false + # DEBIAN ================================================================= sid: - platform_ident: - lxd: - alias: debian/sid/default + # EOL: N/A + default: + # tests should work on sid, however it is not always stable + enabled: false + feature_groups: + - base + - debian_base + lxd: + alias: debian/sid/default stretch: - platform_ident: - lxd: - alias: debian/stretch/default + # EOL: Not yet released + default: + enabled: true + feature_groups: + - base + - debian_base + lxd: + alias: debian/stretch/default + jessie: + # EOL: Jun 2020 + # NOTE: the cloud-init version shipped with jessie is out of date + # tests work if an up to date deb is used + default: + enabled: true + feature_groups: + - base + - debian_base + lxd: + alias: debian/jessie/default wheezy: - platform_ident: - lxd: - alias: debian/wheezy/default + # EOL: May 2018 (Apr 2016 - end of full updates) + default: + # this is old enough that it is no longer relevant for development + enabled: false + feature_groups: + - base + - debian_base + lxd: + alias: debian/wheezy/default + # CENTOS ================================================================= centos70: - timeout: 180 - platform_ident: - lxd: - alias: centos/7/default + # EOL: Jun 2024 (2020 - end of full updates) + default: + enabled: true + feature_groups: + - base + - rhel_base + user_data_overrides: + preserve_hostname: true + lxd: + features: + # NOTE: (LP: #1575779) + hostname: false + alias: centos/7/default centos66: - timeout: 180 - platform_ident: - lxd: - alias: centos/6/default + # EOL: Nov 2020 + default: + enabled: true + feature_groups: + - base + - rhel_base + # still supported, but only bugfixes after may 2017 + system_ready_script: | + #!/bin/bash + [ $(runlevel | awk '{print $2}') = '3' ] + user_data_overrides: + preserve_hostname: true + lxd: + features: + # NOTE: (LP: #1575779) + hostname: false + alias: centos/6/default # vi: ts=4 expandtab diff --git a/tests/cloud_tests/setup_image.py b/tests/cloud_tests/setup_image.py index 5d6c638..1b74ceb 100644 --- a/tests/cloud_tests/setup_image.py +++ b/tests/cloud_tests/setup_image.py @@ -7,6 +7,30 @@ from functools import partial import os +def installed_version(image, package, ensure_installed=True): + """ + get installed version of package + image: cloud_tests.images instance to operate on + package: name of package + ensure_installed: raise error if not installed + return_value: cloud-init version string + """ + # get right cmd for os family + os_family = util.get_os_family(image.properties['os']) + if os_family == 'debian': + cmd = ['dpkg-query', '-W', "--showformat='${Version}'", package] + elif os_family == 'redhat': + cmd = ['rpm', '-q', '--queryformat', "'%{VERSION}'", package] + else: + raise NotImplementedError + + # query version + msg = 'query version for package: {}'.format(package) + (out, err, exit) = image.execute( + cmd, description=msg, rcs=(0,) if ensure_installed else range(0, 256)) + return out.strip() + + def install_deb(args, image): """ install deb into image @@ -21,20 +45,18 @@ def install_deb(args, image): 'family: {}'.format(args.deb, os_family)) # install deb - LOG.debug('installing deb: %s into target', args.deb) + msg = 'install deb: "{}" into target'.format(args.deb) + LOG.debug(msg) remote_path = os.path.join('/tmp', os.path.basename(args.deb)) image.push_file(args.deb, remote_path) - (out, err, exit) = image.execute(['dpkg', '-i', remote_path]) - if exit != 0: - raise OSError('failed install deb: {}\n\tstdout: {}\n\tstderr: {}' - .format(args.deb, out, err)) + cmd = 'dpkg -i {} || apt-get install --yes -f'.format(remote_path) + image.execute(['/bin/sh', '-c', cmd], description=msg) # check installed deb version matches package fmt = ['-W', "--showformat='${Version}'"] (out, err, exit) = image.execute(['dpkg-deb'] + fmt + [remote_path]) expected_version = out.strip() - (out, err, exit) = image.execute(['dpkg-query'] + fmt + ['cloud-init']) - found_version = out.strip() + found_version = installed_version(image, 'cloud-init') if expected_version != found_version: raise OSError('install deb version "{}" does not match expected "{}"' .format(found_version, expected_version)) @@ -52,24 +74,21 @@ def install_rpm(args, image): """ # ensure system is compatible with package format os_family = util.get_os_family(image.properties['os']) - if os_family not in ['redhat', 'sles']: + if os_family != 'redhat': raise NotImplementedError('install rpm: {} not supported on os ' 'family: {}'.format(args.rpm, os_family)) # install rpm - LOG.debug('installing rpm: %s into target', args.rpm) + msg = 'install rpm: "{}" into target'.format(args.rpm) + LOG.debug(msg) remote_path = os.path.join('/tmp', os.path.basename(args.rpm)) image.push_file(args.rpm, remote_path) - (out, err, exit) = image.execute(['rpm', '-U', remote_path]) - if exit != 0: - raise OSError('failed to install rpm: {}\n\tstdout: {}\n\tstderr: {}' - .format(args.rpm, out, err)) + image.execute(['rpm', '-U', remote_path], description=msg) fmt = ['--queryformat', '"%{VERSION}"'] (out, err, exit) = image.execute(['rpm', '-q'] + fmt + [remote_path]) expected_version = out.strip() - (out, err, exit) = image.execute(['rpm', '-q'] + fmt + ['cloud-init']) - found_version = out.strip() + found_version = installed_version(image, 'cloud-init') if expected_version != found_version: raise OSError('install rpm version "{}" does not match expected "{}"' .format(found_version, expected_version)) @@ -80,13 +99,34 @@ def install_rpm(args, image): def upgrade(args, image): """ - run the system's upgrade command + upgrade or install cloud-init from repo + args: cmdline arguments + image: cloud_tests.images instance to operate on + return_value: None, may raise errors + """ + # determine command for os_family + os_family = util.get_os_family(image.properties['os']) + if os_family == 'debian': + cmd = 'apt-get update && apt-get install cloud-init --yes' + elif os_family == 'redhat': + cmd = 'yum install cloud-init --assumeyes' + else: + raise NotImplementedError + + # upgrade cloud-init + msg = 'upgrading cloud-init' + LOG.debug(msg) + image.execute(['/bin/sh', '-c', cmd], description=msg) + + +def upgrade_full(args, image): + """ + run the system's full upgrade command args: cmdline arguments image: cloud_tests.images instance to operate on return_value: None, may raise errors """ # determine appropriate upgrade command for os_family - # TODO: maybe use cloudinit.distros for this? os_family = util.get_os_family(image.properties['os']) if os_family == 'debian': cmd = 'apt-get update && apt-get upgrade --yes' @@ -97,11 +137,9 @@ def upgrade(args, image): 'from family: {}'.format(os_family)) # upgrade system - LOG.debug('upgrading system') - (out, err, exit) = image.execute(['/bin/sh', '-c', cmd]) - if exit != 0: - raise OSError('failed to upgrade system\n\tstdout: {}\n\tstderr:{}' - .format(out, err)) + msg = 'full system upgrade' + LOG.debug(msg) + image.execute(['/bin/sh', '-c', cmd], description=msg) def run_script(args, image): @@ -111,9 +149,9 @@ def run_script(args, image): image: cloud_tests.images instance to operate on return_value: None, may raise errors """ - # TODO: get exit status back from script and add error handling here - LOG.debug('running setup image script in target image') - image.run_script(args.script) + msg = 'run setup image script in target image' + LOG.debug(msg) + image.run_script(args.script, description=msg) def enable_ppa(args, image): @@ -124,17 +162,15 @@ def enable_ppa(args, image): return_value: None, may raise errors """ # ppa only supported on ubuntu (maybe debian?) - if image.properties['os'] != 'ubuntu': + if image.properties['os'].lower() != 'ubuntu': raise NotImplementedError('enabling a ppa is only available on ubuntu') # add ppa with add-apt-repository and update ppa = 'ppa:{}'.format(args.ppa) - LOG.debug('enabling %s', ppa) + msg = 'enable ppa: "{}" in target'.format(ppa) + LOG.debug(msg) cmd = 'add-apt-repository --yes {} && apt-get update'.format(ppa) - (out, err, exit) = image.execute(['/bin/sh', '-c', cmd]) - if exit != 0: - raise OSError('enable ppa for {} failed\n\tstdout: {}\n\tstderr: {}' - .format(ppa, out, err)) + image.execute(['/bin/sh', '-c', cmd], description=msg) def enable_repo(args, image): @@ -155,11 +191,9 @@ def enable_repo(args, image): raise NotImplementedError('enable repo command not configured for ' 'distro from family: {}'.format(os_family)) - LOG.debug('enabling repo: "%s"', args.repo) - (out, err, exit) = image.execute(['/bin/sh', '-c', cmd]) - if exit != 0: - raise OSError('enable repo {} failed\n\tstdout: {}\n\tstderr: {}' - .format(args.repo, out, err)) + msg = 'enable repo: "{}" in target'.format(args.repo) + LOG.debug(msg) + image.execute(['/bin/sh', '-c', cmd], description=msg) def setup_image(args, image): @@ -169,6 +203,11 @@ def setup_image(args, image): image: cloud_tests.image instance to operate on return_value: tuple of results and fail count """ + # update the args if necessary for this image + overrides = image.setup_overrides + LOG.debug('updating args for setup with: %s', overrides) + args = util.update_args(args, overrides, preserve_old=True) + # mapping of setup cmdline arg name to setup function # represented as a tuple rather than a dict or odict as lookup by name not # needed, and order is important as --script and --upgrade go at the end @@ -179,17 +218,19 @@ def setup_image(args, image): ('repo', enable_repo, 'setup func for --repo, enable repo'), ('ppa', enable_ppa, 'setup func for --ppa, enable ppa'), ('script', run_script, 'setup func for --script, run script'), - ('upgrade', upgrade, 'setup func for --upgrade, upgrade pkgs'), + ('upgrade', upgrade, 'setup func for --upgrade, upgrade cloud-init'), + ('upgrade-full', upgrade_full, 'setup func for --upgrade-full'), ) # determine which setup functions needed calls = [partial(stage.run_single, desc, partial(func, args, image)) for name, func, desc in handlers if getattr(args, name, None)] - image_name = 'image: distro={}, release={}'.format( - image.properties['os'], image.properties['release']) - LOG.info('setting up %s', image_name) - return stage.run_stage('set up for {}'.format(image_name), calls, - continue_after_error=False) + LOG.info('setting up %s', image) + res = stage.run_stage( + 'set up for {}'.format(image), calls, continue_after_error=False) + LOG.debug('after setup complete, installed cloud-init version is: %s', + installed_version(image, 'cloud-init')) + return res # vi: ts=4 expandtab diff --git a/tests/cloud_tests/snapshots/base.py b/tests/cloud_tests/snapshots/base.py index d715f03..cbe3f5f 100644 --- a/tests/cloud_tests/snapshots/base.py +++ b/tests/cloud_tests/snapshots/base.py @@ -7,12 +7,18 @@ class Snapshot(object): """ platform_name = None - def __init__(self, properties, config): + def __init__(self, platform, properties, config, features): """ Set up snapshot + platform: platform object + properties: image properties + config: image config + features: supported feature flags """ + self.platform = platform self.properties = properties self.config = config + self.features = features def __str__(self): """ diff --git a/tests/cloud_tests/snapshots/lxd.py b/tests/cloud_tests/snapshots/lxd.py index eabbce3..2241035 100644 --- a/tests/cloud_tests/snapshots/lxd.py +++ b/tests/cloud_tests/snapshots/lxd.py @@ -9,13 +9,18 @@ class LXDSnapshot(base.Snapshot): """ platform_name = "lxd" - def __init__(self, properties, config, platform, pylxd_frozen_instance): + def __init__(self, platform, properties, config, features, + pylxd_frozen_instance): """ Set up snapshot + platform: platform object + properties: image properties + config: image config + features: supported feature flags """ - self.platform = platform self.pylxd_frozen_instance = pylxd_frozen_instance - super(LXDSnapshot, self).__init__(properties, config) + super(LXDSnapshot, self).__init__( + platform, properties, config, features) def launch(self, user_data, meta_data=None, block=True, start=True, use_desc=None): @@ -34,10 +39,11 @@ class LXDSnapshot(base.Snapshot): if meta_data: inst_config['user.meta-data'] = meta_data instance = self.platform.launch_container( - container=self.pylxd_frozen_instance.name, config=inst_config, - block=block, image_desc=str(self), use_desc=use_desc) + self.properties, self.config, self.features, block=block, + image_desc=str(self), container=self.pylxd_frozen_instance.name, + use_desc=use_desc, container_config=inst_config) if start: - instance.start(wait=True, wait_time=self.config.get('timeout')) + instance.start() return instance def destroy(self): diff --git a/tests/cloud_tests/testcases.yaml b/tests/cloud_tests/testcases.yaml index c22b08e..7183e01 100644 --- a/tests/cloud_tests/testcases.yaml +++ b/tests/cloud_tests/testcases.yaml @@ -2,6 +2,7 @@ base_test_data: script_timeout: 20 enabled: True + required_features: [] cloud_config: | #cloud-config collect_scripts: diff --git a/tests/cloud_tests/util.py b/tests/cloud_tests/util.py index 64a8667..4c89c9a 100644 --- a/tests/cloud_tests/util.py +++ b/tests/cloud_tests/util.py @@ -1,5 +1,6 @@ # This file is part of cloud-init. See LICENSE file for license information. +import copy import glob import os import random @@ -7,10 +8,18 @@ import string import tempfile import yaml -from cloudinit.distros import OSFAMILIES from cloudinit import util as c_util from tests.cloud_tests import LOG +OS_FAMILY_MAPPING = { + 'debian': ['debian', 'ubuntu'], + 'redhat': ['centos', 'rhel', 'fedora'], + 'gentoo': ['gentoo'], + 'freebsd': ['freebsd'], + 'suse': ['sles'], + 'arch': ['arch'], +} + def list_test_data(data_dir): """ @@ -68,7 +77,7 @@ def gen_instance_name(prefix='cloud-test', image_desc=None, use_desc=None, """ filter bad characters out of elem and trim to length """ - elem = elem[:max_len] if elem else unknown + elem = elem.lower()[:max_len] if elem else unknown return ''.join(c if c in valid else delim for c in elem) return next(name for name in @@ -88,7 +97,8 @@ def get_os_family(os_name): """ get os family type for os_name """ - return next((k for k, v in OSFAMILIES.items() if os_name in v), None) + return next((k for k, v in OS_FAMILY_MAPPING.items() + if os_name.lower() in v), None) def current_verbosity(): @@ -127,12 +137,17 @@ def configure_yaml(): 'tag:yaml.org,2002:str', data, style='|' if '\n' in data else ''))) -def yaml_format(data): +def yaml_format(data, content_type=None): """ format data as yaml + data: data to dump + header: is specified, add a header to the dumped data + return_value: yaml string """ configure_yaml() - return yaml.dump(data, indent=2, default_flow_style=False) + content_type = ( + '#{}\n'.format(content_type.strip('#\n')) if content_type else '') + return content_type + yaml.dump(data, indent=2, default_flow_style=False) def yaml_dump(data, path): @@ -158,6 +173,95 @@ def write_file(*args, **kwargs): """ write a file using cloudinit.util.write_file """ - c_util.write_file(*args, **kwargs) + return c_util.write_file(*args, **kwargs) + + +def read_conf(*args, **kwargs): + """ + read configuration using cloudinit.util.read_conf + """ + return c_util.read_conf(*args, **kwargs) + + +def subp(*args, **kwargs): + """ + execute a command on the system shell using cloudinit.util.subp + """ + return c_util.subp(*args, **kwargs) + + +def tmpdir(prefix='cloud_test_util_'): + return tempfile.mkdtemp(prefix=prefix) + + +def rel_files(basedir): + """ + list of files under directory by relative path, not including directories + return_value: list or relative paths + """ + basedir = os.path.normpath(basedir) + return [path[len(basedir) + 1:] for path in + glob.glob(os.path.join(basedir, '**'), recursive=True) + if not os.path.isdir(path)] + + +def flat_tar(output, basedir, owner='root', group='root'): + """ + create a flat tar archive (no leading ./) from basedir + output: output tar file to write + basedir: base directory for archive + owner: owner of archive files + group: group archive files belong to + return_value: none + """ + c_util.subp(['tar', 'cf', output, '--owner', owner, '--group', group, + '-C', basedir] + rel_files(basedir), capture=True) + + +def update_args(args, updates, preserve_old=True): + """ + update cmdline arguments from a dictionary + args: cmdline arguments + updates: dictionary of {arg_name: new_value} mappings + preserve_old: if true, create a deep copy of args before updating + return_value: updated cmdline arguments, as new object if preserve_old=True + """ + args = copy.deepcopy(args) if preserve_old else args + if updates: + vars(args).update(updates) + return args + + +def update_user_data(user_data, updates, dump_to_yaml=True): + """ + user_data: user data as yaml string or dict + updates: dictionary to merge with user data + dump_to_yaml: return as yaml dumped string if true + return_value: updated user data, as yaml string if dump_to_yaml is true + """ + user_data = (c_util.load_yaml(user_data) + if isinstance(user_data, str) else copy.deepcopy(user_data)) + user_data.update(updates) + return (yaml_format(user_data, content_type='cloud-config') + if dump_to_yaml else user_data) + + +class InTargetExecuteError(c_util.ProcessExecutionError): + """ + Error type for in target commands that fail + """ + default_desc = 'Unexpected error while running command in target instance' + + def __init__(self, stdout, stderr, exit_code, cmd, instance, + description=None): + """ + init error and parent error class + """ + if isinstance(cmd, (tuple, list)): + cmd = ' '.join(cmd) + super(InTargetExecuteError, self).__init__( + stdout=stdout, stderr=stderr, exit_code=exit_code, cmd=cmd, + reason="Instance: {}".format(instance), + description=description if description else self.default_desc) # vi: ts=4 expandtab diff --git a/tox.ini b/tox.ini index bf9046a..5cf8d22 100644 --- a/tox.ini +++ b/tox.ini @@ -101,4 +101,4 @@ basepython = python3 commands = {envpython} -m tests.cloud_tests {posargs} passenv = HOME deps = - pylxd==2.1.3 + pylxd==2.2.3
_______________________________________________ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp