[GitHub] incubator-ariatosca pull request #134: fix getting started documentation for...
Github user pono closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/134 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca pull request #184: ARIA-307 Automate release process
GitHub user ran-z opened a pull request: https://github.com/apache/incubator-ariatosca/pull/184 ARIA-307 Automate release process Created a bash script to help with the releaes process You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-307-automate-release-process Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/184.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #184 commit dfc81ccf8a8ab4f0e03f8df6d8e4dc8daa6dc387 Author: Ran ZivDate: 2017-07-10T13:29:22Z ARIA-307 Automate release process Created a bash script to help with the releaes process --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca issue #184: ARIA-307 Automate release process
Github user asfgit commented on the issue: https://github.com/apache/incubator-ariatosca/pull/184 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
incubator-ariatosca git commit: ARIA-307 Automate release process
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-307-automate-release-process [created] dfc81ccf8 ARIA-307 Automate release process Created a bash script to help with the releaes process Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/dfc81ccf Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/dfc81ccf Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/dfc81ccf Branch: refs/heads/ARIA-307-automate-release-process Commit: dfc81ccf8a8ab4f0e03f8df6d8e4dc8daa6dc387 Parents: f903006 Author: Ran ZivAuthored: Mon Jul 10 16:29:22 2017 +0300 Committer: Ran Ziv Committed: Tue Jul 11 03:49:49 2017 +0300 -- release.sh | 235 1 file changed, 235 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/dfc81ccf/release.sh -- diff --git a/release.sh b/release.sh new file mode 100755 index 000..8925bc4 --- /dev/null +++ b/release.sh @@ -0,0 +1,235 @@ +#!/bin/bash +set -e + + +function create_apache_release_candidate { + if [ ! -f VERSION ]; then + echo "Must run this from ARIA's root dir" >&2 + return 1 + fi + + if [ "$#" -lt 1 ]; then + echo "Must provide git branch for release candidate" >&2 + return 1 + fi + + local GIT_BRANCH=$1 + local OPTIONAL_ARIATOSCA_DIST_DEV_PATH=$2 + + git checkout ${GIT_BRANCH} + local VERSION=$(cat VERSION) + + echo "Creating Apache release candidate for version ${VERSION}..." + + make clean + _create_source_package ${GIT_BRANCH} ${VERSION} + _create_sdist_and_bdist_packages + _publish_to_apache_dev ${VERSION} ${OPTIONAL_ARIATOSCA_DIST_DEV_PATH} + _publish_to_test_pypi + git checkout - +} + + +function finalize_apache_release { + if [ ! -f VERSION ]; then + echo "Must run this from ARIA's root dir" >&2 + return 1 + fi + + if [ "$#" -ne 1 ]; then + echo "Must provide git branch for release tagging" >&2 + return 1 + fi + + local GIT_BRANCH=$1 + + git checkout ${GIT_BRANCH} + local VERSION=$(cat VERSION) + + read -p "Enter 'Yes' to confirm release finalization for version ${VERSION}: " yn + case $yn in + Yes ) echo "Finalizing Apache release...";; + * ) git checkout -; return;; + esac + + _publish_to_apache_release ${VERSION} + _publish_to_real_pypi + _create_git_tag ${VERSION} + git checkout - +} + + +function _create_source_package { + local GIT_BRANCH=$1 + local VERSION=$2 + local INCUBATING_ARCHIVE_CONTENT_DIR=apache-ariatosca-${VERSION}-incubating # e.g. apache-ariatosca-0.1.0-incubating + local INCUBATING_ARCHIVE=${INCUBATING_ARCHIVE_CONTENT_DIR}.tar.gz # e.g. apache-ariatosca-0.1.0-incubating.tar.gz + local SOURCE_PACKAGE_DIR="source" + + echo "Creating source package..." + mkdir -p dist/${SOURCE_PACKAGE_DIR} + pushd dist/${SOURCE_PACKAGE_DIR} + # re-cloning repository, to ensure repo snapshot is clean and not environment-dependent + wget https://github.com/apache/incubator-ariatosca/archive/${GIT_BRANCH}.zip + unzip ${GIT_BRANCH}.zip > /dev/null + mv incubator-ariatosca-${GIT_BRANCH} ${INCUBATING_ARCHIVE_CONTENT_DIR} + tar -czvf ${INCUBATING_ARCHIVE} ${INCUBATING_ARCHIVE_CONTENT_DIR} > /dev/null + rm -rf ${INCUBATING_ARCHIVE_CONTENT_DIR} + rm ${GIT_BRANCH}.zip + + _sign_package ${INCUBATING_ARCHIVE} + popd +} + +function _sign_package { + local ARCHIVE_NAME=$1 + + echo "Signing archive ${ARCHIVE_NAME}..." + gpg --armor --output ${ARCHIVE_NAME}.asc --detach-sig ${ARCHIVE_NAME} + gpg --print-md MD5 ${ARCHIVE_NAME} > ${ARCHIVE_NAME}.md5 + gpg --print-md SHA512 ${ARCHIVE_NAME} > ${ARCHIVE_NAME}.sha +} + + +function _create_sdist_and_bdist_packages { + local SDIST_PACKAGE_DIR="sdist" + local BDIST_PACKAGE_DIR="bdist" + + echo "Creating sdist and bdist packages..." + make docs + python setup.py sdist -d dist/${SDIST_PACKAGE_DIR} bdist_wheel -d dist/${BDIST_PACKAGE_DIR} + + # pushing LICENSE and additional files into the binary distribution archive + find dist/${BDIST_PACKAGE_DIR} -type f -name '*.whl' -exec zip -u {} LICENSE NOTICE DISCLAIMER \; + + pushd dist/${SDIST_PACKAGE_DIR} + local SDIST_ARCHIVE=$(find . -type f -name "*.tar.gz" -printf '%P\n') + _sign_package
svn commit: r20393 - in /dev/incubator/ariatosca/0.1.1-incubating: ./ bdist/ sdist/ source/
Author: ran Date: Mon Jul 10 16:46:10 2017 New Revision: 20393 Log: ARIA 0.1.1 release candidate Added: dev/incubator/ariatosca/0.1.1-incubating/ dev/incubator/ariatosca/0.1.1-incubating/bdist/ dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl (with props) dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.asc dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.md5 dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.sha dev/incubator/ariatosca/0.1.1-incubating/sdist/ dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz (with props) dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz.asc dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz.md5 dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz.sha dev/incubator/ariatosca/0.1.1-incubating/source/ dev/incubator/ariatosca/0.1.1-incubating/source/apache-ariatosca-0.1.1-incubating.tar.gz (with props) dev/incubator/ariatosca/0.1.1-incubating/source/apache-ariatosca-0.1.1-incubating.tar.gz.asc dev/incubator/ariatosca/0.1.1-incubating/source/apache-ariatosca-0.1.1-incubating.tar.gz.md5 dev/incubator/ariatosca/0.1.1-incubating/source/apache-ariatosca-0.1.1-incubating.tar.gz.sha Added: dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl == Binary file - no diff available. Propchange: dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.asc == --- dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.asc (added) +++ dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.asc Mon Jul 10 16:46:10 2017 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v1 + +iQIcBAABCgAGBQJZY68kAAoJEJcu02+B9rrFDxUP/3Qt4y68bwzmBkAE2EPlnc/o ++SHh1uifI4TucStluLl4vUuJWxbAgyzUecJAtnst8vwon0NqVUnN6YMtcvqLhqJS +V0T7XwFRv32DI2TYNVKWh6B95K+dZR7xpfmULZPIvpfgeGNEwZ60kLAfxMaxxJek +TKgHnnTcg22tvlShhyI6tfd3N4/kZOeo7/L1aUxheKHpYsTwHWEOtltSJCMhEcyy +Yz8+E5CeC09K21NMbek6MGqtWu0tbuKJOxL9lRL+RRxJGCS2t6GUVYzmnXoc3lIS +YNy4C2F3V/5GKxrOiThnMrf/bTTvojMuzbUczUwMWunV7cImXNrwyYSzJjY6ro91 +b0qmmgC/HwEhLiW9ETTvqTkL8ZwiKGlpQqJvQSCmWZ8twNjWE3LkrPNzvHQ9O7CN +JN5NHVUGsUXPHIKl/RJgDyP4BevXm8UTQXjt8psniHhcjYu5zRexqaxfns2xN+AD +FnZy/I1xuOgHL+aJiEjfoWqUq4M908erjs71Scy4nM7PeIGOU316CslHnNJoCMGN +JqeUpVpAhqXv3bLUWndIUPxq+FkviBFK0ZU/kk8V3opEd4tUBzAbcO82cokAWPE3 +kIc5RN/H7D+JVekQL6WmR51IXaBke1sdeEOxibV2HboDQi6P+7S1KWIpPKuXuGPK +W5Wqnb+JIfo4qDBFUAmj +=o2g3 +-END PGP SIGNATURE- Added: dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.md5 == --- dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.md5 (added) +++ dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.md5 Mon Jul 10 16:46:10 2017 @@ -0,0 +1,2 @@ +apache_ariatosca-0.1.1-py2-none-any.whl: +95 F0 DA 8D 23 D1 C8 22 E6 6F 7A 5B 67 7B 9F 7D Added: dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.sha == --- dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.sha (added) +++ dev/incubator/ariatosca/0.1.1-incubating/bdist/apache_ariatosca-0.1.1-py2-none-any.whl.sha Mon Jul 10 16:46:10 2017 @@ -0,0 +1,3 @@ +apache_ariatosca-0.1.1-py2-none-any.whl: +CE009DA5 4876A2AA 4DE51EE2 87A6C8BA D3D2B790 EB9DAD17 DAF9361A 51753966 02D895B7 + 6FA12D43 73D61C38 54CDCB96 1568D20C FF0D17DF 6703EA44 8600697B Added: dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz == Binary file - no diff available. Propchange: dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz -- svn:mime-type = application/octet-stream Added: dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz.asc == --- dev/incubator/ariatosca/0.1.1-incubating/sdist/apache-ariatosca-0.1.1.tar.gz.asc (added) +++
[2/2] incubator-ariatosca git commit: ARIA-311 Update CHANGELOG with 0.1.1 issues
ARIA-311 Update CHANGELOG with 0.1.1 issues Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/8e4f4e24 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/8e4f4e24 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/8e4f4e24 Branch: refs/heads/0.1.1-rc Commit: 8e4f4e24e125aff21889d899605c80fc36187455 Parents: 615868c Author: Ran ZivAuthored: Mon Jul 10 18:02:59 2017 +0300 Committer: Ran Ziv Committed: Mon Jul 10 18:02:59 2017 +0300 -- CHANGELOG.rst | 12 1 file changed, 12 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/8e4f4e24/CHANGELOG.rst -- diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 6abb1af..a0ca089 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,3 +1,15 @@ +0.1.1 +- + +[ARIA-312] Validation of workflow and operation kwargs raise false alarms +[ARIA-301] Environment-marked dependencies are installed regardless of environment when installing from wheel +[ARIA-299] Resuming canceled execution with non-finished tasks fails +[ARIA-298] Test suite sometimes fails or freezes despite all tests passing +[ARIA-296] Process termination test fails on windows +[ARIA-287] New tox suite to make sure that Sphinx documentation generation isn't broken +[ARIA-202] Execution plugin assumes '/tmp' for temp directory on the local/remote machine + + 0.1.0 -
[2/3] incubator-ariatosca git commit: ARIA-312 Validation of workflow and operation kwargs raise False alarms
ARIA-312 Validation of workflow and operation kwargs raise False alarms Workflow and Operation function kwargs validation failed in some scenarios. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/6c2f35ec Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/6c2f35ec Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/6c2f35ec Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: 6c2f35ecd4898ac605787f729aff48fa8dd46097 Parents: f903006 Author: max-orlovAuthored: Mon Jul 10 17:12:00 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 17:22:29 2017 +0300 -- aria/utils/validation.py | 2 +- tests/utils/test_validation.py | 35 +++ 2 files changed, 36 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/6c2f35ec/aria/utils/validation.py -- diff --git a/aria/utils/validation.py b/aria/utils/validation.py index 3452dcc..06989a7 100644 --- a/aria/utils/validation.py +++ b/aria/utils/validation.py @@ -78,7 +78,7 @@ def validate_function_arguments(func, func_kwargs): # all args without the ones with default values args = func.func_code.co_varnames[:args_count] -non_default_args = args[:len(func.func_defaults)] if func.func_defaults else args +non_default_args = args[:len(args) - len(func.func_defaults)] if func.func_defaults else args # Check if any args without default values is missing in the func_kwargs for arg in non_default_args: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/6c2f35ec/tests/utils/test_validation.py -- diff --git a/tests/utils/test_validation.py b/tests/utils/test_validation.py new file mode 100644 index 000..8e35f22 --- /dev/null +++ b/tests/utils/test_validation.py @@ -0,0 +1,35 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import pytest + +from aria.utils import validation + + +def test_function_kwargs_validation(): + +def mock_function(arg1, arg2=1, arg3=1): +pass + +with pytest.raises(ValueError): +validation.validate_function_arguments(mock_function, dict(arg2=1)) +with pytest.raises(ValueError): +validation.validate_function_arguments(mock_function, dict(arg3=3)) +with pytest.raises(ValueError): +validation.validate_function_arguments(mock_function, dict(arg2=2, arg3=3)) + +validation.validate_function_arguments(mock_function, dict(arg1=1, arg3=3)) +validation.validate_function_arguments(mock_function, dict(arg1=1, arg2=2)) +validation.validate_function_arguments(mock_function, dict(arg1=1, arg2=2, arg3=3))
[1/3] incubator-ariatosca git commit: ARIA-103 Remove dependency on Clint [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions ef1419cd0 -> 123a55ce7 (forced update) ARIA-103 Remove dependency on Clint We no longer require this third-party library, instead the utils/console module uses the existing cli/color module. This commit also fixes the cli/color module to properly support Unicode, and also properly deinitialize Colorama in Windows. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/f903006b Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/f903006b Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/f903006b Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: f903006b013fdc9c77b7be42a915dfb72fb16b96 Parents: b30a7ed Author: Tal LironAuthored: Mon Jul 10 12:28:23 2017 +0300 Committer: Tal Liron Committed: Mon Jul 10 15:56:01 2017 +0300 -- aria/cli/color.py | 21 -- aria/utils/console.py | 53 +++--- requirements.in | 1 - requirements.txt | 4 +--- 4 files changed, 52 insertions(+), 27 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f903006b/aria/cli/color.py -- diff --git a/aria/cli/color.py b/aria/cli/color.py index 03381ba..d6a4cd6 100644 --- a/aria/cli/color.py +++ b/aria/cli/color.py @@ -18,11 +18,20 @@ Terminal colorization utilities. """ from StringIO import StringIO +import atexit import re import colorama +from ..utils.formatting import safe_str + + +def _restore_terminal(): +colorama.deinit() + + colorama.init() +atexit.register(_restore_terminal) class StringStylizer(object): @@ -33,20 +42,20 @@ class StringStylizer(object): def __repr__(self): if self._color_spec: return '{schema}{str}{reset}'.format( -schema=self._color_spec, str=str(self._str), reset=Colors.Style.RESET_ALL) +schema=self._color_spec, str=safe_str(self._str), reset=Colors.Style.RESET_ALL) return self._str def __add__(self, other): -return str(self) + other +return safe_str(self) + other def __radd__(self, other): -return other + str(self) +return other + safe_str(self) def color(self, color_spec): self._color_spec = color_spec def replace(self, old, new, **kwargs): -self._str = self._str.replace(str(old), str(new), **kwargs) +self._str = self._str.replace(safe_str(old), safe_str(new), **kwargs) def format(self, *args, **kwargs): self._str = self._str.format(*args, **kwargs) @@ -79,8 +88,8 @@ class Colors(object): class ColorSpec(object): def __init__(self, fore=None, back=None, style=None): """ -It is possible to provide fore, back and style arguments. each could be either -the color is lower case letter, or the actual color from Colorama. +It is possible to provide fore, back and style arguments. Each could be either the color as +lowercase letters, or the full color name for Colorama. """ self._kwargs = dict(fore=fore, back=back, style=style) self._str = StringIO() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f903006b/aria/utils/console.py -- diff --git a/aria/utils/console.py b/aria/utils/console.py index 642cbb1..2f6f622 100644 --- a/aria/utils/console.py +++ b/aria/utils/console.py @@ -17,52 +17,71 @@ Abstraction API above terminal color libraries. """ -from clint.textui.core import STDOUT -from clint.textui import puts as _puts -from clint.textui.colored import ColoredString as _ColoredString -from clint.textui import indent # pylint: disable=unused-import +import os +import sys + +from contextlib import contextmanager from .formatting import safe_str +from ..cli import color + + +_indent_string = '' -class ColoredString(_ColoredString): -def __init__(self, color, str_, always_color=False, bold=False): -super(ColoredString, self).__init__(color, safe_str(str_), always_color, bold) +def puts(string='', newline=True, stream=sys.stdout): +stream.write(_indent_string) +stream.write(safe_str(string)) +if newline: +stream.write(os.linesep) -def puts(string='', newline=True, stream=STDOUT): -_puts(safe_str(string), newline, stream) +@contextmanager +def indent(size=4): +global _indent_string +original_indent_string = _indent_string +try: +_indent_string += ' ' * size +yield +
[GitHub] incubator-ariatosca issue #183: ARIA-76 Parallelize PyTest
Github user asfgit commented on the issue: https://github.com/apache/incubator-ariatosca/pull/183 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca pull request #183: ARIA-76 Parallelize PyTest
GitHub user tliron opened a pull request: https://github.com/apache/incubator-ariatosca/pull/183 ARIA-76 Parallelize PyTest Use the PyTest xdist plugin to parallelize tests in boxed subprocesses. Through benchmarking we discovered that paralellizing on the number of CPU cores ("-n auto") provides the best times. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-76-parallel-pytest Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/183.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #183 commit b2310536e00894ec781cd12ee20a620257cbb437 Author: Tal LironDate: 2017-07-10T15:06:05Z ARIA-76 Parallelize PyTest Use the PyTest xdist plugin to parallelize tests in boxed subprocesses. Through benchmarking we discovered that paralellizing on the number of CPU cores ("-n auto") provides the best times. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[3/3] incubator-ariatosca git commit: ARIA-237 Support for resuming failed workflow executions
ARIA-237 Support for resuming failed workflow executions Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/123a55ce Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/123a55ce Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/123a55ce Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: 123a55ce7f111f4bd113b1c4e8af8fd1e85b295e Parents: 6c2f35e Author: max-orlovAuthored: Sun Jul 2 21:43:43 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 18:01:44 2017 +0300 -- aria/modeling/orchestration.py | 4 +- aria/orchestrator/workflow_runner.py| 7 +- aria/orchestrator/workflows/core/engine.py | 18 ++- .../workflows/core/events_handler.py| 9 +- tests/modeling/test_models.py | 3 +- tests/orchestrator/test_workflow_runner.py | 135 +-- 6 files changed, 125 insertions(+), 51 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/123a55ce/aria/modeling/orchestration.py -- diff --git a/aria/modeling/orchestration.py b/aria/modeling/orchestration.py index df2643e..4d4f0fe 100644 --- a/aria/modeling/orchestration.py +++ b/aria/modeling/orchestration.py @@ -65,7 +65,9 @@ class ExecutionBase(mixins.ModelMixin): PENDING: (STARTED, CANCELLED), STARTED: END_STATES + (CANCELLING,), CANCELLING: END_STATES, -CANCELLED: PENDING +# Retrying +CANCELLED: PENDING, +FAILED: PENDING } # region one_to_many relationships http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/123a55ce/aria/orchestrator/workflow_runner.py -- diff --git a/aria/orchestrator/workflow_runner.py b/aria/orchestrator/workflow_runner.py index 47270c0..2bd3043 100644 --- a/aria/orchestrator/workflow_runner.py +++ b/aria/orchestrator/workflow_runner.py @@ -38,7 +38,8 @@ DEFAULT_TASK_RETRY_INTERVAL = 30 class WorkflowRunner(object): def __init__(self, model_storage, resource_storage, plugin_manager, - execution_id=None, service_id=None, workflow_name=None, inputs=None, executor=None, + execution_id=None, retry_failed=False, + service_id=None, workflow_name=None, inputs=None, executor=None, task_max_attempts=DEFAULT_TASK_MAX_ATTEMPTS, task_retry_interval=DEFAULT_TASK_RETRY_INTERVAL): """ @@ -62,6 +63,7 @@ class WorkflowRunner(object): "and service id with inputs") self._is_resume = execution_id is not None +self._retry_failed = retry_failed self._model_storage = model_storage self._resource_storage = resource_storage @@ -116,7 +118,8 @@ class WorkflowRunner(object): return self._model_storage.service.get(self._service_id) def execute(self): -self._engine.execute(ctx=self._workflow_context, resuming=self._is_resume) +self._engine.execute( +ctx=self._workflow_context, resuming=self._is_resume, retry_failed=self._retry_failed) def cancel(self): self._engine.cancel_execution(ctx=self._workflow_context) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/123a55ce/aria/orchestrator/workflows/core/engine.py -- diff --git a/aria/orchestrator/workflows/core/engine.py b/aria/orchestrator/workflows/core/engine.py index d9c77e9..69505fc 100644 --- a/aria/orchestrator/workflows/core/engine.py +++ b/aria/orchestrator/workflows/core/engine.py @@ -41,14 +41,15 @@ class Engine(logger.LoggerMixin): self._executors = executors.copy() self._executors.setdefault(StubTaskExecutor, StubTaskExecutor()) -def execute(self, ctx, resuming=False): +def execute(self, ctx, resuming=False, retry_failed=False): """ Executes the workflow. """ if resuming: -events.on_resume_workflow_signal.send(ctx) +events.on_resume_workflow_signal.send(ctx, retry_failed=retry_failed) tasks_tracker = _TasksTracker(ctx) + try: events.start_workflow_signal.send(ctx) while True: @@ -68,8 +69,15 @@ class Engine(logger.LoggerMixin): if cancel: self._terminate_tasks(tasks_tracker.executing_tasks) events.on_cancelled_workflow_signal.send(ctx) -else: +elif all(task.status == task.SUCCESS or task.ignore_failure + for
[incubator-ariatosca] Git Push Summary
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-312-Validation-of-workflow-and-operation-kwargs-raise-False-alarms [deleted] 6c2f35ecd
[1/2] incubator-ariatosca git commit: ARIA-312 Validation of workflow and operation kwargs raise False alarms [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/0.1.1-rc 328da3f06 -> 8e4f4e24e (forced update) ARIA-312 Validation of workflow and operation kwargs raise False alarms Workflow and Operation function kwargs validation failed in some scenarios. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/615868c1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/615868c1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/615868c1 Branch: refs/heads/0.1.1-rc Commit: 615868c1ddd569114f5d2eee81851059cca1af7a Parents: c63059a Author: max-orlovAuthored: Mon Jul 10 17:12:00 2017 +0300 Committer: Ran Ziv Committed: Mon Jul 10 18:02:32 2017 +0300 -- aria/utils/validation.py | 2 +- tests/utils/test_validation.py | 35 +++ 2 files changed, 36 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/615868c1/aria/utils/validation.py -- diff --git a/aria/utils/validation.py b/aria/utils/validation.py index 3452dcc..06989a7 100644 --- a/aria/utils/validation.py +++ b/aria/utils/validation.py @@ -78,7 +78,7 @@ def validate_function_arguments(func, func_kwargs): # all args without the ones with default values args = func.func_code.co_varnames[:args_count] -non_default_args = args[:len(func.func_defaults)] if func.func_defaults else args +non_default_args = args[:len(args) - len(func.func_defaults)] if func.func_defaults else args # Check if any args without default values is missing in the func_kwargs for arg in non_default_args: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/615868c1/tests/utils/test_validation.py -- diff --git a/tests/utils/test_validation.py b/tests/utils/test_validation.py new file mode 100644 index 000..8e35f22 --- /dev/null +++ b/tests/utils/test_validation.py @@ -0,0 +1,35 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import pytest + +from aria.utils import validation + + +def test_function_kwargs_validation(): + +def mock_function(arg1, arg2=1, arg3=1): +pass + +with pytest.raises(ValueError): +validation.validate_function_arguments(mock_function, dict(arg2=1)) +with pytest.raises(ValueError): +validation.validate_function_arguments(mock_function, dict(arg3=3)) +with pytest.raises(ValueError): +validation.validate_function_arguments(mock_function, dict(arg2=2, arg3=3)) + +validation.validate_function_arguments(mock_function, dict(arg1=1, arg3=3)) +validation.validate_function_arguments(mock_function, dict(arg1=1, arg2=2)) +validation.validate_function_arguments(mock_function, dict(arg1=1, arg2=2, arg3=3))
[GitHub] incubator-ariatosca pull request #182: ARIA-312 Validation of workflow and o...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/182 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
incubator-ariatosca git commit: removed operations and workflows, and fixed minor issue in validation.py [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions 1ad2a0db6 -> ef1419cd0 (forced update) removed operations and workflows, and fixed minor issue in validation.py Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/ef1419cd Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/ef1419cd Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/ef1419cd Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: ef1419cd068ca109f2493f940c6d15e11964c9c4 Parents: 91cbb78 Author: max-orlovAuthored: Mon Jul 10 16:59:03 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 17:28:23 2017 +0300 -- aria/utils/validation.py | 2 +- tests/orchestrator/test_workflow_runner.py | 82 ++--- 2 files changed, 32 insertions(+), 52 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ef1419cd/aria/utils/validation.py -- diff --git a/aria/utils/validation.py b/aria/utils/validation.py index 3452dcc..0c2af10 100644 --- a/aria/utils/validation.py +++ b/aria/utils/validation.py @@ -78,7 +78,7 @@ def validate_function_arguments(func, func_kwargs): # all args without the ones with default values args = func.func_code.co_varnames[:args_count] -non_default_args = args[:len(func.func_defaults)] if func.func_defaults else args +non_default_args = args[:len(func.func_defaults) - 1] if func.func_defaults else args # Check if any args without default values is missing in the func_kwargs for arg in non_default_args: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/ef1419cd/tests/orchestrator/test_workflow_runner.py -- diff --git a/tests/orchestrator/test_workflow_runner.py b/tests/orchestrator/test_workflow_runner.py index 112f894..adb19e6 100644 --- a/tests/orchestrator/test_workflow_runner.py +++ b/tests/orchestrator/test_workflow_runner.py @@ -359,10 +359,11 @@ class TestResumableWorkflows(object): def test_resume_workflow(self, workflow_context, thread_executor): node = workflow_context.model.node.get_by_name(tests_mock.models.DEPENDENCY_NODE_NAME) node.attributes['invocations'] = models.Attribute.wrap('invocations', 0) -self._create_interface(workflow_context, node, mock_resuming_task) +self._create_interface(workflow_context, node, mock_failed_task) wf_runner = self._create_initial_workflow_runner( -workflow_context, mock_two_parallel_tasks_workflow, thread_executor) +workflow_context, mock_parallel_tasks_workflow, thread_executor, +inputs={'number_of_tasks': 2}) wf_thread = Thread(target=wf_runner.execute) wf_thread.daemon = True @@ -403,7 +404,8 @@ class TestResumableWorkflows(object): self._create_interface(workflow_context, node, mock_stuck_task) wf_runner = self._create_initial_workflow_runner( -workflow_context, mock_single_task_workflow, thread_executor) +workflow_context, mock_parallel_tasks_workflow, thread_executor, +inputs={'number_of_tasks': 1}) wf_thread = Thread(target=wf_runner.execute) wf_thread.daemon = True @@ -444,8 +446,9 @@ class TestResumableWorkflows(object): node.attributes['invocations'] = models.Attribute.wrap('invocations', 0) self._create_interface(workflow_context, node, mock_failed_before_resuming) -wf_runner = self._create_initial_workflow_runner( -workflow_context, mock_single_task_workflow, thread_executor) +wf_runner = self._create_initial_workflow_runner(workflow_context, + mock_parallel_tasks_workflow, + thread_executor) wf_thread = Thread(target=wf_runner.execute) wf_thread.setDaemon(True) wf_thread.start() @@ -488,10 +491,14 @@ class TestResumableWorkflows(object): def test_resume_failed_task_and_successful_task(self, workflow_context, thread_executor): node = workflow_context.model.node.get_by_name(tests_mock.models.DEPENDENCY_NODE_NAME) node.attributes['invocations'] = models.Attribute.wrap('invocations', 0) -self._create_interface(workflow_context, node, mock_two_different_tasks) +self._create_interface(workflow_context, node, mock_failed_task) wf_runner = self._create_initial_workflow_runner( -
[incubator-ariatosca] Git Push Summary [forced push!] [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions f07e95edd -> 1ad2a0db6 (forced update)
[GitHub] incubator-ariatosca issue #182: ARIA-312 Validation of workflow and operatio...
Github user asfgit commented on the issue: https://github.com/apache/incubator-ariatosca/pull/182 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[5/8] incubator-ariatosca git commit: ARIA-287 Add tox environment for docs
ARIA-287 Add tox environment for docs Also adds the environment to "make test". Involves fixing Sphinx to properly exclude SSH documentation when Fabric is not installed. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/f0bd8b4b Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/f0bd8b4b Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/f0bd8b4b Branch: refs/heads/0.1.1-rc Commit: f0bd8b4b5d454934f4bc30e725404fca30ad25b8 Parents: c22166c Author: Tal LironAuthored: Fri Jun 30 17:08:23 2017 -0500 Committer: Ran Ziv Committed: Mon Jul 10 17:00:34 2017 +0300 -- Makefile | 9 +++-- docs/conf.py | 33 + tox.ini | 28 ++-- 3 files changed, 54 insertions(+), 16 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f0bd8b4b/Makefile -- diff --git a/Makefile b/Makefile index 71a1968..a894747 100644 --- a/Makefile +++ b/Makefile @@ -47,11 +47,16 @@ install-virtual: docs: pip install --requirement "$(DOCS)/requirements.txt" rm -rf "$(HTML)" - sphinx-build -b html "$(DOCS)" "$(HTML)" + sphinx-build -W -T -b html "$(DOCS)" "$(HTML)" test: pip install --upgrade "tox>=2.7.0" - tox -e pylint_code -e pylint_tests -e py$(PYTHON_VERSION) -e py$(PYTHON_VERSION)e2e -e py$(PYTHON_VERSION)ssh + tox -e pylint_code \ + -e pylint_tests \ + -e py$(PYTHON_VERSION) \ + -e py$(PYTHON_VERSION)e2e \ + -e py$(PYTHON_VERSION)ssh \ + -e docs dist: docs python ./setup.py sdist bdist_wheel http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f0bd8b4b/docs/conf.py -- diff --git a/docs/conf.py b/docs/conf.py index 6361621..fd1a066 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -390,8 +390,20 @@ NEVER_SKIP_MEMBERS = ( '__evaluate__', ) -# 'autodoc-skip-member' event -def on_skip_member(app, what, name, obj, skip, options): +SKIP_DOCUMENTS = () + +from sphinx import addnodes +from sphinx.domains.python import PythonDomain + +try: +import fabric +except: +# Note: "exclude_patterns" is not good enough for us, because we still have a TOC entry. +# Unfortunately, there is no way to conditionally exclude a TOC entry, and TOC entries without +# matching documents emit an error. So, we will have to manipulate the doctree directly! +SKIP_DOCUMENTS = ('aria.orchestrator.execution_plugin.ssh',) + +def on_autodoc_skip_member(app, what, name, obj, skip, options): if name in NEVER_SKIP_MEMBERS: return False if name in SKIP_MEMBERS: @@ -401,7 +413,18 @@ def on_skip_member(app, what, name, obj, skip, options): return True return skip -from sphinx.domains.python import PythonDomain +def on_source_read(app, docname, source): +# Empty out source +if docname in SKIP_DOCUMENTS: +source[0] = '' + +def on_doctree_read(app, doctree): +# Remove TOC entry (see: https://gist.github.com/kakawait/9215487) +for toctreenode in doctree.traverse(addnodes.toctree): +for e in toctreenode['entries']: +ref = str(e[1]) +if ref in SKIP_DOCUMENTS: +toctreenode['entries'].remove(e) class PatchedPythonDomain(PythonDomain): # See: https://github.com/sphinx-doc/sphinx/issues/3866 @@ -412,5 +435,7 @@ class PatchedPythonDomain(PythonDomain): env, fromdocname, builder, typ, target, node, contnode) def setup(app): -app.connect('autodoc-skip-member', on_skip_member) +app.connect('autodoc-skip-member', on_autodoc_skip_member) +app.connect('source-read', on_source_read) +app.connect('doctree-read', on_doctree_read) app.override_domain(PatchedPythonDomain) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f0bd8b4b/tox.ini -- diff --git a/tox.ini b/tox.ini index 3e1fb3c..9849b5e 100644 --- a/tox.ini +++ b/tox.ini @@ -11,18 +11,19 @@ # limitations under the License. [tox] -envlist=py27,py26,py27e2e,py26e2e,pywin,py27ssh,pylint_code,pylint_tests +envlist=py27,py26,py27e2e,py26e2e,pywin,py27ssh,pylint_code,pylint_tests,docs [testenv] -passenv = -TRAVIS -PYTHON -PYTHON_VERSION -PYTHON_ARCH -deps = --rrequirements.txt --rtests/requirements.txt -basepython = +whitelist_externals=rm +passenv= + TRAVIS + PYTHON + PYTHON_VERSION + PYTHON_ARCH +deps= + -rrequirements.txt + -rtests/requirements.txt +basepython= py26: python2.6
[7/8] incubator-ariatosca git commit: ARIA-299 Resuming canceled execution with frozen task fails
ARIA-299 Resuming canceled execution with frozen task fails Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c63059a2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c63059a2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c63059a2 Branch: refs/heads/0.1.1-rc Commit: c63059a2061927c10cc32571c161ea3faf9f933a Parents: 9de6bf5 Author: max-orlovAuthored: Wed Jul 5 16:16:39 2017 +0300 Committer: Ran Ziv Committed: Mon Jul 10 17:00:46 2017 +0300 -- .../workflows/core/events_handler.py| 4 + aria/orchestrator/workflows/executor/base.py| 2 +- aria/orchestrator/workflows/executor/thread.py | 8 +- tests/orchestrator/context/test_serialize.py| 6 +- .../orchestrator/execution_plugin/test_local.py | 6 +- tests/orchestrator/execution_plugin/test_ssh.py | 6 +- tests/orchestrator/test_workflow_runner.py | 206 --- .../workflows/executor/test_process_executor.py | 66 +++--- ...process_executor_concurrent_modifications.py | 6 +- .../executor/test_process_executor_extension.py | 6 +- .../test_process_executor_tracked_changes.py| 6 +- 11 files changed, 251 insertions(+), 71 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c63059a2/aria/orchestrator/workflows/core/events_handler.py -- diff --git a/aria/orchestrator/workflows/core/events_handler.py b/aria/orchestrator/workflows/core/events_handler.py index 769c1a8..37801de 100644 --- a/aria/orchestrator/workflows/core/events_handler.py +++ b/aria/orchestrator/workflows/core/events_handler.py @@ -123,6 +123,10 @@ def _workflow_resume(workflow_context, *args, **kwargs): with workflow_context.persist_changes: execution = workflow_context.execution execution.status = execution.PENDING +# Any non ended task would be put back to pending state +for task in execution.tasks: +if not task.has_ended(): +task.status = task.PENDING @events.on_cancelling_workflow_signal.connect http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c63059a2/aria/orchestrator/workflows/executor/base.py -- diff --git a/aria/orchestrator/workflows/executor/base.py b/aria/orchestrator/workflows/executor/base.py index ec1a0c7..e7d03ea 100644 --- a/aria/orchestrator/workflows/executor/base.py +++ b/aria/orchestrator/workflows/executor/base.py @@ -49,7 +49,7 @@ class BaseExecutor(logger.LoggerMixin): """ pass -def terminate(self, ctx): +def terminate(self, task_id): """ Terminate the executing task :return: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c63059a2/aria/orchestrator/workflows/executor/thread.py -- diff --git a/aria/orchestrator/workflows/executor/thread.py b/aria/orchestrator/workflows/executor/thread.py index d9dcdf8..6cef2c0 100644 --- a/aria/orchestrator/workflows/executor/thread.py +++ b/aria/orchestrator/workflows/executor/thread.py @@ -36,9 +36,10 @@ class ThreadExecutor(BaseExecutor): Note: This executor is incapable of running plugin operations. """ -def __init__(self, pool_size=1, *args, **kwargs): +def __init__(self, pool_size=1, close_timeout=5, *args, **kwargs): super(ThreadExecutor, self).__init__(*args, **kwargs) self._stopped = False +self._close_timeout = close_timeout self._queue = Queue.Queue() self._pool = [] for i in range(pool_size): @@ -54,7 +55,10 @@ class ThreadExecutor(BaseExecutor): def close(self): self._stopped = True for thread in self._pool: -thread.join() +if self._close_timeout is None: +thread.join() +else: +thread.join(self._close_timeout) def _processor(self): while not self._stopped: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c63059a2/tests/orchestrator/context/test_serialize.py -- diff --git a/tests/orchestrator/context/test_serialize.py b/tests/orchestrator/context/test_serialize.py index 6046a16..091e23c 100644 --- a/tests/orchestrator/context/test_serialize.py +++ b/tests/orchestrator/context/test_serialize.py @@ -87,8 +87,10 @@ def _operation_mapping(): @pytest.fixture def executor(): result = process.ProcessExecutor(python_path=[tests.ROOT_DIR]) -yield result -result.close() +try: +
incubator-ariatosca git commit: removed operations and workflows, and fixed minor issue in validation.py
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions 91cbb78d3 -> 1ad2a0db6 removed operations and workflows, and fixed minor issue in validation.py Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/1ad2a0db Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/1ad2a0db Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/1ad2a0db Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: 1ad2a0db6f9c2c49f5bef9b3af7568cb4213712c Parents: 91cbb78 Author: max-orlovAuthored: Mon Jul 10 16:59:03 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 16:59:03 2017 +0300 -- aria/utils/validation.py | 2 +- tests/orchestrator/test_workflow_runner.py | 82 ++--- 2 files changed, 32 insertions(+), 52 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/1ad2a0db/aria/utils/validation.py -- diff --git a/aria/utils/validation.py b/aria/utils/validation.py index 3452dcc..0c2af10 100644 --- a/aria/utils/validation.py +++ b/aria/utils/validation.py @@ -78,7 +78,7 @@ def validate_function_arguments(func, func_kwargs): # all args without the ones with default values args = func.func_code.co_varnames[:args_count] -non_default_args = args[:len(func.func_defaults)] if func.func_defaults else args +non_default_args = args[:len(func.func_defaults) - 1] if func.func_defaults else args # Check if any args without default values is missing in the func_kwargs for arg in non_default_args: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/1ad2a0db/tests/orchestrator/test_workflow_runner.py -- diff --git a/tests/orchestrator/test_workflow_runner.py b/tests/orchestrator/test_workflow_runner.py index 112f894..adb19e6 100644 --- a/tests/orchestrator/test_workflow_runner.py +++ b/tests/orchestrator/test_workflow_runner.py @@ -359,10 +359,11 @@ class TestResumableWorkflows(object): def test_resume_workflow(self, workflow_context, thread_executor): node = workflow_context.model.node.get_by_name(tests_mock.models.DEPENDENCY_NODE_NAME) node.attributes['invocations'] = models.Attribute.wrap('invocations', 0) -self._create_interface(workflow_context, node, mock_resuming_task) +self._create_interface(workflow_context, node, mock_failed_task) wf_runner = self._create_initial_workflow_runner( -workflow_context, mock_two_parallel_tasks_workflow, thread_executor) +workflow_context, mock_parallel_tasks_workflow, thread_executor, +inputs={'number_of_tasks': 2}) wf_thread = Thread(target=wf_runner.execute) wf_thread.daemon = True @@ -403,7 +404,8 @@ class TestResumableWorkflows(object): self._create_interface(workflow_context, node, mock_stuck_task) wf_runner = self._create_initial_workflow_runner( -workflow_context, mock_single_task_workflow, thread_executor) +workflow_context, mock_parallel_tasks_workflow, thread_executor, +inputs={'number_of_tasks': 1}) wf_thread = Thread(target=wf_runner.execute) wf_thread.daemon = True @@ -444,8 +446,9 @@ class TestResumableWorkflows(object): node.attributes['invocations'] = models.Attribute.wrap('invocations', 0) self._create_interface(workflow_context, node, mock_failed_before_resuming) -wf_runner = self._create_initial_workflow_runner( -workflow_context, mock_single_task_workflow, thread_executor) +wf_runner = self._create_initial_workflow_runner(workflow_context, + mock_parallel_tasks_workflow, + thread_executor) wf_thread = Thread(target=wf_runner.execute) wf_thread.setDaemon(True) wf_thread.start() @@ -488,10 +491,14 @@ class TestResumableWorkflows(object): def test_resume_failed_task_and_successful_task(self, workflow_context, thread_executor): node = workflow_context.model.node.get_by_name(tests_mock.models.DEPENDENCY_NODE_NAME) node.attributes['invocations'] = models.Attribute.wrap('invocations', 0) -self._create_interface(workflow_context, node, mock_two_different_tasks) +self._create_interface(workflow_context, node, mock_failed_task) wf_runner = self._create_initial_workflow_runner( -workflow_context,
incubator-ariatosca git commit: ARIA-103 Remove dependency on Clint
Repository: incubator-ariatosca Updated Branches: refs/heads/master b30a7edd8 -> f903006b0 ARIA-103 Remove dependency on Clint We no longer require this third-party library, instead the utils/console module uses the existing cli/color module. This commit also fixes the cli/color module to properly support Unicode, and also properly deinitialize Colorama in Windows. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/f903006b Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/f903006b Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/f903006b Branch: refs/heads/master Commit: f903006b013fdc9c77b7be42a915dfb72fb16b96 Parents: b30a7ed Author: Tal LironAuthored: Mon Jul 10 12:28:23 2017 +0300 Committer: Tal Liron Committed: Mon Jul 10 15:56:01 2017 +0300 -- aria/cli/color.py | 21 -- aria/utils/console.py | 53 +++--- requirements.in | 1 - requirements.txt | 4 +--- 4 files changed, 52 insertions(+), 27 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f903006b/aria/cli/color.py -- diff --git a/aria/cli/color.py b/aria/cli/color.py index 03381ba..d6a4cd6 100644 --- a/aria/cli/color.py +++ b/aria/cli/color.py @@ -18,11 +18,20 @@ Terminal colorization utilities. """ from StringIO import StringIO +import atexit import re import colorama +from ..utils.formatting import safe_str + + +def _restore_terminal(): +colorama.deinit() + + colorama.init() +atexit.register(_restore_terminal) class StringStylizer(object): @@ -33,20 +42,20 @@ class StringStylizer(object): def __repr__(self): if self._color_spec: return '{schema}{str}{reset}'.format( -schema=self._color_spec, str=str(self._str), reset=Colors.Style.RESET_ALL) +schema=self._color_spec, str=safe_str(self._str), reset=Colors.Style.RESET_ALL) return self._str def __add__(self, other): -return str(self) + other +return safe_str(self) + other def __radd__(self, other): -return other + str(self) +return other + safe_str(self) def color(self, color_spec): self._color_spec = color_spec def replace(self, old, new, **kwargs): -self._str = self._str.replace(str(old), str(new), **kwargs) +self._str = self._str.replace(safe_str(old), safe_str(new), **kwargs) def format(self, *args, **kwargs): self._str = self._str.format(*args, **kwargs) @@ -79,8 +88,8 @@ class Colors(object): class ColorSpec(object): def __init__(self, fore=None, back=None, style=None): """ -It is possible to provide fore, back and style arguments. each could be either -the color is lower case letter, or the actual color from Colorama. +It is possible to provide fore, back and style arguments. Each could be either the color as +lowercase letters, or the full color name for Colorama. """ self._kwargs = dict(fore=fore, back=back, style=style) self._str = StringIO() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f903006b/aria/utils/console.py -- diff --git a/aria/utils/console.py b/aria/utils/console.py index 642cbb1..2f6f622 100644 --- a/aria/utils/console.py +++ b/aria/utils/console.py @@ -17,52 +17,71 @@ Abstraction API above terminal color libraries. """ -from clint.textui.core import STDOUT -from clint.textui import puts as _puts -from clint.textui.colored import ColoredString as _ColoredString -from clint.textui import indent # pylint: disable=unused-import +import os +import sys + +from contextlib import contextmanager from .formatting import safe_str +from ..cli import color + + +_indent_string = '' -class ColoredString(_ColoredString): -def __init__(self, color, str_, always_color=False, bold=False): -super(ColoredString, self).__init__(color, safe_str(str_), always_color, bold) +def puts(string='', newline=True, stream=sys.stdout): +stream.write(_indent_string) +stream.write(safe_str(string)) +if newline: +stream.write(os.linesep) -def puts(string='', newline=True, stream=STDOUT): -_puts(safe_str(string), newline, stream) +@contextmanager +def indent(size=4): +global _indent_string +original_indent_string = _indent_string +try: +_indent_string += ' ' * size +yield +finally: +_indent_string = original_indent_string class Colored(object): @staticmethod def
[GitHub] incubator-ariatosca pull request #181: ARIA-103 Remove dependency on Clint
Github user asfgit closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/181 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[3/5] incubator-ariatosca git commit: ARIA-305 Advance ARIA version
ARIA-305 Advance ARIA version Set package version to 0.2.0 Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c46c94bd Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c46c94bd Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c46c94bd Branch: refs/heads/ARIA-103-remove-clint-dependency Commit: c46c94bd722541492f9cff5e37741d864a74cb41 Parents: 2089655 Author: Ran ZivAuthored: Mon Jul 10 12:36:08 2017 +0300 Committer: Ran Ziv Committed: Mon Jul 10 12:40:49 2017 +0300 -- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c46c94bd/VERSION -- diff --git a/VERSION b/VERSION index 6c6aa7c..341cf11 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.0 \ No newline at end of file +0.2.0 \ No newline at end of file
[5/5] incubator-ariatosca git commit: ARIA-103 Remove dependency on Clint
ARIA-103 Remove dependency on Clint We no longer require this third-party library, instead the utils/console module uses the existing cli/color module. This commit also fixes the cli/color module to properly support Unicode, and also properly deinitialize Colorama in Windows. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/f903006b Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/f903006b Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/f903006b Branch: refs/heads/ARIA-103-remove-clint-dependency Commit: f903006b013fdc9c77b7be42a915dfb72fb16b96 Parents: b30a7ed Author: Tal LironAuthored: Mon Jul 10 12:28:23 2017 +0300 Committer: Tal Liron Committed: Mon Jul 10 15:56:01 2017 +0300 -- aria/cli/color.py | 21 -- aria/utils/console.py | 53 +++--- requirements.in | 1 - requirements.txt | 4 +--- 4 files changed, 52 insertions(+), 27 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f903006b/aria/cli/color.py -- diff --git a/aria/cli/color.py b/aria/cli/color.py index 03381ba..d6a4cd6 100644 --- a/aria/cli/color.py +++ b/aria/cli/color.py @@ -18,11 +18,20 @@ Terminal colorization utilities. """ from StringIO import StringIO +import atexit import re import colorama +from ..utils.formatting import safe_str + + +def _restore_terminal(): +colorama.deinit() + + colorama.init() +atexit.register(_restore_terminal) class StringStylizer(object): @@ -33,20 +42,20 @@ class StringStylizer(object): def __repr__(self): if self._color_spec: return '{schema}{str}{reset}'.format( -schema=self._color_spec, str=str(self._str), reset=Colors.Style.RESET_ALL) +schema=self._color_spec, str=safe_str(self._str), reset=Colors.Style.RESET_ALL) return self._str def __add__(self, other): -return str(self) + other +return safe_str(self) + other def __radd__(self, other): -return other + str(self) +return other + safe_str(self) def color(self, color_spec): self._color_spec = color_spec def replace(self, old, new, **kwargs): -self._str = self._str.replace(str(old), str(new), **kwargs) +self._str = self._str.replace(safe_str(old), safe_str(new), **kwargs) def format(self, *args, **kwargs): self._str = self._str.format(*args, **kwargs) @@ -79,8 +88,8 @@ class Colors(object): class ColorSpec(object): def __init__(self, fore=None, back=None, style=None): """ -It is possible to provide fore, back and style arguments. each could be either -the color is lower case letter, or the actual color from Colorama. +It is possible to provide fore, back and style arguments. Each could be either the color as +lowercase letters, or the full color name for Colorama. """ self._kwargs = dict(fore=fore, back=back, style=style) self._str = StringIO() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/f903006b/aria/utils/console.py -- diff --git a/aria/utils/console.py b/aria/utils/console.py index 642cbb1..2f6f622 100644 --- a/aria/utils/console.py +++ b/aria/utils/console.py @@ -17,52 +17,71 @@ Abstraction API above terminal color libraries. """ -from clint.textui.core import STDOUT -from clint.textui import puts as _puts -from clint.textui.colored import ColoredString as _ColoredString -from clint.textui import indent # pylint: disable=unused-import +import os +import sys + +from contextlib import contextmanager from .formatting import safe_str +from ..cli import color + + +_indent_string = '' -class ColoredString(_ColoredString): -def __init__(self, color, str_, always_color=False, bold=False): -super(ColoredString, self).__init__(color, safe_str(str_), always_color, bold) +def puts(string='', newline=True, stream=sys.stdout): +stream.write(_indent_string) +stream.write(safe_str(string)) +if newline: +stream.write(os.linesep) -def puts(string='', newline=True, stream=STDOUT): -_puts(safe_str(string), newline, stream) +@contextmanager +def indent(size=4): +global _indent_string +original_indent_string = _indent_string +try: +_indent_string += ' ' * size +yield +finally: +_indent_string = original_indent_string class Colored(object): @staticmethod def black(string, always=False, bold=False): -return
[1/5] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-103-remove-clint-dependency d81d734da -> f903006b0 (forced update) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/modeling/test_models.py -- diff --git a/tests/modeling/test_models.py b/tests/modeling/test_models.py index bbc7352..e1167fc 100644 --- a/tests/modeling/test_models.py +++ b/tests/modeling/test_models.py @@ -509,19 +509,15 @@ class TestServiceModification(object): class TestNodeTemplate(object): @pytest.mark.parametrize( -'is_valid, name, default_instances, max_instances, min_instances, properties', +'is_valid, name, properties', [ -(False, m_cls, 1, 1, 1, {}), -(False, 'name', m_cls, 1, 1, {}), -(False, 'name', 1, m_cls, 1, {}), -(False, 'name', 1, 1, m_cls, {}), -(False, 'name', 1, 1, 1, m_cls), +(False, m_cls, {}), +(False, 'name', m_cls), -(True, 'name', 1, 1, 1, {}), +(True, 'name', {}), ] ) -def test_node_template_model_creation(self, service_storage, is_valid, name, default_instances, - max_instances, min_instances, properties): +def test_node_template_model_creation(self, service_storage, is_valid, name, properties): node_template = _test_model( is_valid=is_valid, storage=service_storage, @@ -529,9 +525,6 @@ class TestNodeTemplate(object): model_kwargs=dict( name=name, type=service_storage.type.list()[0], -default_instances=default_instances, -max_instances=max_instances, -min_instances=min_instances, properties=properties, service_template=service_storage.service_template.list()[0] )) @@ -620,9 +613,6 @@ class TestNodeHostAddress(object): kwargs = dict( name='node_template', type=storage.type.list()[0], -default_instances=1, -max_instances=1, -min_instances=1, service_template=storage.service_template.list()[0] ) if host_address: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/orchestrator/context/test_operation.py -- diff --git a/tests/orchestrator/context/test_operation.py b/tests/orchestrator/context/test_operation.py index 9550d12..111e121 100644 --- a/tests/orchestrator/context/test_operation.py +++ b/tests/orchestrator/context/test_operation.py @@ -466,7 +466,7 @@ def operation_common(ctx, holder): holder['actor_name'] = ctx.task.actor.name holder['task_name'] = ctx.task.name holder['function'] = ctx.task.function -holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.values()) +holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.itervalues()) @operation http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml -- diff --git a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml index a5df3e8..5a46532 100644 --- a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml +++ b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml @@ -169,7 +169,7 @@ topology_template: capabilities: - scalable: properties: -- max_instances: { greater_or_equal: 8 } +- max_instances: { greater_or_equal: 5 } mongodb: description: >- @@ -185,7 +185,7 @@ topology_template: capabilities: - scalable: properties: - - max_instances: { greater_or_equal: 8 } + - max_instances: { greater_or_equal: 5 } relationship: interfaces: Configure: @@ -239,7 +239,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 5 # overrides the policy data_host: copy: loadbalancer_host @@ -260,7 +260,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 6 # overrides the policy data_volume: type: openstack.Volume @@ -287,10 +287,21 @@ topology_template: policies: -scaling: +app_scaling: + type: aria.Scaling + properties: +max_instances: 10 +
incubator-ariatosca git commit: wip
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions [created] bf753679a wip Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/bf753679 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/bf753679 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/bf753679 Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: bf753679a8fcbd7e2dbe6afc0f013fa80ad60720 Parents: b30a7ed Author: max-orlovAuthored: Sun Jul 2 21:43:43 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 15:49:53 2017 +0300 -- aria/modeling/orchestration.py | 4 +- aria/orchestrator/workflow_runner.py| 7 +- aria/orchestrator/workflows/core/engine.py | 18 ++- .../workflows/core/events_handler.py| 9 +- tests/modeling/test_models.py | 3 +- tests/orchestrator/test_workflow_runner.py | 116 +-- 6 files changed, 138 insertions(+), 19 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bf753679/aria/modeling/orchestration.py -- diff --git a/aria/modeling/orchestration.py b/aria/modeling/orchestration.py index df2643e..4d4f0fe 100644 --- a/aria/modeling/orchestration.py +++ b/aria/modeling/orchestration.py @@ -65,7 +65,9 @@ class ExecutionBase(mixins.ModelMixin): PENDING: (STARTED, CANCELLED), STARTED: END_STATES + (CANCELLING,), CANCELLING: END_STATES, -CANCELLED: PENDING +# Retrying +CANCELLED: PENDING, +FAILED: PENDING } # region one_to_many relationships http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bf753679/aria/orchestrator/workflow_runner.py -- diff --git a/aria/orchestrator/workflow_runner.py b/aria/orchestrator/workflow_runner.py index 47270c0..2bd3043 100644 --- a/aria/orchestrator/workflow_runner.py +++ b/aria/orchestrator/workflow_runner.py @@ -38,7 +38,8 @@ DEFAULT_TASK_RETRY_INTERVAL = 30 class WorkflowRunner(object): def __init__(self, model_storage, resource_storage, plugin_manager, - execution_id=None, service_id=None, workflow_name=None, inputs=None, executor=None, + execution_id=None, retry_failed=False, + service_id=None, workflow_name=None, inputs=None, executor=None, task_max_attempts=DEFAULT_TASK_MAX_ATTEMPTS, task_retry_interval=DEFAULT_TASK_RETRY_INTERVAL): """ @@ -62,6 +63,7 @@ class WorkflowRunner(object): "and service id with inputs") self._is_resume = execution_id is not None +self._retry_failed = retry_failed self._model_storage = model_storage self._resource_storage = resource_storage @@ -116,7 +118,8 @@ class WorkflowRunner(object): return self._model_storage.service.get(self._service_id) def execute(self): -self._engine.execute(ctx=self._workflow_context, resuming=self._is_resume) +self._engine.execute( +ctx=self._workflow_context, resuming=self._is_resume, retry_failed=self._retry_failed) def cancel(self): self._engine.cancel_execution(ctx=self._workflow_context) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bf753679/aria/orchestrator/workflows/core/engine.py -- diff --git a/aria/orchestrator/workflows/core/engine.py b/aria/orchestrator/workflows/core/engine.py index d9c77e9..69505fc 100644 --- a/aria/orchestrator/workflows/core/engine.py +++ b/aria/orchestrator/workflows/core/engine.py @@ -41,14 +41,15 @@ class Engine(logger.LoggerMixin): self._executors = executors.copy() self._executors.setdefault(StubTaskExecutor, StubTaskExecutor()) -def execute(self, ctx, resuming=False): +def execute(self, ctx, resuming=False, retry_failed=False): """ Executes the workflow. """ if resuming: -events.on_resume_workflow_signal.send(ctx) +events.on_resume_workflow_signal.send(ctx, retry_failed=retry_failed) tasks_tracker = _TasksTracker(ctx) + try: events.start_workflow_signal.send(ctx) while True: @@ -68,8 +69,15 @@ class Engine(logger.LoggerMixin): if cancel: self._terminate_tasks(tasks_tracker.executing_tasks) events.on_cancelled_workflow_signal.send(ctx) -else: +
[GitHub] incubator-ariatosca pull request #175: ARIA-299 Resuming canceled execution ...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/175 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca issue #181: ARIA-103 Remove dependency on Clint
Github user asfgit commented on the issue: https://github.com/apache/incubator-ariatosca/pull/181 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca pull request #181: ARIA-103 Remove dependency on Clint
GitHub user tliron opened a pull request: https://github.com/apache/incubator-ariatosca/pull/181 ARIA-103 Remove dependency on Clint We no longer require this third-party library, instead the utils/console module uses the existing cli/color module. This commit also fixes the cli/color module to properly support Unicode, and also properly deinitialize Colorama in Windows. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-103-remove-clint-dependency Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/181.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #181 commit d81d734da451d8ffa82c61830947589e378b76e5 Author: Tal LironDate: 2017-07-10T09:28:23Z ARIA-103 Remove dependency on Clint We no longer require this third-party library, instead the utils/console module uses the existing cli/color module. This commit also fixes the cli/color module to properly support Unicode, and also properly deinitialize Colorama in Windows. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
incubator-ariatosca git commit: ARIA-103 Remove dependency on Clint
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-103-remove-clint-dependency [created] d81d734da ARIA-103 Remove dependency on Clint We no longer require this third-party library, instead the utils/console module uses the existing cli/color module. This commit also fixes the cli/color module to properly support Unicode, and also properly deinitialize Colorama in Windows. Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/d81d734d Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/d81d734d Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/d81d734d Branch: refs/heads/ARIA-103-remove-clint-dependency Commit: d81d734da451d8ffa82c61830947589e378b76e5 Parents: a93a5df Author: Tal LironAuthored: Mon Jul 10 12:28:23 2017 +0300 Committer: Tal Liron Committed: Mon Jul 10 15:22:49 2017 +0300 -- aria/cli/color.py | 21 -- aria/utils/console.py | 53 +++--- requirements.in | 1 - requirements.txt | 4 +--- 4 files changed, 52 insertions(+), 27 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d81d734d/aria/cli/color.py -- diff --git a/aria/cli/color.py b/aria/cli/color.py index 03381ba..80c5604 100644 --- a/aria/cli/color.py +++ b/aria/cli/color.py @@ -18,11 +18,20 @@ Terminal colorization utilities. """ from StringIO import StringIO +import atexit import re import colorama +from ..utils.formatting import safe_str + + +def _restore_terminal(): +colorama.deinit() + + colorama.init() +atexit.register(_restore_terminal) class StringStylizer(object): @@ -33,20 +42,20 @@ class StringStylizer(object): def __repr__(self): if self._color_spec: return '{schema}{str}{reset}'.format( -schema=self._color_spec, str=str(self._str), reset=Colors.Style.RESET_ALL) +schema=self._color_spec, str=safe_str(self._str), reset=Colors.Style.RESET_ALL) return self._str def __add__(self, other): -return str(self) + other +return safe_str(self) + other def __radd__(self, other): -return other + str(self) +return other + safe_str(self) def color(self, color_spec): self._color_spec = color_spec def replace(self, old, new, **kwargs): -self._str = self._str.replace(str(old), str(new), **kwargs) +self._str = self._str.replace(safe_str(old), safe_str(new), **kwargs) def format(self, *args, **kwargs): self._str = self._str.format(*args, **kwargs) @@ -79,8 +88,8 @@ class Colors(object): class ColorSpec(object): def __init__(self, fore=None, back=None, style=None): """ -It is possible to provide fore, back and style arguments. each could be either -the color is lower case letter, or the actual color from Colorama. +It is possible to provide fore, back and style arguments. Each could be either the color as +a lowercase letter, or the full color name for Colorama. """ self._kwargs = dict(fore=fore, back=back, style=style) self._str = StringIO() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d81d734d/aria/utils/console.py -- diff --git a/aria/utils/console.py b/aria/utils/console.py index 642cbb1..2f6f622 100644 --- a/aria/utils/console.py +++ b/aria/utils/console.py @@ -17,52 +17,71 @@ Abstraction API above terminal color libraries. """ -from clint.textui.core import STDOUT -from clint.textui import puts as _puts -from clint.textui.colored import ColoredString as _ColoredString -from clint.textui import indent # pylint: disable=unused-import +import os +import sys + +from contextlib import contextmanager from .formatting import safe_str +from ..cli import color + + +_indent_string = '' -class ColoredString(_ColoredString): -def __init__(self, color, str_, always_color=False, bold=False): -super(ColoredString, self).__init__(color, safe_str(str_), always_color, bold) +def puts(string='', newline=True, stream=sys.stdout): +stream.write(_indent_string) +stream.write(safe_str(string)) +if newline: +stream.write(os.linesep) -def puts(string='', newline=True, stream=STDOUT): -_puts(safe_str(string), newline, stream) +@contextmanager +def indent(size=4): +global _indent_string +original_indent_string = _indent_string +try: +_indent_string += ' ' * size +yield +finally: +_indent_string = original_indent_string
[1/2] incubator-ariatosca git commit: ARIA-305 Advance ARIA version [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails d31c011df -> b30a7edd8 (forced update) ARIA-305 Advance ARIA version Set package version to 0.2.0 Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c46c94bd Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c46c94bd Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c46c94bd Branch: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails Commit: c46c94bd722541492f9cff5e37741d864a74cb41 Parents: 2089655 Author: Ran ZivAuthored: Mon Jul 10 12:36:08 2017 +0300 Committer: Ran Ziv Committed: Mon Jul 10 12:40:49 2017 +0300 -- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c46c94bd/VERSION -- diff --git a/VERSION b/VERSION index 6c6aa7c..341cf11 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.0 \ No newline at end of file +0.2.0 \ No newline at end of file
[incubator-ariatosca] Git Push Summary
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-287-tox-for-docs [deleted] 6c0842429
[incubator-ariatosca] Git Push Summary
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-260-send-interface-inputs [deleted] 2195e1fb5
[incubator-ariatosca] Git Push Summary
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-305-advance-aria-version [deleted] c46c94bd7
incubator-ariatosca git commit: ARIA-305 Advance ARIA version
Repository: incubator-ariatosca Updated Branches: refs/heads/master 208965586 -> c46c94bd7 ARIA-305 Advance ARIA version Set package version to 0.2.0 Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/c46c94bd Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/c46c94bd Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/c46c94bd Branch: refs/heads/master Commit: c46c94bd722541492f9cff5e37741d864a74cb41 Parents: 2089655 Author: Ran ZivAuthored: Mon Jul 10 12:36:08 2017 +0300 Committer: Ran Ziv Committed: Mon Jul 10 12:40:49 2017 +0300 -- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c46c94bd/VERSION -- diff --git a/VERSION b/VERSION index 6c6aa7c..341cf11 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.0 \ No newline at end of file +0.2.0 \ No newline at end of file
[GitHub] incubator-ariatosca pull request #180: ARIA-305 Advance ARIA version
Github user asfgit closed the pull request at: https://github.com/apache/incubator-ariatosca/pull/180 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca pull request #180: ARIA-305 Advance ARIA version
GitHub user ran-z opened a pull request: https://github.com/apache/incubator-ariatosca/pull/180 ARIA-305 Advance ARIA version Set package version to 0.2.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-ariatosca ARIA-305-advance-aria-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-ariatosca/pull/180.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #180 commit c46c94bd722541492f9cff5e37741d864a74cb41 Author: Ran ZivDate: 2017-07-10T09:36:08Z ARIA-305 Advance ARIA version Set package version to 0.2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca issue #180: ARIA-305 Advance ARIA version
Github user asfgit commented on the issue: https://github.com/apache/incubator-ariatosca/pull/180 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
incubator-ariatosca git commit: review 1 fixes
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails 62875b56c -> 05d2b73ae review 1 fixes Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/05d2b73a Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/05d2b73a Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/05d2b73a Branch: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails Commit: 05d2b73ae20d144de581d9a225fdf9ab19dcd0bd Parents: 62875b5 Author: max-orlovAuthored: Mon Jul 10 12:36:22 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 12:36:22 2017 +0300 -- .../workflows/core/events_handler.py| 8 +- .../orchestrator/execution_plugin/test_local.py | 6 +- tests/orchestrator/execution_plugin/test_ssh.py | 6 +- tests/orchestrator/test_workflow_runner.py | 98 +++- 4 files changed, 87 insertions(+), 31 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/05d2b73a/aria/orchestrator/workflows/core/events_handler.py -- diff --git a/aria/orchestrator/workflows/core/events_handler.py b/aria/orchestrator/workflows/core/events_handler.py index eb6f271..37801de 100644 --- a/aria/orchestrator/workflows/core/events_handler.py +++ b/aria/orchestrator/workflows/core/events_handler.py @@ -114,10 +114,6 @@ def _workflow_cancelled(workflow_context, *args, **kwargs): elif execution.status in (execution.SUCCEEDED, execution.FAILED): _log_tried_to_cancel_execution_but_it_already_ended(workflow_context, execution.status) else: -# Any non ended task would be put back to pending state -for task in execution.tasks: -if not task.has_ended(): -task.status = task.PENDING execution.status = execution.CANCELLED execution.ended_at = datetime.utcnow() @@ -127,6 +123,10 @@ def _workflow_resume(workflow_context, *args, **kwargs): with workflow_context.persist_changes: execution = workflow_context.execution execution.status = execution.PENDING +# Any non ended task would be put back to pending state +for task in execution.tasks: +if not task.has_ended(): +task.status = task.PENDING @events.on_cancelling_workflow_signal.connect http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/05d2b73a/tests/orchestrator/execution_plugin/test_local.py -- diff --git a/tests/orchestrator/execution_plugin/test_local.py b/tests/orchestrator/execution_plugin/test_local.py index 5b94917..e64e998 100644 --- a/tests/orchestrator/execution_plugin/test_local.py +++ b/tests/orchestrator/execution_plugin/test_local.py @@ -509,8 +509,10 @@ if __name__ == '__main__': @pytest.fixture def executor(self): result = process.ProcessExecutor() -yield result -result.close() +try: +yield result +finally: +result.close() @pytest.fixture def workflow_context(self, tmpdir): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/05d2b73a/tests/orchestrator/execution_plugin/test_ssh.py -- diff --git a/tests/orchestrator/execution_plugin/test_ssh.py b/tests/orchestrator/execution_plugin/test_ssh.py index 4fa8184..a96c91d 100644 --- a/tests/orchestrator/execution_plugin/test_ssh.py +++ b/tests/orchestrator/execution_plugin/test_ssh.py @@ -277,8 +277,10 @@ class TestWithActualSSHServer(object): @pytest.fixture def executor(self): result = process.ProcessExecutor() -yield result -result.close() +try: +yield result +finally: +result.close() @pytest.fixture def workflow_context(self, tmpdir): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/05d2b73a/tests/orchestrator/test_workflow_runner.py -- diff --git a/tests/orchestrator/test_workflow_runner.py b/tests/orchestrator/test_workflow_runner.py index 3527f34..a77d727 100644 --- a/tests/orchestrator/test_workflow_runner.py +++ b/tests/orchestrator/test_workflow_runner.py @@ -14,6 +14,7 @@ # limitations under the License. import json +import time from threading import Thread, Event from datetime import datetime @@ -58,6 +59,10 @@ class TimeoutError(BaseException): pass +class FailingTask(BaseException): +pass + + def
[6/6] incubator-ariatosca git commit: review 1 fixes
review 1 fixes Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/d31c011d Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/d31c011d Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/d31c011d Branch: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails Commit: d31c011dfc483050c87a3a025465b0ffc753944b Parents: bd619bd Author: max-orlovAuthored: Mon Jul 10 12:36:22 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 12:41:08 2017 +0300 -- .../workflows/core/events_handler.py| 8 +- .../orchestrator/execution_plugin/test_local.py | 6 +- tests/orchestrator/execution_plugin/test_ssh.py | 6 +- tests/orchestrator/test_workflow_runner.py | 98 +++- 4 files changed, 87 insertions(+), 31 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d31c011d/aria/orchestrator/workflows/core/events_handler.py -- diff --git a/aria/orchestrator/workflows/core/events_handler.py b/aria/orchestrator/workflows/core/events_handler.py index eb6f271..37801de 100644 --- a/aria/orchestrator/workflows/core/events_handler.py +++ b/aria/orchestrator/workflows/core/events_handler.py @@ -114,10 +114,6 @@ def _workflow_cancelled(workflow_context, *args, **kwargs): elif execution.status in (execution.SUCCEEDED, execution.FAILED): _log_tried_to_cancel_execution_but_it_already_ended(workflow_context, execution.status) else: -# Any non ended task would be put back to pending state -for task in execution.tasks: -if not task.has_ended(): -task.status = task.PENDING execution.status = execution.CANCELLED execution.ended_at = datetime.utcnow() @@ -127,6 +123,10 @@ def _workflow_resume(workflow_context, *args, **kwargs): with workflow_context.persist_changes: execution = workflow_context.execution execution.status = execution.PENDING +# Any non ended task would be put back to pending state +for task in execution.tasks: +if not task.has_ended(): +task.status = task.PENDING @events.on_cancelling_workflow_signal.connect http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d31c011d/tests/orchestrator/execution_plugin/test_local.py -- diff --git a/tests/orchestrator/execution_plugin/test_local.py b/tests/orchestrator/execution_plugin/test_local.py index 5b94917..e64e998 100644 --- a/tests/orchestrator/execution_plugin/test_local.py +++ b/tests/orchestrator/execution_plugin/test_local.py @@ -509,8 +509,10 @@ if __name__ == '__main__': @pytest.fixture def executor(self): result = process.ProcessExecutor() -yield result -result.close() +try: +yield result +finally: +result.close() @pytest.fixture def workflow_context(self, tmpdir): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d31c011d/tests/orchestrator/execution_plugin/test_ssh.py -- diff --git a/tests/orchestrator/execution_plugin/test_ssh.py b/tests/orchestrator/execution_plugin/test_ssh.py index 4fa8184..a96c91d 100644 --- a/tests/orchestrator/execution_plugin/test_ssh.py +++ b/tests/orchestrator/execution_plugin/test_ssh.py @@ -277,8 +277,10 @@ class TestWithActualSSHServer(object): @pytest.fixture def executor(self): result = process.ProcessExecutor() -yield result -result.close() +try: +yield result +finally: +result.close() @pytest.fixture def workflow_context(self, tmpdir): http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/d31c011d/tests/orchestrator/test_workflow_runner.py -- diff --git a/tests/orchestrator/test_workflow_runner.py b/tests/orchestrator/test_workflow_runner.py index 3527f34..a77d727 100644 --- a/tests/orchestrator/test_workflow_runner.py +++ b/tests/orchestrator/test_workflow_runner.py @@ -14,6 +14,7 @@ # limitations under the License. import json +import time from threading import Thread, Event from datetime import datetime @@ -58,6 +59,10 @@ class TimeoutError(BaseException): pass +class FailingTask(BaseException): +pass + + def test_undeclared_workflow(request): # validating a proper error is raised when the workflow is not declared in the service with
[4/6] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies
ARIA-254 Scaling capabilities and policies * New aria.Scaling policy (and "scaling" role) * NodeTemplate model no longer stores scaling values (default_instances, etc.) but instead fetches them from applicable scaling capabilities and policies * Some code cleanup Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/20896558 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/20896558 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/20896558 Branch: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails Commit: 208965586997f2335ec242547dd0ee39bd9beb42 Parents: a93a5df Author: Tal LironAuthored: Thu Jun 1 14:17:17 2017 -0500 Committer: Tal Liron Committed: Mon Jul 10 11:59:08 2017 +0300 -- aria/cli/commands/workflows.py | 4 +- aria/cli/execution_logging.py | 4 +- aria/core.py| 2 +- aria/modeling/orchestration.py | 6 +- aria/modeling/relationship.py | 2 +- aria/modeling/service_instance.py | 134 ++-- aria/modeling/service_template.py | 336 ++- aria/modeling/types.py | 4 +- aria/orchestrator/workflow_runner.py| 2 +- aria/orchestrator/workflows/executor/celery.py | 4 +- aria/orchestrator/workflows/executor/process.py | 4 +- aria/orchestrator/workflows/executor/thread.py | 2 +- aria/parser/presentation/presentation.py| 12 +- aria/storage/core.py| 2 +- .../profiles/aria-1.0/aria-1.0.yaml | 38 ++- .../profiles/tosca-simple-1.0/capabilities.yaml | 1 + .../simple_v1_0/assignments.py | 8 + .../simple_v1_0/data_types.py | 22 +- .../simple_v1_0/definitions.py | 8 + .../aria_extension_tosca/simple_v1_0/filters.py | 2 + .../aria_extension_tosca/simple_v1_0/misc.py| 10 + .../simple_v1_0/modeling/__init__.py| 3 - .../simple_v1_0/modeling/artifacts.py | 4 +- .../simple_v1_0/modeling/capabilities.py| 56 +++- .../simple_v1_0/modeling/interfaces.py | 8 +- .../simple_v1_0/modeling/parameters.py | 5 +- .../simple_v1_0/presentation/extensible.py | 1 + .../presentation/field_validators.py| 20 ++ .../simple_v1_0/presentation/types.py | 2 + .../simple_v1_0/presenter.py| 3 +- .../simple_v1_0/templates.py| 8 +- .../aria_extension_tosca/simple_v1_0/types.py | 10 +- tests/end2end/test_hello_world.py | 2 +- tests/end2end/test_nodecellar.py| 2 +- tests/mock/models.py| 8 +- tests/modeling/test_mixins.py | 3 - tests/modeling/test_models.py | 20 +- tests/orchestrator/context/test_operation.py| 2 +- .../node-cellar/node-cellar.yaml| 21 +- .../node-cellar/types/nodejs.yaml | 1 + .../node-cellar/types/openstack.yaml| 3 +- 41 files changed, 446 insertions(+), 343 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/aria/cli/commands/workflows.py -- diff --git a/aria/cli/commands/workflows.py b/aria/cli/commands/workflows.py index 03cf00e..ca191aa 100644 --- a/aria/cli/commands/workflows.py +++ b/aria/cli/commands/workflows.py @@ -50,7 +50,7 @@ def show(workflow_name, service_name, model_storage, logger): logger.info('Retrieving workflow {0} for service {1}'.format( workflow_name, service_name)) service = model_storage.service.get_by_name(service_name) -workflow = next((wf for wf in service.workflows.values() if +workflow = next((wf for wf in service.workflows.itervalues() if wf.name == workflow_name), None) if not workflow: raise AriaCliError( @@ -102,7 +102,7 @@ def list(service_name, model_storage, logger): """ logger.info('Listing workflows for service {0}...'.format(service_name)) service = model_storage.service.get_by_name(service_name) -workflows_list = sorted(service.workflows.values(), key=lambda w: w.name) +workflows_list = sorted(service.workflows.itervalues(), key=lambda w: w.name) defaults = { 'service_template_name': service.service_template_name, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/aria/cli/execution_logging.py -- diff --git
[5/6] incubator-ariatosca git commit: ARIA-299 Resuming canceled execution with frozen task fails
ARIA-299 Resuming canceled execution with frozen task fails Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/bd619bd6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/bd619bd6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/bd619bd6 Branch: refs/heads/ARIA-299-Resuming-canceled-execution-with-frozen-task-fails Commit: bd619bd63837124a160d1c53243b4cd550628078 Parents: 2089655 Author: max-orlovAuthored: Wed Jul 5 16:16:39 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 12:41:08 2017 +0300 -- .../workflows/core/events_handler.py| 4 + aria/orchestrator/workflows/executor/base.py| 2 +- aria/orchestrator/workflows/executor/thread.py | 8 +- tests/orchestrator/context/test_serialize.py| 6 +- tests/orchestrator/test_workflow_runner.py | 144 --- .../workflows/executor/test_process_executor.py | 66 + ...process_executor_concurrent_modifications.py | 6 +- .../executor/test_process_executor_extension.py | 6 +- .../test_process_executor_tracked_changes.py| 6 +- 9 files changed, 186 insertions(+), 62 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bd619bd6/aria/orchestrator/workflows/core/events_handler.py -- diff --git a/aria/orchestrator/workflows/core/events_handler.py b/aria/orchestrator/workflows/core/events_handler.py index 769c1a8..eb6f271 100644 --- a/aria/orchestrator/workflows/core/events_handler.py +++ b/aria/orchestrator/workflows/core/events_handler.py @@ -114,6 +114,10 @@ def _workflow_cancelled(workflow_context, *args, **kwargs): elif execution.status in (execution.SUCCEEDED, execution.FAILED): _log_tried_to_cancel_execution_but_it_already_ended(workflow_context, execution.status) else: +# Any non ended task would be put back to pending state +for task in execution.tasks: +if not task.has_ended(): +task.status = task.PENDING execution.status = execution.CANCELLED execution.ended_at = datetime.utcnow() http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bd619bd6/aria/orchestrator/workflows/executor/base.py -- diff --git a/aria/orchestrator/workflows/executor/base.py b/aria/orchestrator/workflows/executor/base.py index ec1a0c7..e7d03ea 100644 --- a/aria/orchestrator/workflows/executor/base.py +++ b/aria/orchestrator/workflows/executor/base.py @@ -49,7 +49,7 @@ class BaseExecutor(logger.LoggerMixin): """ pass -def terminate(self, ctx): +def terminate(self, task_id): """ Terminate the executing task :return: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bd619bd6/aria/orchestrator/workflows/executor/thread.py -- diff --git a/aria/orchestrator/workflows/executor/thread.py b/aria/orchestrator/workflows/executor/thread.py index 26484dc..170620e 100644 --- a/aria/orchestrator/workflows/executor/thread.py +++ b/aria/orchestrator/workflows/executor/thread.py @@ -36,9 +36,10 @@ class ThreadExecutor(BaseExecutor): Note: This executor is incapable of running plugin operations. """ -def __init__(self, pool_size=1, *args, **kwargs): +def __init__(self, pool_size=1, close_timeout=5, *args, **kwargs): super(ThreadExecutor, self).__init__(*args, **kwargs) self._stopped = False +self._close_timeout = close_timeout self._queue = Queue.Queue() self._pool = [] for i in range(pool_size): @@ -54,7 +55,10 @@ class ThreadExecutor(BaseExecutor): def close(self): self._stopped = True for thread in self._pool: -thread.join() +if self._close_timeout is None: +thread.join() +else: +thread.join(self._close_timeout) def _processor(self): while not self._stopped: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/bd619bd6/tests/orchestrator/context/test_serialize.py -- diff --git a/tests/orchestrator/context/test_serialize.py b/tests/orchestrator/context/test_serialize.py index 6046a16..091e23c 100644 --- a/tests/orchestrator/context/test_serialize.py +++ b/tests/orchestrator/context/test_serialize.py @@ -87,8 +87,10 @@ def _operation_mapping(): @pytest.fixture def executor(): result =
[1/2] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies
Repository: incubator-ariatosca Updated Branches: refs/heads/master a93a5dfaf -> 208965586 http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/modeling/test_models.py -- diff --git a/tests/modeling/test_models.py b/tests/modeling/test_models.py index bbc7352..e1167fc 100644 --- a/tests/modeling/test_models.py +++ b/tests/modeling/test_models.py @@ -509,19 +509,15 @@ class TestServiceModification(object): class TestNodeTemplate(object): @pytest.mark.parametrize( -'is_valid, name, default_instances, max_instances, min_instances, properties', +'is_valid, name, properties', [ -(False, m_cls, 1, 1, 1, {}), -(False, 'name', m_cls, 1, 1, {}), -(False, 'name', 1, m_cls, 1, {}), -(False, 'name', 1, 1, m_cls, {}), -(False, 'name', 1, 1, 1, m_cls), +(False, m_cls, {}), +(False, 'name', m_cls), -(True, 'name', 1, 1, 1, {}), +(True, 'name', {}), ] ) -def test_node_template_model_creation(self, service_storage, is_valid, name, default_instances, - max_instances, min_instances, properties): +def test_node_template_model_creation(self, service_storage, is_valid, name, properties): node_template = _test_model( is_valid=is_valid, storage=service_storage, @@ -529,9 +525,6 @@ class TestNodeTemplate(object): model_kwargs=dict( name=name, type=service_storage.type.list()[0], -default_instances=default_instances, -max_instances=max_instances, -min_instances=min_instances, properties=properties, service_template=service_storage.service_template.list()[0] )) @@ -620,9 +613,6 @@ class TestNodeHostAddress(object): kwargs = dict( name='node_template', type=storage.type.list()[0], -default_instances=1, -max_instances=1, -min_instances=1, service_template=storage.service_template.list()[0] ) if host_address: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/orchestrator/context/test_operation.py -- diff --git a/tests/orchestrator/context/test_operation.py b/tests/orchestrator/context/test_operation.py index 9550d12..111e121 100644 --- a/tests/orchestrator/context/test_operation.py +++ b/tests/orchestrator/context/test_operation.py @@ -466,7 +466,7 @@ def operation_common(ctx, holder): holder['actor_name'] = ctx.task.actor.name holder['task_name'] = ctx.task.name holder['function'] = ctx.task.function -holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.values()) +holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.itervalues()) @operation http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml -- diff --git a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml index a5df3e8..5a46532 100644 --- a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml +++ b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml @@ -169,7 +169,7 @@ topology_template: capabilities: - scalable: properties: -- max_instances: { greater_or_equal: 8 } +- max_instances: { greater_or_equal: 5 } mongodb: description: >- @@ -185,7 +185,7 @@ topology_template: capabilities: - scalable: properties: - - max_instances: { greater_or_equal: 8 } + - max_instances: { greater_or_equal: 5 } relationship: interfaces: Configure: @@ -239,7 +239,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 5 # overrides the policy data_host: copy: loadbalancer_host @@ -260,7 +260,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 6 # overrides the policy data_volume: type: openstack.Volume @@ -287,10 +287,21 @@ topology_template: policies: -scaling: +app_scaling: + type: aria.Scaling + properties: +max_instances: 10 +default_instances: 2 + targets: +
[2/2] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies
ARIA-254 Scaling capabilities and policies * New aria.Scaling policy (and "scaling" role) * NodeTemplate model no longer stores scaling values (default_instances, etc.) but instead fetches them from applicable scaling capabilities and policies * Some code cleanup Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/20896558 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/20896558 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/20896558 Branch: refs/heads/master Commit: 208965586997f2335ec242547dd0ee39bd9beb42 Parents: a93a5df Author: Tal LironAuthored: Thu Jun 1 14:17:17 2017 -0500 Committer: Tal Liron Committed: Mon Jul 10 11:59:08 2017 +0300 -- aria/cli/commands/workflows.py | 4 +- aria/cli/execution_logging.py | 4 +- aria/core.py| 2 +- aria/modeling/orchestration.py | 6 +- aria/modeling/relationship.py | 2 +- aria/modeling/service_instance.py | 134 ++-- aria/modeling/service_template.py | 336 ++- aria/modeling/types.py | 4 +- aria/orchestrator/workflow_runner.py| 2 +- aria/orchestrator/workflows/executor/celery.py | 4 +- aria/orchestrator/workflows/executor/process.py | 4 +- aria/orchestrator/workflows/executor/thread.py | 2 +- aria/parser/presentation/presentation.py| 12 +- aria/storage/core.py| 2 +- .../profiles/aria-1.0/aria-1.0.yaml | 38 ++- .../profiles/tosca-simple-1.0/capabilities.yaml | 1 + .../simple_v1_0/assignments.py | 8 + .../simple_v1_0/data_types.py | 22 +- .../simple_v1_0/definitions.py | 8 + .../aria_extension_tosca/simple_v1_0/filters.py | 2 + .../aria_extension_tosca/simple_v1_0/misc.py| 10 + .../simple_v1_0/modeling/__init__.py| 3 - .../simple_v1_0/modeling/artifacts.py | 4 +- .../simple_v1_0/modeling/capabilities.py| 56 +++- .../simple_v1_0/modeling/interfaces.py | 8 +- .../simple_v1_0/modeling/parameters.py | 5 +- .../simple_v1_0/presentation/extensible.py | 1 + .../presentation/field_validators.py| 20 ++ .../simple_v1_0/presentation/types.py | 2 + .../simple_v1_0/presenter.py| 3 +- .../simple_v1_0/templates.py| 8 +- .../aria_extension_tosca/simple_v1_0/types.py | 10 +- tests/end2end/test_hello_world.py | 2 +- tests/end2end/test_nodecellar.py| 2 +- tests/mock/models.py| 8 +- tests/modeling/test_mixins.py | 3 - tests/modeling/test_models.py | 20 +- tests/orchestrator/context/test_operation.py| 2 +- .../node-cellar/node-cellar.yaml| 21 +- .../node-cellar/types/nodejs.yaml | 1 + .../node-cellar/types/openstack.yaml| 3 +- 41 files changed, 446 insertions(+), 343 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/aria/cli/commands/workflows.py -- diff --git a/aria/cli/commands/workflows.py b/aria/cli/commands/workflows.py index 03cf00e..ca191aa 100644 --- a/aria/cli/commands/workflows.py +++ b/aria/cli/commands/workflows.py @@ -50,7 +50,7 @@ def show(workflow_name, service_name, model_storage, logger): logger.info('Retrieving workflow {0} for service {1}'.format( workflow_name, service_name)) service = model_storage.service.get_by_name(service_name) -workflow = next((wf for wf in service.workflows.values() if +workflow = next((wf for wf in service.workflows.itervalues() if wf.name == workflow_name), None) if not workflow: raise AriaCliError( @@ -102,7 +102,7 @@ def list(service_name, model_storage, logger): """ logger.info('Listing workflows for service {0}...'.format(service_name)) service = model_storage.service.get_by_name(service_name) -workflows_list = sorted(service.workflows.values(), key=lambda w: w.name) +workflows_list = sorted(service.workflows.itervalues(), key=lambda w: w.name) defaults = { 'service_template_name': service.service_template_name, http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/aria/cli/execution_logging.py -- diff --git a/aria/cli/execution_logging.py b/aria/cli/execution_logging.py
[3/6] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies
http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/modeling/test_models.py -- diff --git a/tests/modeling/test_models.py b/tests/modeling/test_models.py index bbc7352..e1167fc 100644 --- a/tests/modeling/test_models.py +++ b/tests/modeling/test_models.py @@ -509,19 +509,15 @@ class TestServiceModification(object): class TestNodeTemplate(object): @pytest.mark.parametrize( -'is_valid, name, default_instances, max_instances, min_instances, properties', +'is_valid, name, properties', [ -(False, m_cls, 1, 1, 1, {}), -(False, 'name', m_cls, 1, 1, {}), -(False, 'name', 1, m_cls, 1, {}), -(False, 'name', 1, 1, m_cls, {}), -(False, 'name', 1, 1, 1, m_cls), +(False, m_cls, {}), +(False, 'name', m_cls), -(True, 'name', 1, 1, 1, {}), +(True, 'name', {}), ] ) -def test_node_template_model_creation(self, service_storage, is_valid, name, default_instances, - max_instances, min_instances, properties): +def test_node_template_model_creation(self, service_storage, is_valid, name, properties): node_template = _test_model( is_valid=is_valid, storage=service_storage, @@ -529,9 +525,6 @@ class TestNodeTemplate(object): model_kwargs=dict( name=name, type=service_storage.type.list()[0], -default_instances=default_instances, -max_instances=max_instances, -min_instances=min_instances, properties=properties, service_template=service_storage.service_template.list()[0] )) @@ -620,9 +613,6 @@ class TestNodeHostAddress(object): kwargs = dict( name='node_template', type=storage.type.list()[0], -default_instances=1, -max_instances=1, -min_instances=1, service_template=storage.service_template.list()[0] ) if host_address: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/orchestrator/context/test_operation.py -- diff --git a/tests/orchestrator/context/test_operation.py b/tests/orchestrator/context/test_operation.py index 9550d12..111e121 100644 --- a/tests/orchestrator/context/test_operation.py +++ b/tests/orchestrator/context/test_operation.py @@ -466,7 +466,7 @@ def operation_common(ctx, holder): holder['actor_name'] = ctx.task.actor.name holder['task_name'] = ctx.task.name holder['function'] = ctx.task.function -holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.values()) +holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.itervalues()) @operation http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/20896558/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml -- diff --git a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml index a5df3e8..5a46532 100644 --- a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml +++ b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml @@ -169,7 +169,7 @@ topology_template: capabilities: - scalable: properties: -- max_instances: { greater_or_equal: 8 } +- max_instances: { greater_or_equal: 5 } mongodb: description: >- @@ -185,7 +185,7 @@ topology_template: capabilities: - scalable: properties: - - max_instances: { greater_or_equal: 8 } + - max_instances: { greater_or_equal: 5 } relationship: interfaces: Configure: @@ -239,7 +239,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 5 # overrides the policy data_host: copy: loadbalancer_host @@ -260,7 +260,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 6 # overrides the policy data_volume: type: openstack.Volume @@ -287,10 +287,21 @@ topology_template: policies: -scaling: +app_scaling: + type: aria.Scaling + properties: +max_instances: 10 +default_instances: 2 + targets: +- node_cellar +- nodejs + +host_scaling: type: openstack.Scaling
[GitHub] incubator-ariatosca pull request #175: ARIA-299 Resuming canceled execution ...
Github user mxmrlv commented on a diff in the pull request: https://github.com/apache/incubator-ariatosca/pull/175#discussion_r126370435 --- Diff: tests/orchestrator/test_workflow_runner.py --- @@ -441,8 +509,42 @@ def mock_resuming_task(ctx): ctx.node.attributes['invocations'] += 1 if ctx.node.attributes['invocations'] != 1: -events['is_active'].set() -if not events['is_resumed'].isSet(): +custom_events['is_active'].set() +if not custom_events['is_resumed'].isSet(): # if resume was called, increase by one. o/w fail the execution - second task should # fail as long it was not a part of resuming the workflow raise BaseException("wasn't resumed yet") + + +@workflow +def mock_sequential_workflow(ctx, graph): +node = ctx.model.node.get_by_name(tests_mock.models.DEPENDENCY_NODE_NAME) +graph.sequence( +api.task.OperationTask(node, + interface_name='aria.interfaces.lifecycle', + operation_name='create', + retry_interval=1, + max_attempts=10), +) + + +@operation +def mock_failed_first_task(ctx): +""" +The task should run atmost ctx.task.max_attempts - 1 times, and only then pass. +overall, the number of invocations should be ctx.task.max_attempts - 1 +""" +ctx.node.attributes['invocations'] += 1 + +if ctx.node.attributes['invocations'] == 2: +custom_events['is_active'].set() +# unfreeze the thread only when all of the invocations are done +while ctx.node.attributes['invocations'] < ctx.task.max_attempts - 1: +pass + +elif ctx.node.attributes['invocations'] == ctx.task.max_attempts - 1: +# pass only just before the end. +return +else: +# fail o.w. +raise BaseException("stop this task") --- End diff -- new exception --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca pull request #175: ARIA-299 Resuming canceled execution ...
Github user mxmrlv commented on a diff in the pull request: https://github.com/apache/incubator-ariatosca/pull/175#discussion_r126370031 --- Diff: tests/orchestrator/test_workflow_runner.py --- @@ -441,8 +509,42 @@ def mock_resuming_task(ctx): ctx.node.attributes['invocations'] += 1 if ctx.node.attributes['invocations'] != 1: -events['is_active'].set() -if not events['is_resumed'].isSet(): +custom_events['is_active'].set() +if not custom_events['is_resumed'].isSet(): # if resume was called, increase by one. o/w fail the execution - second task should # fail as long it was not a part of resuming the workflow raise BaseException("wasn't resumed yet") + + +@workflow +def mock_sequential_workflow(ctx, graph): --- End diff -- rename --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[1/2] incubator-ariatosca git commit: ARIA-254 Scaling capabilities and policies [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-254-multiple-nodes-per-template 1240b5a4f -> c31ea62c0 (forced update) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c31ea62c/tests/modeling/test_models.py -- diff --git a/tests/modeling/test_models.py b/tests/modeling/test_models.py index bbc7352..e1167fc 100644 --- a/tests/modeling/test_models.py +++ b/tests/modeling/test_models.py @@ -509,19 +509,15 @@ class TestServiceModification(object): class TestNodeTemplate(object): @pytest.mark.parametrize( -'is_valid, name, default_instances, max_instances, min_instances, properties', +'is_valid, name, properties', [ -(False, m_cls, 1, 1, 1, {}), -(False, 'name', m_cls, 1, 1, {}), -(False, 'name', 1, m_cls, 1, {}), -(False, 'name', 1, 1, m_cls, {}), -(False, 'name', 1, 1, 1, m_cls), +(False, m_cls, {}), +(False, 'name', m_cls), -(True, 'name', 1, 1, 1, {}), +(True, 'name', {}), ] ) -def test_node_template_model_creation(self, service_storage, is_valid, name, default_instances, - max_instances, min_instances, properties): +def test_node_template_model_creation(self, service_storage, is_valid, name, properties): node_template = _test_model( is_valid=is_valid, storage=service_storage, @@ -529,9 +525,6 @@ class TestNodeTemplate(object): model_kwargs=dict( name=name, type=service_storage.type.list()[0], -default_instances=default_instances, -max_instances=max_instances, -min_instances=min_instances, properties=properties, service_template=service_storage.service_template.list()[0] )) @@ -620,9 +613,6 @@ class TestNodeHostAddress(object): kwargs = dict( name='node_template', type=storage.type.list()[0], -default_instances=1, -max_instances=1, -min_instances=1, service_template=storage.service_template.list()[0] ) if host_address: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c31ea62c/tests/orchestrator/context/test_operation.py -- diff --git a/tests/orchestrator/context/test_operation.py b/tests/orchestrator/context/test_operation.py index 9550d12..111e121 100644 --- a/tests/orchestrator/context/test_operation.py +++ b/tests/orchestrator/context/test_operation.py @@ -466,7 +466,7 @@ def operation_common(ctx, holder): holder['actor_name'] = ctx.task.actor.name holder['task_name'] = ctx.task.name holder['function'] = ctx.task.function -holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.values()) +holder['arguments'] = dict(i.unwrapped for i in ctx.task.arguments.itervalues()) @operation http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/c31ea62c/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml -- diff --git a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml index a34301c..2d39967 100644 --- a/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml +++ b/tests/resources/service-templates/tosca-simple-1.0/node-cellar/node-cellar.yaml @@ -155,7 +155,7 @@ topology_template: capabilities: - scalable: properties: -- max_instances: { greater_or_equal: 8 } +- max_instances: { greater_or_equal: 5 } mongodb: description: >- @@ -171,7 +171,7 @@ topology_template: capabilities: - scalable: properties: - - max_instances: { greater_or_equal: 8 } + - max_instances: { greater_or_equal: 5 } relationship: interfaces: Configure: @@ -225,7 +225,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 5 # overrides the policy data_host: copy: loadbalancer_host @@ -246,7 +246,7 @@ topology_template: capabilities: scalable: properties: -max_instances: 10 +max_instances: 6 # overrides the policy data_volume: type: openstack.Volume @@ -273,10 +273,21 @@ topology_template: policies: -scaling: +app_scaling: + type: aria.Scaling + properties: +max_instances: 10 +
incubator-ariatosca git commit: PR fixes [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-254-multiple-nodes-per-template c0537bf13 -> 1240b5a4f (forced update) PR fixes Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/1240b5a4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/1240b5a4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/1240b5a4 Branch: refs/heads/ARIA-254-multiple-nodes-per-template Commit: 1240b5a4fe51831bdbb341032b448d97045deddb Parents: 9d99fed Author: Tal LironAuthored: Wed Jul 5 16:01:40 2017 -0500 Committer: Tal Liron Committed: Mon Jul 10 11:48:08 2017 +0300 -- aria/modeling/service_template.py | 64 ++-- .../simple_v1_0/modeling/capabilities.py| 8 ++- 2 files changed, 38 insertions(+), 34 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/1240b5a4/aria/modeling/service_template.py -- diff --git a/aria/modeling/service_template.py b/aria/modeling/service_template.py index df2f1a0..22912e2 100644 --- a/aria/modeling/service_template.py +++ b/aria/modeling/service_template.py @@ -622,7 +622,7 @@ class NodeTemplateBase(TemplateModelMixin): def instantiate(self, container): from . import models -node = models.Node(name=self.next_name, +node = models.Node(name=self._next_name, type=self.type, description=deepcopy_with_locators(self.description), state=models.Node.INITIAL, @@ -674,31 +674,6 @@ class NodeTemplateBase(TemplateModelMixin): utils.dump_list_values(self.requirement_templates, 'Requirement templates') @property -def next_index(self): -""" -Next available node index. - -:returns: node index -:rtype: int -""" - -max_index = 0 -if self.nodes: -max_index = max(int(n.name.rsplit('_', 1)[-1]) for n in self.nodes) -return max_index + 1 - -@property -def next_name(self): -""" -Next available node name. - -:returns: node name -:rtype: basestring -""" - -return '{name}_{index}'.format(name=self.name, index=self.next_index) - -@property def scaling(self): scaling = {} @@ -736,12 +711,12 @@ class NodeTemplateBase(TemplateModelMixin): # Validate # pylint: disable=too-many-boolean-expressions -if (scaling['min_instances'] < 0) or \ -(scaling['max_instances'] < 0) or \ -(scaling['default_instances'] < 0) or \ -(scaling['max_instances'] < scaling['min_instances']) or \ -(scaling['default_instances'] < scaling['min_instances']) or \ -(scaling['default_instances'] > scaling['max_instances']): +if ((scaling['min_instances'] < 0) or +(scaling['max_instances'] < 0) or +(scaling['default_instances'] < 0) or +(scaling['max_instances'] < scaling['min_instances']) or +(scaling['default_instances'] < scaling['min_instances']) or +(scaling['default_instances'] > scaling['max_instances'])): context = ConsumptionContext.get_thread_local() context.validation.report('invalid scaling parameters for node template "{0}": ' 'min={1}, max={2}, default={3}'.format( @@ -764,6 +739,31 @@ class NodeTemplateBase(TemplateModelMixin): return False return True +@property +def _next_index(self): +""" +Next available node index. + +:returns: node index +:rtype: int +""" + +max_index = 0 +if self.nodes: +max_index = max(int(n.name.rsplit('_', 1)[-1]) for n in self.nodes) +return max_index + 1 + +@property +def _next_name(self): +""" +Next available node name. + +:returns: node name +:rtype: basestring +""" + +return '{name}_{index}'.format(name=self.name, index=self._next_index) + class GroupTemplateBase(TemplateModelMixin): """ http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/1240b5a4/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py -- diff --git a/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py b/extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py index 760541a..5427c7e 100644 ---
incubator-ariatosca git commit: wip [Forced Update!]
Repository: incubator-ariatosca Updated Branches: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions c1daa4a96 -> 7f0b80176 (forced update) wip Project: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/commit/7f0b8017 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/tree/7f0b8017 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/diff/7f0b8017 Branch: refs/heads/ARIA-237-Support-for-resuming-failed-workflow-executions Commit: 7f0b80176556aea653c651edf6908fe1aaf9a45b Parents: 62875b5 Author: max-orlovAuthored: Sun Jul 2 21:43:43 2017 +0300 Committer: max-orlov Committed: Mon Jul 10 11:00:06 2017 +0300 -- aria/modeling/orchestration.py | 4 +- aria/orchestrator/workflow_runner.py| 7 +- aria/orchestrator/workflows/core/engine.py | 18 ++- .../workflows/core/events_handler.py| 9 +- tests/modeling/test_models.py | 3 +- tests/orchestrator/test_workflow_runner.py | 128 --- 6 files changed, 144 insertions(+), 25 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/7f0b8017/aria/modeling/orchestration.py -- diff --git a/aria/modeling/orchestration.py b/aria/modeling/orchestration.py index 7068557..4d7a5b5 100644 --- a/aria/modeling/orchestration.py +++ b/aria/modeling/orchestration.py @@ -65,7 +65,9 @@ class ExecutionBase(mixins.ModelMixin): PENDING: (STARTED, CANCELLED), STARTED: END_STATES + (CANCELLING,), CANCELLING: END_STATES, -CANCELLED: PENDING +# Retrying +CANCELLED: PENDING, +FAILED: PENDING } # region one_to_many relationships http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/7f0b8017/aria/orchestrator/workflow_runner.py -- diff --git a/aria/orchestrator/workflow_runner.py b/aria/orchestrator/workflow_runner.py index df1725f..5592326 100644 --- a/aria/orchestrator/workflow_runner.py +++ b/aria/orchestrator/workflow_runner.py @@ -38,7 +38,8 @@ DEFAULT_TASK_RETRY_INTERVAL = 30 class WorkflowRunner(object): def __init__(self, model_storage, resource_storage, plugin_manager, - execution_id=None, service_id=None, workflow_name=None, inputs=None, executor=None, + execution_id=None, retry_failed=False, + service_id=None, workflow_name=None, inputs=None, executor=None, task_max_attempts=DEFAULT_TASK_MAX_ATTEMPTS, task_retry_interval=DEFAULT_TASK_RETRY_INTERVAL): """ @@ -62,6 +63,7 @@ class WorkflowRunner(object): "and service id with inputs") self._is_resume = execution_id is not None +self._retry_failed = retry_failed self._model_storage = model_storage self._resource_storage = resource_storage @@ -116,7 +118,8 @@ class WorkflowRunner(object): return self._model_storage.service.get(self._service_id) def execute(self): -self._engine.execute(ctx=self._workflow_context, resuming=self._is_resume) +self._engine.execute( +ctx=self._workflow_context, resuming=self._is_resume, retry_failed=self._retry_failed) def cancel(self): self._engine.cancel_execution(ctx=self._workflow_context) http://git-wip-us.apache.org/repos/asf/incubator-ariatosca/blob/7f0b8017/aria/orchestrator/workflows/core/engine.py -- diff --git a/aria/orchestrator/workflows/core/engine.py b/aria/orchestrator/workflows/core/engine.py index d9c77e9..69505fc 100644 --- a/aria/orchestrator/workflows/core/engine.py +++ b/aria/orchestrator/workflows/core/engine.py @@ -41,14 +41,15 @@ class Engine(logger.LoggerMixin): self._executors = executors.copy() self._executors.setdefault(StubTaskExecutor, StubTaskExecutor()) -def execute(self, ctx, resuming=False): +def execute(self, ctx, resuming=False, retry_failed=False): """ Executes the workflow. """ if resuming: -events.on_resume_workflow_signal.send(ctx) +events.on_resume_workflow_signal.send(ctx, retry_failed=retry_failed) tasks_tracker = _TasksTracker(ctx) + try: events.start_workflow_signal.send(ctx) while True: @@ -68,8 +69,15 @@ class Engine(logger.LoggerMixin): if cancel: self._terminate_tasks(tasks_tracker.executing_tasks) events.on_cancelled_workflow_signal.send(ctx) -
[GitHub] incubator-ariatosca pull request #143: ARIA-254 Create multiple nodes per te...
Github user tliron commented on a diff in the pull request: https://github.com/apache/incubator-ariatosca/pull/143#discussion_r126353998 --- Diff: extensions/aria_extension_tosca/simple_v1_0/modeling/capabilities.py --- @@ -162,6 +164,30 @@ def convert_capability_from_definition_to_assignment(context, presentation, cont return CapabilityAssignment(name=presentation._name, raw=raw, container=container) +def merge_capability_definition(context, presentation, capability_definition, +from_capability_definition): --- End diff -- Because I prefer it to fit with the style of the rest of code right now, to keep it consistent. I would like to change it, but everything together and give it some more thought. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-ariatosca pull request #143: ARIA-254 Create multiple nodes per te...
Github user tliron commented on a diff in the pull request: https://github.com/apache/incubator-ariatosca/pull/143#discussion_r126353859 --- Diff: aria/modeling/service_template.py --- @@ -690,19 +666,104 @@ def dump(self): console.puts(context.style.meta(self.description)) with context.style.indent: console.puts('Type: {0}'.format(context.style.type(self.type.name))) -console.puts('Instances: {0:d} ({1:d}{2})'.format( -self.default_instances, -self.min_instances, -' to {0:d}'.format(self.max_instances) -if self.max_instances is not None -else ' or more')) utils.dump_dict_values(self.properties, 'Properties') utils.dump_dict_values(self.attributes, 'Attributes') utils.dump_interfaces(self.interface_templates) utils.dump_dict_values(self.artifact_templates, 'Artifact templates') utils.dump_dict_values(self.capability_templates, 'Capability templates') utils.dump_list_values(self.requirement_templates, 'Requirement templates') +@property +def next_index(self): +""" +Next available node index. + +:returns: node index +:rtype: int +""" + +max_index = 0 +if self.nodes: +max_index = max(int(n.name.rsplit('_', 1)[-1]) for n in self.nodes) +return max_index + 1 + +@property +def next_name(self): +""" +Next available node name. + +:returns: node name +:rtype: basestring +""" + +return '{name}_{index}'.format(name=self.name, index=self.next_index) + +@property +def scaling(self): +scaling = {} + +def extract_property(properties, name): +if name in scaling: +return +prop = properties.get(name) +if (prop is not None) and (prop.type_name == 'integer') and (prop.value is not None): +scaling[name] = prop.value + +def extract_properties(properties): +extract_property(properties, 'min_instances') +extract_property(properties, 'max_instances') +extract_property(properties, 'default_instances') + +def default_property(name, value): +if name not in scaling: +scaling[name] = value + +# From our scaling capabilities +for capability_template in self.capability_templates.itervalues(): +if capability_template.type.role == 'scaling': +extract_properties(capability_template.properties) + +# From service scaling policies +for policy_template in self.service_template.policy_templates.itervalues(): +if policy_template.type.role == 'scaling': +if policy_template.is_for_node_template(self.name): +extract_properties(policy_template.properties) + +# Defaults +default_property('min_instances', 0) +default_property('max_instances', 1) +default_property('default_instances', 1) + +# Validate +# pylint: disable=too-many-boolean-expressions +if (scaling['min_instances'] < 0) or \ --- End diff -- I see no difference in readability between this and using `or`, seems to me just to be bowing to PyLint's weirdness... but I don't mind switching. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---