[GitHub] houshengbo closed pull request #40: Fix the signature issue
houshengbo closed pull request #40: Fix the signature issue URL: https://github.com/apache/incubator-openwhisk-release/pull/40 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tools/install_dependencies.sh b/tools/install_dependencies.sh index ca365bc..38a7c4f 100755 --- a/tools/install_dependencies.sh +++ b/tools/install_dependencies.sh @@ -16,8 +16,7 @@ elif [ $sysOS == "Linux" ];then fi if [ $DISTRO == "Ubuntu" ];then echo "This is Ubuntu." -sudo apt-get install jq -sudo apt-get install gnupg +sudo apt-get install jq gnupg expect fi else echo "Other OS: $sysOS. Please manually install jq library." diff --git a/tools/sign_artifacts.sh b/tools/sign_artifacts.sh index a4b0992..c43a166 100755 --- a/tools/sign_artifacts.sh +++ b/tools/sign_artifacts.sh @@ -40,8 +40,7 @@ for artifact in *.tar.gz; do # The option --passphrase-fd does not work on Mac. `gpg --yes --armor --output ${artifact}.asc --detach-sig ${artifact}` elif [ $sysOS == "Linux" ];then -echo "started to sign it" -`echo $passphrase | gpg -c -q --passphrase-fd 0 --yes --armor --output ${artifact}.asc --detach-sig ${artifact}` +`echo $passphrase | gpg --passphrase-fd 0 --yes --armor --output ${artifact}.asc --detach-sig ${artifact}` fi done diff --git a/tools/travis/import_pgp_key.sh b/tools/travis/import_pgp_key.sh index 01e64e5..3211baf 100755 --- a/tools/travis/import_pgp_key.sh +++ b/tools/travis/import_pgp_key.sh @@ -14,4 +14,4 @@ gpg --import $PARENTDIR/key_pub.gpg echo "Load the private key." gpg --allow-secret-key-import --import $PARENTDIR/key_sec.gpg -gpg --list-keys +expect -c "spawn gpg --edit-key s...@us.ibm.com trust quit; send \"5\ry\r\"; expect eof" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] violagao opened a new pull request #40: Fix the signature issue
violagao opened a new pull request #40: Fix the signature issue URL: https://github.com/apache/incubator-openwhisk-release/pull/40 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats
chetanmeh commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats URL: https://github.com/apache/incubator-openwhisk/pull/3249#issuecomment-365150100 @markusthoemmes The PR now has changes required for travis integration and codecov integration is also working. You can see the report here at https://codecov.io/gh/apache/incubator-openwhisk This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats
codecov-io commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats URL: https://github.com/apache/incubator-openwhisk/pull/3249#issuecomment-365147821 # [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3249?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@d7e59b5`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-openwhisk/pull/3249/graphs/tree.svg?height=150&width=650&token=l0YmsiSAso&src=pr)](https://codecov.io/gh/apache/incubator-openwhisk/pull/3249?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master#3249 +/- ## = Coverage ? 72.46% = Files ? 109 Lines ? 4911 Branches ? 310 = Hits ? 3559 Misses? 1352 Partials ?0 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3249?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `? = absolute (impact)`, `? = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-openwhisk/pull/3249?src=pr&el=footer). Last update [d7e59b5...78acf34](https://codecov.io/gh/apache/incubator-openwhisk/pull/3249?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] pritidesai opened a new pull request #732: adding env. variable support in trigger feed
pritidesai opened a new pull request #732: adding env. variable support in trigger feed URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/732 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] pritidesai commented on issue #730: Multiple projects management
pritidesai commented on issue #730: Multiple projects management URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/730#issuecomment-365102367 so for now, instead of creating annotation as a JSON value, we can have an array of JSON blob, something similar to: ``` "annotations": [ { "key": "managed", "value": [ { "k1": "v1", "k2": "v2" }, { "k1": "v1", "k2": "v2" } ] }, ``` thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo closed pull request #39: Validate the secret key
houshengbo closed pull request #39: Validate the secret key URL: https://github.com/apache/incubator-openwhisk-release/pull/39 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tools/sign_artifacts.sh b/tools/sign_artifacts.sh index 8695d11..a4b0992 100755 --- a/tools/sign_artifacts.sh +++ b/tools/sign_artifacts.sh @@ -40,6 +40,7 @@ for artifact in *.tar.gz; do # The option --passphrase-fd does not work on Mac. `gpg --yes --armor --output ${artifact}.asc --detach-sig ${artifact}` elif [ $sysOS == "Linux" ];then +echo "started to sign it" `echo $passphrase | gpg -c -q --passphrase-fd 0 --yes --armor --output ${artifact}.asc --detach-sig ${artifact}` fi done diff --git a/tools/travis/import_pgp_key.sh b/tools/travis/import_pgp_key.sh index 2998758..01e64e5 100755 --- a/tools/travis/import_pgp_key.sh +++ b/tools/travis/import_pgp_key.sh @@ -13,3 +13,5 @@ gpg --import $PARENTDIR/key_pub.gpg echo "Load the private key." gpg --allow-secret-key-import --import $PARENTDIR/key_sec.gpg + +gpg --list-keys diff --git a/tools/util.sh b/tools/util.sh index 36ddc8b..17720c6 100755 --- a/tools/util.sh +++ b/tools/util.sh @@ -12,9 +12,6 @@ function json_by_key() { function import_key_verify_signature() { key_url=$1 -dir=$2 -cd $dir - echo "Importing PGP keys" curl $key_url | gpg --import && \ echo "[?] GPG keys imported" \ diff --git a/tools/verify_local_artifacts.sh b/tools/verify_local_artifacts.sh index 3a83484..78a94d7 100755 --- a/tools/verify_local_artifacts.sh +++ b/tools/verify_local_artifacts.sh @@ -7,7 +7,6 @@ echo "Verify the local artifacts with the KEYS" SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" source "$SCRIPTDIR/load_config.sh" $1 $2 $3 -mkdir -p $OPENWHISK_SVN cd $OPENWHISK_SVN/$REMOTE_PATH -import_key_verify_signature $STAGE_URL/KEYS $OPENWHISK_SVN/$REMOTE_PATH +import_key_verify_signature $STAGE_URL/KEYS diff --git a/tools/verify_remote_artifacts.sh b/tools/verify_remote_artifacts.sh index ad4f330..0d50d24 100755 --- a/tools/verify_remote_artifacts.sh +++ b/tools/verify_remote_artifacts.sh @@ -7,7 +7,8 @@ echo "Verify the remote artifacts with the KEYS" SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" source "$SCRIPTDIR/load_config.sh" $1 $2 $3 -mkdir -p $OPENWHISK_SVN +rm -rf $OPENWHISK_SVN +mkdir -p $OPENWHISK_SVN/$REMOTE_PATH cd $OPENWHISK_SVN # Remove the local folder, because we are about to download the artifacts from the staging folder. @@ -15,7 +16,6 @@ rm -rf $REMOTE_PATH # Check out the artifacts. svn co $CURRENT_VERSION_URL $REMOTE_PATH - cd $REMOTE_PATH -import_key_verify_signature $STAGE_URL/KEYS $OPENWHISK_SVN/$REMOTE_PATH +import_key_verify_signature $STAGE_URL/KEYS This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] violagao opened a new pull request #39: Validate the secret key
violagao opened a new pull request #39: Validate the secret key URL: https://github.com/apache/incubator-openwhisk-release/pull/39 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo closed pull request #38: Add credentials to the svn mkdir command
houshengbo closed pull request #38: Add credentials to the svn mkdir command URL: https://github.com/apache/incubator-openwhisk-release/pull/38 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tools/upload_artifacts.sh b/tools/upload_artifacts.sh index 2b58ecf..df68d8e 100755 --- a/tools/upload_artifacts.sh +++ b/tools/upload_artifacts.sh @@ -37,7 +37,7 @@ if [[ `wget -S --spider $CURRENT_VERSION_URL 2>&1 | grep 'HTTP/1.1 200 OK'` ]]; fi # Create an empty folder named ${REMOTE_PATH} in the remote staging folder -svn mkdir -m "Create the directory for ${version} in staging." $CURRENT_VERSION_URL +svn mkdir -m "Create the directory for ${version} in staging." $CURRENT_VERSION_URL $CREDENTIALS cd $OPENWHISK_SVN This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] violagao opened a new pull request #38: Add credentials to the svn mkdir command
violagao opened a new pull request #38: Add credentials to the svn mkdir command URL: https://github.com/apache/incubator-openwhisk-release/pull/38 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows closed pull request #731: Adding rule annotations for managed project
mrutkows closed pull request #731: Adding rule annotations for managed project URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/731 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/Godeps/Godeps.json b/Godeps/Godeps.json index 1758c5d9..d6613b32 100644 --- a/Godeps/Godeps.json +++ b/Godeps/Godeps.json @@ -105,11 +105,11 @@ }, { "ImportPath": "github.com/apache/incubator-openwhisk-client-go/whisk", - "Rev": "d7cab4297a25e2cc25492b85b64e7ec000da9ffb" + "Rev": "a81a9be21faae877e858c6ab9122a40419577f95" }, { "ImportPath": "github.com/apache/incubator-openwhisk-client-go/wski18n", - "Rev": "d7cab4297a25e2cc25492b85b64e7ec000da9ffb" + "Rev": "a81a9be21faae877e858c6ab9122a40419577f95" }, { "ImportPath": "github.com/pelletier/go-buffruneio", diff --git a/deployers/manifestreader.go b/deployers/manifestreader.go index 0442abc4..6c433fb3 100644 --- a/deployers/manifestreader.go +++ b/deployers/manifestreader.go @@ -88,7 +88,7 @@ func (deployer *ManifestReader) HandleYaml(sdeployer *ServiceDeployer, manifestP return wskderrors.NewYAMLFileFormatError(manifestName, err) } - rules, err := manifestParser.ComposeRulesFromAllPackages(manifest) + rules, err := manifestParser.ComposeRulesFromAllPackages(manifest, ma) if err != nil { return wskderrors.NewYAMLFileFormatError(manifestName, err) } diff --git a/deployers/servicedeployer.go b/deployers/servicedeployer.go index 14fc1c8b..1bb2f5e4 100644 --- a/deployers/servicedeployer.go +++ b/deployers/servicedeployer.go @@ -603,9 +603,44 @@ func (deployer *ServiceDeployer) RefreshManagedTriggers(ma map[string]interface{ return nil } -// TODO() engage community to allow metadata (annotations) on Rules // TODO() display "update" | "synced" messages pre/post func (deployer *ServiceDeployer) RefreshManagedRules(ma map[string]interface{}) error { + options := whisk.RuleListOptions{} + // Get list of rules in your namespace + rules, _, err := deployer.Client.Rules.List(&options) + if err != nil { + return err + } + // iterate over the list of rules to determine whether any of them was part of managed project + // and now deleted from manifest file we can determine that from the managed annotation + // If a rule has attached managed annotation with the project name equals to the current project name + // but the project hash is different (project hash differs since the rule is deleted from the manifest file) + for _, rule := range rules { + // rule has attached managed annotation + if a := rule.Annotations.GetValue(utils.MANAGED); a != nil { + // decode the JSON blob and retrieve __OW_PROJECT_NAME and __OW_PROJECT_HASH + ta := a.(map[string]interface{}) + if ta[utils.OW_PROJECT_NAME] == ma[utils.OW_PROJECT_NAME] && ta[utils.OW_PROJECT_HASH] != ma[utils.OW_PROJECT_HASH] { + // we have found a trigger which was earlier part of the current project + output := wski18n.T(wski18n.ID_MSG_MANAGED_FOUND_DELETED_X_key_X_name_X_project_X, + map[string]interface{}{ + wski18n.KEY_KEY: parsers.YAML_KEY_RULE, + wski18n.KEY_NAME:rule.Name, + wski18n.KEY_PROJECT: ma[utils.OW_PROJECT_NAME]}) + wskprint.PrintOpenWhiskWarning(output) + + var err error + err = retry(DEFAULT_ATTEMPTS, DEFAULT_INTERVAL, func() error { + _, err := deployer.Client.Rules.Delete(rule.Name) + return err + }) + + if err != nil { + return err + } + } + } + } return nil } diff --git a/parsers/manifest_parser.go b/parsers/manifest_parser.go index b8405b12..88a3b032 100644 --- a/parsers/manifest_parser.go +++ b/parsers/manifest_parser.go @@ -827,12 +827,12 @@ func (dm *YAMLParser) ComposeTriggers(filePath string, pkg Package, ma whisk.Key return t1, nil } -func (dm *YAMLParser
[GitHub] houshengbo closed pull request #37: Point the correct path of the public key and private key
houshengbo closed pull request #37: Point the correct path of the public key and private key URL: https://github.com/apache/incubator-openwhisk-release/pull/37 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tools/travis/import_pgp_key.sh b/tools/travis/import_pgp_key.sh index fca5112..2998758 100755 --- a/tools/travis/import_pgp_key.sh +++ b/tools/travis/import_pgp_key.sh @@ -5,10 +5,11 @@ set -e echo "Import the PGP key." SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" +PARENTDIR="$(dirname "$SCRIPTDIR")" # Load the public key located in the repo of openwhisk release. echo "Load the public key." -gpg --import $SCRIPTDIR/key_pub.gpg +gpg --import $PARENTDIR/key_pub.gpg echo "Load the private key." -gpg --allow-secret-key-import --import $SCRIPTDIR/key_sec.gpg +gpg --allow-secret-key-import --import $PARENTDIR/key_sec.gpg This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] violagao opened a new pull request #37: Point the correct path of the public key and private key
violagao opened a new pull request #37: Point the correct path of the public key and private key URL: https://github.com/apache/incubator-openwhisk-release/pull/37 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo closed pull request #36: Point the correct path of the public key and private key
houshengbo closed pull request #36: Point the correct path of the public key and private key URL: https://github.com/apache/incubator-openwhisk-release/pull/36 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tools/travis/import_pgp_key.sh b/tools/travis/import_pgp_key.sh index fca5112..2998758 100755 --- a/tools/travis/import_pgp_key.sh +++ b/tools/travis/import_pgp_key.sh @@ -5,10 +5,11 @@ set -e echo "Import the PGP key." SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" +PARENTDIR="$(dirname "$SCRIPTDIR")" # Load the public key located in the repo of openwhisk release. echo "Load the public key." -gpg --import $SCRIPTDIR/key_pub.gpg +gpg --import $PARENTDIR/key_pub.gpg echo "Load the private key." -gpg --allow-secret-key-import --import $SCRIPTDIR/key_sec.gpg +gpg --allow-secret-key-import --import $PARENTDIR/key_sec.gpg This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on issue #3275: WIP: GET trigger now returns matched rules in JSON payload
rabbah commented on issue #3275: WIP: GET trigger now returns matched rules in JSON payload URL: https://github.com/apache/incubator-openwhisk/pull/3275#issuecomment-365079127 We should impose a limit on how many rules a single trigger can activate. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mdeuser opened a new pull request #3275: WIP: GET trigger now returns matched rules in JSON payload
mdeuser opened a new pull request #3275: WIP: GET trigger now returns matched rules in JSON payload URL: https://github.com/apache/incubator-openwhisk/pull/3275 Augment existing GET trigger payload with a new field, `rules` that describes the trigger's matched rules and the rule status. ``` { "namespace": "guest", "name": "trig-with-inactive-rule", "version": "0.0.1", "limits": {}, "publish": false, "rules": { "guest/inactiverule": { "action": { "name": "web-echo-env", "path": "guest" }, "status": "inactive" } } } ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo opened a new pull request #36: Point the correct path of the public key and private key
houshengbo opened a new pull request #36: Point the correct path of the public key and private key URL: https://github.com/apache/incubator-openwhisk-release/pull/36 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr closed pull request #35: Fix the encryption issue
csantanapr closed pull request #35: Fix the encryption issue URL: https://github.com/apache/incubator-openwhisk-release/pull/35 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/.travis.yml b/.travis.yml index de6ffd0..2f095b8 100644 --- a/.travis.yml +++ b/.travis.yml @@ -19,6 +19,11 @@ jobs: - stage: package script: ./tools/travis/package_source_code.sh $(dirname "$TRAVIS_BUILD_DIR") $SVN_USERNAME $SVN_PASSWORD +before_install: + - if [ "$TRAVIS_PULL_REQUEST" == "false" ] ; then + openssl aes-256-cbc -K $encrypted_2030e681f34a_key -iv $encrypted_2030e681f34a_iv -in tools/key_sec.gpg.enc -out tools/key_sec.gpg -d; +fi + after_failure: - echo "failed build" - ./tools/travis/cleanup.sh diff --git a/tools/travis/package_source_code.sh b/tools/travis/package_source_code.sh index e14244e..695e28f 100755 --- a/tools/travis/package_source_code.sh +++ b/tools/travis/package_source_code.sh @@ -21,7 +21,6 @@ fi "$PARENTDIR/package_source_code.sh" $WORK_DIR $SVN_USERNAME $SVN_PASSWORD if [ "$TRAVIS_EVENT_TYPE" == "push" ] ; then -openssl aes-256-cbc -K $encrypted_2030e681f34a_key -iv $encrypted_2030e681f34a_iv -in $PARENTDIR/key_sec.gpg.enc -out $PARENTDIR/key_sec.gpg -d "$SCRIPTDIR/import_pgp_key.sh" "$PARENTDIR/sign_artifacts.sh" $WORK_DIR "$PARENTDIR/upload_artifacts.sh" $WORK_DIR $SVN_USERNAME $SVN_PASSWORD This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo opened a new pull request #35: Fix the encryption issue
houshengbo opened a new pull request #35: Fix the encryption issue URL: https://github.com/apache/incubator-openwhisk-release/pull/35 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] csantanapr closed pull request #33: Add the support to verify the artifacts with the key
csantanapr closed pull request #33: Add the support to verify the artifacts with the key URL: https://github.com/apache/incubator-openwhisk-release/pull/33 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tools/clean_remote_stage_artifacts.sh b/tools/clean_remote_stage_artifacts.sh new file mode 100755 index 000..0577f7a --- /dev/null +++ b/tools/clean_remote_stage_artifacts.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash + +set -e + +echo "Clean the remote artifacts in staging directory" + +SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" +source "$SCRIPTDIR/util.sh" + +CONFIG=$(read_file $SCRIPTDIR/config.json) +version_key="version" +version_major=$(json_by_key "$CONFIG" ${version_key}.major) +version_minor=$(json_by_key "$CONFIG" ${version_key}.minor) + +version=$version_major-$version_minor +REMOTE_PATH="openwhisk-$version" +STAGE_URL=$(json_by_key "$CONFIG" "stage_url") +CURRENT_VERSION_URL="$STAGE_URL/${REMOTE_PATH}/" +CREDENTIALS="" + +SVN_USERNAME=$1 +SVN_PASSWORD=$2 + +if [ ! -z "$SVN_USERNAME" ] && [ ! -z "$SVN_PASSWORD" ];then +CREDENTIALS="--username $SVN_USERNAME --password $SVN_PASSWORD --non-interactive" +fi + +if [[ `wget -S --spider $CURRENT_VERSION_URL 2>&1 | grep 'HTTP/1.1 200 OK'` ]]; then +svn delete $CURRENT_VERSION_URL -m "Removing Apache OpenWhisk release ${version} from staging." $CREDENTIALS +fi diff --git a/tools/install_dependencies.sh b/tools/install_dependencies.sh index f48e33f..ca365bc 100755 --- a/tools/install_dependencies.sh +++ b/tools/install_dependencies.sh @@ -7,7 +7,6 @@ if [ $sysOS == "Darwin" ];then echo "This is MacOS." brew install jq brew install gpg - brew install md5sha1sum elif [ $sysOS == "Linux" ];then echo "This is Linux." if [ -f /etc/lsb-release -o -d /etc/lsb-release.d ]; then diff --git a/tools/key_pub.gpg b/tools/key_pub.gpg new file mode 100644 index 000..febbeaf --- /dev/null +++ b/tools/key_pub.gpg @@ -0,0 +1,29 @@ +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBFqB+RMBEACeKz2rzESI9Hch8ZUEY2mrTsCumXsFn8YAUkiuMN4g6Q5PvoRU +k0tkD0wdQDg9Tqd5DlOaJMFaP25rvchR7OCgygf5DaKW4IsUh7FN5uID94ozwNvD +oznyl5OTwzCB8jdRz5pMTRNx989yi0z0kMhIqXULQeCBWMdbv6wVcRlGmwWO6T42 +b2hi8gPZJjP++577WjGZWTV/NgOLyFPRYIn7phjBLkCfD15fGVzy+icXCxeunTgK +T0qxD/r+6iTtxyWMkLQxLByZWxRUJCdt03oQVVwrL7SJHdKYvU5ElOUr1J4/axN+ +x43+Z5kz06ZZghewzdCMvnwf3IaEdJmrksY1U3wije1wXGKs7f9Y+eS+E9tVDuI/ +yLrhFs1/A6uNtuvfSqvHzaWWNUUl4/YP8VgPttaWKBBNw/EL2i3di9RQAfTMqRsk +JBx2bLORu/MjAnH3nBztw3MHI6ll4u2xb03k1iW9Uc+lh76V63DcykVlhL0renCR +ccZ3cGGi9vrfZ8pQHcPTLxK/l++QRUzewHEUM2nPOSW9DRe1jR128DhTr4p5yaKF +z5vvtjU+GP+cZFM8HkY1RLrNA2/a4G/gHGQqdPybomSeq7hC0GtX6U5ESHeOqyH1 +hDblT7nldvyw1nb52+yzYjuhiJo/TB/F/7teAmHyDmOIot6EEAx+Onh6/wARAQAB +tDxWaW5jZW50IEhvdSAoUmVsZWFzZSBtYW5hZ2VyIG9mIE9wZW5XaGlzaykgPHNo +b3VAdXMuaWJtLmNvbT6JAk4EEwEIADgWIQT2AFplgI3xoq7hv/aeJ0HSiuatCgUC +WoH5EwIbLwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRCeJ0HSiuatClniD/99 +FDXY/Ju8i7+wmnpQpJof+242KhJEumttKn/SRkU79zCrsV3jT+z9Il8CbpPYyPVl +BZPcHYs+1goky3yVJm+tDATtxXYmyeLvU+LcmZA2ftufWaakJti6uAt6gl/CvrPN +Xdu44hcISCZs4b725A3InfGQbBGEppJfa0PxQ8Yx5yktNTom/DuzuaII70DoIffe +rFIs0Bge4m9RDQ21VLxZGyg5l8xhc/viXzASisCiXGpXnRMiwcXwRgUd11VHsTQ+ +iueFBxkfk7O1whobs232iUy2Db42/OtL39fn8HRlkfhV6fzUieX0Z7lcc+hpzLMc +HP/1LGxH5I+LnTN0iZpgZzDiv8HS7toQ3DzMDyMDypskKyrQty+Z0FOLuGFOY06y +rbE6yc9doQBhTugVYQznia+v0G8rrwQwPVsKZnBmEzo1GT16jzGpse2NfPOMpbLk +WJ3a1SNb8mtGS+XFFGQ/y9QNquBFD5kLjptSDdVbNexyxZ6SDpQFzulByonGDpqe +Xez7Ho9kklOb3/1sH918zw6SlWWIhf4HOmZeYyucS6bIGBFnu+r+3wzSvhmJ2IlX +53rX4F/n4PYfS5TEa5rmjxzy+sww1nEdo+/sYF3KiPysLn5h/Y9VtzSh1dsh1mV0 +O/9Ulqw3TsDrGa2k7Kx2PVHVx3KYMvpvskyP51U2EA== +=/f4p +-END PGP PUBLIC KEY BLOCK- diff --git a/tools/key_sec.gpg.enc b/tools/key_sec.gpg.enc new file mode 100644 index 000..7761b7f Binary files /dev/null and b/tools/key_sec.gpg.enc differ diff --git a/tools/export_pgp_key.sh b/tools/load_config.sh similarity index 63% rename from tools/export_pgp_key.sh rename to tools/load_config.sh index 8eafcac..0d6b2a3 100755 --- a/tools/export_pgp_key.sh +++ b/tools/load_config.sh @@ -1,27 +1,31 @@ #!/usr/bin/env bash -set -e +WORK_DIR=${1:-"$HOME"} +SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" -echo "Export the PGP key." +SVN_USERNAME=$2 +SVN_PASSWORD=$3 +CREDENTIALS="" + +if [ ! -z "$SVN_USERNAME" ] && [ ! -z "$SVN_PASSWORD" ];then +CREDENTIALS="--username $SVN_USERNAME --password $SVN_PASSWORD --non-interactive" +fi -WORK_DIR=${1:-"$HOME"} -PGP_EMAIL=${2:-"s...@us.ibm.com"} OPENWHISK_SOURCE_DIR="$WORK_DIR/openwhisk_sources" OPENWHISK_SVN="$OPENWHISK_SOURCE_DIR/openwhisk" -SCRIPTDIR="$(cd $(dirname "$0")/ && pwd)" source "$SCRIPTDIR/util.sh" CONFIG=$(read_file $SCRIPTDIR/config.json) repos=$(echo $(json_by_key "$CONFIG" "RepoList") | sed
[GitHub] dgrove-oss opened a new pull request #158: Add simple loadtesting jobs
dgrove-oss opened a new pull request #158: Add simple loadtesting jobs URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/158 Add simple latency and throughput microbenchmaking jobs based on incubator-openwhisk-performance. Also add pod affinity/anti-affinity annotations to help guide pods to specific kinds of nodes when deploying on larger clusters with mixes of worker nodes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] pritidesai opened a new pull request #731: Adding rule annotations for managed project
pritidesai opened a new pull request #731: Adding rule annotations for managed project URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/731 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2957: splunk logstore
tysonnorris commented on a change in pull request #2957: splunk logstore URL: https://github.com/apache/incubator-openwhisk/pull/2957#discussion_r167671124 ## File path: core/invoker/src/main/resources/application.conf ## @@ -27,4 +27,6 @@ whisk { extra-args: {} } + logstore { + } Review comment: not needed; will remove This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2957: splunk logstore
tysonnorris commented on a change in pull request #2957: splunk logstore URL: https://github.com/apache/incubator-openwhisk/pull/2957#discussion_r167670797 ## File path: common/scala/src/main/resources/application.conf ## @@ -108,3 +108,15 @@ whisk { std = 256 m } } + +//whisk.logstore { +//splunk { +//host = "splunk-host" +//port = 8080 +//user = "splunk-user" +//password = "splunk-pass" +//index = "splunk-index" +//log-message-field = "log_message" +//activation-id-field = "activation_id" +//} +//} Review comment: just for sample config, will remove till there is a better place to locate this This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2957: splunk logstore
tysonnorris commented on a change in pull request #2957: splunk logstore URL: https://github.com/apache/incubator-openwhisk/pull/2957#discussion_r167670126 ## File path: common/scala/src/main/scala/whisk/core/containerpool/logging/SplunkLogStore.scala ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool.logging + +import akka.actor.ActorSystem +import akka.http.scaladsl.Http +import java.time.ZoneId +import java.time.format.DateTimeFormatter + +import akka.http.scaladsl.client.RequestBuilding.Post +import akka.http.scaladsl.model.FormData +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.headers.Authorization +import akka.http.scaladsl.model.headers.BasicHttpCredentials +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.stream.ActorMaterializer +import akka.stream.OverflowStrategy +import akka.stream.QueueOfferResult +import akka.stream.scaladsl.Flow +import akka.stream.scaladsl.Keep +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import com.typesafe.sslconfig.akka.AkkaSSLConfig +import scala.concurrent.Future +import scala.concurrent.Promise +import scala.util.Failure +import scala.util.Success +import scala.util.Try +import spray.json.DefaultJsonProtocol._ +import spray.json.JsArray +import spray.json._ +import whisk.core.entity.ActivationLogs +import whisk.core.entity.WhiskActivation +import pureconfig._ + +case class SplunkLogStoreConfig(host: String, +port: Int, +username: String, +password: String, +index: String, +logMessageField: String, +activationIdField: String, +disableSNI: Boolean) + +/** + * A Splunk based impl of LogDriverLogStore. Logs are routed to splunk via docker log driver, and retrieved via Splunk REST API + * + * @param actorSystem + * @param httpFlow Optional Flow to use for HttpRequest handling (to enable stream based tests) + */ +class SplunkLogStore( + actorSystem: ActorSystem, + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + splunkConfig: SplunkLogStoreConfig = loadConfigOrThrow[SplunkLogStoreConfig]("whisk.logstore.splunk")) +extends LogDriverLogStore(actorSystem) { + implicit val as = actorSystem + implicit val ec = as.dispatcher + implicit val materializer = ActorMaterializer() + + private val splunkApi = "/services/search/jobs" //see http://docs.splunk.com/Documentation/Splunk/6.6.3/RESTREF/RESTsearch#search.2Fjobs Review comment: Can, but have to use `Post(Uri(path = splunkApi))` in that case? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #2957: splunk logstore
markusthoemmes commented on a change in pull request #2957: splunk logstore URL: https://github.com/apache/incubator-openwhisk/pull/2957#discussion_r167669851 ## File path: common/scala/src/main/scala/whisk/core/containerpool/logging/SplunkLogStore.scala ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool.logging + +import akka.actor.ActorSystem +import akka.http.scaladsl.Http +import java.time.ZoneId +import java.time.format.DateTimeFormatter + +import akka.http.scaladsl.client.RequestBuilding.Post +import akka.http.scaladsl.model.FormData +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.headers.Authorization +import akka.http.scaladsl.model.headers.BasicHttpCredentials +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.stream.ActorMaterializer +import akka.stream.OverflowStrategy +import akka.stream.QueueOfferResult +import akka.stream.scaladsl.Flow +import akka.stream.scaladsl.Keep +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import com.typesafe.sslconfig.akka.AkkaSSLConfig +import scala.concurrent.Future +import scala.concurrent.Promise +import scala.util.Failure +import scala.util.Success +import scala.util.Try +import spray.json.DefaultJsonProtocol._ +import spray.json.JsArray +import spray.json._ +import whisk.core.entity.ActivationLogs +import whisk.core.entity.WhiskActivation +import pureconfig._ + +case class SplunkLogStoreConfig(host: String, +port: Int, +username: String, +password: String, +index: String, +logMessageField: String, +activationIdField: String, +disableSNI: Boolean) + +/** + * A Splunk based impl of LogDriverLogStore. Logs are routed to splunk via docker log driver, and retrieved via Splunk REST API + * + * @param actorSystem + * @param httpFlow Optional Flow to use for HttpRequest handling (to enable stream based tests) + */ +class SplunkLogStore( + actorSystem: ActorSystem, + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + splunkConfig: SplunkLogStoreConfig = loadConfigOrThrow[SplunkLogStoreConfig]("whisk.logstore.splunk")) +extends LogDriverLogStore(actorSystem) { + implicit val as = actorSystem + implicit val ec = as.dispatcher + implicit val materializer = ActorMaterializer() + + private val splunkApi = "/services/search/jobs" //see http://docs.splunk.com/Documentation/Splunk/6.6.3/RESTREF/RESTsearch#search.2Fjobs + + val log = actorSystem.log + val maxPendingRequests = 500 + + val defaultHttpFlow = Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]]( +host = splunkConfig.host, +port = splunkConfig.port, +connectionContext = + if (splunkConfig.disableSNI) +Http().createClientHttpsContext(AkkaSSLConfig().mapSettings(s => s.withLoose(s.loose.withDisableSNI(true + else Http().defaultClientHttpsContext) + + override def fetchLogs(activation: WhiskActivation): Future[ActivationLogs] = { + +//example curl request: +//curl -u username:password -k https://splunkhost:port/services/search/jobs -d exec_mode=oneshot -d output_mode=json -d "search=search index=\"someindex\" | spath=activation_id | search activation_id=a930e5ae4ad4455c8f2505d665aad282 | table log_message" -d "earliest_time=2017-08-29T12:00:00" -d "latest_time=2017-10-29T12:00:00" +//example response: +// {"preview":false,"init_offset":0,"messages":[],"fields":[{"name":"log_message"}],"results":[{"log_message":"some log message"}], "highlighted":{}} +val search = + s"""search index="${splunkConfig.index}"| spath ${splunkConfig.activationIdField}| search ${splunkConfig.activationIdField}=${activation.activationId.toString}| table ${splunkConfig.logMessageField}""" + +val formatter = DateTimeFormatter.ofPattern(
[GitHub] dubeejw commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData
dubeejw commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData URL: https://github.com/apache/incubator-openwhisk/pull/3109#discussion_r167669280 ## File path: common/scala/src/main/scala/whisk/core/entity/Exec.scala ## @@ -95,9 +95,23 @@ sealed abstract class CodeExec[+T <% SizeConversion] extends Exec { sealed abstract class ExecMetaData extends ExecMetaDataBase { + /** An entrypoint (typically name of 'main' function). 'None' means a default value will be used. */ + val entryPoint: Option[String] + + /** The runtime image (either built-in or a public image). */ + val image: ImageName + /** Indicates if a container image is required from the registry to execute the action. */ val pull: Boolean + /** + * Indicates whether the code is stored in a text-readable or binary format. + * The binary bit may be read from the database but currently it is always computed + * when the "code" is moved to an attachment this may get changed to avoid recomputing + * the binary property. + */ + val binary: Boolean Review comment: Hmm, I think they all should be there except `code`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2957: splunk logstore
tysonnorris commented on a change in pull request #2957: splunk logstore URL: https://github.com/apache/incubator-openwhisk/pull/2957#discussion_r167667771 ## File path: common/scala/src/main/scala/whisk/core/containerpool/logging/SplunkLogStore.scala ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool.logging + +import akka.actor.ActorSystem +import akka.http.scaladsl.Http +import java.time.ZoneId +import java.time.format.DateTimeFormatter + +import akka.http.scaladsl.client.RequestBuilding.Post +import akka.http.scaladsl.model.FormData +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.headers.Authorization +import akka.http.scaladsl.model.headers.BasicHttpCredentials +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.stream.ActorMaterializer +import akka.stream.OverflowStrategy +import akka.stream.QueueOfferResult +import akka.stream.scaladsl.Flow +import akka.stream.scaladsl.Keep +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import com.typesafe.sslconfig.akka.AkkaSSLConfig +import scala.concurrent.Future +import scala.concurrent.Promise +import scala.util.Failure +import scala.util.Success +import scala.util.Try +import spray.json.DefaultJsonProtocol._ +import spray.json.JsArray +import spray.json._ +import whisk.core.entity.ActivationLogs +import whisk.core.entity.WhiskActivation +import pureconfig._ + +case class SplunkLogStoreConfig(host: String, +port: Int, +username: String, +password: String, +index: String, +logMessageField: String, +activationIdField: String, +disableSNI: Boolean) + +/** + * A Splunk based impl of LogDriverLogStore. Logs are routed to splunk via docker log driver, and retrieved via Splunk REST API + * + * @param actorSystem + * @param httpFlow Optional Flow to use for HttpRequest handling (to enable stream based tests) + */ +class SplunkLogStore( + actorSystem: ActorSystem, + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + splunkConfig: SplunkLogStoreConfig = loadConfigOrThrow[SplunkLogStoreConfig]("whisk.logstore.splunk")) +extends LogDriverLogStore(actorSystem) { + implicit val as = actorSystem + implicit val ec = as.dispatcher + implicit val materializer = ActorMaterializer() + + private val splunkApi = "/services/search/jobs" //see http://docs.splunk.com/Documentation/Splunk/6.6.3/RESTREF/RESTsearch#search.2Fjobs + + val log = actorSystem.log + val maxPendingRequests = 500 + + val defaultHttpFlow = Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]]( +host = splunkConfig.host, +port = splunkConfig.port, +connectionContext = + if (splunkConfig.disableSNI) +Http().createClientHttpsContext(AkkaSSLConfig().mapSettings(s => s.withLoose(s.loose.withDisableSNI(true + else Http().defaultClientHttpsContext) + + override def fetchLogs(activation: WhiskActivation): Future[ActivationLogs] = { + +//example curl request: +//curl -u username:password -k https://splunkhost:port/services/search/jobs -d exec_mode=oneshot -d output_mode=json -d "search=search index=\"someindex\" | spath=activation_id | search activation_id=a930e5ae4ad4455c8f2505d665aad282 | table log_message" -d "earliest_time=2017-08-29T12:00:00" -d "latest_time=2017-10-29T12:00:00" +//example response: +// {"preview":false,"init_offset":0,"messages":[],"fields":[{"name":"log_message"}],"results":[{"log_message":"some log message"}], "highlighted":{}} +val search = + s"""search index="${splunkConfig.index}"| spath ${splunkConfig.activationIdField}| search ${splunkConfig.activationIdField}=${activation.activationId.toString}| table ${splunkConfig.logMessageField}""" + +val formatter = DateTimeFormatter.ofPattern("YY
[GitHub] houshengbo closed pull request #62: Adding support for annotations in Rules
houshengbo closed pull request #62: Adding support for annotations in Rules URL: https://github.com/apache/incubator-openwhisk-client-go/pull/62 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/whisk/rule.go b/whisk/rule.go index 624d66ca..c9b56a09 100644 --- a/whisk/rule.go +++ b/whisk/rule.go @@ -31,13 +31,14 @@ type RuleService struct { } type Rule struct { - Namespace string `json:"namespace,omitempty"` - Name string `json:"name,omitempty"` - Version string `json:"version,omitempty"` - Statusstring `json:"status"` - Trigger interface{} `json:"trigger"` - Actioninterface{} `json:"action"` - Publish *bool `json:"publish,omitempty"` + Namespace string `json:"namespace,omitempty"` + Namestring `json:"name,omitempty"` + Version string `json:"version,omitempty"` + Annotations KeyValueArr `json:"annotations,omitempty"` + Status string `json:"status"` + Trigger interface{} `json:"trigger"` + Action interface{} `json:"action"` + Publish *bool `json:"publish,omitempty"` } type RuleListOptions struct { @@ -113,11 +114,20 @@ func (s *RuleService) Insert(rule *Rule, overwrite bool) (*Rule, *http.Response, ruleName := (&url.URL{Path: rule.Name}).String() route := fmt.Sprintf("rules/%s?overwrite=%t", ruleName, overwrite) - req, err := s.client.NewRequest("PUT", route, rule, IncludeNamespaceInUrl) + routeUrl, err := url.Parse(route) if err != nil { - Debug(DbgError, "http.NewRequest(PUT, %s); error: '%s'\n", route, err) + Debug(DbgError, "url.Parse(%s) error: %s\n", route, err) + errStr := wski18n.T("Invalid request URL '{{.url}}': {{.err}}", + map[string]interface{}{"url": route, "err": err}) + werr := MakeWskError(errors.New(errStr), EXIT_CODE_ERR_GENERAL, DISPLAY_MSG, NO_DISPLAY_USAGE) + return nil, nil, werr + } + + req, err := s.client.NewRequestUrl("PUT", routeUrl, rule, IncludeNamespaceInUrl, AppendOpenWhiskPathPrefix, EncodeBodyAsJson, AuthRequired) + if err != nil { + Debug(DbgError, "http.NewRequestUrl(PUT, %s, %+v, IncludeNamespaceInUrl, AppendOpenWhiskPathPrefix, EncodeBodyAsJson, AuthRequired); error: '%s'\n", routeUrl, rule, err) errStr := wski18n.T("Unable to create HTTP request for PUT '{{.route}}': {{.err}}", - map[string]interface{}{"route": route, "err": err}) + map[string]interface{}{"route": routeUrl, "err": err}) werr := MakeWskErrorFromWskError(errors.New(errStr), err, EXIT_CODE_ERR_GENERAL, DISPLAY_MSG, NO_DISPLAY_USAGE) return nil, nil, werr } This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] tysonnorris commented on a change in pull request #2957: splunk logstore
tysonnorris commented on a change in pull request #2957: splunk logstore URL: https://github.com/apache/incubator-openwhisk/pull/2957#discussion_r167666011 ## File path: common/scala/src/main/scala/whisk/core/containerpool/logging/SplunkLogStore.scala ## @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package whisk.core.containerpool.logging + +import akka.actor.ActorSystem +import akka.http.scaladsl.Http +import java.time.ZoneId +import java.time.format.DateTimeFormatter + +import akka.http.scaladsl.client.RequestBuilding.Post +import akka.http.scaladsl.model.FormData +import akka.http.scaladsl.model.HttpRequest +import akka.http.scaladsl.model.HttpResponse +import akka.http.scaladsl.model.headers.Authorization +import akka.http.scaladsl.model.headers.BasicHttpCredentials +import akka.http.scaladsl.unmarshalling.Unmarshal +import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._ +import akka.stream.ActorMaterializer +import akka.stream.OverflowStrategy +import akka.stream.QueueOfferResult +import akka.stream.scaladsl.Flow +import akka.stream.scaladsl.Keep +import akka.stream.scaladsl.Sink +import akka.stream.scaladsl.Source +import com.typesafe.sslconfig.akka.AkkaSSLConfig +import scala.concurrent.Future +import scala.concurrent.Promise +import scala.util.Failure +import scala.util.Success +import scala.util.Try +import spray.json.DefaultJsonProtocol._ +import spray.json.JsArray +import spray.json._ +import whisk.core.entity.ActivationLogs +import whisk.core.entity.WhiskActivation +import pureconfig._ + +case class SplunkLogStoreConfig(host: String, +port: Int, +username: String, +password: String, +index: String, +logMessageField: String, +activationIdField: String, +disableSNI: Boolean) + +/** + * A Splunk based impl of LogDriverLogStore. Logs are routed to splunk via docker log driver, and retrieved via Splunk REST API + * + * @param actorSystem + * @param httpFlow Optional Flow to use for HttpRequest handling (to enable stream based tests) + */ +class SplunkLogStore( + actorSystem: ActorSystem, + httpFlow: Option[Flow[(HttpRequest, Promise[HttpResponse]), (Try[HttpResponse], Promise[HttpResponse]), Any]] = None, + splunkConfig: SplunkLogStoreConfig = loadConfigOrThrow[SplunkLogStoreConfig]("whisk.logstore.splunk")) +extends LogDriverLogStore(actorSystem) { + implicit val as = actorSystem + implicit val ec = as.dispatcher + implicit val materializer = ActorMaterializer() + + private val splunkApi = "/services/search/jobs" //see http://docs.splunk.com/Documentation/Splunk/6.6.3/RESTREF/RESTsearch#search.2Fjobs + + val log = actorSystem.log + val maxPendingRequests = 500 + + val defaultHttpFlow = Http().cachedHostConnectionPoolHttps[Promise[HttpResponse]]( +host = splunkConfig.host, +port = splunkConfig.port, +connectionContext = + if (splunkConfig.disableSNI) +Http().createClientHttpsContext(AkkaSSLConfig().mapSettings(s => s.withLoose(s.loose.withDisableSNI(true + else Http().defaultClientHttpsContext) + + override def fetchLogs(activation: WhiskActivation): Future[ActivationLogs] = { + +//example curl request: +//curl -u username:password -k https://splunkhost:port/services/search/jobs -d exec_mode=oneshot -d output_mode=json -d "search=search index=\"someindex\" | spath=activation_id | search activation_id=a930e5ae4ad4455c8f2505d665aad282 | table log_message" -d "earliest_time=2017-08-29T12:00:00" -d "latest_time=2017-10-29T12:00:00" +//example response: +// {"preview":false,"init_offset":0,"messages":[],"fields":[{"name":"log_message"}],"results":[{"log_message":"some log message"}], "highlighted":{}} +val search = + s"""search index="${splunkConfig.index}"| spath ${splunkConfig.activationIdField}| search ${splunkConfig.activationIdField}=${activation.activationId.toString}| table ${splunkConfig.logMessageField}""" + +val formatter = DateTimeFormatter.ofPattern("YY
[GitHub] markusthoemmes commented on issue #3240: Add a loadbalancer with local state and horizontal invoker sharding.
markusthoemmes commented on issue #3240: Add a loadbalancer with local state and horizontal invoker sharding. URL: https://github.com/apache/incubator-openwhisk/pull/3240#issuecomment-365036832 PG3 1842 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3254: Preserve rule status when it is updated
dubeejw commented on a change in pull request #3254: Preserve rule status when it is updated URL: https://github.com/apache/incubator-openwhisk/pull/3254#discussion_r167662102 ## File path: core/controller/src/main/scala/whisk/core/controller/Rules.scala ## @@ -313,7 +314,7 @@ trait WhiskRulesApi extends WhiskCollectionAPI with ReferencedEntities { WhiskTrigger.put(entityStore, oldTrigger.removeRule(ruleName)) } - val triggerLink = ReducedRule(actionName, Status.INACTIVE) + val triggerLink = ReducedRule(actionName, status) Review comment: @rabbah, fixed the tests. Can take care of the problem above when this is done https://github.com/apache/incubator-openwhisk/issues/3274. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dubeejw commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData
dubeejw commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData URL: https://github.com/apache/incubator-openwhisk/pull/3109#discussion_r167661815 ## File path: tests/src/test/scala/whisk/core/controller/test/ActionsApiTests.scala ## @@ -159,26 +159,136 @@ class ActionsApiTests extends ControllerTestCommon with WhiskActionsApi { } } + def getExecPermutations() = { +implicit val tid = transid() + +// BlackBox: binary: true, main: bbMain +val bbAction1 = WhiskAction(namespace, aname(), bb("bb", "RHViZWU=", Some("bbMain"))) +val bbAction1Content = Map("exec" -> Map( + "kind" -> Exec.BLACKBOX, + "code" -> "RHViZWU=", + "image" -> "bb", + "main" -> "bbMain")).toJson.asJsObject +val bbAction1ExecMetaData = blackBoxMetaData("bb", Some("bbMain"), true) + +// BlackBox: binary: false, main: bbMain +val bbAction2 = WhiskAction(namespace, aname(), bb("bb", "", Some("bbMain"))) +val bbAction2Content = + Map("exec" -> Map("kind" -> Exec.BLACKBOX, "code" -> "", "image" -> "bb", "main" -> "bbMain")).toJson.asJsObject +val bbAction2ExecMetaData = blackBoxMetaData("bb", Some("bbMain"), false) + +// BlackBox: binary: true, no main +val bbAction3 = WhiskAction(namespace, aname(), bb("bb", "RHViZWU=")) +val bbAction3Content = + Map("exec" -> Map("kind" -> Exec.BLACKBOX, "code" -> "RHViZWU=", "image" -> "bb")).toJson.asJsObject +val bbAction3ExecMetaData = blackBoxMetaData("bb", None, true) + +// BlackBox: binary: false, no main +val bbAction4 = WhiskAction(namespace, aname(), bb("bb", "")) +val bbAction4Content = Map("exec" -> Map("kind" -> Exec.BLACKBOX, "code" -> "", "image" -> "bb")).toJson.asJsObject +val bbAction4ExecMetaData = blackBoxMetaData("bb", None, false) + +// Attachment: binary: true, main: javaMain +val javaAction1 = WhiskAction(namespace, aname(), javaDefault("RHViZWU=", Some("javaMain"))) +val javaAction1Content = + Map("exec" -> Map("kind" -> JAVA_DEFAULT, "code" -> "RHViZWU=", "main" -> "javaMain")).toJson.asJsObject +val javaAction1ExecMetaData = javaMetaData(Some("javaMain"), true) + +// String: binary: true, main: jsMain +val jsAction1 = WhiskAction(namespace, aname(), jsDefault("RHViZWU=", Some("jsMain"))) +val jsAction1Content = + Map("exec" -> Map("kind" -> NODEJS6, "code" -> "RHViZWU=", "main" -> "jsMain")).toJson.asJsObject +val jsAction1ExecMetaData = js6MetaData(Some("jsMain"), true) + +// String: binary: false, main: jsMain +val jsAction2 = WhiskAction(namespace, aname(), jsDefault("", Some("jsMain"))) +val jsAction2Content = Map("exec" -> Map("kind" -> NODEJS6, "code" -> "", "main" -> "jsMain")).toJson.asJsObject +val jsAction2ExecMetaData = js6MetaData(Some("jsMain"), false) + +// String: binary: true, no main +val jsAction3 = WhiskAction(namespace, aname(), jsDefault("RHViZWU=")) +val jsAction3Content = Map("exec" -> Map("kind" -> NODEJS6, "code" -> "RHViZWU=")).toJson.asJsObject +val jsAction3ExecMetaData = js6MetaData(None, true) + +// String: binary: false, no main +val jsAction4 = WhiskAction(namespace, aname(), jsDefault("")) +val jsAction4Content = Map("exec" -> Map("kind" -> NODEJS6, "code" -> "")).toJson.asJsObject +val jsAction4ExecMetaData = js6MetaData(None, false) + +// Sequence +val component = WhiskAction(namespace, aname(), jsDefault("??")) +put(entityStore, component) +val components = Vector(s"/$namespace/${component.name}").map(stringToFullyQualifiedName(_)) +val seqAction = WhiskAction(namespace, aname(), sequence(components), seqParameters(components)) +val seqActionContent = JsObject( + "exec" -> JsObject("kind" -> "sequence".toJson, "components" -> JsArray(s"/$namespace/${component.name}".toJson))) +val seqActionExecMetaData = sequenceMetaData(components) + +Seq( + (bbAction1, bbAction1Content, bbAction1ExecMetaData), + (bbAction2, bbAction2Content, bbAction2ExecMetaData), + (bbAction3, bbAction3Content, bbAction3ExecMetaData), + (bbAction4, bbAction4Content, bbAction4ExecMetaData), + (javaAction1, javaAction1Content, javaAction1ExecMetaData), + (jsAction1, jsAction1Content, jsAction1ExecMetaData), + (jsAction2, jsAction2Content, jsAction2ExecMetaData), + (jsAction3, jsAction3Content, jsAction3ExecMetaData), + (jsAction4, jsAction4Content, jsAction4ExecMetaData), + (seqAction, seqActionContent, seqActionExecMetaData)) + } + it should "get action using code query parameter" in { implicit val tid = transid() -val action = WhiskAction(namespace, aname(), jsDefault("??"), Parameters("x", "b")) -put(entityStore, action) +getExecPermutations.foreach { + case (action, content, execMetaData) => +val expectedWhiskAction = WhiskAction( + action.namespace, +
[GitHub] dubeejw opened a new issue #3274: Controller should not return a payload for created and deleted entities
dubeejw opened a new issue #3274: Controller should not return a payload for created and deleted entities URL: https://github.com/apache/incubator-openwhisk/issues/3274 When creating entities, the controller should respond with just a status code, 200 OK or 201 CREATED, omitting the created entity from the response. Similarly, when deleting entities a status code, 200 OK or 202 ACCEPTED, should be returned without including the deleted entity in the response. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows commented on issue #34: List BSD / MIT licenses in LICENSE.txt if needed
mrutkows commented on issue #34: List BSD / MIT licenses in LICENSE.txt if needed URL: https://github.com/apache/incubator-openwhisk-release/issues/34#issuecomment-365030344 @daisy-ycguo is this confirmed by Apache whiskers utility? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows opened a new issue #34: List BSD / MIT licenses in LICENSE.txt
mrutkows opened a new issue #34: List BSD / MIT licenses in LICENSE.txt URL: https://github.com/apache/incubator-openwhisk-release/issues/34 See what Apache Spark has done (not sure what criteria they used; assume not just simple reference/import, but repackage with their releases? [https://github.com/apache/spark/blob/master/LICENSE](https://github.com/apache/spark/blob/master/LICENSE) ``` BSD-style licenses The following components are provided under a BSD-style license. See project link for details. The text of each license is also included at licenses/LICENSE-[project].txt. (BSD 3 Clause) netlib core (com.github.fommil.netlib:core:1.1.2 - https://github.com/fommil/netlib-java/core) (BSD 3 Clause) JPMML-Model (org.jpmml:pmml-model:1.2.7 - https://github.com/jpmml/jpmml-model) (BSD License) AntLR Parser Generator (antlr:antlr:2.7.7 - http://www.antlr.org/) (BSD License) ANTLR 4.5.2-1 (org.antlr:antlr4:4.5.2-1 - http://.antlr.org/) (BSD licence) ANTLR ST4 4.0.4 (org.antlr:ST4:4.0.4 - http://www.stringtemplate.org) (BSD licence) ANTLR StringTemplate (org.antlr:stringtemplate:3.2.1 - http://www.stringtemplate.org) (BSD License) Javolution (javolution:javolution:5.5.1 - http://javolution.org) (BSD) JLine (jline:jline:0.9.94 - http://jline.sourceforge.net) (BSD) ParaNamer Core (com.thoughtworks.paranamer:paranamer:2.3 - http://paranamer.codehaus.org/paranamer) (BSD) ParaNamer Core (com.thoughtworks.paranamer:paranamer:2.6 - http://paranamer.codehaus.org/paranamer) (BSD 3 Clause) Scala (http://www.scala-lang.org/download/#License) (Interpreter classes (all .scala files in repl/src/main/scala except for Main.Scala, SparkHelper.scala and ExecutorClassLoader.scala), and for SerializableMapWrapper in JavaUtils.scala) (BSD-like) Scala Actors library (org.scala-lang:scala-actors:2.11.8 - http://www.scala-lang.org/) (BSD-like) Scala Compiler (org.scala-lang:scala-compiler:2.11.8 - http://www.scala-lang.org/) (BSD-like) Scala Compiler (org.scala-lang:scala-reflect:2.11.8 - http://www.scala-lang.org/) (BSD-like) Scala Library (org.scala-lang:scala-library:2.11.8 - http://www.scala-lang.org/) (BSD-like) Scalap (org.scala-lang:scalap:2.11.8 - http://www.scala-lang.org/) (BSD-style) scalacheck (org.scalacheck:scalacheck_2.11:1.10.0 - http://www.scalacheck.org) (BSD-style) spire (org.spire-math:spire_2.11:0.7.1 - http://spire-math.org) (BSD-style) spire-macros (org.spire-math:spire-macros_2.11:0.7.1 - http://spire-math.org) (New BSD License) Kryo (com.esotericsoftware:kryo:3.0.3 - https://github.com/EsotericSoftware/kryo) (New BSD License) MinLog (com.esotericsoftware:minlog:1.3.0 - https://github.com/EsotericSoftware/minlog) (New BSD license) Protocol Buffer Java API (com.google.protobuf:protobuf-java:2.5.0 - http://code.google.com/p/protobuf) (New BSD license) Protocol Buffer Java API (org.spark-project.protobuf:protobuf-java:2.4.1-shaded - http://code.google.com/p/protobuf) (The BSD License) Fortran to Java ARPACK (net.sourceforge.f2j:arpack_combined_all:0.1 - http://f2j.sourceforge.net) (The BSD License) xmlenc Library (xmlenc:xmlenc:0.52 - http://xmlenc.sourceforge.net) (The New BSD License) Py4J (net.sf.py4j:py4j:0.10.6 - http://py4j.sourceforge.net/) (Two-clause BSD-style license) JUnit-Interface (com.novocode:junit-interface:0.10 - http://github.com/szeiger/junit-interface/) (BSD licence) sbt and sbt-launch-lib.bash (BSD 3 Clause) d3.min.js (https://github.com/mbostock/d3/blob/master/LICENSE) (BSD 3 Clause) DPark (https://github.com/douban/dpark/blob/master/LICENSE) (BSD 3 Clause) CloudPickle (https://github.com/cloudpipe/cloudpickle/blob/master/LICENSE) (BSD 2 Clause) Zstd-jni (https://github.com/luben/zstd-jni/blob/master/LICENSE) (BSD license) Zstd (https://github.com/facebook/zstd/blob/v1.3.1/LICENSE) MIT licenses The following components are provided under the MIT License. See project link for details. The text of each license is also included at licenses/LICENSE-[project].txt. (MIT License) JCL 1.1.1 implemented over SLF4J (org.slf4j:jcl-over-slf4j:1.7.5 - http://www.slf4j.org) (MIT License) JUL to SLF4J bridge (org.slf4j:jul-to-slf4j:1.7.5 - http://www.slf4j.org) (MIT License) SLF4J API Module (org.slf4j:slf4j-api:1.7.5 - http://www.slf4j.org) (MIT License) SLF4J LOG4J-12 Binding (org.slf4j:
[GitHub] pritidesai opened a new pull request #62: Adding support for annotations in Rules
pritidesai opened a new pull request #62: Adding support for annotations in Rules URL: https://github.com/apache/incubator-openwhisk-client-go/pull/62 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lionelvillard commented on issue #730: Multiple projects management
lionelvillard commented on issue #730: Multiple projects management URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/730#issuecomment-365022690 Are your projects statically dependent to each other or not? If yes, then `dependencies` might solve your problem by putting the common assets into a "library manifest". Otherwise can you deploy each project to their own namespace? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss opened a new pull request #157: update runtimes.json for OW#3259
dgrove-oss opened a new pull request #157: update runtimes.json for OW#3259 URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/157 Upstream PR moved bypassPullForLocalImages, defaultImagePrefix, and defaultImageTag from runtimes.json to application.conf. Make the same change in the kube copy of runtimes.json. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] pritidesai commented on issue #708: redeploy manifest with rule leaves as disables
pritidesai commented on issue #708: redeploy manifest with rule leaves as disables URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/708#issuecomment-365018487 :( @csantanapr I didn't check your comment here, ended up debugging it :( yup, indeed its a backend issue: `r1` doesn't exist: ``` curl --user user:password -X GET http://192.168.33.13:10001/api/v1/namespaces/guest/rules/r1 {"error":"The requested resource does not exist.","code":1430} ``` Creating `r1` using `PUT`: ``` curl --user user:password -X PUT http://192.168.33.13:10001/api/v1/namespaces/guest/rules/r1 -H "Content-Type: application/json" -d '{"name":"r1","status":"active","trigger":"/guest/trigger1","action":"/guest/action1","publish":false}' {"name":"r1","publish":false,"annotations":[],"version":"0.0.1","status":"active","action":{"path":"guest/","name":"action1"},"namespace":"guest","trigger":{"path":"guest/","name":"trigger1"}} ``` Getting `r1`: ``` curl --user user:password -X GET http://192.168.33.13:10001/api/v1/namespaces/guest/rules/r1 {"name":"r1","publish":false,"annotations":[],"version":"0.0.1","status":"active","action":{"path":"guest/","name":"action1"},"namespace":"guest","trigger":{"path":"guest","name":"trigger1"}} ``` Update rule `PUT r1`: ``` curl --user user:password -X PUT http://192.168.33.13:10001/api/v1/namespaces/guest/rules/r1?overwrite=true -H "Content-Type: application/json" -d '{"name":"r1","status":"active","trigger":"/guest/trigger1","action":"/guest/action1","publish":false}' {"name":"r1","publish":false,"annotations":[],"version":"0.0.2","status":"active","action":{"path":"guest","name":"action1"},"namespace":"guest","trigger":{"path":"guest","name":"trigger1"}} ``` Get `r1`: ``` curl --user user:password -X GET http://192.168.33.13:10001/api/v1/namespaces/guest/rules/r1 {"name":"r1","publish":false,"annotations":[],"version":"0.0.2","status":"inactive","action":{"path":"guest","name":"action1"},"namespace":"guest","trigger":{"path":"guest","name":"trigger1"}} ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData
rabbah commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData URL: https://github.com/apache/incubator-openwhisk/pull/3109#discussion_r167641380 ## File path: tests/src/test/scala/whisk/core/controller/test/ActionsApiTests.scala ## @@ -159,26 +159,136 @@ class ActionsApiTests extends ControllerTestCommon with WhiskActionsApi { } } + def getExecPermutations() = { +implicit val tid = transid() + +// BlackBox: binary: true, main: bbMain +val bbAction1 = WhiskAction(namespace, aname(), bb("bb", "RHViZWU=", Some("bbMain"))) +val bbAction1Content = Map("exec" -> Map( + "kind" -> Exec.BLACKBOX, + "code" -> "RHViZWU=", + "image" -> "bb", + "main" -> "bbMain")).toJson.asJsObject +val bbAction1ExecMetaData = blackBoxMetaData("bb", Some("bbMain"), true) + +// BlackBox: binary: false, main: bbMain +val bbAction2 = WhiskAction(namespace, aname(), bb("bb", "", Some("bbMain"))) +val bbAction2Content = + Map("exec" -> Map("kind" -> Exec.BLACKBOX, "code" -> "", "image" -> "bb", "main" -> "bbMain")).toJson.asJsObject +val bbAction2ExecMetaData = blackBoxMetaData("bb", Some("bbMain"), false) + +// BlackBox: binary: true, no main +val bbAction3 = WhiskAction(namespace, aname(), bb("bb", "RHViZWU=")) +val bbAction3Content = + Map("exec" -> Map("kind" -> Exec.BLACKBOX, "code" -> "RHViZWU=", "image" -> "bb")).toJson.asJsObject +val bbAction3ExecMetaData = blackBoxMetaData("bb", None, true) + +// BlackBox: binary: false, no main +val bbAction4 = WhiskAction(namespace, aname(), bb("bb", "")) +val bbAction4Content = Map("exec" -> Map("kind" -> Exec.BLACKBOX, "code" -> "", "image" -> "bb")).toJson.asJsObject +val bbAction4ExecMetaData = blackBoxMetaData("bb", None, false) + +// Attachment: binary: true, main: javaMain +val javaAction1 = WhiskAction(namespace, aname(), javaDefault("RHViZWU=", Some("javaMain"))) +val javaAction1Content = + Map("exec" -> Map("kind" -> JAVA_DEFAULT, "code" -> "RHViZWU=", "main" -> "javaMain")).toJson.asJsObject +val javaAction1ExecMetaData = javaMetaData(Some("javaMain"), true) + +// String: binary: true, main: jsMain +val jsAction1 = WhiskAction(namespace, aname(), jsDefault("RHViZWU=", Some("jsMain"))) +val jsAction1Content = + Map("exec" -> Map("kind" -> NODEJS6, "code" -> "RHViZWU=", "main" -> "jsMain")).toJson.asJsObject +val jsAction1ExecMetaData = js6MetaData(Some("jsMain"), true) + +// String: binary: false, main: jsMain +val jsAction2 = WhiskAction(namespace, aname(), jsDefault("", Some("jsMain"))) +val jsAction2Content = Map("exec" -> Map("kind" -> NODEJS6, "code" -> "", "main" -> "jsMain")).toJson.asJsObject +val jsAction2ExecMetaData = js6MetaData(Some("jsMain"), false) + +// String: binary: true, no main +val jsAction3 = WhiskAction(namespace, aname(), jsDefault("RHViZWU=")) +val jsAction3Content = Map("exec" -> Map("kind" -> NODEJS6, "code" -> "RHViZWU=")).toJson.asJsObject +val jsAction3ExecMetaData = js6MetaData(None, true) + +// String: binary: false, no main +val jsAction4 = WhiskAction(namespace, aname(), jsDefault("")) +val jsAction4Content = Map("exec" -> Map("kind" -> NODEJS6, "code" -> "")).toJson.asJsObject +val jsAction4ExecMetaData = js6MetaData(None, false) + +// Sequence +val component = WhiskAction(namespace, aname(), jsDefault("??")) +put(entityStore, component) +val components = Vector(s"/$namespace/${component.name}").map(stringToFullyQualifiedName(_)) +val seqAction = WhiskAction(namespace, aname(), sequence(components), seqParameters(components)) +val seqActionContent = JsObject( + "exec" -> JsObject("kind" -> "sequence".toJson, "components" -> JsArray(s"/$namespace/${component.name}".toJson))) +val seqActionExecMetaData = sequenceMetaData(components) + +Seq( + (bbAction1, bbAction1Content, bbAction1ExecMetaData), + (bbAction2, bbAction2Content, bbAction2ExecMetaData), + (bbAction3, bbAction3Content, bbAction3ExecMetaData), + (bbAction4, bbAction4Content, bbAction4ExecMetaData), + (javaAction1, javaAction1Content, javaAction1ExecMetaData), + (jsAction1, jsAction1Content, jsAction1ExecMetaData), + (jsAction2, jsAction2Content, jsAction2ExecMetaData), + (jsAction3, jsAction3Content, jsAction3ExecMetaData), + (jsAction4, jsAction4Content, jsAction4ExecMetaData), + (seqAction, seqActionContent, seqActionExecMetaData)) + } + it should "get action using code query parameter" in { implicit val tid = transid() -val action = WhiskAction(namespace, aname(), jsDefault("??"), Parameters("x", "b")) -put(entityStore, action) +getExecPermutations.foreach { + case (action, content, execMetaData) => +val expectedWhiskAction = WhiskAction( + action.namespace, +
[GitHub] rabbah commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData
rabbah commented on a change in pull request #3109: Add binary, image, and main properties to WhiskActionMetaData URL: https://github.com/apache/incubator-openwhisk/pull/3109#discussion_r167638650 ## File path: common/scala/src/main/scala/whisk/core/entity/Exec.scala ## @@ -95,9 +95,23 @@ sealed abstract class CodeExec[+T <% SizeConversion] extends Exec { sealed abstract class ExecMetaData extends ExecMetaDataBase { + /** An entrypoint (typically name of 'main' function). 'None' means a default value will be used. */ + val entryPoint: Option[String] + + /** The runtime image (either built-in or a public image). */ + val image: ImageName + /** Indicates if a container image is required from the registry to execute the action. */ val pull: Boolean + /** + * Indicates whether the code is stored in a text-readable or binary format. + * The binary bit may be read from the database but currently it is always computed + * when the "code" is moved to an attachment this may get changed to avoid recomputing + * the binary property. + */ + val binary: Boolean Review comment: do you need the other properties? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rabbah closed pull request #3259: use PureConfig for misc. runtimesManifest configuration
rabbah closed pull request #3259: use PureConfig for misc. runtimesManifest configuration URL: https://github.com/apache/incubator-openwhisk/pull/3259 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/ansible/environments/docker-machine/group_vars/all b/ansible/environments/docker-machine/group_vars/all index efd0b5628e..6f68e730f9 100644 --- a/ansible/environments/docker-machine/group_vars/all +++ b/ansible/environments/docker-machine/group_vars/all @@ -3,7 +3,7 @@ config_root_dir: /Users/Shared/wskconf whisk_logs_dir: /Users/Shared/wsklogs docker_registry: "" docker_dns: "" -bypass_pull_for_local_images: true +runtimes_bypass_pull_for_local_images: true env_hosts_dir: "{{ playbook_dir }}/environments/docker-machine" diff --git a/ansible/environments/local/group_vars/all b/ansible/environments/local/group_vars/all index fbed10ebe1..bcec410650 100755 --- a/ansible/environments/local/group_vars/all +++ b/ansible/environments/local/group_vars/all @@ -4,7 +4,7 @@ config_root_dir: "{{ openwhisk_tmp_dir }}/wskconf" whisk_logs_dir: "{{ openwhisk_tmp_dir }}/wsklogs" docker_registry: "" docker_dns: "" -bypass_pull_for_local_images: true +runtimes_bypass_pull_for_local_images: true db_prefix: whisk_local_ diff --git a/ansible/files/runtimes.json b/ansible/files/runtimes.json index 8030fa2405..42858d6945 100644 --- a/ansible/files/runtimes.json +++ b/ansible/files/runtimes.json @@ -1,7 +1,4 @@ { -"bypassPullForLocalImages": false, -"defaultImagePrefix": "openwhisk", -"defaultImageTag": "latest", "runtimes": { "nodejs": [ { diff --git a/ansible/group_vars/all b/ansible/group_vars/all index 6978fb97ac..54c2d68b0f 100644 --- a/ansible/group_vars/all +++ b/ansible/group_vars/all @@ -24,16 +24,8 @@ whisk: version: date: "{{ansible_date_time.iso8601}}" -## -# list of supported runtimes (see whisk.core.entity.ExecManifest for schema). -# briefly: -# defaultImagePrefix: the default image prefix when not given explicitly -# defaultImageTag: the default image tag -# runtimes: set of language runtime families grouped by language (e.g., nodejs, python) -# blackboxes: list of pre-populated docker action images as "name" with optional "prefix" and "tag" -# bypassPullForLocalImages: optional, if true, allow images with a prefix that matches {{ docker.image.prefix }} -# to skip docker pull in invoker even if the image is not part of the blackbox set -# + +# list of supported runtimes (see whisk.core.entity.ExecManifest for schema) runtimesManifest: "{{ runtimes_manifest | default(lookup('file', '{{ openwhisk_home }}/ansible/files/runtimes.json') | from_json) }}" limits: diff --git a/ansible/roles/controller/tasks/deploy.yml b/ansible/roles/controller/tasks/deploy.yml index 8601a2985d..9c19dd4255 100644 --- a/ansible/roles/controller/tasks/deploy.yml +++ b/ansible/roles/controller/tasks/deploy.yml @@ -133,6 +133,11 @@ "CONFIG_whisk_activation_payload_max": "{{ limit_activation_payload | default() }}" "RUNTIMES_MANIFEST": "{{ runtimesManifest | to_json }}" + "CONFIG_whisk_runtimes_defaultImagePrefix": "{{ runtimes_default_image_prefix | default() }}" + "CONFIG_whisk_runtimes_defaultImageTag": "{{ runtimes_default_image_tag | default() }}" + "CONFIG_whisk_runtimes_bypassPullForLocalImages": "{{ runtimes_bypass_pull_for_local_images | default() }}" + "CONFIG_whisk_runtimes_localImagePrefix": "{{ runtimes_local_image_prefix | default() }}" + "CONTROLLER_LOCALBOOKKEEPING": "{{ controller.localBookkeeping }}" "AKKA_CLUSTER_SEED_NODES": "{{seed_nodes_list | join(' ') }}" "CONTROLLER_HA": "{{ controller.ha }}" diff --git a/ansible/roles/invoker/tasks/deploy.yml b/ansible/roles/invoker/tasks/deploy.yml index c215e395a8..be2a251346 100644 --- a/ansible/roles/invoker/tasks/deploy.yml +++ b/ansible/roles/invoker/tasks/deploy.yml @@ -165,6 +165,10 @@ -e WHISK_API_HOST_PORT='{{ whisk_api_host_port | default('443') }}' -e WHISK_API_HOST_NAME='{{ whisk_api_host_name | default(groups['edge'] | first) }}' -e RUNTIMES_MANIFEST='{{ runtimesManifest | to_json }}' +-e CONFIG_whisk_runtimes_defaultImagePrefix='{{ runtimes_default_image_prefix | default() }}' +-e CONFIG_whisk_runtimes_defaultImageTag='{{ runtimes_default_image_tag | default() }}' +-e CONFIG_whisk_runtimes_bypassPullForLocalImages='{{ runtimes_bypass_pull_for_local_images | default() }}' +-e CONFIG_whisk_runtimes_localImagePrefix='{{ runtimes_local_image_prefix | default() }}' -e DOCKER_REGISTRY='{{ docker_registry }}' -e DOCKER_IMAGE_PREFIX='{{ docker.image.prefix }}' -e DOCKER_IMAGE_TAG=
[GitHub] rabbah closed issue #3245: bypassPullForLocalImages setting no longer overridable
rabbah closed issue #3245: bypassPullForLocalImages setting no longer overridable URL: https://github.com/apache/incubator-openwhisk/issues/3245 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss commented on issue #3259: use PureConfig for misc. runtimesManifest configuration
dgrove-oss commented on issue #3259: use PureConfig for misc. runtimesManifest configuration URL: https://github.com/apache/incubator-openwhisk/pull/3259#issuecomment-365006493 PG3 / 1841 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows commented on issue #730: Multiple projects management
mrutkows commented on issue #730: Multiple projects management URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/730#issuecomment-364995703 @kpavel as discussed elsewhere... the plan would be to use URI "tags" where there is no key (each project name would be clearly part of the URI) along with domain/path to indicate that this is a "managed project" tag. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats
markusthoemmes commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats URL: https://github.com/apache/incubator-openwhisk/pull/3249#issuecomment-364995590 @chetanmeh can you hook this up into the travis build? (If that makes any sense). So we can know it actually runs/works? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] dgrove-oss commented on issue #3259: use PureConfig for misc. runtimesManifest configuration
dgrove-oss commented on issue #3259: use PureConfig for misc. runtimesManifest configuration URL: https://github.com/apache/incubator-openwhisk/pull/3259#issuecomment-364787479 rebased to latest master; PG1/2612 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] houshengbo opened a new pull request #33: Verify artifacts
houshengbo opened a new pull request #33: Verify artifacts URL: https://github.com/apache/incubator-openwhisk-release/pull/33 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kpavel opened a new issue #730: Multiple projects management
kpavel opened a new issue #730: Multiple projects management URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/730 Currently it is impossible to use wskdeploy to deploy OW assets (action, package...etc) of the same signature from different manifests having different project names. The problem is that wskdeploy overrides the "managed project" annotation. The workaround could be to use different package names, which results in assets duplication. Duplication is undesirable, because it complicates maintenance. Introducing a feature allowing to manage OW assets by multiple projects would help to avoid it. Suggestion: - Use different annotation key instead of "manage" for the sake of backward compatibility. E.g. `managedList` holding an array of managed project names. - On manifest deployment, if a deployed asset already exists in OW, check whether the manifest asset is identical to the existing one. If not: override the existing asset with the manifest one (similar to the current functionality) If yes: add the project specified in the manifest to the `managedList`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chetanmeh commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats
chetanmeh commented on issue #3249: #3243 - Use gradle scoverage to collect test coverage stats URL: https://github.com/apache/incubator-openwhisk/pull/3249#issuecomment-364971921 I have rebased the PR to current master. Its ready to be merged from my side. Let me know if anything else needs to be addressed as part of this PR (coverage in docker runs would be dealt later) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows commented on issue #715: Adding support for export managed project back to manifest
mrutkows commented on issue #715: Adding support for export managed project back to manifest URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/715#issuecomment-364950351 @kpavel have you signed the Apache ICLA? I think this is you here: http://people.apache.org/unlistedclas.html This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows commented on issue #715: Adding support for export managed project back to manifest
mrutkows commented on issue #715: Adding support for export managed project back to manifest URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/715#issuecomment-364950351 @kpavel have you signed the Apache ICLA? I think this is you here: Open Source 201 presentations to be made external with Videos week after Index This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows closed pull request #715: Adding support for export managed project back to manifest
mrutkows closed pull request #715: Adding support for export managed project back to manifest URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/715 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/cmd/export.go b/cmd/export.go new file mode 100644 index ..c6ea0e33 --- /dev/null +++ b/cmd/export.go @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package cmd + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/apache/incubator-openwhisk-client-go/whisk" + "github.com/apache/incubator-openwhisk-wskdeploy/deployers" + "github.com/apache/incubator-openwhisk-wskdeploy/parsers" + "github.com/apache/incubator-openwhisk-wskdeploy/utils" + "github.com/apache/incubator-openwhisk-wskdeploy/wskderrors" + "github.com/spf13/cobra" +) + +var exportCmd = &cobra.Command{ + Use:"export", + SuggestFor: []string{"capture"}, + Short: "Export project assets from OpenWhisk", + Long: `Exports managed project assets from OpenWhisk to manifest and function files`, + RunE: ExportCmdImp, +} + +var config *whisk.Config + +func ExportRule(wskRule whisk.Rule, pkgName string, maniyaml *parsers.YAML) { + if maniyaml.Packages[pkgName].Rules == nil { + pkg := maniyaml.Packages[pkgName] + pkg.Rules = make(map[string]parsers.Rule) + maniyaml.Packages[pkgName] = pkg + } + + // export rule to manifest + maniyaml.Packages[pkgName].Rules[wskRule.Name] = *maniyaml.ComposeParsersRule(wskRule) +} + +func ExportAction(actionName string, packageName string, maniyaml *parsers.YAML) error { + + pkg := maniyaml.Packages[packageName] + if pkg.Actions == nil { + pkg.Actions = make(map[string]parsers.Action) + maniyaml.Packages[packageName] = pkg + } + + wskAction, _, err := client.Actions.Get(actionName) + if err != nil { + return err + } + if wskAction.Exec.Kind == "sequence" { + seq := new(parsers.Sequence) + for _, component := range wskAction.Exec.Components { + // must ommit namespace from seq component name + ExportAction(strings.SplitN(component, "/", 3)[2], packageName, maniyaml) + slices := strings.Split(component, "/") + + // save in the seq list only action names + if len(seq.Actions) > 0 { + seq.Actions += "," + } + + seq.Actions += slices[len(slices)-1] + } + + pkg = maniyaml.Packages[packageName] + if pkg.Sequences == nil { + pkg.Sequences = make(map[string]parsers.Sequence) + } + + pkg.Sequences[wskAction.Name] = *seq + } else { + parsedAction := *maniyaml.ComposeParsersAction(*wskAction) + + // get the action file extension according to action kind (e.g. js for nodejs) + ext := utils.FileRuntimeExtensionsMap[wskAction.Exec.Kind] + + manifestDir := filepath.Dir(utils.Flags.ManifestPath) + + // store function file under action package name subdirectory in the specified manifest folder + functionDir := filepath.Join(manifestDir, packageName) + os.MkdirAll(functionDir, os.ModePerm) + + // store function in manifest under path relative to manifest root + functionFile := filepath.Join(packageName, wskAction.Name) + "." + ext + parsedAction.Function = functionFile + + // create function file at the full path + functionFile = filepath.Join(manifestDir, functionFile) + f, err := os.Create(functionFile) + if err !=
[GitHub] mrutkows commented on issue #715: Adding support for export managed project back to manifest
mrutkows commented on issue #715: Adding support for export managed project back to manifest URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/715#issuecomment-364949429 @pritidesai @kpavel Would like to remove empty maps and interfaces from what the yaml parser generates from our schema. We can look at in a separate issue; want to add this function. Added this issue https://github.com/apache/incubator-openwhisk-wskdeploy/issues/729 to track this future work. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows opened a new issue #729: Remove empty maps/interfaces from "export" manifest yaml
mrutkows opened a new issue #729: Remove empty maps/interfaces from "export" manifest yaml URL: https://github.com/apache/incubator-openwhisk-wskdeploy/issues/729 all empty maps and interfaces should be removed from the final manifest; also, we should remove any deprecated keys in favor of latest keys (this may mean moving maps (subtrees) from deprecated keys to the ones now in spec. This is what we produce today: ``` application: name: "" namespace: "" credential: "" apiHost: "" version: "" packages: {} package: name: "" version: "" license: "" dependencies: {} namespace: "" credential: "" apiHost: "" actions: {} triggers: {} feeds: {} rules: {} inputs: {} sequences: {} apis: {} project: name: MyFirstManagedProject namespace: "" credential: "" apiHost: "" version: "" packages: {} package: name: "" version: "" license: "" dependencies: {} namespace: "" credential: "" apiHost: "" actions: {} triggers: {} feeds: {} rules: {} inputs: {} sequences: {} apis: {} packages: ManagedPackage-1: name: ManagedPackage-1 version: 0.0.2 license: "" dependencies: {} namespace: guest credential: "" apiHost: "" actions: HelloWorld-1: version: 0.0.2 location: "" function: ManagedPackage-1/HelloWorld-1.js runtime: nodejs:6 namespace: guest/ManagedPackage-1 credential: "" inputs: name: value: "" place: value: "" outputs: {} name: HelloWorld-1 exposedUrl: "" web-export: "" main: "" limits: null HelloWorld-2: version: 0.0.2 location: "" function: ManagedPackage-1/HelloWorld-2.js runtime: nodejs:6 namespace: guest/ManagedPackage-1 credential: "" inputs: name: value: "" place: value: "" outputs: {} name: HelloWorld-2 exposedUrl: "" web-export: "" main: "" limits: null HelloWorld-3: version: 0.0.2 location: "" function: ManagedPackage-1/HelloWorld-3.js runtime: nodejs:6 namespace: guest/ManagedPackage-1 credential: "" inputs: name: value: "" place: value: "" outputs: {} name: HelloWorld-3 exposedUrl: "" web-export: "" main: "" limits: null triggers: ManagedTrigger-1: feed: "" namespace: guest credential: "" inputs: {} name: ManagedTrigger-1 source: "" feeds: {} rules: {} inputs: {} sequences: ManagedSequence-1: actions: HelloWorld-1,HelloWorld-2,HelloWorld-3 ManagedSequence-2: actions: HelloWorld-1,HelloWorld-2,HelloWorld-3 apis: {} package: name: "" version: "" license: "" dependencies: {} namespace: "" credential: "" apiHost: "" actions: {} triggers: {} feeds: {} rules: {} inputs: {} sequences: {} apis: {} filepath: "" ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows commented on issue #715: Adding support for export managed project back to manifest
mrutkows commented on issue #715: Adding support for export managed project back to manifest URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/715#issuecomment-364950351 @kpavel have you signed the Apache ICLA? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrutkows commented on issue #715: Adding support for export managed project back to manifest
mrutkows commented on issue #715: Adding support for export managed project back to manifest URL: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/715#issuecomment-364949429 @pritidesai @kpavel Would like to remove empty maps and interfaces from what the yaml parser generates from our schema. We can look at in a separate issue; want to add this function. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3077: Enable ssl on the path between edge and controllers
markusthoemmes commented on a change in pull request #3077: Enable ssl on the path between edge and controllers URL: https://github.com/apache/incubator-openwhisk/pull/3077#discussion_r167554513 ## File path: tests/src/test/scala/common/rest/WskRest.scala ## @@ -85,15 +81,42 @@ import common.WskActorSystem import common.WskProps import whisk.core.entity.ByteSize import whisk.utils.retry -import javax.net.ssl.{HostnameVerifier, KeyManager, SSLContext, SSLSession, X509TrustManager} +import javax.net.ssl._ + +import akka.actor.ActorSystem import com.typesafe.sslconfig.akka.AkkaSSLConfig import java.nio.charset.StandardCharsets +import java.security.KeyStore class AcceptAllHostNameVerifier extends HostnameVerifier { override def verify(s: String, sslSession: SSLSession): Boolean = true } object SSL { + + val controllerProtocol = WhiskProperties.getProperty("controller.protocol") + val controllerKeyStorePath = WhiskProperties.getProperty("controller.ssl.keystore.path") + val controllerKeyStorePassword = WhiskProperties.getProperty("controller.ssl.keystore.password") + val controllerKeyStoreFlavor = WhiskProperties.getProperty("controller.ssl.keystore.flavor") + val controllerClientAuth = WhiskProperties.getProperty("controller.ssl.clientAuth").toBoolean Review comment: Please refer to https://github.com/apache/incubator-openwhisk/pull/3111/commits/04579a3b786c582d0ff35f679208e63069473b68 for our future path of solving this. I back that we should move these to pureconfig and write the `.conf` the same way we write whisk.properties today. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] markusthoemmes commented on a change in pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding.
markusthoemmes commented on a change in pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding. URL: https://github.com/apache/incubator-openwhisk/pull/3240#discussion_r167554625 ## File path: core/controller/src/main/scala/whisk/core/entitlement/Entitlement.scala ## @@ -82,42 +80,63 @@ protected[core] abstract class EntitlementProvider(config: WhiskConfig, loadBala implicit actorSystem: ActorSystem, logging: Logging) { - private implicit val executionContext = actorSystem.dispatcher - - /** - * The number of controllers if HA is enabled, 1 otherwise - */ - private val diviser = if (config.controllerHighAvailability) config.controllerInstances.toInt else 1 + private implicit val executionContext: ExecutionContext = actorSystem.dispatcher /** * Allows 20% of additional requests on top of the limit to mitigate possible unfair round-robin loadbalancing between * controllers */ private val overcommit = if (config.controllerHighAvailability) 1.2 else 1 + private def dilateLimit(limit: Int): Int = Math.ceil(limit.toDouble * overcommit).toInt /** - * Adjust the throttles for a single controller with the diviser and the overcommit. + * Calculates a possibly dilated limit relative to the current user. * - * @param originalThrottle The throttle that needs to be adjusted for this controller. + * @param defaultLimit the default limit across the whole system + * @param user the user to apply that limit to + * @return a calculated limit */ - private def dilateThrottle(originalThrottle: Int): Int = { -Math.ceil((originalThrottle.toDouble / diviser.toDouble) * overcommit).toInt + private def calculateLimit(defaultLimit: Int, overrideLimit: Identity => Option[Int])(user: Identity): Int = { +val absoluteLimit = overrideLimit(user).getOrElse(defaultLimit) +dilateLimit(absoluteLimit) + } + + /** + * Calculates a limit which applies only to this instance individually. + * + * The state needed to correctly check this limit is not shared between all instances, which want to check that + * limit, so it needs to be divided between the parties who want to perform that check. + * + * @param defaultLimit the default limit across the whole system + * @param user the user to apply that limit to + * @return a calculated limit + */ + private def calculateIndividualLimit(defaultLimit: Int, overrideLimit: Identity => Option[Int])( +user: Identity): Int = { +val limit = calculateLimit(defaultLimit, overrideLimit)(user) +limit / loadBalancer.clusterSize Review comment: Good one! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vvraskin commented on a change in pull request #3077: Enable ssl on the path between edge and controllers
vvraskin commented on a change in pull request #3077: Enable ssl on the path between edge and controllers URL: https://github.com/apache/incubator-openwhisk/pull/3077#discussion_r167543012 ## File path: tests/src/test/scala/common/rest/WskRest.scala ## @@ -85,15 +81,42 @@ import common.WskActorSystem import common.WskProps import whisk.core.entity.ByteSize import whisk.utils.retry -import javax.net.ssl.{HostnameVerifier, KeyManager, SSLContext, SSLSession, X509TrustManager} +import javax.net.ssl._ + +import akka.actor.ActorSystem import com.typesafe.sslconfig.akka.AkkaSSLConfig import java.nio.charset.StandardCharsets +import java.security.KeyStore class AcceptAllHostNameVerifier extends HostnameVerifier { override def verify(s: String, sslSession: SSLSession): Boolean = true } object SSL { + + val controllerProtocol = WhiskProperties.getProperty("controller.protocol") + val controllerKeyStorePath = WhiskProperties.getProperty("controller.ssl.keystore.path") + val controllerKeyStorePassword = WhiskProperties.getProperty("controller.ssl.keystore.password") + val controllerKeyStoreFlavor = WhiskProperties.getProperty("controller.ssl.keystore.flavor") + val controllerClientAuth = WhiskProperties.getProperty("controller.ssl.clientAuth").toBoolean Review comment: The variables in the `application.conf` are only aim for the trigger api tests to make sure HttpConfig is instantiated and they are not actually used (the test is executed in memory). For other tests that run against controller API I'd like to leave the possibility to configure the test via `whisk.properties`, do we test only the config which is currently deployed and do not enmesh pure config here. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] vvraskin commented on a change in pull request #3077: Enable ssl on the path between edge and controllers
vvraskin commented on a change in pull request #3077: Enable ssl on the path between edge and controllers URL: https://github.com/apache/incubator-openwhisk/pull/3077#discussion_r167552300 ## File path: common/scala/src/main/scala/whisk/core/WhiskConfig.scala ## @@ -98,6 +98,8 @@ class WhiskConfig(requiredProperties: Map[String, String], val controllerSeedNodes = this(WhiskConfig.controllerSeedNodes) val controllerLocalBookkeeping = getAsBoolean(WhiskConfig.controllerLocalBookkeeping, false) val controllerHighAvailability = getAsBoolean(WhiskConfig.controllerHighAvailability, false) + val controllerProtocol = this(WhiskConfig.controllerProtocol) + val controllerSSLEnabled = if (this.controllerProtocol == "https") true else false Review comment: I wanted to keep it in the Config, so we don't create two identical objects in Trigger and Controller or pass that property around. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cbickel commented on a change in pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding.
cbickel commented on a change in pull request #3240: Add a loadbalancer with local state and horizontal invoker sharding. URL: https://github.com/apache/incubator-openwhisk/pull/3240#discussion_r167550757 ## File path: core/controller/src/main/scala/whisk/core/entitlement/Entitlement.scala ## @@ -82,42 +80,63 @@ protected[core] abstract class EntitlementProvider(config: WhiskConfig, loadBala implicit actorSystem: ActorSystem, logging: Logging) { - private implicit val executionContext = actorSystem.dispatcher - - /** - * The number of controllers if HA is enabled, 1 otherwise - */ - private val diviser = if (config.controllerHighAvailability) config.controllerInstances.toInt else 1 + private implicit val executionContext: ExecutionContext = actorSystem.dispatcher /** * Allows 20% of additional requests on top of the limit to mitigate possible unfair round-robin loadbalancing between * controllers */ private val overcommit = if (config.controllerHighAvailability) 1.2 else 1 + private def dilateLimit(limit: Int): Int = Math.ceil(limit.toDouble * overcommit).toInt /** - * Adjust the throttles for a single controller with the diviser and the overcommit. + * Calculates a possibly dilated limit relative to the current user. * - * @param originalThrottle The throttle that needs to be adjusted for this controller. + * @param defaultLimit the default limit across the whole system + * @param user the user to apply that limit to + * @return a calculated limit */ - private def dilateThrottle(originalThrottle: Int): Int = { -Math.ceil((originalThrottle.toDouble / diviser.toDouble) * overcommit).toInt + private def calculateLimit(defaultLimit: Int, overrideLimit: Identity => Option[Int])(user: Identity): Int = { +val absoluteLimit = overrideLimit(user).getOrElse(defaultLimit) +dilateLimit(absoluteLimit) + } + + /** + * Calculates a limit which applies only to this instance individually. + * + * The state needed to correctly check this limit is not shared between all instances, which want to check that + * limit, so it needs to be divided between the parties who want to perform that check. + * + * @param defaultLimit the default limit across the whole system + * @param user the user to apply that limit to + * @return a calculated limit + */ + private def calculateIndividualLimit(defaultLimit: Int, overrideLimit: Identity => Option[Int])( +user: Identity): Int = { +val limit = calculateLimit(defaultLimit, overrideLimit)(user) +limit / loadBalancer.clusterSize Review comment: If the limit is 1 and we have more than 1 controller, the user won't be able to invoke any actions. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ningyougang commented on issue #3272: improve loadbalancer's schedule algorithm
ningyougang commented on issue #3272: improve loadbalancer's schedule algorithm URL: https://github.com/apache/incubator-openwhisk/pull/3272#issuecomment-364905029 @markusthoemmes ,i have added the test case, please review again. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jthomas commented on issue #3244: Move parameter docs to separate page, add package params docs
jthomas commented on issue #3244: Move parameter docs to separate page, add package params docs URL: https://github.com/apache/incubator-openwhisk/pull/3244#issuecomment-364903176 Great PR! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cbickel closed pull request #3273: Wait for logs based on intervals not based on total processing time.
cbickel closed pull request #3273: Wait for logs based on intervals not based on total processing time. URL: https://github.com/apache/incubator-openwhisk/pull/3273 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/core/invoker/src/main/scala/whisk/core/containerpool/docker/DockerContainer.scala b/core/invoker/src/main/scala/whisk/core/containerpool/docker/DockerContainer.scala index 265a45031d..5d3083c620 100644 --- a/core/invoker/src/main/scala/whisk/core/containerpool/docker/DockerContainer.scala +++ b/core/invoker/src/main/scala/whisk/core/containerpool/docker/DockerContainer.scala @@ -18,6 +18,7 @@ package whisk.core.containerpool.docker import java.time.Instant +import java.util.concurrent.TimeoutException import java.util.concurrent.atomic.AtomicLong import akka.actor.ActorSystem @@ -80,7 +81,7 @@ object DockerContainer { as: ActorSystem, ec: ExecutionContext, log: Logging): Future[DockerContainer] = { -implicit val tid = transid +implicit val tid: TransactionId = transid val environmentArgs = environment.flatMap { case (key, value) => Seq("-e", s"$key=$value") @@ -246,18 +247,21 @@ class DockerContainer(protected val id: ContainerId, size } .via(new CompleteAfterOccurrences(_.containsSlice(DockerContainer.ActivationSentinel), 2, waitForSentinel)) + // As we're reading the logs after the activation has finished the invariant is that all loglines are already + // written and we mostly await them being flushed by the docker daemon. Therefore we can timeout based on the time + // between two loglines appear without relying on the log frequency in the action itself. + .idleTimeout(waitForLogs) .recover { case _: StreamLimitReachedException => // While the stream has already ended by failing the limitWeighted stage above, we inject a truncation // notice downstream, which will be processed as usual. This will be the last element of the stream. ByteString(LogLine(Instant.now.toString, "stderr", Messages.truncateLogs(limit)).toJson.compactPrint) -case _: OccurrencesNotFoundException | _: FramingException => +case _: OccurrencesNotFoundException | _: FramingException | _: TimeoutException => // Stream has already ended and we insert a notice that data might be missing from the logs. While a // FramingException can also mean exceeding the limits, we cannot decide which case happened so we resort // to the general error message. This will be the last element of the stream. ByteString(LogLine(Instant.now.toString, "stderr", Messages.logFailure).toJson.compactPrint) } - .takeWithin(waitForLogs) } /** Delimiter used to split log-lines as written by the json-log-driver. */ @@ -279,9 +283,9 @@ class DockerContainer(protected val id: ContainerId, */ class CompleteAfterOccurrences[T](isInEvent: T => Boolean, neededOccurrences: Int, errorOnNotEnough: Boolean) extends GraphStage[FlowShape[T, T]] { - val in = Inlet[T]("WaitForOccurances.in") - val out = Outlet[T]("WaitForOccurances.out") - override val shape = FlowShape.of(in, out) + val in: Inlet[T] = Inlet[T]("WaitForOccurrences.in") + val out: Outlet[T] = Outlet[T]("WaitForOccurrences.out") + override val shape: FlowShape[T, T] = FlowShape.of(in, out) override def createLogic(inheritedAttributes: Attributes): GraphStageLogic = new GraphStageLogic(shape) with InHandler with OutHandler { diff --git a/tests/src/test/scala/whisk/core/containerpool/docker/test/DockerContainerTests.scala b/tests/src/test/scala/whisk/core/containerpool/docker/test/DockerContainerTests.scala index 5f9898e6b2..ea7781036b 100644 --- a/tests/src/test/scala/whisk/core/containerpool/docker/test/DockerContainerTests.scala +++ b/tests/src/test/scala/whisk/core/containerpool/docker/test/DockerContainerTests.scala @@ -34,7 +34,6 @@ import org.scalamock.scalatest.MockFactory import org.scalatest.BeforeAndAfterEach import org.scalatest.FlatSpec import whisk.core.containerpool.logging.{DockerToActivationLogStore, LogLine} - import org.scalatest.junit.JUnitRunner import org.scalatest.Matchers import common.{StreamLogging, WskActorSystem} @@ -49,7 +48,6 @@ import whisk.core.entity.ActivationResponse.ContainerResponse import whisk.core.entity.ActivationResponse.Timeout import whisk.core.entity.size._ import whisk.http.Messages - import whisk.core.entity.size._ /** @@ -583,8 +581,9 @@ class DockerContainerTests docker.rawCo
[GitHub] cbickel commented on a change in pull request #3077: Enable ssl on the path between edge and controllers
cbickel commented on a change in pull request #3077: Enable ssl on the path between edge and controllers URL: https://github.com/apache/incubator-openwhisk/pull/3077#discussion_r167503054 ## File path: tests/src/test/scala/common/rest/WskRest.scala ## @@ -85,15 +81,42 @@ import common.WskActorSystem import common.WskProps import whisk.core.entity.ByteSize import whisk.utils.retry -import javax.net.ssl.{HostnameVerifier, KeyManager, SSLContext, SSLSession, X509TrustManager} +import javax.net.ssl._ + +import akka.actor.ActorSystem import com.typesafe.sslconfig.akka.AkkaSSLConfig import java.nio.charset.StandardCharsets +import java.security.KeyStore class AcceptAllHostNameVerifier extends HostnameVerifier { override def verify(s: String, sslSession: SSLSession): Boolean = true } object SSL { + + val controllerProtocol = WhiskProperties.getProperty("controller.protocol") + val controllerKeyStorePath = WhiskProperties.getProperty("controller.ssl.keystore.path") + val controllerKeyStorePassword = WhiskProperties.getProperty("controller.ssl.keystore.password") + val controllerKeyStoreFlavor = WhiskProperties.getProperty("controller.ssl.keystore.flavor") + val controllerClientAuth = WhiskProperties.getProperty("controller.ssl.clientAuth").toBoolean Review comment: You already put these variables into `application.conf`. Can we use them instead? Afterwards you can remove the entries from `whisk.properties` again. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cbickel commented on a change in pull request #3077: Enable ssl on the path between edge and controllers
cbickel commented on a change in pull request #3077: Enable ssl on the path between edge and controllers URL: https://github.com/apache/incubator-openwhisk/pull/3077#discussion_r167501744 ## File path: core/controller/src/main/scala/whisk/core/controller/Controller.scala ## @@ -161,7 +161,7 @@ object Controller { // a value, and whose values are default values. A null value in the Map means there is // no default value specified, so it must appear in the properties file def requiredProperties = -Map(WhiskConfig.controllerInstances -> null) ++ +Map(WhiskConfig.controllerInstances -> null, WhiskConfig.controllerProtocol -> null) ++ Review comment: Pure config as well. But if you really need to keep it in WhiskConfig, it is already in `RestApiCommons.requiredProperties `. So you don't need to add it here. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cbickel commented on a change in pull request #3077: Enable ssl on the path between edge and controllers
cbickel commented on a change in pull request #3077: Enable ssl on the path between edge and controllers URL: https://github.com/apache/incubator-openwhisk/pull/3077#discussion_r167496514 ## File path: common/scala/src/main/scala/whisk/core/WhiskConfig.scala ## @@ -98,6 +98,8 @@ class WhiskConfig(requiredProperties: Map[String, String], val controllerSeedNodes = this(WhiskConfig.controllerSeedNodes) val controllerLocalBookkeeping = getAsBoolean(WhiskConfig.controllerLocalBookkeeping, false) val controllerHighAvailability = getAsBoolean(WhiskConfig.controllerHighAvailability, false) + val controllerProtocol = this(WhiskConfig.controllerProtocol) + val controllerSSLEnabled = if (this.controllerProtocol == "https") true else false Review comment: Can we move this to pureconfig as well? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services