Re: New OpenWhisk PMC Members: Brendan Doyle and Cosmin Stanciu

[Dascalita Dragos]

Huge congrats Cosmin, and Brendan 

On Wed, Feb 23, 2022 at 3:00 PM Dominic Kim  wrote:

> Nice!
> Congratulations Brendan and Cosmin!
>
> Best regards
> Dominic
>
> 2022년 2월 24일 (목) 오전 2:39, Tyson Norris 님이 작성:
>
> > Congratulations Cosmin and Brendan!!!
> >
> > Best,
> > Tyson
> >
> > On Wed, Feb 23, 2022 at 6:57 AM Rodric Rabbah  wrote:
> >
> > > Congratulations Brendan and Cosmin!
> > >
> > > -r
> > >
> > > On Wed, Feb 23, 2022 at 6:30 AM Matt Rutkowski 
> > > wrote:
> > >
> > > > Welcome and congratulations Brendan and Cosmin!
> > > >
> > > > Kind regards,
> > > > Matt
> > > >
> > >
> >
>

Re: [VOTE] Release Apache OpenWhisk Client Js (v3.21.5, rc1)

[Dascalita Dragos]

+1 to release it ! I also verified it successfully with rcverify !

Thanks Cosmin !

On Wed, Nov 3, 2021 at 12:12 PM Dave Grove  wrote:

> +1 to release openwhisk-client-js 3.21.5 from rc1.
>
> --dave
>
> Daves-MacBook-Pro:tools dgrove$ ./rcverify.sh openwhisk-client-js 3.21.5
> rc1
> rcverify.sh (script SHA1: 7FC5 5DBE 1809 6D92 DEFF  0E31 D138 059B 8F27
> 20F7)
> working in the following directory:
> /var/folders/8c/zvj0nsxx2rgc_km8nvf8k0c0gn/T/tmp.SN47Q7vm
> fetching tarball and signatures from
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1
> fetching openwhisk-client-js-3.21.5-sources.tar.gz... ok
> fetching openwhisk-client-js-3.21.5-sources.tar.gz.asc... ok
> fetching openwhisk-client-js-3.21.5-sources.tar.gz.sha512... ok
> fetching apache license... ok
> fetching release keys... ok
> importing keys... ok (keys already imported (processed 13 unchanged 13))
> unpacking tar ball... ok
> cloning scancode... ok
> computing sha512 for openwhisk-client-js-3.21.5-sources.tar.gz... ok
> openwhisk-client-js-3.21.5-sources.tar.gz:
> 06B7D8FB 6F5DC5CC A3BDFB08 6952FD10 8478EC3D 3A16B2C9 200D0157 0A3D2548
> 01DD9030
>  6FDDA0E7 EEF39D8E 594504AD 069AFD40 C85F3F09 CB5E4809 153F6D2B
> validating sha512... passed
> verifying asc... passed (signed-by: Cosmin Stanciu (CODE SIGNING KEY) <
> stan...@apache.org>)
> verifying notice... passed
> verifying absence of DISCLAIMER.txt passed
> verifying license... passed
> verifying sources have proper headers... passed
> scanning for executable files... passed
> scanning for unexpected file types... passed
> scanning for archives... passed
> scanning for packages... passed
> scanning package.json for version match... passed
> scanning package-lock.json for version match... passed
>
>
> On 2021/11/03 07:39:26 Cosmin Stanciu wrote:
> > Hi,
> >
> > This is a call to vote on releasing version 3.21.5 release candidate rc1
> of the following project module with artifacts built from the Git
> repositories and commit IDs listed below.
> >
> > * OpenWhisk Client Js: cb4b147ada707fddc34f37a5bdc4413eb06b2474
> >
> https://github.com/apache/openwhisk-client-js/commit/cb4b147ada707fddc34f37a5bdc4413eb06b2474
> >
> https://github.com/apache/openwhisk-client-js/commit/4dfc896d230f2d705c300ee1520ef25b8008762d
> >
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-client-js-3.21.5-sources.tar.gz
> >
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-client-js-3.21.5-sources.tar.gz.asc
> >
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-client-js-3.21.5-sources.tar.gz.sha512
> >
> > This release is comprised of source code distribution only.
> >
> > You can use this UNIX script to download the release and verify the
> checklist below:
> >
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=a071b1b
> >
> > Usage:
> > curl -s "
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=a071b1b";
> -o rcverify.sh
> > chmod +x rcverify.sh
> > ./rcverify.sh openwhisk-client-js 3.21.5 rc1
> >
> > Please vote to approve this release:
> >
> >   [ ] +1 Approve the release
> >   [ ]  0 Don't care
> >   [ ] -1 Don't release, because ...
> >
> > Release verification checklist for reference:
> >   [ ] Download links are valid.
> >   [ ] Checksums and PGP signatures are valid.
> >   [ ] Source code artifacts have correct names matching the current
> release.
> >   [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
> >   [ ] All files have license headers as specified by OpenWhisk project
> policy [1].
> >   [ ] No compiled archives bundled in source archive.
> >
> > This majority vote is open for at least 72 hours.
> >
> >
> > [1]
> https://github.com/apache/openwhisk-release/blob/master/docs/license_compliance.md
> >
> >
>

Re: [DISCUSS] Upgrade Akka to 2.6.12

[Dascalita Dragos]

Just letting you know that this PR is now merged. Special thanks to Eugene
Tulika for working on this PR, and also for fixing the Travis build which
had issues due to a dependent lib [1]

[1] - https://github.com/apache/openwhisk/pull/5121

On Mon, Mar 29, 2021 at 12:59 PM Dascalita Dragos 
wrote:

> Hi,
> We have this nice contribution by Eugene Tulika [1] which has recently
> taken the endeavor of upgrading our project to Akka 2.6.12. Thanks Eugene !
>
> Eugene and I work together; we wanna share some improvements for sync web
> actions on a separate PR, but before that, there are two PRs to be merged
> as prereqs: [1] and [2].
>
> I'm opening this thread to discuss only the Akka upgrade at [1]. The PR
> was approved a few weeks ago (thanks Rodric !). I'm leaving this thread
> open in case anyone has any comments to add to this change, otherwise I
> plan on merging it either later in the week, or early next week.
>
> Thanks,
> dragos
>
> [1] - https://github.com/apache/openwhisk/pull/5065
> [2] - https://github.com/apache/openwhisk/pull/5091
>

Re: [VOTE] Release Apache OpenWhisk Command-line Interface (CLI) (v1.2.0, rc1)

[Dascalita Dragos]

+1 to release  !

I tested with rcverify and all tests passed as expected.

My highlight:  .rs files are automatically recognized as Rust kind !

On Mon, Mar 29, 2021 at 1:57 PM Dave Grove  wrote:

> +1 to release openwhisk-cli v1.2.0 from rc1.
>
> reverify log appended.
>
> --Dave
>
> Daves-MacBook-Pro:tools dgrove$ ./rcverify.sh openwhisk-cli 'OpenWhisk
> Command-line Interface (CLI)' 1.2.0 rc1
> rcverify.sh (script SHA1: DE44 311A 861F 6965 8E52  9B21 96EB 949F 3E76
> A29C)
> working in the following directory:
> /var/folders/8c/zvj0nsxx2rgc_km8nvf8k0c0gn/T/tmp.Ffq25W7s
> fetching tarball and signatures from
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1
> fetching openwhisk-cli-1.2.0-sources.tar.gz... ok
> fetching openwhisk-cli-1.2.0-sources.tar.gz.asc... ok
> fetching openwhisk-cli-1.2.0-sources.tar.gz.sha512... ok
> fetching apache license... ok
> fetching release keys... ok
> importing keys... ok (keys already imported (processed 10 unchanged 10))
> unpacking tar ball... ok
> cloning scancode... ok
> computing sha512 for openwhisk-cli-1.2.0-sources.tar.gz... ok
> openwhisk-cli-1.2.0-sources.tar.gz: 6556A143 A5972A60 AF2143EE FECA3935
> B1D536CA
> 183AFD70 60F97B83 C417A2E0 56A576D0
> 72182E3D
> 8A84CAAC A109D98F 10E9A8EC B9ABCD25
> 42C77669
> 812052C6
> validating sha512... passed
> verifying asc... passed (signed-by: Matt Rutkowski  >)
> verifying notice... passed
> verifying absence of DISCLAIMER.txt passed
> verifying license... passed
> verifying sources have proper headers... passed
> scanning for executable files... passed
> scanning for unexpected file types... passed
> scanning for archives... passed
> scanning for packages... passed
> scanning package.json for version match... passed (none detected)
> scanning package-lock.json for version match... passed (none detected)
> removing the scratch space
> (/var/folders/8c/zvj0nsxx2rgc_km8nvf8k0c0gn/T/tmp.Ffq25W7s)... ok
>
> On 2021/03/29 15:02:29, Matt Rutkowski  wrote:
> > Hi Whiskians,
> >
> > This is a call to vote on releasing version 1.2.0 release candidate rc1
> of the following project module with artifacts built from the Git
> repositories and commit IDs listed below.
> >
> > OpenWhisk Command-line Interface (CLI):
> 7c47ef1dbad550566114baf976c091b4cdd9e678
> >
> > Details of the commits can be found here:
> >
> >
> https://github.com/apache/openwhisk-cli/commit/7c47ef1dbad550566114baf976c091b4cdd9e678
> >
> > The corresponding candidate release artifacts can be found here:
> >
> >
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-cli-1.2.0-sources.tar.gz
> >
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-cli-1.2.0-sources.tar.gz.asc
> >
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-cli-1.2.0-sources.tar.gz.sha512
> >
> > This release is comprised of source code distribution only.
> >
> > You can use this UNIX script to download the release and verify the
> checklist below:
> >
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=927cdc6aa2b07fd043bba225b89fbdc0c7e204df
> >
> > Usage:
> > curl -s "
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=927cdc6aa2b07fd043bba225b89fbdc0c7e204df";
> -o rcverify.sh
> > chmod +x rcverify.sh
> > ./rcverify.sh openwhisk-cli 'OpenWhisk Command-line Interface (CLI)'
> 1.2.0 rc1
> >
> > Please vote to approve this release:
> >
> >   [ ] +1 Approve the release
> >   [ ]  0 Don't care
> >   [ ] -1 Don't release, because ...
> >
> > Release verification checklist for reference:
> >   [ ] Download links are valid.
> >   [ ] Checksums and PGP signatures are valid.
> >   [ ] Source code artifacts have correct names matching the current
> release.
> >   [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
> >   [ ] All files have license headers as specified by OpenWhisk project
> policy [1].
> >   [ ] No compiled archives bundled in source archive.
> >
> > This majority vote is open for at least 72 hours.
> >
> > [1]
> https://github.com/apache/openwhisk-release/blob/master/docs/license_compliance.md
> >
> > Thanks,
> > Matt
> >
>

[DISCUSS] Upgrade Akka to 2.6.12

[Dascalita Dragos]

Hi,
We have this nice contribution by Eugene Tulika [1] which has recently
taken the endeavor of upgrading our project to Akka 2.6.12. Thanks Eugene !

Eugene and I work together; we wanna share some improvements for sync web
actions on a separate PR, but before that, there are two PRs to be merged
as prereqs: [1] and [2].

I'm opening this thread to discuss only the Akka upgrade at [1]. The PR was
approved a few weeks ago (thanks Rodric !). I'm leaving this thread open in
case anyone has any comments to add to this change, otherwise I plan on
merging it either later in the week, or early next week.

Thanks,
dragos

[1] - https://github.com/apache/openwhisk/pull/5065
[2] - https://github.com/apache/openwhisk/pull/5091

Re: [Review]Change return status code for ApplicationError and DeveloperError

[Dascalita Dragos]

While we're on the status code topic I'd like to share some feedback we got
from developers using OpenWhisk.

They have complained that for blocking activations, when the system can't
execute the action, they get a 202 instead. The feedback was that it's not
correct for a system to override the intention of the developer, and that
the correct response should be an error status code. The system should also
stop the execution to avoid wasting resources. Since OpenWhisk actions can
be invoked from a browser too (HTML content, or other content type that can
be displayed by the browser), dealing with a 202 becomes complicated, and
it's not in line with its purpose either [1] "cases where another process
or server handles the request, or for batch processing".

[1] - https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/202

On Mon, Jan 25, 2021 at 7:00 AM Rodric Rabbah  wrote:

> Thanks Jiang for addressing this long standing bug.
>
> I reviewed the PR. The 400 status code for developer errors also matches
> the web action response. I think the only real debate is whether
> application error should be 207 or 400. If a web action responds with an
> error and no explicit status code, is it treated as a 20x response. So I
> find the change in your PR consistent.
>
> There may be some effects to check in the npm client and the wsk (or other
> clis).
>
> -r
>
> On Mon, Jan 25, 2021 at 1:19 AM 蒋鹏程  wrote:
>
> > Dear whiskers:
> > ​
> >   Currently we are using 502 BadGateway for ApplicationError and
> > DeveloperError, which is not very appropriate I think, so I submit a PR
> to
> > change this:
> > ​
> > 1. Use MultiStatus for ApplicationError
> > 2. Use BadRequest for DeveloperError
> > ​
> > Any comments and suggestions are welcomed
> > ​
> > refs:
> > 1. https://github.com/apache/openwhisk/issues/645#issuecomment-232534948
> > 2. https://github.com/apache/openwhisk/pull/5047
> > ​
> > Best Regards
> > Jiang Pengcheng
> >
> > ​
> >
>

Re: Proposal: Adding Support for Juypter Notebooks

[Dascalita Dragos]

Michele, this is awesome. I'm so appreciative for building on the existing
"pythonaiaction".
Thanks for updating it to the latest Tensorflow.

I think there's a lot of value in running AI Actions in a serverless
fashion to quickly build AI APIs (inference mainly; training is a
separate topic that's worth investigating; I mean training batches in
parallel, using AI actions). For inference however teams can experiment
with many different AI models in a very (cost) efficient way. Paying only
for what you use is what makes the serverless execution model so attractive
for AI.

Kudos for moving this ahead ! A BIG thank you !

On Sat, Oct 24, 2020 at 2:21 AM Michele Sciabarra 
wrote:

> I implemented support for Jupyter Notebooks for a Python 3.8 runtime (that
> I temporarily published as actionloop/action-python-v3.8) as you can see
> here:
>
> https://imgur.com/dTjj6UJ
>
> It was easy as I leveraged the "compilation" script of Actionloop that is
> written in Python and also the nbconvert library that is also Python-based.
>
> However since I believe this is a useful feature, and since Jupyter
> Notebooks are available for all the languages I was thinking to make it a
> standard feature of the Go Proxy (aka AcionLoop) so it is inherited by all
> the runtimes based on ActionLoop (Python, PHP, Rust, Go, and the flavors of
> Java and Node that uses it).
>
> So what the community thinks? Is widespread support for Jupyter Notebook
> as a "source" format, that I detect and handle the init level and to
> extract the code from it, a worthy addition?
>
> --
>   Michele Sciabarra
>   mich...@sciabarra.com
>

Re: [VOTE] Release Apache OpenWhisk Client Js (v3.21.2, rc1)

[Dascalita Dragos]

+1

Release verification checklist (tested with rcverify.sh):
  [x] Download links are valid.
  [x] Checksums and PGP signatures are valid.
  [x] Source code artifacts have correct names matching the current release.
  [x] LICENSE and NOTICE files are correct for each OpenWhisk repository.
  [x] All files have license headers as specified by OpenWhisk project
policy [1].
  [x] No compiled archives bundled in source archive.



On Mon, May 4, 2020 at 6:00 PM Rodric Rabbah  wrote:

> Hi,
>
> This is a call to vote on releasing version 3.21.2 release candidate rc1 of
> the following project module with artifacts built from the Git repositories
> and commit IDs listed below.
>
> * OpenWhisk Client Js: eaa43743648c4ff69f53af95befc9bd314178d57
>
>
> https://github.com/apache/openwhisk-client-js/commits/eaa43743648c4ff69f53af95befc9bd314178d57
>
>
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-client-js-3.21.2-sources.tar.gz
>
>
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-client-js-3.21.2-sources.tar.gz.asc
>
>
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-client-js-3.21.2-sources.tar.gz.sha512
>
> This release is comprised of source code distribution only.
>
> You can use this UNIX script to download the release and verify the
> checklist below:
>
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=56445f1
>
> Usage:
> curl -s "
>
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=56445f1
> "
> -o rcverify.sh
> chmod +x rcverify.sh
> rcverify.sh openwhisk-client-js 'OpenWhisk Client Js' 3.21.2 rc1
>
> Please vote to approve this release:
>
>   [ ] +1 Approve the release
>   [ ]  0 Don't care
>   [ ] -1 Don't release, because ...
>
> Release verification checklist for reference:
>   [ ] Download links are valid.
>   [ ] Checksums and PGP signatures are valid.
>   [ ] Source code artifacts have correct names matching the current
> release.
>   [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
>   [ ] All files have license headers as specified by OpenWhisk project
> policy [1].
>   [ ] No compiled archives bundled in source archive.
>
> This majority vote is open for at least 72 hours.
>
>
> [1]
>
> https://github.com/apache/openwhisk-release/blob/master/docs/license_compliance.md
>

Re: Welcome new committer Alex Klimetschek

[Dascalita Dragos]

Warm welcome Alex !

On Thu, Mar 12, 2020 at 6:38 AM David P Grove  wrote:

>
>
> I'm happy to share that the Apache OpenWhisk PMC has invited Alex
> Klimetschek to become a committer and he has accepted.
>
> Alex drove the development of wskdebug, a debugging and live development
> tool for Apache OpenWhisk.
>
> For more information about wskdebug, see
> https://github.com/apache/openwhisk-wskdebug.  The project just made the
> first Apache release of wskdebug earlier this week, so give it a try if you
> haven't already.
>

Re: [VOTE] Accept wskdebug contribution from Adobe

[Dascalita Dragos]

Big +1 too !

On Fri, Feb 21, 2020 at 8:28 AM Rob Allen  wrote:

> +1
>
> Thanks Adobe team.
>
> Regards,
>
> Rob
>
> > On 20 Feb 2020, at 17:35, David P Grove  wrote:
> >
> > 
> >
> > This is a call to vote to accept the donation of the wskdebug code base
> > from Adobe as discussed in [1]. This majority vote will remain open for
> at
> > least 72 hours.
> >
> > The IP clearance form [2] has been filed and I will initiate an IPMC IP
> > clearance vote shortly (in parallel with this vote).  (Note that there
> > appears to be a lag in rebuilding the html version of the clearance at
> [2];
> > consult the .xml file in svn [3] if you want to see the latest)
> >
> > If the vote is successful, we will create a new git repo,
> > openwhisk-wskdebug, to contain the donated code to allow the OpenWhisk
> > community to continue its development.
> >
> > --dave
> >
> > [1]
> >
> https://lists.apache.org/thread.html/3cb01abdf0a6a0f2d577d08ed687ab0c06918b7ce958a27b0654b145%40%3Cdev.openwhisk.apache.org%3E
> > [2] https://incubator.apache.org/ip-clearance/openwhisk-wskdebug.html
> > [3]
> >
> https://svn.apache.org/repos/asf/incubator/public/trunk/content/ip-clearance/openwhisk-wskdebug.xml
>
>

Re: recent openwhisk-runtime-* releases

[Dascalita Dragos]

Ditto. Many thanks Dave !

On Tue, Feb 18, 2020 at 6:51 AM Rodric Rabbah  wrote:

> Thank you Dave for leading the marathon.
>
> -r
>
> > On Feb 18, 2020, at 9:10 AM, David P Grove  wrote:
> >
> > 
> >
> > Thanks to all that have helped with the recent round of releases of our
> > action runtimes!
> >
> > There are now updated releases of OpenWhisk runtimes for NodeJS, Python,
> > Rust, Swift, Ruby, PHP, and Java available at
> > http://openwhisk.apache.org/downloads.html
> >
> > --dave
>

Re: [VOTE] Release Apache OpenWhisk Runtime Go (v1.15.0, rc1)

[Dascalita Dragos]

+1
Verified with rcverify tool, log appended.

Thanks,
dragos

rcverify.sh (script SHA1: 8DD4 A0E6 7974 CB8F E859  B403 9463 852F 2749
E23D)

working in the following directory:

/var/folders/zy/xhbjcqg12w1182qx447mdz10gn/T/tmp.ywS1on4g

fetching tarball and signatures from
https://dist.apache.org/repos/dist/dev/openwhisk/rc1

fetching openwhisk-runtime-go-1.15.0-sources.tar.gz

fetching openwhisk-runtime-go-1.15.0-sources.tar.gz.asc

fetching openwhisk-runtime-go-1.15.0-sources.tar.gz.sha512

fetching release keys

importing keys

gpg: key 72AF0CC22C4CF320: "Vincent Hou (Release manager of OpenWhisk) <
houshen...@apache.org>" not changed

gpg: key 22907064147F886E: "Dave Grove " not changed

gpg: key 44667BC927C86D51: "Rodric Rabbah " not changed

gpg: key B1457C3D7101CC78: public key "James Thomas "
imported

gpg: key A600E3331427515D: public key "Olivier Tardieu "
imported

gpg: Total number processed: 5

gpg:   imported: 2

gpg:  unchanged: 3

unpacking tar ball

cloning scancode

Cloning into 'openwhisk-utilities'...

remote: Enumerating objects: 55, done.

remote: Counting objects: 100% (55/55), done.

remote: Compressing objects: 100% (41/41), done.

remote: Total 55 (delta 21), reused 30 (delta 11), pack-reused 0

Unpacking objects: 100% (55/55), done.

computing sha512 for openwhisk-runtime-go-1.15.0-sources.tar.gz

SHA512: openwhisk-runtime-go-1.15.0-sources.tar.gz:

6483FB2A 09312CDE DD78996E 641F330F 21CF314A EA73EF68 80ADBB1E 5EA3CC22
0DBD1BCC

 FA6B3C24 4A33204A ED1195E6 8D179B55 EBF18FEA F12E61B2 1A534926

validating sha512... passed

verifying asc... passed (signed-by: Dave Grove )

verifying notice... passed

verifying absence of DISCLAIMER.txt passed

verifying license... passed

verifying sources have proper headers... passed

scanning for executable files... passed

scanning for unexpected file types... passed

scanning for archives... passed

scanning for packages... passed


On Tue, Jan 14, 2020 at 8:46 AM David P Grove  wrote:

>
>
> Hi,
>
> This is a call to vote on releasing version 1.15.0 release candidate rc1 of
> the following project module with artifacts built from the Git repositories
> and commit IDs listed below.
>
> * OpenWhisk Runtime Go: ddfded27004b1042bdccde9c75b7404f439ebc2a
>
>
> https://github.com/apache/openwhisk-runtime-go/commits/ddfded27004b1042bdccde9c75b7404f439ebc2a
>
>
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-go-1.15.0-sources.tar.gz
>
>
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-go-1.15.0-sources.tar.gz.asc
>
>
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-go-1.15.0-sources.tar.gz.sha512
>
> This release is comprised of source code distribution only.
>
> You can use this UNIX script to download the release and verify the
> checklist below:
>
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=5c13e39
>
> Usage:
> curl -s "
>
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=5c13e39
> " -o rcverify.sh
> chmod +x rcverify.sh
> rcverify.sh openwhisk-runtime-go 'OpenWhisk Runtime Go' 1.15.0 rc1
>
> Please vote to approve this release:
>
>   [ ] +1 Approve the release
>   [ ]  0 Don't care
>   [ ] -1 Don't release, because ...
>
> Release verification checklist for reference:
>   [ ] Download links are valid.
>   [ ] Checksums and PGP signatures are valid.
>   [ ] Source code artifacts have correct names matching the current
> release.
>   [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
>   [ ] All files have license headers as specified by OpenWhisk project
> policy [1].
>   [ ] No compiled archives bundled in source archive.
>
> This majority vote is open for at least 72 hours.
>
>
> [1]
>
> https://github.com/apache/openwhisk-release/blob/master/docs/license_compliance.md
>

Re: Requesting feedback about the new typescript runtime

[Dascalita Dragos]

Thanks for the details Rodric.
I’m then +1 for a dedicated repo.

On Thu, Jan 9, 2020 at 5:18 PM Rodric Rabbah  wrote:

> It wasn’t done previously because there was no advantage - the go proxy
> lead to performance gains for the python runtime and a few others iirc but
> was the same for node. There was also no support for intra container
> concurrency.
>
> -r
>
> > On Jan 9, 2020, at 8:04 PM, Dascalita Dragos  wrote:
> >
> > The argument that the way this new runtime is constructed is very
> different
> > from the existing  nodejs one, seems a strong one in my opinion to
> consider
> > a dedicated repo.
> >
> > The unclear part to me is whether it would make sense to consider
> applying
> > the “action loop” pattern to the existing NodeJs Runtime. Are there any
> > thoughts on this ?
> >
> >
> >
> >
> >
> >> On Wed, Jan 8, 2020 at 12:43 PM Michele Sciabarra <
> mich...@sciabarra.com>
> >> wrote:
> >>
> >> Hello,
> >> I developed a new openwhisk runtime with native support for typescript
> as
> >> part of my work at Nimbella and I intend to donate this code the Apache
> >> Openwhisk project. The new runtime works for node/javascript as well.
> Since
> >> typescript is compiled, I reused the compilation support from the
> "action
> >> loop" proxy. This runtime is intended as a new action kind, which can
> >> support both node and typescript.
> >> This email is to solicit feedback for the next 48 hrs on the question of
> >> new repo vs reuse the existing repo for node.js. I will decide based on
> >> community feedback and follow up with the next steps.
> >> Features of the new runtime:
> >> - supports precompilation, meaning you use the runtime as a compiler to
> >> "precompile" the sources. Compilation here means translating typescript
> to
> >> javascript, resolve the package.json downloading all the node_modules
> and
> >> producing a zip that includes everything ready to run.
> >> You do this with a command like:
> >>
> >> docker run openwhisk/action-typescript-v3.7 bin.zip
> >>
> >> then you can create the action using bin.zip.
> >> - supports on the fly compilation, so that an action made up of
> typescript
> >> sources will also work.
> >> - supports debugging, but in a different way than adobe/wskdebug. If you
> >> set an environment variable __OW_DEBUG_PORT it will start the runtime
> >> listening on a port for a debugger to attach.
> >>
> >> - an action must always "export" the main function. So in typescript you
> >> need to do: `export function main(args) { .. }' and in javascript
> >> `export.main = function args(...)`. I did manage to make sure a single
> file
> >> with 'function main(args)' works but it does not work when there are
> >> multiple files zipped. I think the current apache openwhisk nodejs
> runtime
> >> uses eval, I use a `require` to import action code.
> >>
> >> Since this is a new language runtime (typescript) and also differs in
> the
> >> node.js support from the existing apache openwhisk nodejs runtime, I am
> >> proposing a new apache repo to host this code. It's possible to include
> the
> >> new sources in the existing node.js runtime repo to avoid adding yet
> >> another repository, but may be confusing since the code base is very
> >> different.
> >>
> >> --
> >>  Michele Sciabarra
> >>  mich...@sciabarra.com
> >>
>

Re: Requesting feedback about the new typescript runtime

[Dascalita Dragos]

The argument that the way this new runtime is constructed is very different
from the existing  nodejs one, seems a strong one in my opinion to consider
a dedicated repo.

The unclear part to me is whether it would make sense to consider applying
the “action loop” pattern to the existing NodeJs Runtime. Are there any
thoughts on this ?





On Wed, Jan 8, 2020 at 12:43 PM Michele Sciabarra 
wrote:

> Hello,
> I developed a new openwhisk runtime with native support for typescript as
> part of my work at Nimbella and I intend to donate this code the Apache
> Openwhisk project. The new runtime works for node/javascript as well. Since
> typescript is compiled, I reused the compilation support from the "action
> loop" proxy. This runtime is intended as a new action kind, which can
> support both node and typescript.
> This email is to solicit feedback for the next 48 hrs on the question of
> new repo vs reuse the existing repo for node.js. I will decide based on
> community feedback and follow up with the next steps.
> Features of the new runtime:
> - supports precompilation, meaning you use the runtime as a compiler to
> "precompile" the sources. Compilation here means translating typescript to
> javascript, resolve the package.json downloading all the node_modules and
> producing a zip that includes everything ready to run.
> You do this with a command like:
>
>  docker run openwhisk/action-typescript-v3.7 bin.zip
>
> then you can create the action using bin.zip.
> - supports on the fly compilation, so that an action made up of typescript
> sources will also work.
> - supports debugging, but in a different way than adobe/wskdebug. If you
> set an environment variable __OW_DEBUG_PORT it will start the runtime
> listening on a port for a debugger to attach.
>
> - an action must always "export" the main function. So in typescript you
> need to do: `export function main(args) { .. }' and in javascript
> `export.main = function args(...)`. I did manage to make sure a single file
> with 'function main(args)' works but it does not work when there are
> multiple files zipped. I think the current apache openwhisk nodejs runtime
> uses eval, I use a `require` to import action code.
>
> Since this is a new language runtime (typescript) and also differs in the
> node.js support from the existing apache openwhisk nodejs runtime, I am
> proposing a new apache repo to host this code. It's possible to include the
> new sources in the existing node.js runtime repo to avoid adding yet
> another repository, but may be confusing since the code base is very
> different.
>
> --
>   Michele Sciabarra
>   mich...@sciabarra.com
>

Re: Welcome new Committer Shawn Black

[Dascalita Dragos]

Warm welcome Shawn !

On Sun, Dec 15, 2019 at 7:46 AM Chetan Mehrotra 
wrote:

> Welcome Shawn!!
>
> On Tue, Dec 3, 2019, 4:49 AM Rodric Rabbah  wrote:
>
> > It is my pleasure to share that the OpenWhisk PPMC has elected Shawn
> Black
> > as a Committer, based on his ongoing and valuable contributions to the
> > project around the .NET runtime. Shawn has accepted the invitation.
> >
> > Please join me in welcoming Shawn to his new role on the project.
> >
> > -r
> >
>

Re: API Gateway routing on Mac

[Dascalita Dragos]

I didn’t check the recording but How is the URL for the action retrieved ?

On Mon, Nov 18, 2019 at 10:05 AM Tom Barber  wrote:

> Hmm, good catch I didn't spot that. Its all created dynamically by the
> serverless deploy routine the entirety of which looks like:
> https://gist.github.com/buggtb/34163a72f8ec88a3df6b2b80f3d79fbc so its
> not in my serverless.yaml so I guess its something under the hood in
> serverless. I'll dig more.
>
> On Mon, Nov 18, 2019 at 5:37 PM Matt Hamann 
> wrote:
> >
> > In the video when Pixelize action is brought up, I see this output:
> >
> > ```
> > endpoints (web actions):
> > https://
> https://10.0.0.106/api/v1/web/guest/default/pixlize-serverless-dev-login
> > ```
> >
> > The double "https" is likely causing the issue here, as the API
> > gateway doesn't know what to do with that.
> >
> > Must be a configuration issue somewhere...
> >
> > 
> > -Matt
> > matthew.ham...@gmail.com
> >
> >
> > On Mon, Nov 18, 2019 at 12:21 PM Tom Barber  wrote:
> > >
> > > I replicated it from scratch here's a recording of what I'm doing:
> > >
> > > https://asciinema.org/a/Zwoim3sFOSUXAC9ZCAYtdbPUV
> > >
> > > Thanks!
> > >
> > > Tom
> > >
> > >
> > > On Mon, Nov 18, 2019 at 4:33 PM Rodric Rabbah 
> wrote:
> > > >
> > > > Hi Tom. I’ve used quick start on Mac with api gateway successfully
> in the past. I’ll see if I can reproduce your issue.
> > > >
> > > > -r
> > > >
> > > > > On Nov 18, 2019, at 11:26 AM, Tom Barber 
> wrote:
> > > > >
> > > > > Just to expand on my deployment question, I appreciate make
> > > > > quick-start isn't production ready. I'm just curious if its not API
> > > > > gateway ready either for Mac users?
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > Spicule Limited is registered in England & Wales. Company Number:
> > > > > 09954122. Registered office: First Floor, Telecom House, 125-135
> Preston
> > > > > Road, Brighton, England, BN1 6AF. VAT No. 251478891.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > All engagements
> > > > > are subject to Spicule Terms and Conditions of Business. This
> email and its
> > > > > contents are intended solely for the individual to whom it is
> addressed and
> > > > > may contain information that is confidential, privileged or
> otherwise
> > > > > protected from disclosure, distributing or copying. Any views or
> opinions
> > > > > presented in this email are solely those of the author and do not
> > > > > necessarily represent those of Spicule Limited. The company
> accepts no
> > > > > liability for any damage caused by any virus transmitted by this
> email. If
> > > > > you have received this message in error, please notify us
> immediately by
> > > > > reply email before deleting it from your system. Service of legal
> notice
> > > > > cannot be effected on Spicule Limited by email.
> > >
> > > --
> > >
> > >
> > > Spicule Limited is registered in England & Wales. Company Number:
> > > 09954122. Registered office: First Floor, Telecom House, 125-135
> Preston
> > > Road, Brighton, England, BN1 6AF. VAT No. 251478891.
> > >
> > >
> > >
> > >
> > > All engagements
> > > are subject to Spicule Terms and Conditions of Business. This email
> and its
> > > contents are intended solely for the individual to whom it is
> addressed and
> > > may contain information that is confidential, privileged or otherwise
> > > protected from disclosure, distributing or copying. Any views or
> opinions
> > > presented in this email are solely those of the author and do not
> > > necessarily represent those of Spicule Limited. The company accepts no
> > > liability for any damage caused by any virus transmitted by this
> email. If
> > > you have received this message in error, please notify us immediately
> by
> > > reply email before deleting it from your system. Service of legal
> notice
> > > cannot be effected on Spicule Limited by email.
>
> --
>
>
> Spicule Limited is registered in England & Wales. Company Number:
> 09954122. Registered office: First Floor, Telecom House, 125-135 Preston
> Road, Brighton, England, BN1 6AF. VAT No. 251478891.
>
>
>
>
> All engagements
> are subject to Spicule Terms and Conditions of Business. This email and
> its
> contents are intended solely for the individual to whom it is addressed
> and
> may contain information that is confidential, privileged or otherwise
> protected from disclosure, distributing or copying. Any views or opinions
> presented in this email are solely those of the author and do not
> necessarily represent those of Spicule Limited. The company accepts no
> liability for any damage caused by any virus transmitted by this email. If
> you have received this message in error, please notify us immediately by
> reply email before deleting it from your system. Service of legal notice
> cannot be effected on Spicule Limited by email.
>

Re: Composer requires Redis but NodeJS Runtime doesn't include it

[Dascalita Dragos]

Thanks so much for the prompt feedback. I’ll push a PR to include Redis in
Nodejs 10 and 12. Also “uuid” module is needed.

On Tue, Oct 1, 2019 at 4:36 PM Olivier Tardieu  wrote:

> I am planning to update composer to make REDIS optional in any case
> (option 2), but I agree we should go for option 1: include redis in the
> image to help with user experience.
> The redis npm package is very small (Including dependencies) so I don't
> think image size is a concern.
>
> du -k node_modules
> 8   node_modules/double-ended-queue/js
> 40  node_modules/double-ended-queue
> 32  node_modules/redis-parser/lib
> 60  node_modules/redis-parser
> 8   node_modules/redis/.github
> 72  node_modules/redis/lib
> 224 node_modules/redis
> 4   node_modules/redis-commands/tools
> 60  node_modules/redis-commands
> 384 node_modules
>
> Olivier
>
> Rodric Rabbah  wrote on 10/01/2019 10:12:43 AM:
>
> > From: Rodric Rabbah 
> > To: dev@openwhisk.apache.org
> > Date: 10/01/2019 10:26 AM
> > Subject: [EXTERNAL] Re: Composer requires Redis but NodeJS Runtime
> > doesn't include it
> >
> > +1 for including redis in runtimes.
> >
> > -r
> >
> > On Tue, Oct 1, 2019 at 10:06 AM Dascalita Dragos 
> wrote:
> >
> > > Currently there's an issue with Composer which requires Redis [1].
> Redis
> > > module is not installed by default in the NodeJS images.
> > > I see 2 options to unblock this:
> > > 1 -  include Redis in the default nodeJS runtime [2]
> > > 2  - make Redis optional for Composer; parallel combinator in Composer
> > > won't work w/o Redis, but we can assume it's the Openwhisk operator's
> > > responsibility to provide a default nodeJS image that includes Redis
> > > module.
> > >
> > > I'm personally slightly more inclined toward option (1) b/c that
> enables in
> > > theory all features that exist in Composer. I'm saying "in theory" b/c
> the
> > > developers still needs to provision Redis on their own, before using
> > > parallel combinators; so, since the developers need to do something to
> > > enable this feature, we could also assume they could provide a custom
> > > runtime that includes Redis, in case the operator doesn't provide it;
> but
> > > with a blackbox action, the system can't optimize the cold-start.
> > >
> > > The bottom line is that currently, a developer that wants to try
> OpenWhisk
> > > and Composer, by default it won't work; unless at least we implement
> option
> > > (2) . Hence I'm not really sure what's best to do.
> > >
> > > WDYT ?
> > >
> > > Thanks,
> > > dragos
> > >
> > > [1] -
> > > https://urldefense.proofpoint.com/v2/url?
> >
>
> u=https-3A__github.com_apache_openwhisk-2Dcomposer_blob_master_conductor.js-23L45&d=DwIBaQ&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=C3zA0dhyHjF4WaOy8EW8kQHtYUl9-
> > dKPdS8OrjFeQmE&m=bUh0V9YIXGy9eyT6ur-
> >
> pZfEM6vweXiRrz_PM1XCHj_Y&s=lB5jlca6suiqjog31T_VnjCUyY4qSHmamb3ndolh9Kk&e=
> > > [2] -
> > >
> > > https://urldefense.proofpoint.com/v2/url?
> >
>
> u=https-3A__github.com_apache_openwhisk-2Druntime-2Dnodejs_blob_master_core_nodejs10Action_package.json&d=DwIBaQ&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=C3zA0dhyHjF4WaOy8EW8kQHtYUl9-
> > dKPdS8OrjFeQmE&m=bUh0V9YIXGy9eyT6ur-
> >
> pZfEM6vweXiRrz_PM1XCHj_Y&s=haweaGNP4coHQNXoAsX5UBsPizUsKuKgawnBTZgH6pE&e=
> > >
>
>
>

Composer requires Redis but NodeJS Runtime doesn't include it

[Dascalita Dragos]

Currently there's an issue with Composer which requires Redis [1]. Redis
module is not installed by default in the NodeJS images.
I see 2 options to unblock this:
1 -  include Redis in the default nodeJS runtime [2]
2  - make Redis optional for Composer; parallel combinator in Composer
won't work w/o Redis, but we can assume it's the Openwhisk operator's
responsibility to provide a default nodeJS image that includes Redis
module.

I'm personally slightly more inclined toward option (1) b/c that enables in
theory all features that exist in Composer. I'm saying "in theory" b/c the
developers still needs to provision Redis on their own, before using
parallel combinators; so, since the developers need to do something to
enable this feature, we could also assume they could provide a custom
runtime that includes Redis, in case the operator doesn't provide it; but
with a blackbox action, the system can't optimize the cold-start.

The bottom line is that currently, a developer that wants to try OpenWhisk
and Composer, by default it won't work; unless at least we implement option
(2) . Hence I'm not really sure what's best to do.

WDYT ?

Thanks,
dragos

[1] -
https://github.com/apache/openwhisk-composer/blob/master/conductor.js#L45
[2] -
https://github.com/apache/openwhisk-runtime-nodejs/blob/master/core/nodejs10Action/package.json

Re: Can we adjust the time for the community call

[Dascalita Dragos]

I'm PST but I'm ok with 7AM PST. I've been fighting to reschedule early
meetings in order to join OW community calls many times; moving it 1 hr
early would allow me to participate too.

On Thu, Sep 5, 2019 at 6:51 PM Dominic Kim  wrote:

> Thank you all for the warm words.
>
> Actually, 1 hour earlier is enough for me.
> This is because alternating time may introduce additional complexity in
> terms of management. Also, attendees can get confused.
> (I often felt it is cumbersome to remember the different time under a
> similar situation.)
> But if there are many people who want that, it would be worth to bear with
> it.
>
> I would send a poll to see if there are any other people pursuing this as
> Matt suggested.
>
>
> Best regards
> Dominic
>
>
> 2019년 9월 6일 (금) 오전 4:37, Matt Rutkowski 님이 작성:
>
> > Pursuing alternating times may be a good choice, but would like to know
> we
> > have a reliable person to host (fallback) for these slots as our
> > "volunteer" mechanism is barely functioning for the current call/time.
> >
> > Dom, perhaps you can post a poll for contribs. to vote on; perhaps this
> > would give us a sense of who is paying attention, what suits them and get
> > a ballpark on how many would benefit?
> >
> > Kind regards,
> > Matt
> >
> >
> >
> > From:   Tyson Norris 
> > To: "dev@openwhisk.apache.org" 
> > Date:   09/05/2019 10:40 AM
> > Subject:[EXTERNAL] Re: Can we adjust the time for the community
> > call
> >
> >
> >
> > US West coast dweller here - I'm OK with either 1 hour ealier, or
> > alternating times. Anything to get more players on the field :)
> >
> > Thanks
> > Tyson
> >
> > On 9/5/19, 3:53 AM, "Rodric Rabbah"  wrote:
> >
> > Thanks Dominic for bringing this up. +1 from me.
> > What about alternating times (every two weeks) so other time zones
> > aren't
> > always so late in the day?
> >
> > -r
> >
> > On Wed, Sep 4, 2019 at 11:27 PM Dominic Kim 
> > wrote:
> >
> > > Dear community members.
> > >
> > > I wonder whether we can set up our Tech. Int. meeting 1-hour
> > earlier.
> > > Currently, we hold the meeting at 3 PM GMT which is 12:00 AM
> > KST/JST.
> > >
> > > If we hold the meeting 1-hour earlier, the time changes like this:
> > >
> > > - 2 PM GMT.
> > > - 10 AM US eastern time
> > > - 10 CST
> > > - 11 KST/JST
> > >
> > > One issue is the US western time would be 7 AM.
> > > So if anyone lives in that area, I would give up.
> > >
> > > But if it's fine for most of the members, I hope we start the
> > meeting
> > > 1-hour earlier.
> > > It would help me and other folks living in Korea/Japan to join the
> > meeting
> > > better.
> > >
> > >
> > > Best regards
> > > Dominic
> > >
> >
> >
> >
> >
> >
> >
> >
>

Re: Re: Changing JavaScript SDK NPM Module Name: openwhisk => apache-openwhisk?

[Dascalita Dragos]

James, the way I'm imagining what could be done to update the name:
(1) deprecate "openwhisk" module with "npm deprecate" [1] so that everyone
gets the warning and
(2) publish "apache-openwhisk" with the same version as "openwhisk", and
update this one going forward.

For (1) we could plug OW graduation in the deprecation message.

If NPM displaying the deprecation warning is acceptable for tutorials,
docs, blog posts, etc, this could work.

[1] - https://docs.npmjs.com/cli/deprecate

On Wed, Jul 17, 2019 at 10:08 AM Matt Sicker  wrote:

> It’s likely a topic that is brought up by the board once in a while for
> those projects.
>
> On Wed, Jul 17, 2019 at 12:06, James Thomas  wrote:
>
> > I've discovered the Cordova project publishes all their project repos
> > without the `apache-` prefix.
> > https://www.npmjs.com/search?q=cordova
> >
> > Same goes for thrift (https://www.npmjs.com/package/thrift). I've
> > guess there's precedence that maybe this isn't an issue?
> >
> > On Mon, 15 Jul 2019 at 18:17, Matt Sicker  wrote:
> > >
> > > Most or all of the Apache projects that are distributed on Homebrew
> > >  are named apache-foo.
> > >
> > > ...except for `wsk` and `wskdeploy` which are curiously lacking
> > > `apache-` prefixes as well. ;)
> > >
> > > On Mon, 15 Jul 2019 at 12:08, Matt Rutkowski 
> > wrote:
> > > >
> > > > I too like the dash approach unless Apache likes having a domain name
> > > > style which implies (family) membership hierarchy.
> > > >
> > > >
> > > >
> > > > From:   Matt Sicker 
> > > > To: dev@openwhisk.apache.org
> > > > Date:   07/15/2019 12:05 PM
> > > > Subject:[EXTERNAL] Re: Changing JavaScript SDK NPM Module
> Name:
> > > > openwhisk => apache-openwhisk?
> > > >
> > > >
> > > >
> > > > The name with the dash looks nicer, agreed. In migrating from an old
> > > > package name to a new one where you already have existing users, I
> > > > haven't seen a solution to that myself quite yet, though I know that
> > > > Groovy has a similar problem where their packages are still published
> > > > under the `org.codehaus.groovy` group id instead of
> > > > `org.apache.groovy`. While Maven and NPM are quite different, the
> > > > method of migrating a package name is similarly not well-defined in
> > > > both systems.
> > > >
> > > > Does anyone have more info about how NPM runs their repository? Maybe
> > > > they can add in some redirects of some sort.
> > > >
> > > > On Mon, 15 Jul 2019 at 11:11, James Thomas 
> > wrote:
> > > > >
> > > > > Reviewing the ASF guidelines on NPM packages to check our JS SDK
> > > > satifises
> > > > > all the rules[1] - we're supposed to be publishing the NPM package
> as
> > > > > "apacheopenwhisk" and not "openwhisk". This NPM library was
> > published at
> > > > (
> > > > >
> > > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.npmjs.com_package_openwhisk&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=NilRlnhMriE1MNYQW3S_Ni47FW8uu-CTsXNbM3FYkH8&s=C-3wIDNjUO6k1tpWW7WQA9d4c-lbe7KshNS1jAR6jxM&e=
> > > > ) before the project was donated to
> > > > > Apache.
> > > > >
> > > > > Moving from the library to publish at `apache-openwhisk` rather
> than
> > > > > `openwhisk`[2] is not technically challenging (and the new package
> > name
> > > > is
> > > > > available) but will cause numerous issues
> > > > >
> > > > > I'm asking for comments on what to do about this. Would like to
> > engage
> > > > the
> > > > > ASF mentors for advice as well. What does the community think about
> > > > this?
> > > > >
> > > > > The library has significant usage (NPM tells me the library is
> > averaging
> > > > 6k
> > > > > downloads a week) using the existing package name. GitHub lists 38K
> > > > > references to the module.
> > > > >
> > > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_search-3Fq-3Drequire-2528-2522openwhisk-2522-2529-26type-3DCode&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=NilRlnhMriE1MNYQW3S_Ni47FW8uu-CTsXNbM3FYkH8&s=nIOIJxXhbd1TkXzWJVHx9-NAMQV4JuBsXbm1pEkX8u0&e=
> > > >
> > > > >
> > > > > All those external dependent projects, blog posts, documentation
> and
> > > > > tutorials, etc, that reference the library (and are outside of our
> > > > control)
> > > > > will be reliant on the old package name. These will still work (as
> > the
> > > > old
> > > > > library version will still be available from NPM) but never receive
> > new
> > > > > versions on installing the dependency. This may eventually mean the
> > old
> > > > > library doesn't work with future platform changes and/or lead to
> > > > security
> > > > > issues with outdated dependencies.
> > > > >
> > > > > I'm not sure if there's any leeway in the allowing the short-name
> for
> > > > the
> > > > > NPM library (given we follow all the other requirements)? This will
> > be a
> > > > > significant amount of work just changing all the references in

Re: Using Rust for the KnativeWhisk controller

[Dascalita Dragos]

I’m inserting my +1 for Go, based on the community adoption of Go; my
perception is that it found its place as the de-facto language for managing
infrastructure. Akka is amazing for distributed programming model ...
that’s my only argument for Scala, but this argument alone is not strong
enough to go against the stream.


What Controller functionality are you looking to re-implement ? The API
used by WSK CLI ? Or more ?


 dragos


On Wed, Jul 17, 2019 at 3:25 PM Matt Sicker  wrote:

> Ah, yes, Rust would be good for any performance-sensitive code or
> modules needed. I haven't kept up to date with C++ since pre-'10, but
> I'd imagine that it's easier to learn Rust than the latest version of
> C++ at this point.
>
> On Wed, 17 Jul 2019 at 08:06, Michele Sciabarra 
> wrote:
> >
> > Ok guys, I give up  with  rust. Will use Go.
> >
> >
> > --
> >   Michele Sciabarra
> >   mich...@sciabarra.com
> >
> > - Original message -
> > From: "Markus Thömmes" 
> > To: dev@openwhisk.apache.org
> > Subject: Re: Using Rust for the KnativeWhisk controller
> > Date: Wednesday, July 17, 2019 9:16 AM
> >
> > +1 to all arguments. Rust's barrier of entry is considerably higher than
> > that of Scala even. As much as I like the language's design, from a
> > community attraction point-of-view we should absolutely opt for go,
> > especially for things that are built around Kubernetes.
> >
> > That's of course to be taken with a grain of salt: If we have a use-case
> > that requires the performance characteristics of Rust (especially that
> of a
> > lacking garbage collector), we should absolutely choose for the
> > implementation. Implementing a controller however sounds like a
> > business-as-usual REST talk-to-a-database thingy and Go is perfect for
> that.
> >
> > Am Di., 16. Juli 2019 um 08:53 Uhr schrieb Martin Henke <
> martin.he...@web.de
> > >:
> >
> > > Michele,
> > >
> > > Two thoughts:
> > >
> > > 1) For writing a controller in Knative I  recommend to choose Go
> instead
> > > of Rust (even when I like Rust more).
> > > With Go you can leverage the fantastic Operator SDK from Redhat which
> > > makes writing controllers fairly
> > > simple (I had my first one up and running in under an hour).
> > > Link: https://github.com/operator-framework/operator-sdk
> > >
> > > The getting started guide is a good starting point:
> > > https://github.com/operator-framework/getting-started
> > >
> > > It also addresses the lifecycle of an controller with the Lifecycle
> > > Manager:
> > > https://github.com/operator-framework/operator-lifecycle-manager
> > > (I have not yet used this myself)
> > >
> > >
> > > 2) I think we should clearly separate the Knative work from Openwhisk
> and
> > > stay there with  a strict Scala only policy
> > > (with the existing  exceptions).
> > > Introducing more languages would in my opinion  lead to maintenance
> > > problems and the waste of  build up skills.
> > >
> > > Regards,
> > > Martin
> > >
> > >
> > > > On 15. Jul 2019, at 11:58, Michele Sciabarra 
> > > wrote:
> > > >
> > > > Hello all,
> > > >
> > > > In my efforts to work a Kanative Whisk, I reached the point where I
> have
> > > a kit with tekton-pipelines, knatve-serving building an actionlooop
> based
> > > runtime. Now I need to implement a controller, in order to use the
> existing
> > > wsk tooling.
> > > >
> > > > I know there is a prototype kwsk implementation made by redhat,
> written
> > > in Go but looks like it is obsolete and unfinished, and to the best of
> my
> > > knowledge, abandoned.
> > > >
> > > > I would like to resume the effort of writing an updated controller. I
> > > actually already have a prototype using the Julia language. Julia is
> really
> > > awesome, Python simplicity and Go speed, but I feed the community would
> > > disagree on using Julia.  Of course if I am wrong... let me know
> because
> > > that would be my preferred choice.
> > > >
> > > > However, I feel that,  given our Scala background, Rust would be a
> much
> > > better choice for the KnativeWhisk controller.  So I propose to use
> Rust
> > > for the KwhiskController.
> > > >
> > > > What does the community think of the proposal?
> > > >
> > > >
> > > > --
> > > >  Michele Sciabarra
> > > >  mich...@sciabarra.com
> > >
> > >
>
>
>
> --
> Matt Sicker 
>

Re: devtools and make-quickstart

[Dascalita Dragos]

"...we should pin all the dependent images etc and make a release
available"
+1

I also like the standalone JAR. Should we consider adding to that the API
Management features made available through OW GW, or for that we should
keep the docker-compose route ?

On Fri, Jul 12, 2019 at 8:53 AM James Thomas  wrote:

> +1 on the first point. I've seen other issues in the past with the devtools
> project based on image versioning stuff.
>
> Given devtools is more for experimental and first-steps than production
> usage - switching to the standalone controller seems like a good idea.
>
> On Fri, 12 Jul 2019 at 16:15, Rodric Rabbah  wrote:
>
> > this issue was opened against devtools and raises an important point:
> > https://github.com/apache/incubator-openwhisk-devtools/issues/273
> >
> > "On a separate note, will these builds eventually be versioned or will
> they
> > continue to be tagged in a backwards incompatible way? I am trying to
> > create a build process that is deterministic and currently I am unable to
> > achieve this."
> >
> > Given that we're pointing developer to make quick start still as the way
> to
> > startup and some of the dependence in this build on docker latest (now
> > nightly), I think we should pin all the dependent images etc and make a
> > release available.
> >
> > An existential question is whether we should use the standalone
> controller
> > which is faster to startup and adapt our docs accordingly.
> >
> > Thoughts?
> >
> > -r
> >
>
>
> --
> Regards,
> James Thomas
>

Re: api gateway with standalone controller

[Dascalita Dragos]

Rodric, are you running a custom api gateway container ?
The lines at [1] should fix the issue for you.

NGINX has its own internal DNS resolution for performance considerations.
I'm curios what the content of
"/etc/api-gateway/conf.d/includes/resolvers.conf"
 is inside the apigateway container.

Also, in case you use a custom configuration file for OW, it's possible
that the config doesn't include "resolvers.conf" and hence the DNS
resolution doesn't work ?

[1] -
https://github.com/apache/incubator-openwhisk-apigateway/blob/master/init.sh#L58-L59

On Wed, Jun 26, 2019 at 5:28 PM Rodric Rabbah  wrote:

> I tried to use the api gateway with the standalone controller and ran into
> a few issues which I'll document in the relevant docs soon. I'm now hitting
> an issue where the api gateway cannot resolve the host for routing a
> request.
>
> I opened
> https://github.com/apache/incubator-openwhisk-apigateway/issues/345
> to summarize the findings. I created an API successfully which when I curl
> fails in the gateway because it can't resolve the upstream host.
>
> I can nslookup/ping the stanalone controller from the api gateay container,
> and I can successfully the webaction URL from the api gateway container as
> well.
>
> How does the API gateway resolve a hostname and where is that code? Thanks
> for the help.
>
> -r
>

Re: [VOTE] Apache OpenWhisk graduation to Top Level Project

[Dascalita Dragos]

Big +1

Amazing milestone. Many thanks for the the people involved in moving the
project to this state.

On Wed, Jun 5, 2019 at 2:06 PM Priti Desai  wrote:

> +1 yup, definitely, its been a great pleasure working with the community!
>
> Cheers
> Priti
>
> On 2019/06/04 21:25:28, Rodric Rabbah  wrote:
> > Hi all,
> >
> > After a discussion among the Apache OpenWhisk community on the dev
> > mailing list [1], we have completed all Trademark transfers, and we
> > are now in the process of pruning the PMC roster, completing the
> > podling status page and completing the project maturity model [2].
> >
> > Apache OpenWhisk entered the incubator on November 23 2016. Since
> > then, we have grown to be in the top 25 list of Apache projects by
> > GitHub Stars at 4041, have 229 unique contributors across all our
> > project repos, more than 2500 commits, and most importantly, our
> > community has grown and is diversified beyond the initial founding
> > contributors and organization.
> >
> > The project has come a long way in embracing The Apache Way, in no
> > small part to our dedicated mentors and the community spirit that has
> > grown along this journey. We are operating well as an Apache project
> > and so we should take the next step.
> >
> > As such, I am calling a vote for Apache OpenWhisk to graduate to a top
> > level project. If we agree that we should graduate to a top level
> > project, the next step will be to draft a Resolution [3] for the PPMC
> > and IPMC to vote upon.
> >
> > Please take a minute to vote on whether or not Apache OpenWhisk should
> > graduate to a Top Level Project by responding with one of the
> > following:
> >
> >  [ ] +1 Apache OpenWhisk should graduate.
> >  [ ] +0 No opinion
> >  [ ] -1 Apache OpenWhisk should not graduate (please provide the reason)
> >
> > The VOTE is open for a minimum of 72 hours. Per Apache guidelines [4]
> > I will notify the incubator mailing list that a community vote is
> > under way.
> >
> > Thank you.
> > -r
> > (on behalf of the Apache OpenWhisk PPMC)
> >
> > [1]
> https://lists.apache.org/thread.html/8daa3a05148f54ca82458777e2b2b5e25ba99d39dcf8ce7dd85d0188@%3Cdev.openwhisk.apache.org%3E
> > [2]
> https://cwiki.apache.org/confluence/display/OPENWHISK/Project+Maturity+Model
> > [3]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=115526932
> > [4]
> https://incubator.apache.org/guides/graduation.html#community_graduation_vote
> >
>

Re: A plan to (re) implement OpenWhisk on top of Knative

[Dascalita Dragos]

Thanks for taking the time to do this POC Michele. I'm looking forward for
learn about the cold-start times with KNative.

On Fri, May 24, 2019 at 8:48 AM Michele Sciabarra 
wrote:

> I am taking all the suggestions and trying to setup a first implementation.
>
>
> --
>   Michele Sciabarra
>   mich...@sciabarra.com
>
> - Original message -
> From: James Thomas 
> To: dev@openwhisk.apache.org
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative
> Date: Friday, May 24, 2019 5:39 PM
>
> On Mon, 20 May 2019 at 15:50, Markus Thömmes 
> wrote:
> >
> > Can we try to define what the desired end-goal is here? I'm a bit unclear
> > what resembling the OpenWhisk API actually buys us.
> >
> > To me, the desired end-state would be to run OpenWhisk actions as-is on a
> > Knative cluster (similar to OpenFaaS' and Azure's integration). There's
> no
> > good way for us to provide the full API without spinning up a control
> plane
> > and we can only handle so much via the CLI. So to me, the end-goal looks
> > like:
> >
> > 1. *wsk action create* actually doing all the pieces necessary to run a
> > piece of code on Knative.
> > 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
> that
> > action. The action should be reachable via a sensible URL. If we really
> > want to keep the API surface (as I said, I'm dubious here) we can also do
> > that via ingress level abstractions (like VirtualService).
>
> I've been spending a bit more time reading up on knative this week and
> agree with Markus' suggestions. In the short term, being able to run
> an OpenWhisk action on Knative with no changes to the action code is a
> good first step. I think the work Priti has done with others to make
> the Node.js runtime Knative compatible is a sensible step towards
> this. (
> https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/119)
> If we can utilise that approach with the actionloop runtimes - that'd
> be great.
>
> This means people can move (simple) OpenWhisk actions from a platform
> instance to knative with minimal changes. Having a defined build
> pipeline to produce docker images from action code seems like the next
> step... The user has to do this manually but it simplifies the process
> of building those images manually.
>
> The longer-term discussion is whether we allow people to use the
> existing OpenWhisk API (and tools) as a proxy for actions deployed on
> knative. This would means people can switch their clients (CLI)
> between a normal openwhisk instance and a knative-based one.. This
> would be similiar to how projects like Riff operate.
>
> Gloo does have some interesting concepts around "function level"
> routing that are building out that might be useful
> (https://gloo.solo.io/user_guides/cloud_function/)
>
>
> --
> Regards,
> James Thomas
>

Re: [VOTE] Release Apache OpenWhisk API Gateway (v0.10.0-incubating, rc2)

[Dascalita Dragos]

[x] +1 Approve the release

Release verification checklist for reference:
  [x] Download links are valid.
  [x] Checksums and PGP signatures are valid.
  [x] DISCLAIMER is included.
  [x] Source code artifacts have correct names matching the current
release.
  [x] LICENSE and NOTICE files are correct for each OpenWhisk repository.
  [x] All files have license headers as specified by OpenWhisk project
policy [1].
  [x] No compiled archives bundled in source archive.

Thanks,
dragos

PS: rcverify.sh is awesome !

On Wed, May 29, 2019 at 3:22 PM Rodric Rabbah  wrote:

>  [x] +1 Approve the release
>
> Release verification checklist for reference:
>   [x ] Download links are valid.
>   [x ] Checksums and PGP signatures are valid.
>   [x ] DISCLAIMER is included.
>   [x ] Source code artifacts have correct names matching the current
> release.
>   [x ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
>   [x ] All files have license headers as specified by OpenWhisk project
> policy [1].
>   [x ] No compiled archives bundled in source archive.
>
>
> ./tools/rcverify.sh openwhisk-apigateway 'OpenWhisk API Gateway'
> 0.10.0-incubating rc2
> rcverify.sh (script SHA1: 6871 208F 37F8 2352 BA60  E66E 8953 94E8 2832
> AA69)
> working in the following directory:
> /var/folders/q9/s3th42s53d34ftd5wvcypybrgn/T/tmp.vrfdBO8F
> fetching tarball and signatures from
>
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.10.0-incubating-rc2
> fetching openwhisk-apigateway-0.10.0-incubating-sources.tar.gz
> fetching openwhisk-apigateway-0.10.0-incubating-sources.tar.gz.asc
> fetching openwhisk-apigateway-0.10.0-incubating-sources.tar.gz.sha512
> fetching release keys
> importing keys
> gpg: key 72AF0CC22C4CF320: "Vincent Hou (Release manager of OpenWhisk) <
> houshen...@apache.org>" not changed
> gpg: key 22907064147F886E: "Dave Grove " not changed
> gpg: key 44667BC927C86D51: "Rodric Rabbah " not changed
> gpg: Total number processed: 3
> gpg:  unchanged: 3
> unpacking tar ball
> cloning scancode
> Cloning into 'incubator-openwhisk-utilities'...
> remote: Enumerating objects: 57, done.
> remote: Counting objects: 100% (57/57), done.
> remote: Compressing objects: 100% (42/42), done.
> remote: Total 57 (delta 19), reused 37 (delta 12), pack-reused 0
> Unpacking objects: 100% (57/57), done.
> computing sha512 for openwhisk-apigateway-0.10.0-incubating-sources.tar.gz
> SHA512: openwhisk-apigateway-0.10.0-incubating-sources.tar.gz:
> F3A852F8 95982ED3 5735336F D1A85FF6 AE4575AA BAF251A7 0B77AA40 EE4D0727
> 09C40A99
>  6F1BD0ED CDA91B9B 59C9C3AF 19E7AC52 BF05A869 61C113D9 D42BB691
> validating sha512... passed
> verifying asc... passed (signed-by: Dave Grove )
> verifying disclaimer... passed
> verifing notice... passed
> verifying license... passed
> verifying sources have proper headers... passed
> scanning for executable files... passed
> scanning for non-text files... passed
> scanning for archives... passed
> scanning for packages... passed
>
> run the following command to remove the scratch space:
>
> On Wed, May 29, 2019 at 5:50 PM David P Grove  wrote:
>
> >
> >
> > Hi,
> >
> > As discussed in [2], there was a regression in rc1 of apigateway.
> > so
> > we needed to generate a new release candidate once the fix had been
> merged.
> > That has been done. Therefore...
> >
> > This is a call to vote on releasing version 0.10.0-incubating release
> > candidate rc2 of the following project module with artifacts built from
> the
> > Git repositories and commit IDs listed below.
> >
> > * OpenWhisk API Gateway: a737552c
> >
> >
> https://github.com/apache/incubator-openwhisk-apigateway/commits/a737552c
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.10.0-incubating-rc2/openwhisk-apigateway-0.10.0-incubating-sources.tar.gz
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.10.0-incubating-rc2/openwhisk-apigateway-0.10.0-incubating-sources.tar.gz.asc
> >
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.10.0-incubating-rc2/openwhisk-apigateway-0.10.0-incubating-sources.tar.gz.sha512
> >
> > This release is comprised of source code distribution only.
> >
> > You can use this UNIX script to download the release and verify the
> > checklist below:
> >
> >
> https://gitbox.apache.org/repos/asf?p=incubator-openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=HEAD
> >
> > Usage:
> > curl -s "
> >
> >
> https://gitbox.apache.org/repos/asf?p=incubator-openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=HEAD
> > " -o rcverify.sh
> > chmod +x rcverify.sh
> > rcverify.sh openwhisk-apigateway 'OpenWhisk API Gateway'
> 0.10.0-incubating
> > rc2
> >
> > Please vote to approve this release:
> >
> >   [ ] +1 Approve the release
> >   [ ]  0 Don't care
> >   [ ] -1 Don't release, because ...
> >
> > Release verification checklist for reference:
> >  

Re: Dropping supports for Docker Machine.

[Dascalita Dragos]

+1 as well.

On Mon, Apr 22, 2019 at 8:49 AM Rodric Rabbah  wrote:

> +1 to remove old docker machine related docs.
>
> -r
>
> On Mon, Apr 22, 2019 at 8:08 AM Dominic Kim  wrote:
>
> > Dear whiskers.
> >
> > I am working on upgrading Docker version in
> > https://github.com/apache/incubator-openwhisk/pull/4430.
> > And one thing comes up in my mind.
> >
> > These days, it seems Docker implicitly recommends to use Docker for mac(
> > https://docs.docker.com/docker-for-mac/install/) as they name it as
> > `Docker
> > Desktop for Mac` to express it is a standard way to use Docker in Mac.
> > And even they provide the guide to migrate from Docker-machine(Docker
> > Toolbox): https://docs.docker.com/docker-for-mac/docker-toolbox/ to
> Docker
> > For Mac.
> > Considering the relative easiness of setting up Docker for mac, it's not
> > surprising. (Though there are still fundamental limitations in Docker For
> > Mac.)
> >
> > With DockerForMacContainerFactory, it became easier to utilize Docker for
> > mac in mac hosts.
> > So I proposed to drop supports for Docker machine.
> >
> > Please share your opinion if you disagree with this.
> > If there's no objection, I will drop supports for Docker machine in
> > https://github.com/apache/incubator-openwhisk/pull/4430
> >
> > Best regards
> > Dominic
> >
>

Re: New architecture proposal

[Dascalita Dragos]

"...I think GC implementation is orthogonal to my proposal.
So if you can deal with such problems, I would gladly apply it as well..."
I think it's definitely worth trying. The "mark-and-sweep" pattern with an
eventual consistent state may give GC a good premise to make decisions.
If GC is instantiated as a singleton action, through Akka, it can achieve a
highly fault tolerant service, allowing for self-healing with no single
point of failure.

If an action changes state every "2ms" as in the example above, and the GC
gets that change after "100ms", this could work fine; a timeout of "10mins"
may give GC a decent time window to make decisions. The "mark-and-sweep"
then allows containers ( ContainerProxy ) to move that container in the
"survival" space - to use GC G1 language - in case the container processed
an activation in between "mark" and "sweep" commands. If a container
"survived", then GC will prolong its TTL with another 10mins, and so
forth.  At the same time GC doesn't need to know about *all* state changes
(busy->warm->etc), especially if they happen so frequently. This generates
unnecessary noise in the network. GC could use a sampling strategy instead,
or check for "last activation" *only* when the allocated memory in the
cluster is over a threshold; there are a number of strategies that could
ensure GC runs with "minimum enough info" and "as infrequent as needed" to
make the right decisions.

"...If we "must" guarantee the execution of users, we need to have a bigger
cluster than the sum of users' limit

I agree. For the cases when an operator can't run such a large cluster to
accommodate all namespaces regardless of their traffic, that's when a smart
GC becomes critical to achieve cost efficiency. At least in the deployments
I'm involved in, the cluster aims to shut-down most of its VMs when there's
no traffic, and scale back up as quickly as possible as traffic increases.
We run clusters in public clouds; we can't afford to keep thousands of VMs
running if nobody uses them. When cluster scales up, there's a time window
when there could be a congestion of resources; so, as with network
congestion, activations should suffer for a little while, but at least they
all get a fair chance to execute. They execute slower, but at least they
execute; but if containers are left to destroy themselves, then it's hard
to predict cluster behavior when congested, and it's hard to guarantee an
SLA. This is where I was coming from.

I'm sure we'll find a way to accommodate multiple deployment patterns. I'm
describing my setup with the hope that the new architecture will allow a
configuration mechanism or SPIs for such key areas.

On Thu, Apr 11, 2019 at 8:15 PM Dominic Kim  wrote:

> Well, that is not the tradeoff only resides in my proposal.
> If we "must" guarantee the execution of users, we need to have a bigger
> cluster than the sum of users' limit.
> Even though we use current implementation, if the sum of concurrent limit
> exceeds the system resources, we cannot completely guarantee all executions
> under a burst.
> (We cannot guarantee 200 concurrent execution with 100 containers.)
>
> The reason why I used a kind of self-GC is, it's not easy to track the
> resource status in real time.
> It is the same with the reason why I take the pull-based model.
>
> For example, if we control the container deletion in a central way, we
> should track all container status such as how many containers are running
> for each action, which of them are idle, where they are running, and so on.
> But the status of resources changes blazingly fast.
> Below is one example. The execution is over within 2 ms.
> {
> "namespace": "style95",
> "name": "hello-world",
> "version": "0.0.1",
> "subject": "style95",
> "activationId": "ba2cc561fc8e4272acc561fc8ea27210",
> "start": 1554967214351,
> "end": 1554967214353,
> "duration": 2,
> "response": {
> "status": "success",
> "statusCode": 0,
> .
> .
> .
> }
>
> It means the container status(busy, warm) can change every 2 ms.
>
> Also, if you are not thinking of one central component which decides to
> delete containers,(it can be a SPOF) there will be multiple components
> which are making the decision.
> When one of them makes a decision, it should consider decisions made(and
> will be made) by the others at the same time.
> So we need to track down all container status, consider all 

Re: New architecture proposal

[Dascalita Dragos]

Thanks Dominic for the details.

It seems like an operator has to choose between “do I hurt performance(low
timeout) or do I hurt the SLA” ?

If this is the trade off , isn’t this a hard choice to make ? So I’m
wondering whether some alternative designs could be used for this problem.

The key decision here is: should OW be given a cluster wide power to view
and control the resources or not. IIUC the current proposal doesn’t support
this? I’m not saying the proposed model is not good; I’d just feel more
comfortable if OW would allow more options instead of one, in the same way
the JVM allows multiple GC implementations. In the proposed model the GC
would offload the decision to each container, while other implementations
may do it differently. For instance,  I’d implement something dynamic that
adapts the timeout to the load, and maybe try some predictive ML algorithms
to manage resources - if a model suggests that out of 3 actions that could
be removed, 1 has a higher probability to be invoked again, wouldn’t it be
more efficient to remove one of the other 2 ? Such a logic can only be
achieved through an entity with a cluster wide view, as actions don’t know
about each other, to negotiate a dynamic timeout.

- dragos

On Wed, Apr 10, 2019 at 3:46 AM Dominic Kim  wrote:

> Dear Dascalita
>
> That depends on the timeout configuration.
> For example, if you need something similar to the one in the current code
> base, you can just configure the timeout to a small enough value, such as
> 50ms.
>
> The idea behind the longer timeout is, it shows better performance when
> there are highly likely subsequent requests.
> For example, it takes about 100ms ~ 1s to create a new coldstart container.
> If the action execution takes 10ms, it should wait 10 to 100 times more for
> a new container.
> In this case, it is reasonable to wait for the previous execution and reuse
> the existing container rather than creating a new container.
> So 100ms ~ 1s could be good options for the timeout value.
> (Under heavy loads, I even observed it took 2s ~ 5s to create a coldstart
> container.)
> And this implies some changes in the notion of resources.
>
> In the cluster, there would be a different kind of requests.
> There would be both batch and real-time invocation.
> So I think this is a tradeoff.
> Longer timeout will increase the reuse rate of containers but idle
> containers will possess resources longer.
>
> And even in the current implementation, subsequent invocation should wait
> for some time to remove existing(warmed containers) and create a new cold
> start container.
> As I said, it could take up to few seconds under heavy loads.
> With reasonable timeout value, there would be no big performance difference
> in the above situation.
> (Actually, I expect new scheduler would outperform even with 5~10s timeout
> value as it will evenly distribute docker operation.
> In the current implementation, all execution is sent to the home invoker
> first and it could make the situation worse in edge cases.
> I hope I can share performance comparison results as I am doing
> benchmarking.)
>
> Also, I think the above case is an edge case that someone is consuming most
> of the cluster resources and executing two different batch invocation
> alternatively.
> Anyway, we can support such an edge case with some shutdown period.
> This can be controversial, but I believe this is a viable option.
>
>
> If you said that in the view of OpenWhisk operator, I think you meant there
> are more than 1 heavy users.
> Let's say, one user has 60 containers limit and the other has 80 containers
> limit.
> Then can we guarantee both users' execution without any issue in current
> implementation?
> If their invocation requests come together, one or both of their invocation
> will be heavily delayed.
>
> So I think when we(operators) notice there are such heavy users, we should
> scale out our clusters to guarantee their invocation or we should reduce
> their resource limit.
> This is also a tradeoff. If we must guarantee their invocation, we at least
> need a bigger cluster than the sum of their throttling limit.
> If we have weak SLA, we can support both users with smaller cluster though
> their invocation can be delayed a bit.
>
>
> In short, if you prefer the current behavior you can still have a similar
> effect by configuring the timeout as 50ms.
> (So containers will only wait for 50ms, though it may induce some
> performance degradation in other cases.)
>
> Best regards
> Dominic
>
>
> 2019년 4월 10일 (수) 오전 1:36, Dascalita Dragos 님이 작성:
>
> > "...When there is no more activation message, ContainerProxy will be wait
> > for the given time(configurable) and just stop"
> >
> > H

Re: New architecture proposal

[Dascalita Dragos]

er the
> new
> > state of the scheduler it managers and change its policy to distribute
> > queue creation requests?
> >
> > Thanks
> > Mingyu Zhou
> >
> > On Fri, Apr 5, 2019 at 2:56 PM Dominic Kim  wrote:
> >
> > > Dear David, Matt, and Dascalita.
> > > Thank you for your interest in my proposal.
> > >
> > > Let me answer your questions one by one.
> > >
> > > @David
> > > Yes, I will(and actually already did) implement all components based on
> > > SPI.
> > > The reason why I said "breaking changes" is, my proposal will affect
> most
> > > of components drastically.
> > > For example, InvokerReactive will become a SPI and current
> > InvokerReactive
> > > will become one of its concrete implementation.
> > > My load balancer and throttler are also based on the current SPI.
> > > So though my implementation would be included in OpenWhisk, downstreams
> > > still can take advantage of existing implementation such as
> > > ShardingPoolBalancer.
> > >
> > > Regarding Leader/Follower, a fair point.
> > > The reason why I introduced such a model is to prepare for the future
> > > enhancement.
> > > Actually, I reached a conclusion that memory based activation passing
> > would
> > > be enough for OpenWhisk in terms of message persistence.
> > > But it is just my own opinion and community may want more rigid level
> of
> > > persistence.
> > > I naively thought we can add replication and HA logic in the queue
> which
> > > are similar to the one in Kafka.
> > > And Leader/Follower would be a good base building block for this.
> > >
> > > Currently only a leader fetch activation messages from Kafka. Followers
> > > will be idle while watching the leadership change.
> > > Once the leadership is changed, one of followers will become a new
> leader
> > > and at that time, Kafka consumer for the new leader will be created.
> > > This is to minimize the failure handling time in the aspect of clients
> as
> > > you mentioned. It is also correct that this flow does not prevent
> > > activation messages lost on scheduler failure.
> > > But it's not that complex as activation messages are not replicated to
> > > followers and the number of followers are configurable.
> > > If we want, we can configure the number of required queue to 1, there
> > will
> > > be only one leader queue.
> > > (If we ok with the current level of persistence, I think we may not
> need
> > > more than 1 queue as you said.)
> > >
> > > Regarding pulling activation messages, each action will have its own
> > Kafka
> > > topic.
> > > It is same with what I proposed in my previous proposals.
> > > When an action is created, a Kafka topic for the action will be
> created.
> > > So each leader queue(consumer) will fetch activation messages from its
> > own
> > > Kafka topic and there would be no intervention among actions.
> > >
> > > When I and my colleagues open PRs for each component, we will add
> detail
> > > component design.
> > > It would help you guys understand the proposal more.
> > >
> > > @Matt
> > > Thank you for the suggestion.
> > > If I change the name of it now, it would break the link in this thread.
> > > I would use the name you suggested when I open a PR.
> > >
> > >
> > > @Dascalita
> > >
> > > Interesting idea.
> > > Any GC patterns do you keep in your mind to apply in OpenWhisk?
> > >
> > > In my proposal, the container GC is similar to what OpenWhisk does
> these
> > > days.
> > > Each container will autonomously fetch activations from the queue.
> > > Whenever they finish invocation of one activation, they will fetch the
> > next
> > > request and invoke it.
> > > In this sense, we can maximize the container reuse.
> > >
> > > When there is no more activation message, ContainerProxy will be wait
> for
> > > the given time(configurable) and just stop.
> > > One difference is containers are no more paused, they are just removed.
> > > Instead of pausing them, containers are waiting for subsequent requests
> > for
> > > longer time(5~10s) than current implementation.
> > > This is because pausing is also relatively expensive operation in
> > > comparison to short-running invocation.
> > >
> 

Re: New architecture proposal

[Dascalita Dragos]

Hi Dominic,
Thanks for sharing your ideas. IIUC (and pls keep me honest), the goal of
the new design is to improve activation performance. I personally love
this; performance is a critical non-functional feature of any FaaS system.

There’s something I’d like to call out: the management of containers in a
FaaS system could be compared to a JVM. A JVM allocates objects in memory,
and GC them. A FaaS system allocates containers to run actions, and it GCs
them when they become idle. If we could look at OW's scheduling from this
perspective, we could reuse the proven patterns in the JVM vs inventing
something new. I’d be interested on any GC implications in the new design,
meaning how idle actions get removed, and how is that orchestrated.

Thanks,
dragos


On Thu, Apr 4, 2019 at 8:40 AM Matt Sicker  wrote:

> Would it make sense to define an OpenWhisk Improvement/Enhancement
> Propoposal or similar that various other Apache projects do? We could
> call them WHIPs or something. :)
>
> On Thu, 4 Apr 2019 at 09:09, David P Grove  wrote:
> >
> >
> > Dominic Kim  wrote on 04/04/2019 04:37:19 AM:
> > >
> > > I have proposed a new architecture.
> > > https://cwiki.apache.org/confluence/display/OPENWHISK/New+architecture
> > +proposal
> > >
> > > It includes many controversial agendas and breaking changes.
> > > So I would like to form a general consensus on them.
> > >
> >
> > Hi Dominic,
> >
> > There's much to like about the proposal.  Thank you for writing
> it
> > up.
> >
> > One meta-comment is that the work will have to be done in a way
> so
> > there are no actual "breaking changes".  It has to be possible to
> continue
> > to configure the system using the existing architectures while this work
> > proceeds.  I would expect this could be done via a new LoadBalancer and
> > some deployment options (similar to how Lean OpenWhisk was done).  If
> work
> > needs to be done to generalize the LoadBalancer SPI, that could be done
> > early in the process.
> >
> > On the proposal itself, I wonder if the complexity of
> Leader/Follower
> > is actually needed?  If a Scheduler crashes, it could be restarted and
> then
> > resume handling its assigned load.  I think there should be enough
> > information in etcd for it to recover its current set of assigned
> > ContainerProxys and carry on.   Activations in its in memory queues would
> > be lost (bigger blast radius than the current architecture), but I don't
> > see that the Leader/Follower changes that (seems way too expensive to be
> > replicating every activation in the Follower Queues).   The
> Leader/Follower
> > would allow for shorter downtime for those actions assigned to the downed
> > Scheduler, but at the cost of significant complexity.  Is it worth it?
> >
> > Perhaps related to the Leader/Follower, its not clear to me how
> > activation messages are being pulled from the action topic in Kafka
> during
> > the Queue creation window. I think they have to go somewhere (because the
> > is a mix of actions on a single Kafka topic and we can't stall other
> > actions while waiting for a Queue to be created for a new action), but if
> > you don't know yet which Scheduler is going to win the race to be a
> Leader
> > how do you know where to put them?
> >
> > --dave
>
>
>
> --
> Matt Sicker 
>

Re: [Bi-Weekly Tech Interchange] Call for agenda topics

[Dascalita Dragos]

Thanks.

So far we have:
- Vincent - Jenkins CI for OpenWhisk
- Priti - Knative build
- James - Web Action Http Proxy

If you have more topics, it's not too late to send them.

Thanks,
dragos

On Tue, Mar 19, 2019 at 11:06 AM James Thomas  wrote:

> I'd like to show my new web action http proxy - it'll only take 5 - 10
> mins.
>
> On Tue, 19 Mar 2019 at 17:26, Priti Desai  wrote:
>
> > Hi Dragos,
> >
> > Please reserve 10 to 15 minutes for Knative buiid.
> >
> > Cheers
> > Priti
> >
> >
> >
> > From:   Dascalita Dragos 
> > To: dev@openwhisk.apache.org
> > Date:   03/19/2019 09:08 AM
> > Subject:[Bi-Weekly Tech Interchange] Call for agenda topics
> >
> >
> >
> > Hi,
> > I'm starting this thread to start capturing topics to discuss for our
> call
> > tomorrow.
> > Please reply to this email to reserve your spot !
> >
> > Bi-weekly Tech Interchange call details:
> > - Zoom:
> >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__zoom.us_my_asfopenwhisk&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6V3FXFwHvE0SFE3N4Rk7-rlMhS2xkaqR1AlgZ0xtvKY&m=27x2jBqNcasvt0sohR9IvAJ_JZoi4YvKvgdtpOjxtRE&s=AXU7OQ9kktdXLKRBmyqhY63F-s7B-U3h2nElV5sk_WI&e=
> >
> > - Wednesday March 20th
> > 11AM EST(Eastern US)
> > 5PM CET (Central Europe),
> > 4PM UTC, 12AM CST (Beijing)
> >
> > Thanks,
> > dragos
> >
> >
> >
> >
> >
>
> --
> Regards,
> James Thomas
>

[Bi-Weekly Tech Interchange] Call for agenda topics

[Dascalita Dragos]

Hi,
I'm starting this thread to start capturing topics to discuss for our call
tomorrow.
Please reply to this email to reserve your spot !

Bi-weekly Tech Interchange call details:
- Zoom: https://zoom.us/my/asfopenwhisk
- Wednesday March 20th
11AM EST(Eastern US)
5PM CET (Central Europe),
4PM UTC, 12AM CST (Beijing)

Thanks,
dragos

Re: [DISCUSS] Release event providers

[Dascalita Dragos]

+1 as well for 2.0.0

On Mon, Mar 18, 2019 at 10:58 AM James Thomas  wrote:

> +1 on going with 2.0.0 and moving forward with the release.
>
> On Sun, 17 Mar 2019 at 12:57, Carlos Santana  wrote:
>
> > +1
> >
> > - Carlos Santana
> > @csantanapr
> >
> > > On Mar 16, 2019, at 3:54 PM, David P Grove  wrote:
> > >
> > >
> > >
> > > As part of the unified release, we will be doing the first Apache
> release
> > > of the OpenWhisk event providers (openwhisk-package-alarms,
> > > openwhisk-package-cloudant, openwhisk-package-kafka).
> > >
> > > I've taken a look at the pending PRs, and I think only the ones I just
> > > submitted to add DISCLAIMER.txt, etc. are release blocking.  Please
> > comment
> > > if you disagree (or want to get in any other changes before a release).
> > >
> > > These components all have had non-Apache releases in the past, and thus
> > > have a history of version numbering that we need to not confuse.
> > >
> > > If we want to stick with the minimal increment under semvar, the
> initial
> > > Apache releases would be numbered:
> > >alarms: 1.12.5
> > >cloudant: 1.9.4
> > >kafka: 1.4.22
> > >
> > > We could also decide to do a "major" bump to allow a clean Apache vs.
> > > non-Apache break in the version numbers.  In other words, we would make
> > > this the 2.0.0 release of all three event provider modules.
> > >
> > > My strong inclination is to go with 2.0.0, but I would like to hear
> > > thoughts from others.   Assuming there are no technical blockers, I
> would
> > > like to initiate a formal release process for these three components
> > early
> > > next week.
> > >
> > > thanks,
> > >
> > > --dave
> >
>
>
> --
> Regards,
> James Thomas
>

Re: [DISCUSS] graduation from the incubator

[Dascalita Dragos]

Huge +1 from my side as well.
I couldn't agree more. I think the project not only has momentum, but it's
also used in production environments and it's well tested and stable.

In addition, I believe multiple parties have long term visions with
enhancements, which IMO I see it as a positive indicator that this project
will continue to keep the momentum going.



On Mon, Mar 18, 2019 at 11:03 AM James Thomas  wrote:

> +100 on this.
>
> I think the project community has reached a level of maturity that would
> enable us to graduate according to the incubator guidelines. The level of
> community contributions on the mailing list and the slack channel are
> indicative of the succes of the project. We have a broad number of
> committers from different backgrounds and interests. From a technical
> perspective, I think the platform is also relatively stable and has
> multiple production users that have been running over multiple years.
>
> On Fri, 15 Mar 2019 at 22:06, David P Grove  wrote:
>
> >
> >
> > I'd like to kick off a discussion to assess the project's readiness for
> > graduation from the incubator.
> >
> > Per Rodric's recent stats [1], the community has developed nicely in
> terms
> > of code contribution.
> >
> > We've released a number of software components following the Apache
> release
> > process.  We are in the midst of making our first "uber-release" across
> all
> > of our sub-components (expect at least 2 voting threads next week).
> >
> > Overall I think the community is active.  Communication on the project
> > slack is frequent (avg of >160 messages a day) and is now digested daily
> to
> > the dev list. (See [2] for stats).
> >
> > There are a couple procedural tasks we still need to complete, foremost
> > being the formal transfer of the OpenWhisk trademarks from IBM to the
> ASF.
> > But I think we can assume that these tasks will be completed and start
> > considering graduation in parallel.
> >
> > Please share your thoughts,
> >
> > --dave
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/b2217c61caad5c7a0369699d06d44e5cf688d3cba982e354a45b8c78@%3Cdev.openwhisk.apache.org%3E
> > [2]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103091999
> >
>
>
> --
> Regards,
> James Thomas
>

Re: PR to enable actions on YARN

[Dascalita Dragos]

Hi Samuel,
This is an interesting contribution. Do you happen to have any performance
numbers with YARN ? I'd be particularly interested in the cold start
latencies.

Thanks,
dragos

On Fri, Feb 22, 2019 at 5:21 PM Samuel Hjelmfelt
 wrote:

>
> Hi Rodric and Carlos,
>
>
> ApacheHadoop has three major components: HDFS (distributed filesystem),
> MapReduce(distributed batch processing engine), YARN (Yet Another Resource
> Negotiator) (containerengine). While MapReduce has been largely replaced by
> Apache Tez, Apache Spark,and Apache Flink, HDFS and YARN are still widely
> used for data analytics use cases.
>
>
>
> YARN is unique as a container engine because, unlike Mesos and Kubernetes,
> it was designed for ephemeral, short-livedcontainers rather than for long
> running micro-services. The jobs and queries that run on YARN are split
> intosmall tasks that run to completion and generally only last for seconds
> or maybe minutes. Overthe last couple years, YARN has been expanding its
> support for long running usecases, but is still focused on data-driven use
> cases over more generic micro-serviceuse cases (like web apps). The primary
> long running technologies on YARN are currently Spark Streamingand
> TensorFlow. Here is an articlefrom LinkedIn about why they created a
> project for TensorFlow on YARN. Asimilar case could be made for OpenWhisk:
> https://engineering.linkedin.com/blog/2018/09/open-sourcing-tony--native-support-of-tensorflow-on-hadoop.
>
>
>
>
> Bringing OpenWhisk onto YARN makes FaaS more accessible to thethousands of
> organizations with existing Hadoop clusters. Between Cloudera’s 2,000+
> customers; Azure, AWS,and GCP cloud customers; and the organizations
> self-supporting like Netflix, theinstall base of YARN is very high and
> still growing.
>
>
>
> ThisPR is a first level of integration, but YARN’s focus on ephemeral
> containerscould be more fully leveraged by OpenWhisk to improve scalability
> andperformance. Here is an interesting article on the scalability of YARN
> fromMicrosoft:
> https://azure.microsoft.com/en-us/blog/how-microsoft-drives-exabyte-analytics-on-the-world-s-largest-yarn-cluster/
>
> Thanks,
> Sam Hjelmfelt
>

Re: Agenda Items for Tech Interchange this Wed (December 5th)

[Dascalita Dragos]

If time allows I can show PR
https://github.com/apache/incubator-openwhisk/pull/4142 , enabling
developers to run both controller and invoker from IntelliJ locally, for
both docker-compose and ansible deployments (5 mins)

On Tue, Dec 4, 2018 at 2:59 PM Priti Desai  wrote:

> Thanks Moritz, Erez, and Vincent, you are all added to agenda for
> tomorrow.
>
> Here is the list of topics we have in order:
>
> (1) Priti - OpenWhisk Catalog Installation using Whisk Deploy - 5 minutes
> (2) Moritz - API Gateway - 10 minutes
> (3) Erez -  Shared context across invocations in OpenWhisk - 10 minutes
> (4) Vincent - Jenkins pipeline for OpenWhisk - 10 minutes
> (5) May be Someone - Project Updates - 5 minutes
>
> Please use https://zoom.us/j/4297821713 to join the meeting. Please note
> that we are using different zoom account just for tomorrow (December 5th).
> Also, this zoom account is time bound so we are having call for 40 minutes
> instead of an hour.
>
> Looking forward to seeing you all tomorrow.
>
> Cheers
> Priti
>
>
>
>
> From:   "Vincent S Hou" 
> To: dev@openwhisk.apache.org
> Date:   12/04/2018 10:13 AM
> Subject:Re: Agenda Items for Tech Interchange this Wed (December
> 5th)
>
>
>
> Hi Priti,
>
> I would like to update the plan to implement Jenkins pipeline for
> OpenWhisk.
>
> Best wishes.
> Vincent Hou (侯胜博)
>
> Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM
> Cloud
>
> Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
> Phone: +1(919)254-7182
> Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United
> States
>
> -Moritz Raho  wrote: -
> To: "dev@openwhisk.apache.org" 
> From: Moritz Raho 
> Date: 12/04/2018 12:26PM
> Subject: Re: Agenda Items for Tech Interchange this Wed (December 5th)
>
> Hi Priti,
>
> If time permits, I would like to share a demo about a work in progress for
> the api-gateway. This might take around 10 minutes.
> It’s a feature that allows the api-gateway to serve content in the place
> of web actions based on some custom return headers.
> This is useful to web action developers that need to return a payload
> bigger than the response size limit.
>
> Thanks!
> Moritz
>
> On 03.12.18, 21:13, "Priti Desai"  wrote:
>
> Hello Whiskers,
>
> It's time to collect topics for bi-weekly Tech Interchange this week
> (December 5th). Please send your agenda items here or message me
> (@pritidesai) on Slack. Also, please remind other whiskers to join the
>
> call.
>
> I have one on "OpenWhisk Catalog Installation using Whisk Deploy" for
> 5
> minutes.
>
> I am confirming with Matt on what are we using this week, Zoom/Webex.
> Will
> send out update soon.
>
> Cheers
> Priti
>
> P.S.
>
> CWIKI:
>
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FOPENWHISK%2FOpenWhisk%2BProject%2BWiki&data=02%7C01%7Craho%40adobe.com%7C5fc888a9d5da4413536508d6595bc319%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636794647987240122&sdata=UiZ5GhQUHpbIR8wozfG43C%2FepaZLeIfbOqnbU%2BgQZSg%3D&reserved=0
>
>
>
> To view/add the recurring calendar entry click this link:
>
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalendar.google.com%2Fevent%3Faction%3DTEMPLATE%26tmeid%3DM3RmZG04YW12cXVib2xwYzFycmEzYmFicGNfMjAxODA1MjNUMTUwMDAwWiBhcGFjaGVvcGVud2hpc2tAbQ%26tmsrc%3Dapacheopenwhisk%2540gmail.com%26scp%3DALL&data=02%7C01%7Craho%40adobe.com%7C5fc888a9d5da4413536508d6595bc319%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636794647987250131&sdata=ImfytI6av6aNJ4cQPakm5sEpm5%2B0sTU6sJFzZ3%2BuU4Y%3D&reserved=0
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating-rc1: OpenWhisk Composer

[Dascalita Dragos]

+1 on the release.
Just verified the checklist as well.

On Mon, Nov 26, 2018 at 9:01 AM Tyson Norris 
wrote:

> +1 to release Apache OpenWhisk 0.9.0-incubating: OpenWhisk composer
>
> Verified checklist
>
> Thanks
> Tyson
>
> On 11/20/18, 11:30 AM, "David P Grove"  wrote:
>
>
>
> This is call for a vote for the release of Apache OpenWhisk
> 0.9.0-incubating: OpenWhisk composer
>
> List of JIRA ticket(s) resolved for this release can be found at
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FINCUBATOR-227&data=02%7C01%7Ctnorris%40adobe.com%7Cba599811e5d045bea45408d64f1ea215%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636783390319129809&sdata=c5Vvh8FCPIJEXP8VbVAwfpp7mCWwyvDb28s6h2X55GA%3D&reserved=0
> .
>
> To learn more about Apache OpenWhisk, please visit
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopenwhisk.apache.org%2F&data=02%7C01%7Ctnorris%40adobe.com%7Cba599811e5d045bea45408d64f1ea215%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636783390319129809&sdata=Hf90uR%2BNrbuExRoPCJNwzTopx1So7bpIA4hyC46KgZM%3D&reserved=0
>
> This release comprises of source code distribution only. There is only
> one
> module within this release number 0.9.0. The artifact were built from
> the
> following Git commit IDs:
> * openwhisk-composer: 7ae7f08,
>
> The source code artifact of openwhisk composer can be found at:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fapache-openwhisk-0.9.0-incubating-rc1%2Fopenwhisk-composer-0.9.0-incubating-sources.tar.gz&data=02%7C01%7Ctnorris%40adobe.com%7Cba599811e5d045bea45408d64f1ea215%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636783390319129809&sdata=ZtPxn78DZc5VkDVj%2B5%2FObEH%2FU08%2BbL4DRh7x%2FvFWFzk%3D&reserved=0
>
> The SHA-512 checksum for the artifact of openwhisk composer is
> openwhisk-composer-0.9.0-incubating-sources.tar.gz:
> 7D5ECB65 DF653840 C8A33C7B DDF346AD 2AA36507 DD1D6DE6 8CB99238 BDC93EF6
> 425B5BAF
>  1371C4BE 6F1E0DEF 01D60D18 03AADC33 B0B7BD95 40724450 5FF6D131
> which can can be found via:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fapache-openwhisk-0.9.0-incubating-rc1%2Fopenwhisk-composer-0.9.0-incubating-sources.tar.gz.sha512&data=02%7C01%7Ctnorris%40adobe.com%7Cba599811e5d045bea45408d64f1ea215%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636783390319129809&sdata=NNtivScNG4YM7izTvJzoX%2FrZAMHyR0vuScc8usOTa%2Fc%3D&reserved=0
>
> The signature of the artifact of openwhisk composer can be found via:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fapache-openwhisk-0.9.0-incubating-rc1%2Fopenwhisk-composer-0.9.0-incubating-sources.tar.gz.asc&data=02%7C01%7Ctnorris%40adobe.com%7Cba599811e5d045bea45408d64f1ea215%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636783390319129809&sdata=bDNdi6idFpGjgcDIv%2BGjmTgZ0EyyCz2HE4HVCCR7qMo%3D&reserved=0
>
> KEYS file is available here:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2FKEYS&data=02%7C01%7Ctnorris%40adobe.com%7Cba599811e5d045bea45408d64f1ea215%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636783390319129809&sdata=c%2BGNCmz7HSH4v2rJp901wUbdjokQw8K6EsUl%2FBGeOvs%3D&reserved=0
>
>
> Please vote on releasing this package as Apache OpenWhisk
> 0.9.0-incubating:
> OpenWhisk Composer
>
> The vote will be open for at least 72 hours.
> [ ] +1 Release as Apache OpenWhisk 0.9.0-incubating: openwhisk composer
> [ ] +0 no opinion
> [ ] -1 Do not release and the reason
>
> Checklist for reference:
> [ ] Download links are valid.
> [ ] Checksums and PGP signatures are valid.
> [ ] DISCLAIMER is included.
> [ ] Source code artifacts have correct names matching the current
> release.
> [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
> [ ] All files have license headers if necessary.
> [ ] No compiled archives bundled in source archive.
>
> regards,
>
> --dave
>
>
>

Re: [graduation] Maturity Model assessment

[Dascalita Dragos]

Hi Bertrand,
Thanks for guiding us.

It looks like Matt has already started a page to describe the maturity
model. [1] Thanks Matt !

I'll make time to review the items in there, and contribute where there's
missing statuses.

In addition, should we also add this to the agenda for the Tech Interchange
to check the progress ?


[1] -
https://cwiki.apache.org/confluence/display/OPENWHISK/Project+Maturity+Model

On Fri, Nov 9, 2018 at 6:28 AM Bertrand Delacretaz 
wrote:

> Hi OpenWhisk PPMC,
>
> I think the project has been progressing nicely towards graduation,
> electing a few people, making releases etc.
>
> One way to keep track of this progress and to convince the Incubator
> PMC that the project is ready to graduate is to do a self-assessment
> based on
> https://community.apache.org/apache-way/apache-project-maturity-model.html
> - see the "How To Use" section for links to examples.
>
> Would someone from the PPMC take the lead on creating that document
> for OpenWhisk?
>
> We can discuss the details here of course, I'm happy to help in my
> incubation mentor role but would like someone from the PPMC to take
> the lead if possible.
>
> -Bertrand
>

Re: Where and how do you deploy Apache OpenWhisk?

[Dascalita Dragos]

Hi Ben,
Thanks for starting this thread.

I'll chime in with my POV.

I see the deployment largely structured around 3 layers:

1. Container Management. This layer contains logic to spin up a Kubernetes
or Mesos cluster. It could be a managed cluster ( Azure Container Service,
Amazon ECS for Kubernetes, or others ) or a cluster installed using custom
scripts. The latter option gives more control, the first option is faster
to start with.

2. Install OpenWhisk components. This layers uses Helm or DCOS' Universe to
define packages, and kube or dcos CLI to install them.

3. Install system packages such as "whisk.system" actions. This layers uses
wskdeploy.

As you could probably guessed, each layer builds on the previous one.

To our particular setup:
- We use Ansible to orchestrate (1), (2), and (3).
- For (1) and (2) we have A/B deployments.
- (1) is installed with custom scripts to fully control the security in the
cluster. I.e. custom iptables rules isolating actions.



On Mon, Sep 17, 2018 at 7:17 AM Ben Browning  wrote:

> Over the past few months, a number of discussions and proposals on the
> list have popped up on the list proposing varying degrees of
> architectural changes to OpenWhisk. And, we have a large number of
> deployment options with varying levels of maintenance.
>
> With that in mind, I'd like to kick off a thread to gather some data
> about OpenWhisk deployments in the wild. If you run some flavor of
> OpenWhisk - whether straight Apache OpenWhisk or something based on it
> - please answer as many of the questions below as you feel comfortable
> doing. Please indicate whether your answers are for development,
> testing, or production environments. Feel free to answer more than
> once for different environments.
>
> Where do you deploy OpenWhisk? On bare metal, VMs, Kubernetes, Mesos,
> OpenShift, something else? In private datacenters? A cloud provider?
>
> How do you deploy OpenWhisk? Related to the where, but what kind of
> tooling do you use for the deployment itself?
>
> Why did you choose those environments and deployment methods? Are you
> satisfied with those choices?
>
> Is there anything you'd like to see change in OpenWhisk to better
> support your deployment and operational needs?
>
> Is there anything else about your environments that would be helpful
> for the community to know?
>
>
> Thank you in advance!
>
> Ben
>

Bi-weekly Tech Interchange call tomorrow - add agenda topics here

[Dascalita Dragos]

Hi,
Please add to this thread any agenda items you'd like to discuss at the
Tech Interchange call tomorrow.

Call details:

Web Meeting: Tech Interchange (bi-weekly):
- Day-Time: Wednesdays, 11AM EDT (Eastern US), 5PM CEST (Central Europe),
3PM UTC, 11PM CST (Beijing)
- Zoom: https://zoom.us/my/asfopenwhisk

Thanks,
dragos

Re: Bi-weekly Tech Interchange call tomorrow

[Dascalita Dragos]

Hi Tyson, I also had the AI Action proposal from the last meeting.

On Tue, Aug 28, 2018 at 2:58 PM Vincent S Hou  wrote:

> Hi Tyson,
>
> I want to introduce the status of release, the plan to release runtimes,
> and the future version management.
> Thank you.
>
> Best wishes.
> Vincent Hou (侯胜博)
>
> Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM
> Cloud
>
> Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
> Phone: +1(919)254-7182 <(919)%20254-7182>
> Address: 4205 S Miami Blvd
> 
> (Cornwallis Drive), Durham, NC 27703, United States
>
> -"Markus Thömmes"  wrote: -
> To: dev@openwhisk.apache.org
> From: "Markus Thömmes" 
> Date: 08/28/2018 01:14PM
> Subject: Re: Bi-weekly Tech Interchange call tomorrow
>
> Hi Tyson,
>
> shall we discuss the mechanics on how to start the prototype for the
> future-arch discussion?
>
> Cheers,
> Markus
>
> Am Di., 28. Aug. 2018 um 16:59 Uhr schrieb Tyson Norris
> :
>
> > Hi Whiskers!
> >
> > Please send any agenda items you would like to discuss at the Tech
> > Interchange call tomorrow.
> > Thanks
> > Tyson
> >
> > Call details:
> > Web Meeting: Tech Interchange (bi-weekly):
> > - Day-Time: Wednesdays, 11AM EDT (Eastern US), 5PM CEST (Central Europe),
> > 3PM UTC, 11PM CST (Beijing)
> > - Zoom: https://zoom.us/my/asfopenwhisk
> >
>
>

Re: Prototyping for a future architecture

[Dascalita Dragos]

I also click on the clean and lean upgrade path, with gradual improvements
to the existing system (which is production-proof), as Michael and Rodric
are suggesting.

At the same time I see how Go would make sense from the Kubernetes and
Knative impl POV, how the trends favor Go over Scala [1], [2], and the
potential benefits from addressing a larger dev audience. @Markus, feel
free to keep me honest if this is where you're coming from.

Trying to keep our options open, I'm thinking: why not take Markus'
challenge, and design these enhancements now to support polyglot
implementations in the future ? This would be the important decision we can
take. BTW, for the data plane itself, we could go at even lower levels to
Envoy or Nginx, to provide superior performance, should we choose to.

[1] -
https://trends.google.com/trends/explore?geo=US&q=%2Fm%2F091hdj,%2Fm%2F09gbxjr

[2] - https://octoverse.github.com/

On Tue, Aug 28, 2018 at 2:53 PM Rodric Rabbah  wrote:

> Thanks Michael for raising these points. I share the same opinion and
> sentiment and think a branch with a clean migration story is better and
> makes more sense. I am not entirely convinced that the choice of language
> itself will make the difference vs the new architecture which is quite
> different and should in itself be more efficient.
>
> -r
>
> On Tue, Aug 28, 2018 at 4:51 PM Michael Marth 
> wrote:
>
> > Hi Markus,
> >
> > IMHO what you propose below is a rather severe change in scope of this
> > discussion and effort.
> > Up until so far this was about _evolving_ the OW architecture. We have
> not
> > explicitly discussed it, but one could assume that it is at least
> feasible
> > to gradually adopt the new architecture. So there would be a smooth path
> > between the current state of the code base and a future one.
> >
> > Your proposal below breaks this assumption somewhat (by proposing a new
> > repo instead of a branch - which will inevitably make the 2 code bases
> > drift apart) as well as explicitly by suggesting a new implementation
> > language. Especially the latter would create a schism between OW-now and
> > OW-future.
> > This schism has implications like the perception of OW-now being
> > deprecated, the _possibility_ of no clean upgrade path, the immediate
> split
> > of the community between *-now and *-future and of course carries the
> risk
> > of the version 2 syndrome.
> >
> > I would propose to implement the future architecture in a branch and in
> > Scala first. If it turns out to be good, then subsequent experiments can
> > show or not-show if a switch of language is of additional value. That
> would
> > allow to make a decision based on data rather than anything else.
> >
> > My2c
> > Michael
> >
> >
> > On 28.08.18, 14:26, "Markus Thömmes"  wrote:
> >
> > Hi all,
> >
> > Am Mo., 27. Aug. 2018 um 20:04 Uhr schrieb David P Grove <
> > gro...@us.ibm.com
> > >:
> >
> > >
> > >
> > >
> > > "Markus Thömmes"  wrote on 08/23/2018
> > 04:19:33
> > > PM:
> > >
> > > >
> > > > Key point I want to make is: At some point we'll have to start to
> > > prototype
> > > > things out and see if our assumptions actually hold water. For
> > example,
> > > my
> > > > assumption on a work-stealing backend is pretty much in the air.
> > > >
> > > > My proposal for going forward would be:
> > > > 1. Create a playground for the implementation of some parts of
> the
> > system
> > > > (a new repository?)
> > > > 2. Let's build some of the things that are uncontroversial and
> > absolutely
> > > > needed in any case (ContainerRouter, ContainerManager).
> > > > 3. Play around with them, hook them up in different ways, see
> what
> > works
> > > > and what doesn't.
> > > >
> > > > Some things will need some testing out to see the scale that the
> > > components
> > > > can operate at. These things will narrow or widen the solution
> > space for
> > > > the more controversial topics around how to distribute containers
> > in the
> > > > system, how to balance between the routers, work-stealing queue:
> > yes/no
> > > etc.
> > > >
> > > > Having some simple components fleshed out could encourage
> > innovation and
> > > > creates some facts that we need to focus things into a good
> > direction.
> > > >
> > > > What do you think? Too early to start with this and/or the wrong
> > way of
> > > > doing it?
> > > >
> > >
> > > +1 for starting to prototype.  It's been a good discussion and I
> > think
> > > we've identified some things that we know we don't know, so time to
> > > experiment and find out.
> > >
> > >
> > > Not sure what the best logistics are for this.  Would like the work
> > to be
> > > at Apache (community visibility).  I'm not sure if the best way is
> a
> > new
> > > repo or an experimental branch of the main repo (we could dial down
> > the
> >

Re: Prototyping for a future architecture

[Dascalita Dragos]

I highly support the idea to start experimenting to help us make more
informed decisions vs basing decisions on assumptions.

Should we also agree on a performance goal (aside from the other goals you
called out ) ? I'm thinking at setting a performance goal i.e. run
X/requests per second on a machine ? WDYT ?

On Thu, Aug 23, 2018 at 4:19 PM Markus Thömmes 
wrote:

> Hi OpenWhiskers,
>
> We've been discussing a new direction for our architecture based on a
> proposal I made here:
>
> https://cwiki.apache.org/confluence/display/OPENWHISK/OpenWhisk+future+architecture
>
> Discussion threads:
> -
>
> https://lists.apache.org/thread.html/29289006d190b2c68451f7625c13bb8020cc8e9928db66f1b0def18e@%3Cdev.openwhisk.apache.org%3E
> -
>
> https://lists.apache.org/thread.html/02986151fb2cffb7426c0d3b207c35923bc49bf164a967d310b3bbb6@%3Cdev.openwhisk.apache.org%3E
> -
>
> https://lists.apache.org/thread.html/efec3d588e3519ce72b44f2d5ae00cabf1f6a33afaa2f08194290553@%3Cdev.openwhisk.apache.org%3E
>
> This discussion has been veeery fruitful and I enjoyed i'm enjoying it a
> lot! Thanks so far to anybody who continues to contribute here!
>
> I think we are reaching a point where we start assuming a lot of things.
> I'll attempt to diagram out my current view of things and maybe we can
> already agree on some key parts of the current design as-is? Things I have
> in mind that we need:
>
> 1. ContainerRouter: I think there's little debate around us needing an
> efficient router, preferably with an API, that we can add/remove containers
> from.
> 2. ContainerManager: In terms of it being the interface creating containers
> seems pretty accepted as well, from what I read in the discussions.
> 3. A work-stealing backend: A backend (hopefully a pre-canned MQ) that
> allows us to do action-specific pulling and signaling of demand through
> that pulling. This one I think is more subject to a debate than the other
> two.
>
> We should also lay out some of the basic gains we want to have from that
> new system, i.e.:
> 1. Makes streaming payloads possible.
> 2. Has very high warm-ratios.
> 3. Doesn't create (many) more containers than needed.
>
> (These are examples, others will feel different)
>
> Key point I want to make is: At some point we'll have to start to prototype
> things out and see if our assumptions actually hold water. For example, my
> assumption on a work-stealing backend is pretty much in the air.
>
> My proposal for going forward would be:
> 1. Create a playground for the implementation of some parts of the system
> (a new repository?)
> 2. Let's build some of the things that are uncontroversial and absolutely
> needed in any case (ContainerRouter, ContainerManager).
> 3. Play around with them, hook them up in different ways, see what works
> and what doesn't.
>
> Some things will need some testing out to see the scale that the components
> can operate at. These things will narrow or widen the solution space for
> the more controversial topics around how to distribute containers in the
> system, how to balance between the routers, work-stealing queue: yes/no
> etc.
>
> Having some simple components fleshed out could encourage innovation and
> creates some facts that we need to focus things into a good direction.
>
> What do you think? Too early to start with this and/or the wrong way of
> doing it?
>
> Cheers,
> Markus
>

Re: Kafka and Proposal on a future architecture of OpenWhisk

[Dascalita Dragos]

“... FWIW I should change that to no longer say "Kafka" but "buffer" or
"message
queue"...”
+1. One idea could be to use Akka Streams and let the OW operator make a
decision on using Kafka with Akka Streams, or not [1]. This would make OW
deployment easier, Kafka becoming optional, while opening the door for
other connectors like AWS Kinesis, Azure Event Hub, and others (see the
link at [1] for a more complete list of connectors )

[1] - https://developer.lightbend.com/docs/alpakka/current/
On Sun, Aug 19, 2018 at 7:30 AM Markus Thömmes 
wrote:

> Hi Tyson, Carlos,
>
> FWIW I should change that to no longer say "Kafka" but "buffer" or "message
> queue".
>
> I see two use-cases for a queue here:
> 1. What you two are alluding to: Buffering asynchronous requests because of
> a different notion of "latency sensitivity" if the system is in an overload
> scenario.
> 2. As a work-stealing type balancing layer between the ContainerRouters. If
> we assume round-robin/least-connected (essentially random) scheduling
> between ContainerRouters, we will get load discrepancies between them. To
> smoothen those out, a ContainerRouter can put the work on a queue to be
> stolen by a Router that actually has space for that work (for example:
> Router1 requests a new container, puts the work on the queue while it waits
> for that container, Router2 already has a free container and executes the
> action by stealing it from the queue). This does has the added complexity
> of breaking a streaming communication between User and Container (to
> support essentially unbounded payloads). A nasty wrinkle that might render
> this design alternative invalid! We could come up with something smarter
> here, i.e. only putting a reference to the work on the queue and the
> stealer connects to the initial owner directly which then streams the
> payload through to the stealer, rather than persisting it somewhere.
>
> It is important to note, that in this design, blocking invokes could
> potentially gain the ability to have unbounded entities, where
> trigger/non-blocking invokes might need to be subject to a bound here to be
> able to support eventual execution efficiently.
>
> Personally, I'm much more torn to the work-stealing type case. It implies a
> wholy different notion of using the queue though and doesn't have much to
> do with the way we use it today, which might be confusing. It could also
> well be the case, that work-stealing type algorithms are easier to back on
> a proper MQ vs. trying to make it work on Kafka.
>
> It might also be important to note that those two use-cases might require
> different technologies (buffering vs. queue-backend for work-stealing) and
> could well be seperated in the design as well. For instance, buffering
> triggers fires etc. does not necessarily need to be done on the execution
> layer but could instead be pushed to another layer. Having the notion of
> "async" vs "sync" in the execution layer could be benefitial for
> loadbalancing itself though. Something worth exploring imho.
>
> Sorry for the wall of text, I hope this clarifies things!
>
> Cheers,
> Markus
>
> Am Sa., 18. Aug. 2018 um 02:36 Uhr schrieb Carlos Santana <
> csantan...@gmail.com>:
>
> > triggers get responded right away (202) with an activation is and then
> > sent to the queue to be processed async same as async action invokes.
> >
> > I think we would keep same contract as today for this type of activations
> > that are eventually process different from blocking invokes including we
> > Actions were the http client hold a connection waiting for the result
> back.
> >
> > - Carlos Santana
> > @csantanapr
> >
> > > On Aug 17, 2018, at 6:14 PM, Tyson Norris 
> > wrote:
> > >
> > > Hi -
> > > Separate thread regarding the proposal: what is considered for routing
> > activations as overload and destined for kafka?
> > >
> > > In general, if kafka is not on the blocking activation path, why would
> > it be used at all, if the timeouts and processing expectations of
> blocking
> > and non-blocking are the same?
> > >
> > > One case I can imagine: triggers + non-blocking invokes, but only in
> the
> > case where those have some different timeout characteristics. e.g. if a
> > trigger fires an action, is there any case where the activation should be
> > buffered to kafka if it will timeout same as a blocking activation?
> > >
> > > Sorry if I’m missing something obvious.
> > >
> > > Thanks
> > > Tyson
> > >
> > >
> >
>

Re: Notes & Video posted from today's Tech. Interchange meeting

[Dascalita Dragos]

Matt, echoing again my thanks for such great notes. Not being able to
attend today, it was valuable to get the summary in between my flight
connections. Kudos !
On Wed, Aug 15, 2018 at 12:23 PM Matt Rutkowski  wrote:

> Thanks Ben for moderating a very full agenda with lots of good
> discussions.
>
> Notes:
>
> https://cwiki.apache.org/confluence/display/OPENWHISK/2018-08-15+OW+Tech+Interchange+-+Meeting+Notes
> Video: https://youtu.be/ZvESgK88TyQ
>
> Also, a thanks to Tyson for volunteering to be our next moderator.
>
> Cheers,
> mr
>
>

Re: Pluggable API Gateways

[Dascalita Dragos]

I’d +1 Rodric’s suggestion with subdirs until we group a few implementation
together in the repo, and decide at that point on splitting into separate
repos or not.
On Fri, Aug 10, 2018 at 10:48 PM Rodric Rabbah  wrote:

> > One possibility would be to create a layer that sits in between Whisk
> > "proper" and a given gateway implementation that knows how to take a
> common
> > set of directives and translate them into whatever DSL the target gateway
> > understands.
>
> Isn’t this what the route management package does, with the canonical
> input as swagger and the wsk cli for APIs?
>
> -r

Re: Pluggable API Gateways

[Dascalita Dragos]

I assume Henry's suggestion is to colocate (as a start) APIGW + apimgmt
actions in a folder, probably in "incubator-openwhisk-apigateway".
In that repo there would be a folder for each impl: GW + apimgmt together.
Henry please keep me honest here.
I assume that apimgmt tests would also move along.

The existing WSK CLI impl for apimgmt, even though it could be improved,
it's something I was able to work with when writing an alternate
implementation. We could keep it as-is, and ask the other apimgmt actions
to adhere to the existing API.

Supporting other implementations of Gateways ( Traefik, Envoy, and others )
should be a great addition to the ecosystem. I would also mention the
feature of "Caching HTTP Responses" of web actions as an additional
capability of new Gateway impl, besides API Management.

On Fri, Aug 10, 2018 at 10:27 AM Rodric Rabbah  wrote:

> Hi Henry
>
> Fundamentally, I agree with you and think what you're proposing is a good
> way forward.
>
> Specifically, based on my own experience helping others integrate their own
> API gateways, I've found a few issues which I've tried to slowly fix.
>
> 1. The openwhisk deployment and route management packages were tied
> together. A PR that was finally merged yesterday breaks this dependence, so
> that the API gateway and route management package are entirely optional.
> (This affects ansible deployments, doesn't necessarily address other ways
> of deploying the system).
>
> 2. I also posite that the route management package (and all of its
> corresponding tests) should be consolidated in the same place that ties
> them to the implementation (i.e. the actual API gateway being used).
>
> 3. The wsk CLI needs to be refactored to break several hard coded
> assumptions (which are not necessary in my assessment) about the underling
> route management implementation and what the gateway implementation
> supports. Simply put, hard coding of the JSON responses and the parameters
> that can be passed to the gateway is wrong and should be generalized.
>
> I believe that the way OpenWhisk provides a level of indirection to
> impedance match against many API gateways that are out there is a good
> engineering of the system, and that it bootstraps actions to extend the API
> is a good design to preserve... so that from the OpenWhisk clients
> perspective we're defining a canonical experience.
>
> In short, I'm in favor of the proposal but caution there will be a bit of
> grunt work ahead to fully realize this (based on my recent experience).
>
> -r
>
>
> -r
>
>
> On Fri, Aug 10, 2018 at 12:35 PM, japhar81  wrote:
>
> > This is a follow-up to a Slack conversation with @dragos.dascalita and
> > @csantanapr. As I've been struggling to stand up the API Gateway for
> > OpenWhisk, it occurred to me that I would actually prefer to use a
> > different GW entirely. In my case, Traefik fits nicely in one spot, and
> > Istio integration would be very handy in another.
> >
> > I'm happy to implement both, but the current state makes it hard to
> > contribute. A couple options and opinions came out of our discussion that
> > could use more input;
> >
> >1. Splitting the routemgt stuff out of incubator-openwhisk either into
> >openwhisk-apigateway or the cli repo. My preference is a dedicated
> > repo, as
> >it makes more sense from a new users' POV -- I had a rough time
> getting
> >things to work and it was unclear in the current structure.
> >2. We can either ship APIGW+routemgt combos for each implementation,
> or
> >we can potentially make the api commands in wsk a plugin, shipping an
> > APIGW
> >+ a wsk cli plugin to match.
> >
> > My personal opinion is this; if the APIGW is optional, the routemgt
> actions
> > must be as well -- one is pointless without the other. They're also
> tightly
> > coupled to the specific implementation. As such, I would suggest that
> both
> > pieces move to their own repo, with a folder for each implementation, as
> > well as deployment docs for all things APIGW in the same place.
> >
> > Thoughts/comments welcome!
> >
> > Thanks,
> > Henry
> >
>

Re: System env vars in user containers

[Dascalita Dragos]

RE “context” object it’s interesting to observe what other FaaS providers
do in that regard. See, AWS [1], Azure [2], and Google [3].

If we want to later support (a) concurrency AND (b) “raw” requests ( as in
No Objects, imagine the “raw” param being a JPEG image ) then the added
“context” param seem to enable this functionality.

[1] -
https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
[2] -
https://docs.microsoft.com/en-us/azure/azure-functions/functions-triggers-bindings
[3] - https://cloud.google.com/functions/docs/concepts/events-triggers
On Tue, Aug 7, 2018 at 1:25 AM Vadim Raskin  wrote:

> Right, invoker already has the changes provided by the Entitlement spi.
> Additional variables are provided via `authEnvironment`val in the
> ContainerProxy.scala.
> Just want to point out that the new variables will be only available for
> the downstream implementations of the Entitlement SPI, e.g. IBM Functions.
>
> regards,
> Vadim.
>
> On Mon, Aug 6, 2018 at 10:14 PM Carlos Santana 
> wrote:
>
> > There are no changes required to the invoker code to enable this, this is
> > already supported via SPI from what I understand, but Vadim can clarify
> >
> > -- Carlos
> >
> > On Mon, Aug 6, 2018 at 4:11 PM Carlos Santana 
> > wrote:
> >
> > > The runtime changes will allow a downstream like IBM Functions to
> > > implement the authentication and entitlement SPI already discussed in
> the
> > > mailing list by Martin, the controller will pass IAM (Identity and
> Access
> > > Management) data about the namespace to the invoker, and the invoker
> will
> > > pass this data to the runtimes as extra environment variables.
> > >
> > > -- Carlos
> > >
> > >
> > > On Mon, Aug 6, 2018 at 2:22 PM Rodric Rabbah  wrote:
> > >
> > >>
> > >> > So what are the invoker changes that will leverage these runtime
> > >> changes? I’m not sure that context was part of the thread yet, sorry
> if
> > it
> > >> was.
> > >>
> > >> It wasn’t but without invoker changes this in itself isn’t very
> useful.
> > >> You are right.
> > >>
> > >> -r
> > >
> > >
> >
>

Re: Update Kafka to 2.0.0

[Dascalita Dragos]

Hi Markus,
Big +1 for upgrading.

In my case we use blue/green deployments so that would be fine.

The only potential problem is on the DCOS package for Mesos, for supporting
the latest Kafka version; i think it should work OOTB, but until it’s
tested I’m leaving room for unknowns. Would you know if the Kafka client
library is able to handle both versions ?

Thanks,
Dragos
On Mon, Jul 30, 2018 at 12:57 PM Markus Thömmes 
wrote:

> Dear OpenWhiskers,
>
> James Dubee, Sven Lange-Last and I have been on a Kafka-related bug
> hunt throughout the past days and found several issues in our current
> KafkaConsumerConnector. Several PRs have been opened since and several
> are yet to come.
>
> Currently, I'm trying to make the Kafka 2.0.0 clients work with our
> old-dated Kafka 0.11 line Kafka broker. I'm wondering if we should
> just update the brokers to 2.0.0 as well. It has been released today
> and is admittedly fairly young. I feel like we should start tracking
> more recent Kafka versions though, to profit from bug and stability
> fixes.
>
> One wrinkle: Today we don't support a graceful update of the Kafka
> brokers in-place. Is any of the production deployments based on
> OpenWhisk relying on being able to make an in-place update or is a
> blue/green style full replacement of the whole cluster fine for
> everybody?
>
> Cheers,
> Markus
>

Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating

[Dascalita Dragos]

A big +1

On Wed, Jun 20, 2018 at 2:16 PM Vincent S Hou  wrote:

> Hi everyone,
>
> This is to call for a vote for the release of Apache OpenWhisk
> 0.9.0-incubating.
>
> List of JIRA ticket(s) resolved for this release can be found at
> https://issues.apache.org/jira/browse/INCUBATOR-213.
>
> To learn more about Apache OpenWhisk, please visit
> https://openwhisk.apache.org/.
>
> This release comprises of source code distribution only. There are totally
> 13 OpenWhisk projects within this release. The artifacts were built from
> the following Git commit IDs:
> * openwhisk: 071d841, Make test-instances of Exec depend on the loaded
> manifest.
> * openwhisk-client-go: 1e50522, Add the DISCLAIMER file for Apache
> incubator project
> * openwhisk-cli: 461f94f, add OS and CPU architecture to user agent header
> * openwhisk-catalog: 517341d, Add the DISCLAIMER file for Apache incubator
> project
> * openwhisk-wskdeploy: 7620ef7, disabling export integration
> * openwhisk-apigateway: 2b87366, Fix awk command in init.sh generating
> resolvers.conf file
> * openwhisk-deploy-kube: cb9c3f5, Update runtimes for upstream changes.
> * openwhisk-runtime-nodejs: 557c4bd, update nodejs 6 & 8 to latest
> security patch
> * openwhisk-runtime-java: b20f90e, Add skip_pull_runtimes for Travis CI
> * openwhisk-runtime-swift: 06c4972, update travis to push "master" tag to
> Docker on "master" branch merges
> * openwhisk-runtime-python: a2098d9, update travis to push "master" tag to
> Docker on "master" branch merges
> * openwhisk-runtime-php: b0834a5, Fix travis publish 72
> * openwhisk-runtime-docker: 650842a, Add ActionProxyContainer tests
>
> All the source code artifacts can be found at:
>
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc1/
>
> Each source code artifact can be found via:
>
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc1/[project
> name]-0.9.0-incubating-sources.tar.gz
>
> The MD5 checksum for each artifact can be found via:
>
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc1/[project
> name]-0.9.0-incubating-sources.tar.gz.md5
>
> The SHA-512 checksum for each artifact can be found via:
>
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-0.9.0-incubating-rc1/[project
> name]-0.9.0-incubating-sources.tar.gz.sha512
>
> KEYS file is available here:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS
>
>
> Please vote on releasing this package as Apache OpenWhisk 0.9.0-incubating.
>
> The vote will be open for at least 72 hours.
> [ ] +1 Release as Apache OpenWhisk 0.9.0-incubating
> [ ] +0 no opinion
> [ ] -1 Do not release and the reason
>
> Thank you very much.
>
>
> Checklist for reference:
> [ ] Download links are valid.
> [ ] Checksums and PGP signatures are valid.
> [ ] Source code artifacts have correct names matching the current release.
> [ ] LICENSE and NOTICE files are correct for each OpenWhisk repo.
> [ ] All files have license headers if necessary.
> [ ] No compiled archives bundled in source archive.
>
>
> Best wishes.
> Vincent Hou (侯胜博)
>
> Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM
> Cloud
>
> Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
> Phone: +1(919)254-7182 <(919)%20254-7182>
> Address: 4205 S Miami Blvd
> 
> (Cornwallis Drive), Durham, NC 27703, United States
>
>

Re: Supporting user-configurable warm action containers?

[Dascalita Dragos]

Acknowledging that dev cycles, performance tests, or other spiky traffic
may not be initially captured, I would put my $$$ on teaching OW how to
learn the traffic pattern and let it pre-warm based on that. There are
existing algorithms that could be used for prediction, if we just have the
features/data captured. This is a beautiful area, yet unchartered territory
so far ?

A related aspect is "how many VMs should an OW operator keep up". I
intuitively relate both problems to the learning capacity of a system to
adapt to the load, in order to perform both:
1. tear-down or warm-up action containers
2. shut-down or start-up new VMs to run actions


On Thu, May 31, 2018 at 10:36 AM Michele Sciabarra 
wrote:

> The recent work introduced also support to use docker image as a compiler
> (and it is what I normally do).
>
> I "compile" the binary offliine and I send it as a single binary, that is
> started at init time.
>
> However while for Go and Swift it is "easy", I do not see how we can
> "precompile" a nodejs application.
> I checked around and it looks like NodeJS does a Just In Time Compilatoin,
> I have not found a way to cache the compilation.
>
> So this is where Go applications may have an edge...
>
> --
>   Michele Sciabarra
>   mich...@sciabarra.com
>
> On Thu, May 31, 2018, at 1:34 PM, Nick Mitchell wrote:
> > for nodejs at least: the cost of a few requires of common packages can
> > easily get you up to the 150-200ms range (e.g. request is a big hitter;
> and
> > this is all on top of the cost of starting a container!). perhaps, for
> > nodejs at least, there are only a few options, ultimately: user pays more
> > for idle resources; provider pays more for idle stem cells; or users
> take a
> > very hard line on the modules they import.
> >
> > switching to other (compiled) runtimes might help, e.g. with the recent
> > work on precompiled go and swift actions? we'd still be left with the
> > container start times, but at least this is something we can control,
> e.g.
> > by requiring users to pay more for access to a larger prewarmed pool?
> >
> > nick
> >
> >
> > On Thu, May 31, 2018 at 7:22 AM, James Thomas 
> wrote:
> >
> > > One of most frequent complaints[1][2][3] I hear from developers using
> > > serverless platforms is coping with cold-start latency when dealing
> with
> > > sudden bursts of traffic.
> > >
> > > Developers often ask for a feature where they can set the number of
> warm
> > > containers kept in the cache for a function. This would allow them to
> keep
> > > a higher number of warm containers for applications with bursty traffic
> > > and/or upgrade the cached number prior to an anticpated burst of
> traffic
> > > arriving. This would be exposed by the managed platforms as a chargable
> > > feature.
> > >
> > > Is this something we could support on OpenWhisk? Ignoring the
> complexity
> > > and feasibility of any solution, from a developer POV I can image
> having an
> > > action annotation `max-warm` which would set the maximum number of warm
> > > containers allowed in the cache.
> > >
> > > Tyson is currently working on concurrent activation processing, which
> is
> > > one approach to reducing cold-start delays[4]. However, there are some
> > > downsides to concurrent activations, like no runtime isolation for
> request
> > > processing, which might make this feature inappropraite for some users.
> > >
> > > [1]
> > > https://www.reddit.com/r/aws/comments/6w1hip/how_many_
> > > successive_lambda_invocations_will_use_a/
> > > [2]
> > > https://twitter.com/search?f=tweets&vertical=default&q=%20%
> > > 23AWSWishlist%20warm&src=typd
> > > [3]
> > > https://theburningmonk.com/2018/01/im-afraid-youre-
> > > thinking-about-aws-lambda-cold-starts-all-wrong/
> > > [4] - https://github.com/apache/incubator-openwhisk/pull/2795
> > >
> > > --
> > > Regards,
> > > James Thomas
> > >
>

Re: Transactionid in the ErrorResponse

[Dascalita Dragos]

+1 to make the “code” field a string

and +1 should we choose to rename it into what it is: “id” instead of
“code”; “code” may still be used but it’s semantincs would be given by
static codes (I.e HTTP Status Codes )

If we’re interested to bring more structure we can look at some ongoing
specs efforts such as [1]

[1] - http://jsonapi.org/format/#errors


On Thu, Apr 19, 2018 at 12:49 PM Carlos Santana 
wrote:

> We just need to find the floating spec and updated with the new schema.
>
> Also `code` is not documented as top level API or documentation, didn't
> find in the swagger doc [1]
>
> + 1 to change and adjust downstreams (i.e. CLI, etc..)
>
> [1]
>
> https://github.com/apache/incubator-openwhisk/blob/master/core/controller/src/main/resources/apiv1swagger.json
>
>
> On Thu, Apr 19, 2018 at 12:04 PM Nick Mitchell  wrote:
>
> > it's unlikely, for sure, but e.g. there may be a typescript spec floating
> > around out there
> >
> > On Thu, Apr 19, 2018 at 11:58 AM, Rodric Rabbah 
> wrote:
> >
> > > It is changing the schema for the error response, but I'm having a hard
> > > time thinking of a case where a client depends on the value of the
> "code"
> > > (so that a change from int to string becomes a problem) - do you have
> an
> > > example?
> > >
> > > Renaming "code" would be more worrisome to me, but apart from the CLI,
> I
> > > suspect - with no evidence to back this up other than my intuition -
> that
> > > it's not used that heavily.
> > >
> > > -r
> > >
> > > On Thu, Apr 19, 2018 at 11:51 AM, Nick Mitchell 
> > > wrote:
> > >
> > > > this seems like a breaking API change. e.g. in nodejs `===` checks
> > would
> > > > break.
> > > >
> > > > On Thu, Apr 19, 2018 at 11:37 AM, Rodric Rabbah 
> > > wrote:
> > > >
> > > > > Should we also rename “code”?
> > > > >
> > > > > I don’t see the value in using code: 0 and changing the schema
> should
> > > be
> > > > > fine and better in the long run.
> > > > >
> > > > > -r
> > > > >
> > > > > > On Apr 19, 2018, at 11:31 AM, Christian Bickel <
> apa...@cbickel.de>
> > > > > wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm currently working on a PR which basically moves the
> > transactionId
> > > > > generation from the controller to the entrypoint of the system.
> This
> > is
> > > > the
> > > > > nginx or a frontdoor above.
> > > > > > One change in this PR is to change the format of the tid from a
> > > number
> > > > > to a String.
> > > > > > This works pretty well except one change, that could be seen by
> > > users.
> > > > > > If there is an error in our system, we return an error response
> > with
> > > a
> > > > > short description and the tid. Until now the tid was a number, so
> the
> > > > value
> > > > > in the JSON has no quotes. With this change, the response message
> > would
> > > > > change, because the tid is a String.
> > > > > > This means the response would change from
> > > > > > ```
> > > > > > {
> > > > > > "error": "This is the description",
> > > > > > "code": 123
> > > > > > }
> > > > > > ```
> > > > > > to
> > > > > > ```
> > > > > > {
> > > > > > "error": "This is the description",
> > > > > > "code": "123"
> > > > > > }
> > > > > > ```.
> > > > > >
> > > > > > Do you agree, that this change would be OK?
> > > > > > An alternative would be to always return a 0 and add an
> additional
> > > > field
> > > > > with our new tid-format.
> > > > > >
> > > > > > If there are no concerns, I'll go ahead and change the field from
> > the
> > > > > number to a String.
> > > > > >
> > > > > > Greetings
> > > > > > Christian Bickel
> > > > >
> > > >
> > >
> >
>

Re: Sending activation metadata to Kafka

[Dascalita Dragos]

Continuing what Rodric was saying I'd also like to bring into this picture
Distributed Tracing from this older PR [1].
A holistic approach of "what to use, when, and for what purpose" would be
useful.

[1] - https://github.com/apache/incubator-openwhisk/pull/2282

On Tue, Apr 17, 2018 at 2:59 PM Tyson Norris 
wrote:

> I took a brief look at the PR - it looks like “the prior approach” of
> sending to couchdb is still enabled, is that correct?
>
> If so, it may be worthwhile to make the reference impl store to couchdb,
> and remove the activation persistence from controller/invoker?
>
> This would also imply that the “polling” that is controller would also
> need to be replaced?
> Thanks
> Tyson
>
> On Apr 17, 2018, at 12:02 PM, Rodric Rabbah  rod...@gmail.com>> wrote:
>
> It would be useful to provide a reference implementation for consuming
> this data. Can you also capture the goals in an issue (I scanned the PR
> quickly but there’s no corresponding issue).
>
> Further now I think there are multiple ways of recording/reporting some of
> the metrics (log markers which are largely silenced by default, kamon
> metrics, and now kafka). Is that right?
>
> I think we’ll need to also document these and a guide for when to use
> which - I caution that we are proliferating multiple ways of doing similar
> things with no consistency or articulated long term vision.
>
> -r
>
> On Apr 17, 2018, at 12:01 PM, Vadim Raskin  raskinva...@gmail.com>> wrote:
>
> Hi Chetan,
>
> Can you share some details on how this is currently being done with
> CouchDB. Do we have any analytics view configured which computes these
> numbers currently?
>
> To my knowledge we don't have the views that are shared anywhere in open
> repos.
>
> May be we also include a basic default implementation out of the box
> which collect aggregated stats using Kamon metrics already being used
>
> Not a bad idea, I'll consider sharing the peace of code after finishing the
> development (separate from the original PR), it might require some
> post-processing to strip away the ibm specific parts, so it might bide some
> time.
>
> regards,
> Vadim.
>
>
>
> On Tue, Apr 17, 2018 at 8:29 AM Chetan Mehrotra  >
> wrote:
>
> Hi Vadim,
>
> This looks helpful to get better insight in runtime operational stats!
>
> It has some advantages over to the prior approach (sending them to
> CouchDB).
>
> Can you share some details on how this is currently being done with
> CouchDB. Do we have any analytics view configured which computes these
> numbers currently?
>
> Now it would be possible to simply connect a custom micro service
> to Kafka and consume the activations in real-time.
>
> May be we also include a basic default implementation out of the box
> which collect aggregated stats using Kamon metrics already being used
> Chetan Mehrotra
>
>
> On Mon, Apr 16, 2018 at 9:51 PM, Vadim Raskin  >
> wrote:
> Hi everyone,
>
>
> I’ve just opened a PR that enables sending activation metadata to Kafka.
> It has some advantages over to the prior approach (sending them to
> CouchDB). Now it would be possible to simply connect a custom micro
> service
> to Kafka and consume the activations in real-time. Some of the use cases
> it
> might cover: activation metrics - collect the data and push them into a
> custom time-series database; user activity audit; activation analytics -
> potentially get some insights with KSQL.
>
>
> At the moment I’ve created a new kafka topic called events, which will
> include messages from Controllers and Invokers. It encompasses the
> following data collected from a single activation:
>
>
> concurrentActivations
> throttledActivations
> statusCode
> initTime
> waitTime
> duration
> kind
>
>
> Probably some more metadata will affiliate this list soon.
>
>
> Just wanted to give a short heads up here. The PR I mentioned:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fpull%2F3552&data=02%7C01%7Ctnorris%40adobe.com%7Cc908a394ad184dffaefe08d5a495c08c%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636595885445950512&sdata=seMpqJ1pUTYUt2a4XDUcBPTleDNCjxruf3HSjx%2BhDk4%3D&reserved=0
>
>
> Thank you,
>
>
> Vadim.
>
>

Re: Install Api Gateway in dcos

[Dascalita Dragos]

Hi Kumar,
Can you provide details on the DCOS version as well ?

*"...The (api gateway) service status shows waiting and there are no logs;
nothing getting deployed also"*
Depending on the DCOS version, you may find in the DCOS UI debug
information in regards to the resource offered by Mesos and what's missing.
[1] It's possible that some constraints are not matched, and Marathon is
still waiting to get offers, which would explain this effect you're seeing.
Note that by default the Gateway listens on this domain defined in config
at [2] ( .gw.). Once the controller is
installed in the cluster, it's accessible on controller.gw.,
assuming the Marathon app name for the OpenWhisk Controller is called
"controller". Same goes for the "invoker": invoker.gw..
Same for Exhibitor, CouchDB etc, any other app installed in the cluster is
exposed in this way. Of course, these domain names can be easily configured
differently in NGINX, I'm just explaining the default rules.

We might have to provide a DCOS universe for 1.9 or 1.10 as well. We've
tested with both versions successfully; we need to strip customizations
particular to our deployment, prior to sharing the updated universe.

HTH,
Dragos

[1] - https://docs.mesosphere.com/1.10/monitoring/debugging/gui-debugging/
[2] -
https://github.com/adobe-apiplatform/apigateway/blob/master/api-gateway-config/conf.d/marathon_apis.conf#L35



On Wed, Feb 28, 2018 at 6:08 PM Carlos Santana  wrote:

> Hi Kumar welcome to list
>
>   Adobe team is currently deploying OpenWhisk on DCOS/Mesos I bet one of
> them will be able to assist you on this task and point you int he right
> direction
>
> The file you point is no longer on the repo, Maybe  Dragos or Tyson could
> also clarify that and the apigateway issues.
>
> -- Carlos
>
> On Wed, Feb 28, 2018 at 8:27 PM Kumar Subramanian  >
> wrote:
>
> > Hi,
> >
> > I am trying to install OpenWhisk on Mesos Cluster.
> >
> >
> >
> > I followed the steps described at
> >
> https://github.com/apache/incubator-openwhisk-devtools/tree/1f42e6057de42babc2bf88870fabb61d172d728d/dcos-universe
> >
> >
> >
> > *Steps Completed:*
> >
> >1. In the universe home directory, run command ./scripts/build.sh.
> >2. Upload /target/repo-up-to-1.8.json to a host service (e.g. AWS S3)
> >   - Make sure Content-Type =
> >   application/vnd.dcos.universe.repo+json in the file's headers.
> >3. In DC/OS admin console, under System > Overview > Repositories, add
> >the link to the new repository.
> >
> >
> >
> > Note: I’ve attached the repository file repo-uo-to-1.8.json.
> >
> >
> >
> > *Next:*
> >
> >1. I added the repository to dcos
> >2. Installed redis successfully.
> >   1. Container port : 6379
> >   2. Host: 
> >   3. Ports: 3774
> >3. Then I tried to install api gateway package (default settings –
> >mentioned below)
> >
> >
> >
> > *Stage I am stuck at:*
> >
> >
> >
> >1. Api Gateway Installation
> >   1. Environment: MarathonHost: http://marathon.mesos:8080
> >
> >
> >
> > *What’s happening?* The (api gateway) service status shows waiting and
> > there are no logs; nothing getting deployed also.
> >
> >
> >
> > *Questions:*
> >
> >1. Why is it stuck and not even getting deployed?
> >2. How can I debug
> >3. Where can I look for logs
> >4. Is there anything wrong with my settings for api-gateway(from json
> >file attached)?
> >5. Is there anything else that needs to be installed?
> >
> >
> >
> > Thanks,
> >
> > Kumar.
> >
>

Any topics for tomorrow's Tech Interchange meeting? @4pm GMT / 11am EST / 8am PST

[Dascalita Dragos]

Does anyone have any agenda items to discuss?   This can include
discussion of Issues,  PRs, Feature ideas, How-To's, Docs, Demos, just
about anything OW related...

Current (typical) agenda:

1. Introduction/New faces on the call? (5 mins)
2. Asking around for notable changes/updates? (5 min)
   - Main/core OpenWhisk
   - Kube/Mesos/Compose/etc
   - API Gateway
   - Catalog/Packages/Samples
   - Tooling/Utilities
3. Topics
- Project graduation plans and workitems
- *YOUR TOPIC HERE!*
4. Find and confirm moderator for next meeting (2 min)

Kind regards,
Dragos

Reminder: OW Tech Interchange call tomorrow, Jan 17rd, 4pm GMT, 10am US EST, 8am US PST

[Dascalita Dragos]

Just a reminder for our "Tech Interchange" (bi-weekly) tomorrow.

---

Topic: Apache OpenWhisk "Tech. Interchange" (bi-weekly) Zoom Meeting
Time: this is a recurring meeting Meet anytime

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/my/asfopenwhisk

Or iPhone one-tap (US Toll):  +16465588656 <(646)%20558-8656>,,5043933185
<(504)%20393-3185># or
+14086380968 <(408)%20638-0968>,,5043933185 <(504)%20393-3185>#

Or Telephone:
Dial: +1 646 558 8656 <(646)%20558-8656> (US Toll) or +1 408 638 0968
<(408)%20638-0968> (US Toll)
Meeting ID: 504 393 3185 <(504)%20393-3185>
International numbers available:
https://zoom.us/zoomconference?m=T4Ycc4hVdhvPkjU0ZjPjwtzOUOtbAH5W

---

Soliciting topics in another thread.

Kind regards,
Dragos

Re: Performance tests for OpenWhisk

[Dascalita Dragos]

+1 to the idea of having 1 machine dedicated to running action containers
only.
We've also taken Markus' repo a step further with a distributed testing
tool named Locust.io [1] . The tests are at [2].
I've also used Tsung [3] on other projects (it's based on Erlang). Tsung
provides the best performance I've seen out there. It scales to millions of
concurrent users with only a few machines. But b/c of Erlang, I've been
having issues running a cluster in docker containers. Locust.io is much
easier to setup and unlike Tsung, which provides an XML to describe the
tests, Locust.io supports Python and Go scripts.

For running performance tests in a single machine we don't need a
distributed testing tool, but I would still build on such a tool b/c I find
it valuable to reuse the same scripts to generate more load in bigger
OpenWhisk clusters.

[1] - http://locust.io/
[2] - https://github.com/adobe-apiplatform/openwhisk-performance-tests
[3] - http://tsung.erlang-projects.org/



On Thu, Aug 3, 2017 at 1:04 AM Carlos Santana  wrote:

> Following up here I requested a new repo incubator-openwhisk-performance
> INFRA ticket here: https://issues.apache.org/jira/browse/INFRA-14778
>
>
> On Wed, May 3, 2017 at 2:57 PM Michael Marth  wrote:
>
> > Markus,
> >
> > Quick update: sent the below to users@infra. So far no reaction. The
> > archive is here [1] but Bertrand tells me only ASF member have access  -
> > for whatever reason.
> >
> > Michael
> >
> > [1]
> >
> https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E
> >
> >
> > On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth  > mma...@adobe.com>> wrote:
> > Dear Infra team,
> >
> > I am enquiring on behalf of the OpenWhisk project (currently in
> Incubator)
> > [1].
> >
> > We would like to periodically run performance tests on a distributed
> > environment (OpenWhisk typically runs on more than 1 machine). So we are
> > basically looking for an ability to spin up/tear down a number of
> (virtual)
> > machines and exclusively use them for a certain amount of time (so that
> the
> > VMs are not shared and the performance test results are comparable over
> > time).
> > The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week.
> >
> > I would like to find out if there is an ASF-supported mechanism to do
> that.
> > For example, can Infra provide such infrastructure? Or is there a cloud
> > provider (like Azure) that might sponsor such efforts with VMs? Or maybe
> > there is an established way for commercial companies that are interested
> in
> > an ASF project to sponsor (fund) such tests?
> >
> > If none of the above exists, then it would also be helpful for us to get
> to
> > know how other projects run such sort of tests.
> >
> > Thanks a lot!
> > Michael
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E
> >
> >
> >
> >
> > From: Markus Thömmes mailto:markusthoem...@me.com
> >>
> > Reply-To: "dev@openwhisk.apache.org" <
> > dev@openwhisk.apache.org>
> > Date: Wednesday 26 April 2017 12:59
> > To: "dev@openwhisk.apache.org" <
> > dev@openwhisk.apache.org>
> > Subject: Re: Performance tests for OpenWhisk
> >
> > Hi Michael,
> >
> > yeah that sounds pretty much spot on. I'd like to have at least 2 VMs
> with
> > 4+ cores and 8GB memory. One VM would host the management stack while one
> > would be dedicated to an Invoker only. That way we could assert
> > single-invoker performance the easiest.
> >
> > Thanks for helping!
> >
> > Cheers,
> > Markus
> >
> > Am 26. April 2017 um 11:36 schrieb Michael Marth   > mma...@adobe.com>>:
> >
> > Markus,
> >
> > Does what I describe reflect what you are looking for?
> > If yes, I am happy to ask on infra.
> >
> > Let me know
> > Michael
> >
> >
> >
> > On 26/04/17 07:52, "Bertrand Delacretaz"  > bdelacre...@apache.org>> wrote:
> >
> > Hi Michael,
> >
> > On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth  > mma...@adobe.com>> wrote:
> > ...Maybe our mentors can chime in. Has this been discussed in the ASF
> > board or so?...
> >
> > Best would be to ask the ASF infrastructure team via
> > us...@infra.apache.org - briefly describe
> > what you need to see what's
> > possible.
> >
> > -Bertrand
> >
>

Re: terraform wskdeploy plugin demo

[Dascalita Dragos]

Nice work David ! Would you be able to highlight the advantages you
observed from using terraform with wskdeploy ?  Were you able to also
factor in terraform's state ?

Thanks,
Dragos

On Thu, Jul 6, 2017 at 10:34 PM David ZL Liu  wrote:

> Greetings,
>
> I made a simple demo app for wskdeploy as a terraform plugin, so is there
> any one interested
> in integrate wskdeploy or other OpenWhisk related tools to terraform? You
> could reference the code
> here:https://github.com/lzbj/TerraformPlugin_Demo, thanks.
>
>
> Cheers,
> dliu
>
>
>

Re: Improving support for UI driven use cases

[Dascalita Dragos]

> The prototype PR from Tyson was based on a fixed capacity of concurrent
activations per container. From that, I presume once the limit is reached,
the load balancer would roll over to allocate a new container.
+1. This is indeed the intent in this proposal.

> a much higher level of complexity and traditional behavior than what you
described.

Thanks for bringing up this point as well. IMO this also makes sense, and
it's not in conflict with the current proposal, but rather an addition to
it, for a later time.  if OW doesn't monitor the activity at the action
container level, then it's going to be hard to ensure reliable resource
allocation across action containers. Based on my tests Memory is the only
parameter we can correctly allocate for Docker containers. For CPU, unless
"--cpu-set" is used, CPU is a shared resource across actions; one action
may impact other actions. For network, unless we implement custom network
drivers for containers, the bandwidth is shared between actions; one action
can congest the network, impacting other actions as well . Disk I/O, same
problem.

So my point is that without monitoring, resource isolation (beyond memory)
remains theoretical at this point.

In an ideal picture OW would monitor closely any available parameters when
invoking actions, through Tracing, monitoring containers, etc, anything
that's available. Then through machine learning OW can learn what's a
normal "SLA" for an action, maybe by simply learning the normal
distribution of response times, if CPU and other parameters are too much to
analyze. Then if the action doesn't behave normally for an Nth percentile,
take 2 courses of action:
1) observe if the action has been impacted by other actions, and
re-schedule it on other VMs if that's the case. Today OW tries to achieve
some isolation through load balancer and invoker settings, but the rules
are not dynamic.
2) otherwise, notify the developer that an anomaly is happening for one of
the actions

These examples are out of the scope for the current proposal. I only shared
them so that we don't take monitoring out of the picture later. It's worth
a separate conversation on this DL, and it's not as pressing as the
performance topic is right now.

Dragos


On Thu, Jul 6, 2017 at 4:40 AM Michael M Behrendt <
michaelbehre...@de.ibm.com> wrote:

> thx for clarifying, very helpful. The approach you described could be
> really interesting. I was thrown off by Dragos' comment saying:
>
>  "What stops Openwhisk to be smart in observing the response times, CPU
> consumption memory consumption of the running containers ? Doing so it
> could learn automatically how many concurrent requests 1 action can
> handle."
>
> ...which in my mind would have implied a much higher level of complexity
> and traditional behavior than what you described.
>
> Dragos,
> did I misinterpret you?
>
>
>
> Thanks & best regards
> Michael
>
>
>
>
> From:   Rodric Rabbah 
> To: dev@openwhisk.apache.org
> Date:   07/06/2017 01:04 PM
> Subject:Re: Improving support for UI driven use cases
>
>
>
> The prototype PR from Tyson was based on a fixed capacity of concurrent
> activations per container. From that, I presume once the limit is reached,
> the load balancer would roll over to allocate a new container.
>
> -r
>
> > On Jul 6, 2017, at 6:09 AM, Michael M Behrendt
>  wrote:
> >
> > Hi Michael,
> >
> > thx for checking. I wasn't referring to adding/removing VMs, but rather
> > activation contaIners. In today's model that is done intrinsically,
> while
> > I *think* in what Dragos described, the containers would have to be
> > monitored somehow so this new component can decide (based on
> > cpu/mem/io/etc load within the containers) when to add/remove
> containers.
> >
> >
> > Thanks & best regards
> > Michael
>
>
>
>
>
>

Re: Improving support for UI driven use cases

[Dascalita Dragos]

>  how this approach would be different from the many IaaS- and PaaS-centric

I like Adrian Cockcroft's response (
https://twitter.com/intent/like?tweet_id=736553530689998848 ) to this:
*"...If your PaaS can efficiently start instances in 20ms that run for half
a second, then call it serverless..."*

I think none of us here imagines that we're building a PaaS experience for
developers, nor does the current proposal intends to suggest we should. I
also assume that none of us imagines to run in production a scalable system
with millions of concurrent users with the current setup that demands an
incredible amount of resources.

Quoting Michael M, which said it nicely, the intent is to make  "*the
current problem ... not be a problem in reality anymore (and simply remain
as a theoretical problem)*".

I think we have a way to be pragmatic about the current limitations, make
it so that developers don't suffer b/c of this, and buy us enough time to
implement the better model that should be used for serverless, where
monitoring, "crash safety", "scaling" , and all of the concerns listed
previously in this thread are addressed better, but at the same time, the
performance doesn't have to suffer so much. This is the intent of this
proposal.



On Tue, Jul 4, 2017 at 11:15 AM Michael M Behrendt <
michaelbehre...@de.ibm.com> wrote:

> Hi Dragos,
>
> > What stops
> > Openwhisk to be smart in observing the response times, CPU consumption,
> > memory consumption of the running containers ?
>
> What are your thoughts on how this approach would be different from the
> many IaaS- and PaaS-centric autoscaling solutions that have been built over
> the last years? All of them require relatively complex policies (eg scale
> based on cpu or mem utilization, end-user response time, etc.? What are the
> thresholds for when to add/remove capacity?), and a value prop of
> serverless is that folks don't have to care about that.
>
> we should discuss more during the call, but wanted to get this out as food
> for thought.
>
> Sent from my iPhone
>
> On 4. Jul 2017, at 18:50, Dascalita Dragos  wrote:
>
> >> How could a developer understand how many requests per container to set
> >
> > James, this is a good point, along with the other points in your email.
> >
> > I think the developer doesn't need to know this info actually. What stops
> > Openwhisk to be smart in observing the response times, CPU consumption,
> > memory consumption of the running containers ? Doing so it could learn
> > automatically how many concurrent requests 1 action can handle. It might
> be
> > easier to solve this problem efficiently, instead of the other problem
> > which pushes the entire system to its limits when a couple of actions
> get a
> > lot of traffic.
> >
> >
> >
> >> On Mon, Jul 3, 2017 at 10:08 AM James Thomas 
> wrote:
> >>
> >> +1 on Markus' points about "crash safety" and "scaling". I can
> understand
> >> the reasons behind exploring this change but from a developer experience
> >> point of view this adds introduces a large amount of complexity to the
> >> programming model.
> >>
> >> If I have a concurrent container serving 100 requests and one of the
> >> requests triggers a fatal error how does that affect the other requests?
> >> Tearing down the entire runtime environment will destroy all those
> >> requests.
> >>
> >> How could a developer understand how many requests per container to set
> >> without a manual trial and error process? It also means you have to
> start
> >> considering things like race conditions or other challenges of
> concurrent
> >> code execution. This makes debugging and monitoring also more
> challenging.
> >>
> >> Looking at the other serverless providers, I've not seen this featured
> >> requested before. Developers generally ask AWS to raise the concurrent
> >> invocations limit for their application. This keeps the platform doing
> the
> >> hard task of managing resources and being efficient and allows them to
> use
> >> the same programming model.
> >>
> >>> On 2 July 2017 at 11:05, Markus Thömmes  wrote:
> >>>
> >>> ...
> >>>
> >>
> >>>
> >> To Rodric's points I think there are two topics to speak about and
> discuss:
> >>>
> >>> 1. The programming model: The current model encourages users to break
> >>> their actions apart in "functions" that take payload and return
> payload.
>

Re: Improving support for UI driven use cases

[Dascalita Dragos]

Michael , +1 to how you summarized the problem.

> I’d suggest that the first step is to support “multiple heterogeneous
resource pools”

I'd like to reinforce Stephen's idea on "multiple resource pools". We've
been already using this idea in production systems successfully in other
setups, with Mesos, isolating the Spark workloads requiring state, from
other stateless workloads, or from GPU workloads. This idea would be a
perfect fit for Openwhisk. It can also be extended beyond the Invokers, to
other cluster managers like Mesos, and Kube.


On Tue, Jul 4, 2017 at 7:05 AM Stephen Fink  wrote:

> Hi all,
>
> I’ve been lurking a bit on this thread, but haven’t had time to fully
> digest all the issues.
>
> I’d suggest that the first step is to support “multiple heterogeneous
> resource pools”, where a resource pool is a set of invokers managed by a
> load balancer.  There are lots of reasons we may want to support invokers
> with different flavors:  long-running actions, invokers in a VPN, invokers
> with GPUs,  invokers with big memory, invokers which support concurrent
> execution, etc…  .  If we had a general way to plug in a new resource
> pool, folks could feel free to experiment with any new flavors they like
> without having to debate the implications on other flavors.
>
> I tend to doubt that there is a “one size fits all” solution here, so I’d
> suggest we bite the bullet and engineer for heterogeneity.
>
> SJF
>
>
> > On Jul 4, 2017, at 9:55 AM, Michael Marth 
> wrote:
> >
> > Hi Jeremias, all,
> >
> > Tyson and Dragos are travelling this week, so that I don’t know by when
> they get to respond. I have worked with them on this topic, so let me jump
> in and comment until they are able to reply.
> >
> > From my POV having a call like you suggest is a really good idea. Let’s
> wait for Tyson & Dragos to chime in to find a date.
> >
> > As you mention the discussion so far was jumping across different
> topics, especially the use case, the problem to be solved and the proposed
> solution. In preparation of the call I think we can clarify use case and
> problem on the list. Here’s my view:
> >
> > Use Case
> >
> > For us the use case can be summarised with “dynamic, high performance
> websites/mobile apps”. This implies:
> > 1 High concurrency, i.e. Many requests coming in at the same time
> > 2 The code to be executed is the same code across these different
> requests (as opposed to a long tail distribution of many different actions
> being executed concurrently). In our case “many” would mean “hundreds” or a
> few thousand.
> > 3 The latency (time to start execution) matters, because human users are
> waiting for the response. Ideally, in these order of magnitudes of
> concurrent requests the latency should not change much.
> >
> > All 3 requirements need to be satisfied for this use case.
> > In the discussion so far it was mentioned that there are other use cases
> which might have similar requirements. That’s great and I do not want to
> rule them out, obviously. The above is just to make it clear from where we
> are coming from.
> >
> > At this point I would like to mention that it is my understanding that
> this use case is within OpenWhisk’s strike zone, i.e. Something that we all
> think is reasonable to support. Please speak up if you disagree.
> >
> > The Problem
> >
> > One can look at the problem in two ways:
> > Either you keep the resources of the OW system constant (i.e. No
> scaling). In that case latency increases very quickly as demonstrated by
> Tyson’s tests.
> > Or you increase the system’s capacity. In that case the amount of
> machines to satisfy this use case quickly becomes prohibitively expensive
> to run for the OW operator – where expensive is defined as “compared to
> traditional web servers” (in our case a standard Node.js server). Meaning,
> you need 100-1000 concurrent action containers to serve what can be served
> by 1 or 2 Node.js containers.
> >
> > Of course, the proposed solution is not a fundamental “fix” for the
> above. It would only move the needle ~2 orders of magnitude – so that the
> current problem would not be a problem in reality anymore (and simply
> remain as a theoretical problem). For me that would be good enough.
> >
> > The solution approach
> >
> > Would not like to comment on the proposed solution’s details (and leave
> that to Dragos and Tyson). However, it was mentioned that the approach
> would change the programming model for users:
> > Our mindset and approach was that we explicitly do not want  to change
> how OpenWhisk exposes itself to users. Meaning, users should still be able
> to use NPMs, etc  - i.e. This would be an internal implementation detail
> that is not visible for users. (we can make things more explicit to users
> and e.g. Have them requests a special concurrent runtime if we wish to do
> so – so far we tried to make it transparent to users, though).
> >
> > Many thanks
> > Michael
> >
> >
> >
> > On 03/07/17 14:48, "Je

Re: Improving support for UI driven use cases

[Dascalita Dragos]

> How could a developer understand how many requests per container to set

James, this is a good point, along with the other points in your email.

I think the developer doesn't need to know this info actually. What stops
Openwhisk to be smart in observing the response times, CPU consumption,
memory consumption of the running containers ? Doing so it could learn
automatically how many concurrent requests 1 action can handle. It might be
easier to solve this problem efficiently, instead of the other problem
which pushes the entire system to its limits when a couple of actions get a
lot of traffic.



On Mon, Jul 3, 2017 at 10:08 AM James Thomas  wrote:

> +1 on Markus' points about "crash safety" and "scaling". I can understand
> the reasons behind exploring this change but from a developer experience
> point of view this adds introduces a large amount of complexity to the
> programming model.
>
> If I have a concurrent container serving 100 requests and one of the
> requests triggers a fatal error how does that affect the other requests?
> Tearing down the entire runtime environment will destroy all those
> requests.
>
> How could a developer understand how many requests per container to set
> without a manual trial and error process? It also means you have to start
> considering things like race conditions or other challenges of concurrent
> code execution. This makes debugging and monitoring also more challenging.
>
> Looking at the other serverless providers, I've not seen this featured
> requested before. Developers generally ask AWS to raise the concurrent
> invocations limit for their application. This keeps the platform doing the
> hard task of managing resources and being efficient and allows them to use
> the same programming model.
>
> On 2 July 2017 at 11:05, Markus Thömmes  wrote:
>
> > ...
> >
>
> >
> To Rodric's points I think there are two topics to speak about and discuss:
> >
> > 1. The programming model: The current model encourages users to break
> > their actions apart in "functions" that take payload and return payload.
> > Having a deployment model outlined could as noted encourage users to use
> > OpenWhisk as a way to rapidly deploy/undeploy their usual webserver based
> > applications. The current model is nice in that it solves a lot of
> problems
> > for the customer in terms of scalability and "crash safeness".
> >
> > 2. Raw throughput of our deployment model: Setting the concerns aside I
> > think it is valid to explore concurrent invocations of actions on the
> same
> > container. This does not necessarily mean that users start to deploy
> > monolithic apps as noted above, but it certainly could. Keeping our
> > JSON-in/JSON-out at least for now though, could encourage users to
> continue
> > to think in functions. Having a toggle per action which is disabled by
> > default might be a good way to start here, since many users might need to
> > change action code to support that notion and for some applications it
> > might not be valid at all. I think it was also already noted, that this
> > imposes some of the "old-fashioned" problems on the user, like: How many
> > concurrent requests will my action be able to handle? That kinda defeats
> > the seemless-scalability point of serverless.
> >
> > Cheers,
> > Markus
> >
> >
> --
> Regards,
> James Thomas
>

Re: Improving support for UI driven use cases

[Dascalita Dragos]

>  I think the opportunities for packing computation at finer granularity
will be there. In your approach you're tending, it seems, toward taking
monolithic codes and overlapping their computation. I tend to think this
will work better with another approach.

+1 to making the serverless system smarter in managing and running the code
at scale. I don't think the current state is there right now. There are
limitations which could be improved by simply allowing developers to
control which action can be invoked concurrently. We could also consider
designing the system to "learn" this intent by observing how the action is
configured by the developer: if it's an HTTP endpoint, or an event handler.

As long as today we can improve the performance by allowing concurrency in
actions, and by invoking them faster, why would we not benefit from this
now, and update the implementation later, once the system improves ? Or are
there better ways available now to match this performance that are not
captured in the proposal ?


On Sat, Jul 1, 2017 at 10:29 PM Alex Glikson  wrote:

> My main point is - interactive Web applications is certainly not the only
> case which is sensitive to latency (or throughput) under variable load.
> Think of an event that a person presses 'emergency' button in an elevator,
> and we need to respond immediately (it might be even more important than
> occasionally getting a timeout on a web page). So, ideally, the solution
> should address *any* (or as many as possible of) such applications.
>
> Regards,
> Alex
>
>
>
> From:   Tyson Norris 
> To: "dev@openwhisk.apache.org" 
> Date:   02/07/2017 01:35 AM
> Subject:Re: Improving support for UI driven use cases
>
>
>
>
> > On Jul 1, 2017, at 2:07 PM, Alex Glikson  wrote:
> >
> >> a burst of users will quickly exhaust the system, which is only fine
> for
> > event handling cases, and not fine at all for UI use cases.
> >
> > Can you explain why is it fine for event handling cases?
> > I would assume that the key criteria would be, for example, around
> > throughput and/or latency (and their tradeoff with capacity), and not
> > necessarily the nature of the application per se.
> >
> > Regards,
> > Alex
>
> Sure - with event handling, where blocking=false, or where a timeout
> response of 202 (and fetch the response later) is tolerable,  exhausting
> container resources will simply mean that the latency goes up based on the
> number of events generated after the point of saturation.  If you can only
> process 100 events at one time, an arrival of 1000 events at the same time
> means that the second 100 events will only be processed after the first
> 100 (twice normal latency), third 100 events after that (3 times normal
> latency), 4th 100 events after that (4 times normal latency) etc. But if
> no user is sitting at a browser waiting for a response, it is unlikely
> they care whether the processing occurs 10ms or 10min after the triggering
> event. (This is exaggerating, but you get the point)
>
> In the case a user is staring at a browser waiting for response, such a
> variance in latency just due to the raw number of users in the system
> directly relating to the raw number of containers in the system, will not
> be usable. Consider concurrency not as a panacea for exhausting container
> pool resources, but rather a way to dampen the graph of user traffic
> increase vs required container pool increase, making it something like
> 1000:1 (1000 concurrent users requires 1 container) instead of it being a
> 1:1 relationship.
>
> Thanks
> Tyson
>
>
>
>
>

Re: Setting up Feeds w/ a verification step

[Dascalita Dragos]

Thanks for your thoughts Nick. Do you happen to have a sample with the
"router action" ? Does it replace "triggers" by invoking other actions
directly, after understanding what kind of event it is ? Or the router
fires other events, based on the properties of the incoming event ?

On Tue, Jun 20, 2017 at 6:06 PM Nick Mitchell  wrote:

> hi dragos,
>
> an approach we took, in this regard, was the "router action" pattern.
> facebook (similarly for slack) is configured to communicate with the router
> action, which will discriminate challenge invocations from normal event
> flow invocations.
>
> in the case of facebook (and slack, more recently, can be configured to
> operate in this way), you (not facebook) can choose the challenge
> parameter. so, this can be bound to the router action in advance.
>
> a nice thing about this is that it allows some flexibility: re-establishing
> the challenge parameter can be done without tearing down any openwhisk
> assets. also, the router pattern allows for a router that fans out events,
> e.g. further discriminating based on some property of the incoming event
> (actually more truly as a router)
>
> the pattern also can be generalized to the case where the event source
> chooses the challenge. the challenge parameter can then be self-bound
> (dynamically) as a package parameter (to the package in which the router
> action sits). then, even flows can easily validate the event-flow challenge
> parameter (because the pre-established parameter is a package binding, it
> is readily accessible by the router action)
>
> nick
>
> On Tue, Jun 20, 2017 at 8:45 PM, Dragos Dascalita Haut <
> ddas...@adobe.com.invalid> wrote:
>
> > Is there a pattern for registering feeds with providers that require a
> > verification action to be invoked, in order to validate the webhook ?
> >
> >
> > I'm trying to add a new feed provider for FB to the OW catalog [1]. Some
> > providers may enforce a verification step before accepting the webhook;
> it
> > could be as simple as the provider sending a verification token that it
> > expects to be sent back by that webhook. See an example at [2].
> >
> >
> > In OW triggers and rules are separate concerns, a design I personally
> > like. The issue is that when the providers want to verify something,
> > there's no "default verification rule" associated with a trigger.
> >
> >
> > The pattern I'm currently trying to implement is:
> >
> >   1.  When the CREATE event for the feed is called, register the new
> > trigger
> >   2.  Create a "verification" rule for this trigger to invoke an action
> > that satisfies the provider requirements
> >   3.  Invoke the provider API for registering a new webhook
> >   4.  Wait for the provider to verify the webhook, and then delete the
> > rule created at step 2.
> >
> >
> > Do you have any thoughts regarding this approach ?
> >
> >
> > Thanks,
> >
> > dragos
> >
> > [1] - https://github.com/apache/incubator-openwhisk-catalog
> > [2] -
> https://developers.facebook.com/docs/graph-api/webhooks#callback-url
> >
>

Re: PassportJS with OpenWhisk

[Dascalita Dragos]

Thanks everyone for the feedback.

> "...we currently don't have passport installed in nodejs6action runtime,
to use your repo as is today I need to create a zip first, can't use the
.js as your doc show right?..."

Actually no. I've used browserify to combine the dependencies into a single
JS. That's how I highlighted that it's only ~240Kb.

For the fun of it, I've quickly deployed it in Bluemix, in case you want to
try it out for yourself:
https://openwhisk.ng.bluemix.net/api/v1/web/oauth_test_oauth_space/oauth/github

> "...Don't know if you saw last post from Lionel [1] uses a webaction to
wrap an existing express app..."
That looks fantastic. I've also taken a similar approach, adapting the
request and response for Passport. It would be interesting to try it out
with openwhisk-express NPM and see how it goes.

"...very interesting to the point of creating an action that can be shared
as standard middleware in OpenWhisk apps, in terms of composition..."
Yes, at the end of the day w or w/o Express this is what I was trying to
achieve.



On Mon, May 8, 2017 at 11:24 AM Carlos Santana  wrote:

> This is awesome Dragos +1
>
> I think everybody by now realize that with web actions and having access to
> the request and response, there is a lot of modules in ExpressJS that cab
> be leverage in OpenWhisk without re-inventing the wheel (web server) :-)
>
> Don't know if you saw last post from Lionel [1] uses a webaction to wrap an
> existing express app, my proxying the request and response.
> I think he publish some initial work on npm [2] and github [3]
>
> I think he is planning on part 2 of the article to include authentication
>
> You work is very interesting to the point of creating an action that can be
> shared as standard middleware in OpenWhisk apps, in terms of composition, I
> think it feats very well in a sequence composition were this action will be
> the gatekeeper to handle the auth and get the user token which outputs to
> the next action which doesn't have to be a webaction it can be a
> traditional action.
>
> The output of the traditional action, would then be pass to an action that
> knows how to build the http response (i.e. statusCode, headers, body), as
> you can see there can be action in the center of the sequence that never
> new they were in the middle of a request/response flow.
>
> Rodric coined the term "webify" an traditional action, where thissequence
> action can be created with an Action, as you can see we like to eat our dog
> food a lot META !!
>
> I think there is a need for both a library and a package that offers this
> helper functions, that people don't have to re-invent just to make web
> actions handle expressjs middleware.
>
> In terms of delivering this package in openwhisk-catalog repo, I think I
> want to go in opposite direction right now it's a big monolithic repo that
> I want to break out into individual repositories.
>
> So I proposed having a repository named something like
>  "openwhisk-express", we can put the npm libraries/modules and re-usable
> actions in this repository and deploy them to openwhisk by default as part
> of postdeploy step that today we only deploy catalog packages, but we
> should also deploy the other packages in OpenWhisk like alarms, kafka,
> etc..
>
> Having in it's repo also makes it easy to deploy it with wskdeploy in case
> you have your own copy or different version of the package
>
> One minor comment on your README that I didn't follow, we currently don't
> have passport installed in nodejs6action runtime, to use your repo as is
> today I need to create a zip first, can't use the .js as your doc show
> right?
>
> [1]
>
> https://medium.com/openwhisk/deploying-express-js-apps-to-openwhisk-part-1-9133ba5f262c
> [2] https://www.npmjs.com/package/openwhisk-expressjs
> [3] https://github.com/lionelvillard/openwhisk-expressjs
>
>
> --Carlos
>
>
> On Mon, May 8, 2017 at 11:30 AM James Thomas  wrote:
>
> > This looks amazing! Great work - will have to try it out.
> >
> > On 7 May 2017 at 23:03, Dascalita Dragos  wrote:
> >
> > > Recently I’ve been trying to see if we can reuse PassportJS with
> > OpenWhisk
> > > in order to create a User Authentication experience in a Serverless
> > > fashion. PassportJS is a Node library that supports 300+ authentication
> > > providers and this is what made it appealing.
> > >
> > > The use-case I was after was to enable developers to authenticate users
> > > with multiple providers.
> > >
> > > As an example: when users backup, upload, or edit a photo in

PassportJS with OpenWhisk

[Dascalita Dragos]

Recently I’ve been trying to see if we can reuse PassportJS with OpenWhisk
in order to create a User Authentication experience in a Serverless
fashion. PassportJS is a Node library that supports 300+ authentication
providers and this is what made it appealing.

The use-case I was after was to enable developers to authenticate users
with multiple providers.

As an example: when users backup, upload, or edit a photo in a Cloud
Storage service, register a webhook as an action that automatically syncs
the photo with another Cloud Storage service. For such a scenario the
action needs 2 tokens, one for each service, in order to access user's data
in both places.

My first step was to see if we can run PassportJS in an OpenWhisk action,
without having to depend on an Express server. This action would be
responsible with user authentication, without being concerned with storing
any user tokens; the assumption is that other actions could handle this
concern of caching tokens in a secure way, and making them available to the
actions that need them. So the “authentication” action would only reuse
PassportJS, login users, and output the tokens.

Eventually I was able to adapt PassportJS to an OpenWhisk action. You can
find the code here:
https://github.com/ddragosd/experimental-openwhisk-passport-auth

The good news is that with less than a couple of hundreds LOC OpenWhisk &
PassportJS can authenticate users in a lot of systems. So far I’ve tried
authentication with GitHub, Twitter, Facebook, and Google, and it works
flawlessly. The code of the action is ~240kb, including the libs supporting
these providers.

The tricky part was to decouple Passport from Express / Connect, by
providing simple implementations for request/response objects, as they
should work differently in a serverless environment; this resulted in a
fine-grained level of control for the response of the action, which proved
to be useful.

The way the action works is that developers can create multiple OpenWhisk
web actions reusing the same code, an action for each provider; for each
action they get to set default parameters like client_id, client_secret,
scope, corresponding to each provider. Each provider has its own unique
URL.

It would be nice if we can make it straight forward in OpenWhisk for
developers to create actions that listen to events from a variety of 3rd
party providers on behalf of the users, and then be able to do something
useful on user’s behalf; user authentication plays an important role in
this. To some extent I’ve also been thinking what it would be like for
developers if this feature would come with the OW catalog.


Looking forward to hearing your feedback,

Dragos

Re: concurrent requests on actions

[Dascalita Dragos]

Why not considering giving developers options to control the level of
concurrency, instead of deciding on their behalf ? I think that cases such
as the ones Tyson is mentioning make sense; unless we build something that
will estimate the resources needed by an action automatically, letting the
developer specify it instead, might be a mean of "supervised learning" that
the system can use further in order to make decisions at runtime.

Dragos
On Mon, May 1, 2017 at 4:46 PM Tyson Norris  wrote:

> Sure, many of our use cases are mostly stitching together API calls, as
> opposed to being CPU bound - consider a simple javascript action that wraps
> a downstream http API (or many APIs).
>
> What do you mean by “more efficient packing of I/O-bound processes”? For
> example, in the case of actions that wrap an API call, typically the action
> developer is NOT the owner of the API call, so its not clear how to handle
> this more efficiently than by creating a nodejs action that proxies
> (multiple concurrent) network requests around, but does little actual
> computing besides possibly some minor request/response parsing etc. In our
> cases we our much more likely to run into bottlenecks with concurrent users
> without any concurrent container usage support, unless we greatly over
> provision clusters which will provide drastic reduction in efficiency. It
> is much simpler to provision for anticipated or immediate load changes when
> each new container can support multiple concurrent requests, instead of
> each new container supporting a single request.
>
> More tests demonstrating these cases (e.g. API wrappers, and
> compute-centric actions) will help this discussion, I’ll work on providing
> those.
>
> Thanks
> Tyson
>
> > On May 1, 2017, at 3:24 PM, Nick Mitchell  wrote:
> >
> > won't this only be of benefit for invocations that are mostly sleepy?
> e.g.
> > I/O-bound? because if an action uses CPU flat-out, then there is no
> > throughput win to be had (by increasing the parallelism of CPU-bound
> > processes), given the small CPU sliver that each container gets -- unless
> > there is a concomitant increase in concurrency, i.e. CPU slice?
> >
> > if so, then my gut tells me that there are more general solutions to this
> > (i.e. more efficient packing of I/O-bound processes)
> >
> > On Mon, May 1, 2017 at 5:36 PM, Tyson Norris  wrote:
> >
> >> Thanks Markus.
> >>
> >> Can you direct me to the travis job where I can see the 40+RPS? I agree
> >> that is a big gap and would like to take a look - I didn’t see anything
> in
> >>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftravis-ci.org%2Fopenwhisk%2Fopenwhisk%2Fbuilds%2F226918375&data=02%7C01%7C%7C8a29a490bc6545d4460408d490e0c979%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636292742509382993&sdata=2RiV65g7zvR07ditlzosUxsrWvQIo8WfpMvr7g2JHWY%3D&reserved=0
> ; maybe I’m
> >> looking in the wrong place.
> >>
> >> I will work on putting together a PR to discuss.
> >>
> >> Thanks
> >> Tyson
> >>
> >>
> >> On May 1, 2017, at 2:22 PM, Markus Thömmes   >> markusthoem...@me.com>> wrote:
> >>
> >> Hi Tyson,
> >>
> >> Sounds like you did a lot of investigation here, thanks a lot for that
> :)
> >>
> >> Seeing the numbers, 4 RPS in the "off" case seem very odd. The Travis
> >> build that runs the current system as is also reaches 40+ RPS. So we'd
> need
> >> to look at a mismatch here.
> >>
> >> Other than that I'd indeed suspect a great improvement in throughput
> from
> >> your work!
> >>
> >> Implementationwise I don't have a strong opionion but it might be worth
> to
> >> discuss the details first and land your impl. once all my staging is
> done
> >> (the open PRs). That'd ease git operation. If you want to discuss your
> >> impl. now I suggest you send a PR to my new-containerpool branch and
> share
> >> the diff here for discussion.
> >>
> >> Cheers,
> >> Markus
> >>
> >> Von meinem iPhone gesendet
> >>
> >> Am 01.05.2017 um 23:16 schrieb Tyson Norris  tnor
> >> r...@adobe.com>>:
> >>
> >> Hi Michael -
> >> Concurrent requests would only reuse a running/warm container for
> >> same-action requests. So if the action has bad/rogue behavior, it will
> >> limit its own throughput only, not the throughput of other actions.
> >>
> >> This is ignoring the current implementation of the activation feed,
> which
> >> I guess is susceptible to a flood of slow running activations. If those
> >> activations are for the same action, running concurrently should be
> enough
> >> to not starve the system for other activations (with faster actions) to
> be
> >> processed. In case they are all different actions, OR not allowed to
> >> execute concurrently, then in the name of quality-of-service, it may
> also
> >> be desirable to reserve some resources (i.e. separate activation feeds)
> for
> >> known-to-be-faster actions, so that fast-running actions are not
> penalized
> >> for existing alongside the slow-running actions. This would require a
> more
> >> complicat

Re: API Gateway

[Dascalita Dragos]

Hi Jugaadi,
This is a good idea. There is already a Kong Plugin for OpenWhisk - [1];
it's limited in functionality when compared with the current Openwhisk
Gateway - [2] . But the community could enhance the Kong plugin and it can
be used by people who are already familiar with Kong. The current design in
Openwhisk supports this "bring your own gateway" concept.

Thanks,
Dragos

[1] - https://github.com/Mashape/kong-plugin-openwhisk
[2] - https://github.com/openwhisk/openwhisk-apigateway


On Sat, Apr 15, 2017 at 6:50 PM Jugaadi Da 
wrote:

> Team,
>
> I was wondering whether you have considered integrating with Kong as API
> gateway for Openwhisk. The experimental project(Openwhisk API gateway) for
> the same purpose is created along the same architecture as Kong. Kong being
> a mature project and with support for excellent middleware functionalities
> and clustering capabilities, would be a good candidate for this job. The
> 3rd party  plugins ecosystem is also highly beneficial. Do let me know your
> thoughts on this.
>
> Thanks and Regards
>
> ​jugaadi
>  :-)
>

Re: New ContainerPool

[Dascalita Dragos]

Looks nice James !

"...It would be great to have Zipkin or OpenTracing support
embedded within the core OpenWhisk platform that users could enable with a
feature flag for functions"

+1. I was looking to add zipkin for the OW components, mainly Controller
and Invoker. It's very useful to have it for the actions themselves too.

"...Happy to discuss some of the challenges and issues I found getting this
to
work if people are interested"

Interest++ on my side !


Dragos



On Tue, Apr 4, 2017 at 2:57 PM James Thomas  wrote:

> I need to write up my blog post about the zipkin library for OpenWhisk.
>
> Using a client-side function wrapper does work but has a few issues with
> performance. It would be great to have Zipkin or OpenTracing support
> embedded within the core OpenWhisk platform that users could enable with a
> feature flag for functions.
>
> Happy to discuss some of the challenges and issues I found getting this to
> work if people are interested.
>
> On 4 April 2017 at 22:54, Michael M Behrendt 
> wrote:
>
> > Hi Dragos,
> >
> > James Thomas has done some work around zipkin & openwhisk. Is that what
> > you're looking for?
> >
> > https://www.npmjs.com/package/zipkin-instrumentation-openwhisk
> >
> > Sent from my iPhone
> >
> > > On 4 Apr 2017, at 23:50, Dascalita Dragos  wrote:
> > >
> > > This looks very promising Markus ! Great work !
> > >
> > >
> > > I'm wondering if anyone is currently looking into integrating HTrace
> and
> > > Zipkin; if there's no on-going effort I'm interested to do this. At
> least
> > > my team and I are interested in getting a distributed tracing solution
> in
> > > place, helpful in highlighting areas where performance can be improved.
> > It
> > > should also highlight the improvements brought by this new
> ContainerPool.
> > >
> > > dragos
> > >
> > >> On Mon, Apr 3, 2017 at 5:04 AM Markus Thömmes 
> > wrote:
> > >>
> > >> Thanks James,
> > >>
> > >> I will, I already drafted a baseline post :).
> > >>
> > >> Am 03. April 2017 um 13:59 schrieb James Thomas  >:
> > >>
> > >> This looks fantastic - great work.
> > >> You should definitely write this up as a new blog post!
> > >>
> > >> On 1 April 2017 at 14:05, Markus Thömmes 
> wrote:
> > >>
> > >> Hi out there,
> > >>
> > >>
> > >> Over the past couple of weeks, I started working on a new
> ContainerPool
> > >>
> > >> (and thus eventually a new Invoker). It started as a weekend
> > investigation
> > >>
> > >> into how one would write the Invoker if one started on a green field
> and
> > >>
> > >> turned out a valuable step forward after all.
> > >>
> > >>
> > >> The new ContainerPool is modeled around Akka Actors and I put an
> > emphasis
> > >>
> > >> on the testability of the code. Before diving deeper into performance
> > work
> > >>
> > >> on OpenWhisk we need to be able to quickly verify new models of
> > scheduling
> > >>
> > >> and thus I abstracted the "container providing" code away from the
> pool
> > >>
> > >> itself. One can now easily simulate different workloads without
> actually
> > >>
> > >> talking to docker itself. A nice side-effect of this is, that the
> > Container
> > >>
> > >> interface (it's literally a trait) is completely pluggable and can
> also
> > be
> > >>
> > >> filled in by "insert-your-favorite-container-solution-here".
> > >>
> > >>
> > >> In terms of performance I did see a significant improvement in
> > single-user
> > >>
> > >> throughput performance but haven't yet got around to make a proper
> > write-up
> > >>
> > >> of the experiment's setup and methodology, hence I'll not show hard
> > numbers
> > >>
> > >> for now. We're still missing out on a common load test suite which we
> > can
> > >>
> > >> all use to verify our changes.
> > >>
> > >>
> > >> So all in all features:
> > >>
> > >> - Eliminated concurrency behavior through the actor model
> > >>
> > >> - Eliminated busy loop

Re: New ContainerPool

[Dascalita Dragos]

This looks very promising Markus ! Great work !


I'm wondering if anyone is currently looking into integrating HTrace and
Zipkin; if there's no on-going effort I'm interested to do this. At least
my team and I are interested in getting a distributed tracing solution in
place, helpful in highlighting areas where performance can be improved. It
should also highlight the improvements brought by this new ContainerPool.

dragos

On Mon, Apr 3, 2017 at 5:04 AM Markus Thömmes  wrote:

> Thanks James,
>
> I will, I already drafted a baseline post :).
>
> Am 03. April 2017 um 13:59 schrieb James Thomas :
>
> This looks fantastic - great work.
> You should definitely write this up as a new blog post!
>
> On 1 April 2017 at 14:05, Markus Thömmes  wrote:
>
> Hi out there,
>
>
> Over the past couple of weeks, I started working on a new ContainerPool
>
> (and thus eventually a new Invoker). It started as a weekend investigation
>
> into how one would write the Invoker if one started on a green field and
>
> turned out a valuable step forward after all.
>
>
> The new ContainerPool is modeled around Akka Actors and I put an emphasis
>
> on the testability of the code. Before diving deeper into performance work
>
> on OpenWhisk we need to be able to quickly verify new models of scheduling
>
> and thus I abstracted the "container providing" code away from the pool
>
> itself. One can now easily simulate different workloads without actually
>
> talking to docker itself. A nice side-effect of this is, that the Container
>
> interface (it's literally a trait) is completely pluggable and can also be
>
> filled in by "insert-your-favorite-container-solution-here".
>
>
> In terms of performance I did see a significant improvement in single-user
>
> throughput performance but haven't yet got around to make a proper write-up
>
> of the experiment's setup and methodology, hence I'll not show hard numbers
>
> for now. We're still missing out on a common load test suite which we can
>
> all use to verify our changes.
>
>
> So all in all features:
>
> - Eliminated concurrency behavior through the actor model
>
> - Eliminated busy looping to get a container
>
> - Increased testability by drawing sharp-abstraction layers and making
>
> sure each component is testable separately
>
> - Increased "experimentability" with scheduling algorithms (they are
>
> encapsulated themselves)
>
> - Performance increases very likely due implementation of a "pause grace"
>
> (the container is not paused for a defined amount of time and can continue
>
> its work immediately if another request comes in at that time)
>
> - Pluggability of the container interface
>
>
> The pull-request is open at https://github.com/
>
> openwhisk/openwhisk/pull/2021. It's passing tests already. Test
>
> coverage is still a bit low, but Sven Lange-Last and I are working to get
>
> it done quite soon.
>
>
> It will be delivered in multiple smallish pull-requests, the first of
>
> which is open here: https://github.com/openwhisk/openwhisk/pull/2092. At
>
> first, the new pool is going to be behind a feature flag. Once the pool has
>
> battle proven, we can then flip the switch and remove old code as
>
> applicable.
>
>
> Feel free to have a play with it, feedback and code-reviews are very very
>
> welcome!
>
>
> Cheers,
>
> Markus
>
>
>
>
>
> --
> Regards,
> James Thomas
>
>

Re: factoring out "runtimes" into deployment configuration

[Dascalita Dragos]

I was thinking we could avoid the zip.

On Thu, Mar 9, 2017 at 4:13 PM Rodric Rabbah  wrote:

> You can send a zip file to a docker container. So if the image is
> openwhisk/dockerskeleton or openwhisk/swift3action it works as you'd
> expect. In either case there's no docker pull. And this is doable today
> already. What am I missing from your explanation?
>
> -r
>
> On Mar 9, 2017, at 6:44 PM, Dascalita Dragos  wrote:
>
> >> Isn't that what we call "blackbox" (docker) actions now?
> >
> > IIRC, with "blackbox", developers need to rebuild the image every time,
> > push it to a registry then invoke the action:
> > ```
> > wsk action create --docker my-blackbox  container/name
> > ```
> >
> > What I was imagining is a case where developers still deploy "code" but,
> in
> > addition, they get to specify which "container" to use for executing that
> > code. I.e.
> > ```
> > wsk action create my-action ./my-action.js --kind:container/name:tag
> > ```
> > Assuming that `container/name` extends an existing OW container.
> >
> > Now that I'm thinking at this, I'd say that probably a more elegant
> > solution would be for developers to describe the extra libs in the
> > `manifest.yaml` then, at deploy time, a control-plane component of OW
> > builds an optimized container to be used later when invoking the action
> ...
> > not an easier option to implement though.
> >
> >
> >
> >
> > On Thu, Mar 9, 2017 at 2:55 PM Rodric Rabbah  wrote:
> >
> >>> Extending the thought: what if we allow users to specify a custom
> >> container
> >> name when creating / updating actions, in case users want to ?
> >>
> >> Isn't that what we call "blackbox" (docker) actions now?
> >> But in general - the reason for this work - is along these lines.
> >> Rather than having a small number of fixed images per runtime,
> >> we can support many more.
> >>
> >> A concern however is that we don't want to incur docker pull costs for
> some
> >> set of curated images (in the future, you can image that docker actions
> >> come
> >> with an explicit push operation at create time - but we're not there
> yet).
> >>
> >> So by having an extensible list of images, it makes it easier to curate
> >> action runtimes.
> >>
> >> The scenario you describe otherwise is already supported (we know there
> are
> >> users
> >> who extend our nodejs base image with their own artifacts). This has the
> >> benefit
> >> of pulling in fewer layers.
> >>
> >> -r
> >>
>

Re: factoring out "runtimes" into deployment configuration

[Dascalita Dragos]

> Isn't that what we call "blackbox" (docker) actions now?

IIRC, with "blackbox", developers need to rebuild the image every time,
push it to a registry then invoke the action:
```
wsk action create --docker my-blackbox  container/name
```

What I was imagining is a case where developers still deploy "code" but, in
addition, they get to specify which "container" to use for executing that
code. I.e.
```
wsk action create my-action ./my-action.js --kind:container/name:tag
```
Assuming that `container/name` extends an existing OW container.

Now that I'm thinking at this, I'd say that probably a more elegant
solution would be for developers to describe the extra libs in the
`manifest.yaml` then, at deploy time, a control-plane component of OW
builds an optimized container to be used later when invoking the action ...
not an easier option to implement though.




On Thu, Mar 9, 2017 at 2:55 PM Rodric Rabbah  wrote:

>  > Extending the thought: what if we allow users to specify a custom
> container
> name when creating / updating actions, in case users want to ?
>
> Isn't that what we call "blackbox" (docker) actions now?
> But in general - the reason for this work - is along these lines.
> Rather than having a small number of fixed images per runtime,
> we can support many more.
>
> A concern however is that we don't want to incur docker pull costs for some
> set of curated images (in the future, you can image that docker actions
> come
> with an explicit push operation at create time - but we're not there yet).
>
> So by having an extensible list of images, it makes it easier to curate
> action runtimes.
>
> The scenario you describe otherwise is already supported (we know there are
> users
> who extend our nodejs base image with their own artifacts). This has the
> benefit
> of pulling in fewer layers.
>
> -r
>

Re: factoring out "runtimes" into deployment configuration

[Dascalita Dragos]

With the "runtimeManifest" property I'm wondering if we can also expose the
name of the container image instead of having to compute it in the code ?
Extending the thought: what if we allow users to specify a custom container
name when creating / updating actions, in case users want to ?

I'm thinking at the use-case when a package with a group of actions
requires multiple libraries for each action. To simplify my example I'm
going to refer only to JS actions:
-   Today: when actions needs extra libs users have 2 options :
 a) create a zip
 b) browserify/"compile" JS into a single JS file
-   What I'm thinking: allow users to create their own "runtime" image
where they can bundle those dependent libs into. All the actions in the
package would be smaller, they would use less space and would probably flow
easier through the system when being invoked. A potential problem in this
case would be whether it's secure to let users push their own images in the
OW nodes, but at the same time allowing them to upload a ZIP is
conceptually also a "package"/"container" that can contain anything; I
guess the security aspect would still remain even with user-defined
containers.

Dragos



On Tue, Mar 7, 2017 at 4:20 AM Michael Marth  wrote:

> Hi Rodric,
>
> I think that’s great.
>
> Only comment (more to Matt than you):
> If the runtimes are dynamic by deployment (and of course over time) this
> probably should be reflected in the packaging format spec [1].
> (see also previous comment [2]:)
>
> - line 252: not sure if the list of runtime names should be in the
> normative
> section of a spec, because (as you also write) this list will fluctuate
> between
> deployments and over time. OTOH there should be established well known
> names for
> well-known runtimes. Maybe point to a community-curated, separate markdown
> file
> (where new entries get a PR that can be voted upon)?
>
> Cheers
> Michael
>
> [1]
> https://github.com/openwhisk/openwhisk-wskdeploy/blob/master/specification/openwhisk_v0.8.pdf
> [2] http://markmail.org/message/pa35wxxl52tvbfxc
>
>
>
> On 05/03/17 17:39, "Rodric Rabbah"  rod...@gmail.com>> wrote:
>
> I've been thinking about how we can add more action runtimes more easily -
> without changing the backend (API, controller, invoker, CLI). One way I'm
> leaning toward and started prototyping to get a better understanding of the
> feasibility is to strip all runtime types from the backend (and CLI) and
> encode the information about what action runtimes the system supports
> through configuration parameters.
>
> This will make it possible to decouple deploying the core components from
> managing the action runtime images. An example of encoding the runtime
> information in a deployment configuration is
> https://github.com/openwhisk/openwhisk/pull/1948.
>
> In support of this general direction (even if we settle on a different
> approach), I have increasingly flattened the "Exec" type hierarchy in the
> backend[1-3]
>
> [1] https://github.com/openwhisk/openwhisk/pull/1938
> [2] https://github.com/openwhisk/openwhisk/pull/1942
> [3] https://github.com/openwhisk/openwhisk/pull/1911
>
> A sketch of how to use the runtime manifest can be seen in this WIP commit:
>
> https://github.com/rabbah/openwhisk/commit/a8890abe51a3e7e6a34f96a46798de660583e36f
>
> I can see how using this we can add new versions of Node.js, python:3, and
> other handy images by curating a list of these in a separate process.
>
> Feedback, as always, solicited and appreciated.
>
> -r
>
>

Re: Configuration through environment vs. Consul KV

[Dascalita Dragos]

*"...**pushing the images we build to **DockerHub** (a couchdb image
included)"*
I'm working with my team these days at the CouchDB image.
Should we include it now in the repo or push it when we're ready to push
the other images too ?

Thanks,
Dragos