Apply distribution in openwhisk-package-alarm to reduce load

2020-02-11 Thread Jeongmin Yu 
Hi, OpenWhisk developers.


I submitted a PR to openwhisk-package-alarms.
https://github.com/apache/openwhisk-package-alarms/pull/198


With the existing alarm package, the alarms with 5-field crons fire on the
top of the minute. Then a heavy load can occur in the provider worker when
the alarms converge on a specific time. My issue:
https://github.com/apache/openwhisk-package-alarms/issues/196#issue-480059133


My PR suggests handling this problem by applying the distribution of alarms
within one minute. This satisfies the given conditions(cron) and reduces
the load at the same time.


When the provider detects a new 5-field cron alarm with a parameter
strict=false, it converts the cron into 6 fields by adding a specific value
between 0-59 as a second-unit field. This gives a few seconds of spaces
among alarms, the load of the providers gets reduced as a result.

The injected value looks random, but actually, it's a hash of trigger.name.
So the intervals of each alarm are kept consistently before and after the
redeployment or the re-registration.


Please take a look and leave some comments.


Thanks

Jeongmin

Re: Preview of a OpenWhisk IDE & Debugger... and an help request

2020-02-11 Thread Alexander Klimetschek 
And regarding actionloop specifically: when I looked at it last year, it seemed 
that the generic actionloop container would just need a flag (such as an 
environment variable) to enable debugging, which wskdebug could set 
automatically if it sees an actionloop based kind. Then each specific 
actionloop language container would use this to run the language runtime inside 
the container with debugging enabled and open a debug port, which would also be 
passed through as env var.

Environment variables are good for this, as this makes it impossible to change 
them in a real production Openwhisk environment (as only the openwhisk invoker 
can set these, and they would only have an effect on start of the 
container/app). While wskdebug can easily set these as well.

Cheers,
Alex

From: Alexander Klimetschek 
Sent: Tuesday, February 11, 2020 17:06
To: mich...@sciabarra.com ; dev@openwhisk.apache.org 

Subject: Re: Preview of a OpenWhisk IDE & Debugger... and an help request

> : Standalone OpenWhisk running in a docker container, because the debugger 
> (and the IDE) runs in another docker container.

I guess this isn't really a scenario that wskdebug is built for, because in 
this case you have full control over what happens in that local openwhisk and 
debug ports of action containers can be visible in your host. IIUC this might 
be what you are doing already - not sure I fully grasped that however.

wskdebug is generally meant for debugging when you are using a remote Openwhisk 
(say IBM Cloud or Adobe I/O Runtime) and you have no other choice. Many/most 
action developers will not themselves be able to spin up a local openwhisk, and 
sometimes you really need to debug in the real environment, not a local copy.

But ignoring that, I wonder what exactly is failing when you run wskdebug 
inside a container? In theory it should be feasible. Might be something that 
can be fixed.

Cheers,
Alex

From: Michele Sciabarra 
Sent: Tuesday, February 4, 2020 08:18
To: dev@openwhisk.apache.org 
Subject: Re: Preview of a OpenWhisk IDE & Debugger... and an help request

The main reason because I tried my own approach instead of using wskdebug, that 
I tried, is because I was unable to make it work from within a docker container.

My setup is: Standalone OpenWhisk running in a docker container, because the 
debugger (and the IDE) runs in another docker container.

I need this setup because I want to provide a very simple installation,  a 
wskide command that will setup the development environment.  If I run the 
debugger from the standalone openwhisk works. But I have problems to add also 
the IDE that requires also node. So the prerequisites to run the whole thing 
would be a java of a given version, a node of the right version and docker.

So I decided to go for a full dockerized solution. I have a launcher, written 
in go, that starts standalone openwhisk running in a docker container, and 
another container with the debugger and the editor (theia). In this setup, 
wskdebug does not work. It tries to do something with docker and does not work. 
I was unable to understand what is wrong, I tried to read the code but I do not 
follow it.

For this reason I simply put an action in debug mode (using the runtime) and 
then I connected from the ide. It is more straightforward. There are still some 
issues to sync. But everything is open for discussion, I am more than happy if 
I can re-use wskdebug.

--
  Michele Sciabarra
  mich...@sciabarra.com

- Original message -
From: Alexander Klimetschek 
To: "dev@openwhisk.apache.org" 
Subject: Re: Preview of a OpenWhisk IDE & Debugger... and an help request
Date: Tuesday, February 04, 2020 5:02 PM

Hi Michele,

please note that wskdebug [1] is a debugger for any kind. Nodejs has the best 
out of the box support since that’s what we are exclusively using right now. 
Other languages can already be supported using the right command line arguments 
(ports, docker command etc), which is how Java worked. Given this new 
__OW_DEBUG_PORT env variable, it can be set with wskdebug using —dockerArgs. 
Contributions are welcome to make this more automatic based on the kind/image 
version of the action :-)

FYI, On the contribution front, I will work on the legal documents this or next 
week.

[1] https://github.com/adobe/wskdebug

Cheers,
Alex

Von: Michele Sciabarra 
Gesendet: Monday, February 3, 2020 2:27:15 AM
An: dev@openwhisk.apache.org 
Betreff: Re: Preview of a OpenWhisk IDE & Debugger... and an help request

This is basically also what I am trying to do.

I guess the difference is that you are doing this using the standard nodejs 
runtime while I am doing the same using the goproxy. I am sure it is also 
similar to the adobe/wskdebug that I tried to use, and works but it specific to 
the node and java runtimes, without actionloop.

My goal is to get the debugger for the "other" languag

Re: Preview of a OpenWhisk IDE & Debugger... and an help request

2020-02-11 Thread Alexander Klimetschek 
> : Standalone OpenWhisk running in a docker container, because the debugger 
> (and the IDE) runs in another docker container.

I guess this isn't really a scenario that wskdebug is built for, because in 
this case you have full control over what happens in that local openwhisk and 
debug ports of action containers can be visible in your host. IIUC this might 
be what you are doing already - not sure I fully grasped that however.

wskdebug is generally meant for debugging when you are using a remote Openwhisk 
(say IBM Cloud or Adobe I/O Runtime) and you have no other choice. Many/most 
action developers will not themselves be able to spin up a local openwhisk, and 
sometimes you really need to debug in the real environment, not a local copy.

But ignoring that, I wonder what exactly is failing when you run wskdebug 
inside a container? In theory it should be feasible. Might be something that 
can be fixed.

Cheers,
Alex

From: Michele Sciabarra 
Sent: Tuesday, February 4, 2020 08:18
To: dev@openwhisk.apache.org 
Subject: Re: Preview of a OpenWhisk IDE & Debugger... and an help request

The main reason because I tried my own approach instead of using wskdebug, that 
I tried, is because I was unable to make it work from within a docker container.

My setup is: Standalone OpenWhisk running in a docker container, because the 
debugger (and the IDE) runs in another docker container.

I need this setup because I want to provide a very simple installation,  a 
wskide command that will setup the development environment.  If I run the 
debugger from the standalone openwhisk works. But I have problems to add also 
the IDE that requires also node. So the prerequisites to run the whole thing 
would be a java of a given version, a node of the right version and docker.

So I decided to go for a full dockerized solution. I have a launcher, written 
in go, that starts standalone openwhisk running in a docker container, and 
another container with the debugger and the editor (theia). In this setup, 
wskdebug does not work. It tries to do something with docker and does not work. 
I was unable to understand what is wrong, I tried to read the code but I do not 
follow it.

For this reason I simply put an action in debug mode (using the runtime) and 
then I connected from the ide. It is more straightforward. There are still some 
issues to sync. But everything is open for discussion, I am more than happy if 
I can re-use wskdebug.

--
  Michele Sciabarra
  mich...@sciabarra.com

- Original message -
From: Alexander Klimetschek 
To: "dev@openwhisk.apache.org" 
Subject: Re: Preview of a OpenWhisk IDE & Debugger... and an help request
Date: Tuesday, February 04, 2020 5:02 PM

Hi Michele,

please note that wskdebug [1] is a debugger for any kind. Nodejs has the best 
out of the box support since that’s what we are exclusively using right now. 
Other languages can already be supported using the right command line arguments 
(ports, docker command etc), which is how Java worked. Given this new 
__OW_DEBUG_PORT env variable, it can be set with wskdebug using —dockerArgs. 
Contributions are welcome to make this more automatic based on the kind/image 
version of the action :-)

FYI, On the contribution front, I will work on the legal documents this or next 
week.

[1] https://github.com/adobe/wskdebug

Cheers,
Alex

Von: Michele Sciabarra 
Gesendet: Monday, February 3, 2020 2:27:15 AM
An: dev@openwhisk.apache.org 
Betreff: Re: Preview of a OpenWhisk IDE & Debugger... and an help request

This is basically also what I am trying to do.

I guess the difference is that you are doing this using the standard nodejs 
runtime while I am doing the same using the goproxy. I am sure it is also 
similar to the adobe/wskdebug that I tried to use, and works but it specific to 
the node and java runtimes, without actionloop.

My goal is to get the debugger for the "other" languages, most notabily 
typescript python and go.

Let's discuss on the call to see how we can align efforts.


--
  Michele Sciabarra
  mich...@sciabarra.com

- Original message -
From: Dominic Kim 
To: dev@openwhisk.apache.org
Subject: Re: Preview of a OpenWhisk IDE & Debugger... and an help request
Date: Monday, February 03, 2020 10:52 AM

Thank you for the details.
It seems what you are tying now is similar with our approach and can be
aligned with ours too as it is generic like you said.

We enabled the debugging of Nodejs runtime with chrome DevTools protocol
https://chromedevtools.github.io/devtools-protocol/.
Our UI communicates with the real remote OW deployment.
We implemented one proxy server to correlate a debugging session with a
corresponding action container.
Since each action container is not exposed to public, we also have a bridge
component on each invoker machine for location transparency.

If I understood correctly, each container will connect to the IDE in a
reverse way in your v

Re: [RESULT] [VOTE] Release Apache OpenWhisk Runtime Docker, OpenWhisk Runtime Java, OpenWhisk Runtime PHP, OpenWhisk Runtime Ruby, OpenWhisk Runtime Swift (v1.14.0, rc1)

2020-02-11 Thread Dave Grove 
This vote is now closed.  

The vote to release these 5 runtime modules was successful with 3 binding +1 
votes (rabbah, kim, allen) and no other votes cast.

I will proceed with the release process.

--dave

On 2020/02/08 19:00:26, "David P Grove"  wrote: 
> 
> 
> Hi,
> 
> This is a call to vote on releasing version 1.14.0 release candidate rc1 of
> the following 5 project modules with artifacts built from the Git
> repositories and commit IDs listed below.
> 
> * OpenWhisk Runtime Docker: 60a6475c65e16fedd280e03f06cba27831289737
> 
> https://github.com/apache/openwhisk-runtime-docker/commits/60a6475c65e16fedd280e03f06cba27831289737
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-docker-1.14.0-sources.tar.gz
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-docker-1.14.0-sources.tar.gz.asc
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-docker-1.14.0-sources.tar.gz.sha512
> 
> * OpenWhisk Runtime Java: 3faee44ba46151b8548bfbbd7764d89d1b201b54
> 
> https://github.com/apache/openwhisk-runtime-java/commits/3faee44ba46151b8548bfbbd7764d89d1b201b54
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-java-1.14.0-sources.tar.gz
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-java-1.14.0-sources.tar.gz.asc
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-java-1.14.0-sources.tar.gz.sha512
> 
> * OpenWhisk Runtime PHP: 71e7908a1ee57b4f362a147b66b04c6078b13fab
> 
> https://github.com/apache/openwhisk-runtime-php/commits/71e7908a1ee57b4f362a147b66b04c6078b13fab
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-php-1.14.0-sources.tar.gz
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-php-1.14.0-sources.tar.gz.asc
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-php-1.14.0-sources.tar.gz.sha512
> 
> * OpenWhisk Runtime Ruby: 5d7b54ec2e775db5c464347dbb4241cefcc7c470
> 
> https://github.com/apache/openwhisk-runtime-ruby/commits/5d7b54ec2e775db5c464347dbb4241cefcc7c470
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-ruby-1.14.0-sources.tar.gz
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-ruby-1.14.0-sources.tar.gz.asc
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-ruby-1.14.0-sources.tar.gz.sha512
> 
> * OpenWhisk Runtime Swift: 70cfc59f12e38f2b5e98292d8e617397f6796d3d
> 
> https://github.com/apache/openwhisk-runtime-swift/commits/70cfc59f12e38f2b5e98292d8e617397f6796d3d
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-swift-1.14.0-sources.tar.gz
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-swift-1.14.0-sources.tar.gz.asc
> 
> https://dist.apache.org/repos/dist/dev/openwhisk/rc1/openwhisk-runtime-swift-1.14.0-sources.tar.gz.sha512
> 
> This release is comprised of source code distribution only.
> 
> You can use this UNIX script to download the release and verify the
> checklist below:
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=177e177
> 
> Usage:
> curl -s "
> https://gitbox.apache.org/repos/asf?p=openwhisk-release.git;a=blob_plain;f=tools/rcverify.sh;hb=177e177
> " -o rcverify.sh
> chmod +x rcverify.sh
> rcverify.sh openwhisk-runtime-docker 'OpenWhisk Runtime Docker' 1.14.0 rc1
> rcverify.sh openwhisk-runtime-java 'OpenWhisk Runtime Java' 1.14.0 rc1
> rcverify.sh openwhisk-runtime-php 'OpenWhisk Runtime PHP' 1.14.0 rc1
> rcverify.sh openwhisk-runtime-ruby 'OpenWhisk Runtime Ruby' 1.14.0 rc1
> rcverify.sh openwhisk-runtime-swift 'OpenWhisk Runtime Swift' 1.14.0 rc1
> 
> Please vote to approve this release:
> 
>   [ ] +1 Approve the release
>   [ ]  0 Don't care
>   [ ] -1 Don't release, because ...
> 
> Release verification checklist for reference:
>   [ ] Download links are valid.
>   [ ] Checksums and PGP signatures are valid.
>   [ ] Source code artifacts have correct names matching the current
> release.
>   [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
>   [ ] All files have license headers as specified by OpenWhisk project
> policy [1].
>   [ ] No compiled archives bundled in source archive.
> 
> This majority vote is open for at least 72 hours.
> 
> 
> [1]
> https://github.com/apache/openwhisk-release/blob/master/docs/license_compliance.md
>

[slack-digest] [2020-02-10] #general

2020-02-11 Thread OpenWhisk Team Slack 
2020-02-10 05:48:47 UTC - Nitin Nizhawan: Hi, How can I test blackbox action 
using standalone, 
  ? When I 
create the action it tries to download the image although the image exists on 
local machine. I have named image as "whisk/somename".
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581313727207000?thread_ts=1581313727.207000&cid=C3TPCAQG1

2020-02-10 09:04:57 UTC - chetanm: It should work. Whats your manifest json 
looks like
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581325497207400?thread_ts=1581313727.207000&cid=C3TPCAQG1

2020-02-10 10:39:42 UTC - Nitin Nizhawan: manifest was not provided. I thought 
it is required to specify custom runtimes. Do I need to add my docker image as 
a runtime?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581331182207600?thread_ts=1581313727.207000&cid=C3TPCAQG1

2020-02-10 10:41:05 UTC - chetanm: It also captures the blackbox images which I 
think . are then excluded from explicit pull
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581331265207800?thread_ts=1581313727.207000&cid=C3TPCAQG1

2020-02-10 11:28:08 UTC - Nitin Nizhawan: Thanks, it worked!  Had to do 
following things
1. Looks like manifest completely overrides built-in runtimes. Therfore, copy 
json from ansible/files/runtimes.json to manifest.json.  Add to the list of 
blackboxes. 
2. The image prefix must be "whisk" (not openwhisk) otherwise it will still try 
to pull image
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581334088208000?thread_ts=1581313727.207000&cid=C3TPCAQG1

2020-02-10 18:20:07 UTC - Rob Allen: I always feel slightly guilty that I’ve 
broken the process when I fail to verify a build. On the flip side, this is why 
we do it…
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581358807208700

2020-02-10 18:21:30 UTC - Rodric Rabbah: are you using python 2 or 3?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581358890208900

2020-02-10 18:21:57 UTC - Rodric Rabbah: (i made a change related to python 3 
over the weekend, maybe that broke python 2… will need to check)
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581358917209500

2020-02-10 18:22:20 UTC - Rodric Rabbah: i think there’s a flag to “bypass” 
pulling an image starting with whisk
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581358940209600?thread_ts=1581313727.207000&cid=C3TPCAQG1

2020-02-10 18:23:56 UTC - Rob Allen: Py3 I think
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359036209900

2020-02-10 18:23:57 UTC - Rob Allen: hold on
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359037210100

2020-02-10 18:24:05 UTC - Rob Allen: `Python 3.7.6`
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359045210300

2020-02-10 18:26:05 UTC - Rodric Rabbah: boogers there goes my theory
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359165210900

2020-02-10 18:37:46 UTC - Rodric Rabbah: ah looks like it’s working as expected 
- thanks @Dave Grove
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359866211300

2020-02-10 18:38:38 UTC - Rob Allen: yeah - I didn’t get the rc2 vote email
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359918211600

2020-02-10 18:38:58 UTC - Rob Allen: So the one for rc1 from 09:29 this morning 
seemed current
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581359938212100

2020-02-10 21:41:24 UTC - Bilal: I just turned on user metrics grafana board 
for openwhisk-deploy-kube and it's very nice. Is there anything like this for 
looking at logs? perhaps ELK stack? I also noticed some stuff in the 
documentation about kamon and statsD?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581370884213800

2020-02-10 21:46:19 UTC - Bilal: I see 
 , but it hasn't been 
touched in years
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1581371179214200