Re: New OpenWhisk PMC Members: Brendan Doyle and Cosmin Stanciu

2022-02-23 Thread Tyson Norris
Congratulations Cosmin and Brendan!!! Best, Tyson On Wed, Feb 23, 2022 at 6:57 AM Rodric Rabbah wrote: > Congratulations Brendan and Cosmin! > > -r > > On Wed, Feb 23, 2022 at 6:30 AM Matt Rutkowski > wrote: > > > Welcome and congratulations Brendan and Cosmin! > > > > Kind regards, > > Matt >

Re: [VOTE] Release Apache OpenWhisk Client Js (v3.21.6, rc1)

2022-01-04 Thread Tyson Norris
+1 to release Apache OpenWhisk Client Js (v3.21.6, rc1) Thanks Tyson From: Rob Allen Date: Friday, December 31, 2021 at 3:10 AM To: dev@openwhisk.apache.org Subject: Re: [VOTE] Release Apache OpenWhisk Client Js (v3.21.6, rc1) +1 to release Apache OpenWhisk Client Js (v3.21.6, rc1) Checked with

Re: [VOTE] Release Apache OpenWhisk Client Js (v3.21.5, rc1)

2021-11-03 Thread Tyson Norris
ADA933BB2FFF: "Dominic Kim " not changed > gpg: key A1F071AF3F62EEFF: 3 signatures not checked due to missing keys > gpg: key A1F071AF3F62EEFF: "keybase.io/akrabat " not > changed > gpg: key 395282A61D88D0AC: "Matt Rutkowski " not > changed > gpg: key 7050DAD4D8D21A6

Re: [DISCUSS} OpenWhisk 21.05 / OpenWhisk 1.1

2021-06-10 Thread Tyson Norris
I would suggest removing MesosContainerFactory and dependencies on mesos-actor. We are not using it, and I’m sure nobody else is either. If you already have changed to accommodate this, feel free to add it, otherwise I can create a PR just for this. Thanks Tyson From: Rodric Rabbah Date: Thurs

Re: [discuss]take prewarmed container's memory as used memory

2020-05-26 Thread Tyson Norris
I agree this is a good change, but may require operators to update their configured userMemory and/or their prewarm configs to avoid unexpected problems. It might be good to add some log statements to indicate userMemory available AFTER initial prewarm config is applied, to give some hints that

Re: [FeedBack]Adjust prewarm container dynamically

2020-05-19 Thread Tyson Norris
Thank Ning! I have approved the PR - if nobody else has comments or changes requested by tomorrow, I will merge it if nobody else pushes the button first. Tyson On 5/18/20, 8:34 PM, "Rodric Rabbah" wrote: I've watched this PR as it has evolved - very nice work and contribution (and tha

Re: [VOTE] Release Apache OpenWhisk Client Js (v3.21.2, rc1)

2020-05-05 Thread Tyson Norris
+1 to release OW client js v3.21.2, rc1 Please vote to approve this release: [x] +1 Approve the release [ ] 0 Don't care [ ] -1 Don't release, because ... Release verification checklist for reference: [x] Download links are valid. [x] Checksums and PGP sign

Re: planning npm openwhisk release candidate

2020-05-04 Thread Tyson Norris
FYI, I merged this as well https://github.com/apache/openwhisk-client-js/pull/208 Thanks Tyson On 5/4/20, 2:26 PM, "Rodric Rabbah" wrote: I am planning to cut a release candidate for our npm openwhisk package. There was an important bug fix from Jesse https://nam04.safelinks.prote

Re: Transaction ID in js client

2020-04-10 Thread Tyson Norris
know all the details) Cheers, Alex ________ From: Tyson Norris Sent: Monday, April 6, 2020 07:39 To: dev@openwhisk.apache.org Subject: Re: Transaction ID in js client The current impl is that any request invoked via `this.client.

Re: Transaction ID in js client

2020-04-06 Thread Tyson Norris
TTP request? -r On Mon, Apr 6, 2020 at 10:23 AM Tyson Norris wrote: > Hi – > One of our customers wants to reuse the transaction id when a js action > uses the openwhisk js client to invoke another action. > This sounds reasonable to me, b

Transaction ID in js client

2020-04-06 Thread Tyson Norris
Hi – One of our customers wants to reuse the transaction id when a js action uses the openwhisk js client to invoke another action. This sounds reasonable to me, but I’m not sure if there is some argument to keep them as separate transaction ids? There is a PR already here https://github.com/ap

Re: Welcome new committer: Neeraj Mangal

2020-03-13 Thread Tyson Norris
Welcome Neeraj! Congratulations and thanks for all your help! Tyson > On Mar 12, 2020, at 9:13 PM, Dragos Dascalita Haut > wrote: > > The Project Management Committee (PMC) for Apache OpenWhisk > has invited Neeraj to become a committer and we are pleased > to announce that he has accepted. >

Re: Welcome new committer Alex Klimetschek

2020-03-13 Thread Tyson Norris
Welcome Alex! Your support for both Adobe Runtime and OpenWhisk has been awesome! Tyson > On Mar 13, 2020, at 3:35 AM, Bertrand Delacretaz > wrote: > > Welcome Alex! > >> On Fri, Mar 13, 2020 at 4:40 AM Alexander Klimetschek >> wrote: >> ...In not so small parts because it was a fun thing to

Re: Welcome new committer: Dan McWeeny

2020-02-25 Thread Tyson Norris
Woohoo! Welcome Dan! On 2/25/20, 11:33 AM, "Dragos Dascalita Haut" wrote: It is my pleasure to share that the OpenWhisk PPMC has elected Dan McWeeny as a Committer, based on his ongoing and valuable contributions to the project, the most recent ones being around moving to Java

Re: Config runtime

2020-01-09 Thread Tyson Norris
Hi - I would rather see this as an automated + configurable feature, rather than an API that is manually invoked with a user making (possibly bad) decisions. I created an issue to describe this here https://github.com/apache/openwhisk/issues/4725 Part of the reason for automating this, is that

Re: [VOTE] Release Apache OpenWhisk Runtime Dotnet (v1.14.0, rc1)

2020-01-02 Thread Tyson Norris
+1 to release Apache OpenWhisk Runtime Dotnet (v1.14.0, rc1) All tests passed via rcverify.sh Thanks Tyson On 12/29/19, 3:18 PM, "Shawn Black" wrote: +1 computing sha512 for openwhisk-runtime-dotnet-1.14.0-sources.tar.gz SHA512: openwhisk-runtime-dotnet-1.14.0-sources.tar.gz:

Re: Welcome new committer: Cosmin Stanciu

2019-12-16 Thread Tyson Norris
WOO WOO! Welcome and congratulations Cosmin!!! Thanks for all your work! On 12/16/19, 1:47 PM, "Dragos Dascalita Haut" wrote: It is my pleasure to share that the OpenWhisk PPMC has elected Cosmin Stanciu as a Committer, based on his ongoing and valuable contributions to the project

Re: Welcome new Committer Pengcheng Jiang

2019-12-05 Thread Tyson Norris
Welcome Pengcheng! Thank you! On Wed, Dec 4, 2019 at 11:43 PM Rob Allen wrote: > Welcome Pengcheng! > > > On 5 Dec 2019, at 05:31, Dominic Kim wrote: > > > > It is my pleasure to share that the OpenWhisk PMC has elected Pengcheng > > Jiang > > as a Committer, based on his ongoing and valuable c

Re: Adding OW_ACTION_VERSION

2019-12-03 Thread Tyson Norris
Hi - Just to clarify, your examples mention "cache keys" - can you confirm these keys are stored external to the action code/container? I guess the behavior you are after is that a cache is populated the first time a particular version of the action is invoked, and the action is responsible for

Re: Welcome new Committer Shawn Black

2019-12-02 Thread Tyson Norris
Welcome Shawn!!! Best, Tyson On 12/2/19, 5:24 PM, "Dominic Kim" wrote: Congrats! Welcome, Shawn~! :) Regards Dominic 2019년 12월 3일 (화) 오전 8:19, Rodric Rabbah 님이 작성: > It is my pleasure to share that the OpenWhisk PPMC has elected Shawn Black > as a

Re: Pool slot was shut down

2019-12-02 Thread Tyson Norris
Akka-http 10.1.11 is released I created https://github.com/apache/openwhisk/pull/4759 Have not verified that this is fixed with the change, but hopefully it is. On 11/27/19, 11:04 PM, "Markus Thömmes" wrote: The fix you shared seems to be only logging related so it seems that these logs

Re: Tech Interchange Meeting Wednesday!

2019-11-12 Thread Tyson Norris
I think I had the times wrong: can someone confirm after the time change? 7am PST 10am EST 4pm CET 3pm GMT 11pm Beijing 12am Seoul Call: https://zoom.us/my/asfopenwhisk Thanks Tyson On 11/11/19, 5:37 PM, "Tyson Norris" wrote: Hi Whiskers – I will be hosting the biw

Re: ContainerPool buffering changes

2019-11-12 Thread Tyson Norris
Seeking someone to review this PR – I’ve been load testing it this week with good success. https://github.com/apache/openwhisk/pull/4593 We can discuss at the call tomorrow if nobody reviews it before then. Anyone? Thanks Tyson From: Tyson Norris Date: Thursday, November 7, 2019 at 6:34 AM To

Tech Interchange Meeting Wednesday!

2019-11-11 Thread Tyson Norris
Hi Whiskers – I will be hosting the biweekly Tech Interchange Call on Wednesday November 13th at 10am Eastern 7am Pacific, 3pm Central Europe, 2pm GMT, 10pm Beijing, 11pm Seoul. Call on Zoom: https://zoom.us/my/asfopenwhisk Please submit any topics that you might like to cover. Thanks! Tyson

Re: OpenWhisk as a single docker image?

2019-11-09 Thread Tyson Norris
I suspect that due to Docker-in-Docker scenario, it will be easier to use java+jar (+local docker) instead of running the jar in a container. Today you can start the jar with only java , but you will need a bunch of parameters (probably different per OS?) to run it in a container, I think. Loca

Re: Action health checks

2019-11-07 Thread Tyson Norris
ke that. Am Mi., 30. Okt. 2019 um 16:34 Uhr schrieb Tyson Norris : > I don't think "retry" is the right handling for warm connection failures - > if a connection cannot be made due to container crash/removal, it won't > suddenly come back.

ContainerPool buffering changes

2019-11-07 Thread Tyson Norris
Hi – I have a long outstanding PR to change buffer processing at ContainerPool https://github.com/apache/openwhisk/pull/4593 The background is that in cases where container scheduling fails, we should not immediately retry scheduling, but rather wait for a resource-affecting event to occur, and

Re: Action health checks

2019-10-30 Thread Tyson Norris
this case as it doesn't affect the critical path. Am Di., 29. Okt. 2019 um 18:00 Uhr schrieb Tyson Norris : > By "critical path" you mean the path during action invocation? > The current PR only introduces latency on that path for the case of a

Re: Action health checks

2019-10-29 Thread Tyson Norris
east once), then failure at this level would lead to retries which is reasonable. if we added a third health route or introduced a health check, would we increase the critical path? -r On Tue, Oct 29, 2019 at 12:29 PM David P Grove wrote: > Tyso

Re: Action health checks

2019-10-28 Thread Tyson Norris
machine I believe, that allows for such a retrying use-case. The "connection refused" use-case I mentioned should be equivalent to the TCP probe you're doing now. WDYT? Cheers, Markus Am So., 27. Okt. 2019 um 02:56 Uhr schrieb Tyson Norris :

Action health checks

2019-10-26 Thread Tyson Norris
Hi Whiskers – We periodically have an unfortunate problem where a docker container (or worse, many of them) dies off unexpectedly, outside of HTTP usage from invoker. In these cases, prewarm or warm containers may still have references at the Invoker, and eventually if an activation arrives that

Re: Dangers of renaming and removing runtime kinds

2019-09-17 Thread Tyson Norris
I think the PR is a good start, but I also think upgrading OpenWhisk is a hazardous area that has broader scope that operators need to consider when running OpenWhisk with any custom configuration (and nobody should probably run with default config, but I guess technically it would be possible).

Re: Backpressure for slow activation storage in Invoker

2019-09-16 Thread Tyson Norris
On 9/16/19, 8:32 AM, "Chetan Mehrotra" wrote: Hi Tyson, > in case of logs NOT in db: when queue full, publish non-blocking to "completed-non-blocking" The approach I was thinking was to completely disable (configurable) support for persisting activation from Invoker

Re: Please submit topics for this week's (Wed. 18th) Tech. Interchange call!

2019-09-16 Thread Tyson Norris
Hi Matt - Please add: Dan McWeeney - present some prototype code related to execution design discussion. Thanks! Tyson On 9/16/19, 6:03 AM, "Matt Rutkowski" wrote: Hello Whiskers! Please submit items for agenda for this Wednesday’s (Sept 18) Tech Interchange call. So

OpenWhisk Execution Design

2019-09-16 Thread Tyson Norris
Hi – Here is a more detailed document regarding execution design that I briefly discussed at last meeting. https://docs.google.com/document/d/1A8IyQ2Zjjl6WPc41DBWJa28bp7jEs46bvXVO_H77yBY/edit?usp=sharing Please review and comment. Dan McWeeney will provide a brief demo of some prototype code at

Re: Backpressure for slow activation storage in Invoker

2019-09-12 Thread Tyson Norris
I think this sounds good, but want to be clear I understand the consumers and producers involved - is this summary correct? Controller: * consumes "completed-" topic (as usual) Invoker: * in case of logs NOT in db: when queue full, publish non-blocking to "completed-non-blocking" *in case of lo

Re: Allow decision about action result inclusion in logs on a per call basis

2019-09-10 Thread Tyson Norris
he code change is (github.com/apache/openwhisk/pull/4604) - we do not add more complexity to the code with this little change. PR is github.com/apache/openwhisk/pull/4604 Thanks, Ruediger From: Tyson Norris To: "dev@openwhisk.apache.or

Re: Allow decision about action result inclusion in logs on a per call basis

2019-09-06 Thread Tyson Norris
A couple questions: * " Action results may contain sensible data that should not be logged." - I don't think this solves that problem, correct? E.g. the result or error could have sensitive data still, right? Since this is generated based on the user code and that code's error messages which may

Re: Can we adjust the time for the community call

2019-09-05 Thread Tyson Norris
US West coast dweller here - I'm OK with either 1 hour ealier, or alternating times. Anything to get more players on the field :) Thanks Tyson On 9/5/19, 3:53 AM, "Rodric Rabbah" wrote: Thanks Dominic for bringing this up. +1 from me. What about alternating times (every two weeks) so

Tech Interchange meeting tomorrow

2019-09-03 Thread Tyson Norris
Hi All – Please submit items for agenda for tomorrow’s (Sept 4) Tech Interchange call. I would plan to discuss * A forthcoming proposal for action invocation workflows. * Adding a sandbox area to repo for experimentation (directory in main repo vs separate repo). See you then! Tyson Da

Re: OpenWhisk invoker overloads - "Rescheduling Run message"

2019-08-29 Thread Tyson Norris
E-mail: sven.lange-l...@de.ibm.com Find me on: Schoenaicher Str. 220 Boeblingen, 71032 Germany IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft

Re: Passing TransactionId as part of action invocation

2019-08-21 Thread Tyson Norris
I think the point of transaction id in this case is to correlate multiple activations, similar to how a sequence works, but not relying on sequence as the mechanism for doing this. Today, if you launch many activations explicitly, e.g. using OW SDK in your nodejs action, they are not "related"

Re: Recording metadata related to activation

2019-08-21 Thread Tyson Norris
This part (exposing transaction id to action code) is provided via https://github.com/apache/openwhisk/pull/4586 I'm not sure what other meta may exist or planned that does not already follow this pattern, but I agree it should all be included where possible - cannot include the "duration", sin

Re: Passing TransactionId as part of action invocation

2019-08-16 Thread Tyson Norris
I think if OW SDK, and sequences/compositions, propagate X-Request-Id header (using the existing transaction id/X-Request-Id), the parent is not needed? i.e. there may be 2 parts to this effort: - expose the transaction id to runtime container - propagate the transaction id in requests initiated fr

Re: Openwhisk prewar containers

2019-07-24 Thread Tyson Norris
Hi Adam - Currently there is no support for prewarming blackbox containers. If you need prewarm support, you can add them to the runtimes manifest configured in your deployment, define prewarm configs for each there, and reference them from actions using "--kind ". Best Tyson On 7/24/19, 11:

Re: OpenWhisk invoker overloads - "Rescheduling Run message"

2019-07-19 Thread Tyson Norris
The PR you referenced above contains a lot of other changes. It does not only improve this particular area but also includes a lot of other changes - in particular, it adds a different way of managing containers. Due to the PR's size and complexity, it's very hard to understa

Re: Re: OpenWhisk invoker overloads - "Rescheduling Run message"

2019-07-08 Thread Tyson Norris
Related to the "Rescheduling Run message", one problem we have encountered in these cases is that the invoker becomes unstable due ( I think) to a tight message loop, since the message that couldn't run is immediately resent to the pool to be run, which fails again, etc. We saw CPU getting pegge

Re: exporting activation arguments to the environment

2019-06-27 Thread Tyson Norris
BTW, circling back on "can it be done just using -p in current form": I still like the idea that: - action configured params are different the user specified params (and we should only pass them on init, not run) - this is also a bigger convention change for action developers (read from env or c

Re: exporting activation arguments to the environment

2019-06-26 Thread Tyson Norris
Sorry, what I meant was: accumulating issues around the "main" function, which are: - context - use of env vars - your issue: separating user provided params from developer-provided params - completely separate, but worth noting: logging (and env vars) in the face of concurrency On 6/26/19, 5:2

Re: exporting activation arguments to the environment

2019-06-26 Thread Tyson Norris
For the incremental change I would suggest: - include the action-configured params as args to init - optionally: include an invoker flag to optionally remove action-configured params from being sent to run (these would only be available by way of init setting env vars, or exposing some other obje

Re: exporting activation arguments to the environment

2019-06-26 Thread Tyson Norris
I'd like to make sure my explanation is clearer and if this is still controversial. -r On Wed, Jun 26, 2019 at 1:21 PM Tyson Norris wrote: > Are you saying one is exported to environment, during init, based on > parameter name being UPPER case? Forg

Re: exporting activation arguments to the environment

2019-06-26 Thread Tyson Norris
hat you can export global (immutable) variables to the action. This makes it easier to take existing code and containers which might use env vars and use them almost off the shelf. -r > On Jun 25, 2019, at 6:07 PM, Tyson Norris wrote: > > I had to read this severa

Re: exporting activation arguments to the environment

2019-06-25 Thread Tyson Norris
I had to read this several times, but have some suggestions. I think when you say "action's arguments", you mean action-configured params, e.g. `wsk action create --param p1 v1`? My preferences would be: - we should split off "run" args into context and params - this is the convention change fo

Re: Re: Backpressure for slow activation storage in Invoker

2019-06-21 Thread Tyson Norris
The logs issue is mostly separate from the activation records. RE activation records: Can we handle these in same way as user events? Maybe exactly like user events, as in use a single service to process both topics. RE logging: We deal with logs this way (collect from container via fluent), bu

Re: [VOTE] Apache OpenWhisk graduation to Top Level Project

2019-06-04 Thread Tyson Norris
+1 SHIP IT To Rodric, and all of you who have been working hard to shepherd the evolution of OpenWhisk THANK YOU! On Tue, Jun 4, 2019 at 2:51 PM Michele Sciabarra wrote: > +1 What? Apache OpenWhisk is not yet graduated? But there is even an > O’Reilly Book on it! > > -- > Michele Sciabarra >

Re: Change the way the java runtime handles envirorment variable

2019-05-02 Thread Tyson Norris
FWIW, I don't think of this as a huge change, meaning no change is required at invoker/controller. Rather it is just a convention for context value access within functions and function signature that is already unique to each language runtime, and requires possibly supporting 2 runtimes per lang

Bi-weekly meeting tomorrow

2019-04-02 Thread Tyson Norris
Hi Whiskers – I plan to host the bi-weekly tech exchange meeting tomorrow – please send me your agenda items! Call details: Zoom: https://zoom.us/j/5043933185 Wednesday April 3rd - 11AM EST(Eastern US) - 5PM CET (Central Europe), - 4PM UTC, 12AM CST (Beijing) Thanks Tyson

Re: Tech Interchange meeting Notes & Video posted

2019-03-06 Thread Tyson Norris
Thanks Matt + Rodric I opened the invoker clustered resource PR here: https://github.com/apache/incubator-openwhisk/pull/4326 And added a wiki page with diagram and presentation attached here: https://cwiki.apache.org/confluence/display/OPENWHISK/Cluster+Manager+Resources+in+Invoker Thanks Tyson

Re: [Bi-Weekly Tech Interchange] Call for agenda topics

2019-03-04 Thread Tyson Norris
I would like to share details on a forthcoming PR to enable better invoker integration with cluster managed (mesos/k8s) action containers. I don't have the PR open yet, but plan to by sometime tomorrow. Unfortunately I have a Dr appt on March 20, so cannot volunteer - but possibly April 3rd.

Re: change the default action context to omit api key

2019-02-13 Thread Tyson Norris
I agree the api_key is bad, when not using e.g. OW npm within the action. +1 for using an annotation to enable this. activation_id is required to do the right thing for logging with concurrency enabled - but I'm also not sure what risk it is to include that? It will be in the response header an

concurrency tracking PR

2019-01-30 Thread Tyson Norris
Hi – I have a PR #4186 ready to merge that has some changes to critical areas of ContainerPool for tracking concurrency; specifically so that initing containers (not yet warm) can receive activation jobs, instead of current behavior where additional containers will launch. While this only applie

Re: Update KindRestrictor to merge namespace and default whitelists

2018-11-28 Thread Tyson Norris
Hi - We haven't heard any feedback on this, so will plan to merge this change today. Thanks Tyson On 11/14/18, 6:48 PM, "Andy Steed" wrote: Hello Whiskers! I wanted to bring up a potentially breaking change to existing functionality for the KindRestrictor. Given how recent

Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating-rc1: OpenWhisk Composer

2018-11-26 Thread Tyson Norris
+1 to release Apache OpenWhisk 0.9.0-incubating: OpenWhisk composer Verified checklist Thanks Tyson On 11/20/18, 11:30 AM, "David P Grove" wrote: This is call for a vote for the release of Apache OpenWhisk 0.9.0-incubating: OpenWhisk composer List of JIRA ticket(s)

system tests with external logstore

2018-10-08 Thread Tyson Norris
Hi – We are successfully using the openwhisk system tests against our deployments, but as we near deployments with our external log store, there is the conflict that current tests expect: * Logs to be returned with ‘wsk activation get’ * Logs to be available immediately To deal with thi

Re: Logstore usage during `activation get`

2018-10-02 Thread Tyson Norris
on the activation id returns the entire record. I think we can break this because the clients can sugar the underlying calls. -r > On Oct 2, 2018, at 12:07 PM, Tyson Norris wrote: > > Hi – > I created this PR [1] due to noticing that `wsk activation get` does

Re: Travis problem with runtime-docker

2018-10-02 Thread Tyson Norris
already, so my PR is not needed... On 10/1/18, 5:12 PM, "Tyson Norris" wrote: Of course I just noticed when the `./gradlew install` is fast, is because the openwhisk/tools/travis/setup.sh script does ./gradlew :tests:compileTestScala, which is indeed much slower. I also n

Logstore usage during `activation get`

2018-10-02 Thread Tyson Norris
Hi – I created this PR [1] due to noticing that `wsk activation get` does NOT return logs from a LogStore which stores logs outside of the Activation entity. But it bring up a question of: Does IBM or any other operator who might use a custom LogStore desire to have logs included with `activation

Re: Travis problem with runtime-docker

2018-10-01 Thread Tyson Norris
updating runtime-docker to same gradle version... On 10/1/18, 4:45 PM, "Tyson Norris" wrote: Hi – I’m troubleshooting a problem with this PR build [1] where a test fails due to a difference in exception handling between akka http client (now used in runtime test base class),

Travis problem with runtime-docker

2018-10-01 Thread Tyson Norris
Hi – I’m troubleshooting a problem with this PR build [1] where a test fails due to a difference in exception handling between akka http client (now used in runtime test base class), and apache http client. This change was merged to OW master some time ago in [2]. The problem I’m having is that

Re: Bi-weekly Tech Interchange call tomorrow - add agenda topics here

2018-09-11 Thread Tyson Norris
Hi - I'd like to discuss the pending concurrency PR here: https://github.com/apache/incubator-openwhisk/pull/2795 Specific controversial topics: - adding a synchronized block during concurrency>1 action scheduling when the activation requires a new container - not adding indication of which runt

Re: Prototyping for a future architecture

2018-08-28 Thread Tyson Norris
Changing language is a big leap, that seems unrelated to the basic design principles we are discussing in the proposal. If performance is the main concern, what type of performance difference will be worth this effort? It will be hard (or impossible) to measure until it's done, and the only gua

Bi-weekly Tech Interchange call tomorrow

2018-08-28 Thread Tyson Norris
Hi Whiskers! Please send any agenda items you would like to discuss at the Tech Interchange call tomorrow. Thanks Tyson Call details: Web Meeting: Tech Interchange (bi-weekly): - Day-Time: Wednesdays, 11AM EDT (Eastern US), 5PM CEST (Central Europe), 3PM UTC, 11PM CST (Beijing) - Zoom: https://z

Re: Kafka and Proposal on a future architecture of OpenWhisk

2018-08-23 Thread Tyson Norris
> Router is not pulling at queue for "specific actions", just for any action > that might replace idle containers - right? This is complicated with > concurrency though since while a container is not idle (paused + > removable), it may be useable, but only if the action received is

Re: Kafka and Proposal on a future architecture of OpenWhisk

2018-08-23 Thread Tyson Norris
> > And each ContainerRouter has a queue consumer that presumably pulls from > the queue constantly? Or is consumption based on something else? If all > ContainerRouters are consuming at the same rate, then while this does > distribute the load across ContainerRouters, it doesn'

Re: Simplifying configuring OpenWhisk Components (#3984)

2018-08-23 Thread Tyson Norris
Hi Chetan - As mentioned in the linked DCOS issue, using a marathon "uri" for config files (fetched before container is started) is an option - is there any reason that won't work for us? Files will end up in /mnt/mesos/sandbox in the container, not sure if that path matters (we can copy the fi

Re: Kafka and Proposal on a future architecture of OpenWhisk

2018-08-22 Thread Tyson Norris
Hi - thanks for the discussion! More inline... On 8/22/18, 2:55 PM, "Markus Thömmes" wrote: Hi Tyson, Am Mi., 22. Aug. 2018 um 23:37 Uhr schrieb Tyson Norris : > Hi - > > > > When exactly is the case that a ContainerRout

Re: Kafka and Proposal on a future architecture of OpenWhisk

2018-08-22 Thread Tyson Norris
Hi - > > When exactly is the case that a ContainerRouter should put a blocking > activation to a queue for stealing? Since a) it is not spawning containers > and b) it is not parsing request/response bodies, can we say this would > only happen when a ContainerRouter maxes o

Re: Kafka and Proposal on a future architecture of OpenWhisk

2018-08-22 Thread Tyson Norris
the execution system. Cheers, Markus Am Mo., 20. Aug. 2018 um 23:30 Uhr schrieb Tyson Norris : > Thanks for summarizing Markus. > > Yes this is confusing in context of current system, which stores in kafka, > but not to indefinitely wait

Re: Proposal on a future architecture of OpenWhisk

2018-08-21 Thread Tyson Norris
> Tracking these metrics consistently will introduce the same problem as > precisely tracking throttling numbers across multiple controllers, I think, > where either there is delay introduced to use remote data, or eventual > consistency will introduce inaccurate data. >

Re: Proposal on a future architecture of OpenWhisk

2018-08-20 Thread Tyson Norris
On Aug 19, 2018, at 3:59 AM, Markus Thömmes mailto:markusthoem...@apache.org>> wrote: Hi Tyson, Am Fr., 17. Aug. 2018 um 23:45 Uhr schrieb Tyson Norris mailto:tnor...@adobe.com.invalid>>: If the failover of the singleton is too long (I think it will be based on cluster size,

Re: Kafka and Proposal on a future architecture of OpenWhisk

2018-08-20 Thread Tyson Norris
riggers get responded right away (202) with an activation is and then >> sent to the queue to be processed async same as async action invokes. >> >> I think we would keep same contract as today for this type of activations >> that are eventually process different from blockin

Re: Concurrency PR

2018-08-20 Thread Tyson Norris
To better handle the case of images that don’t support concurrency, or don’t support log collection from invoker, I would suggest we change the container protocol to allow containers to broadcast their support either via the /init endpoint, or via a new /info endpoint. This of course would not giv

Kafka and Proposal on a future architecture of OpenWhisk

2018-08-17 Thread Tyson Norris
Hi - Separate thread regarding the proposal: what is considered for routing activations as overload and destined for kafka? In general, if kafka is not on the blocking activation path, why would it be used at all, if the timeouts and processing expectations of blocking and non-blocking are the

Re: Proposal on a future architecture of OpenWhisk

2018-08-17 Thread Tyson Norris
Ugh my reply formatting got removed!!! Trying this again with some >> On Aug 17, 2018, at 2:45 PM, Tyson Norris mailto:tnor...@adobe.com.INVALID>> wrote: If the failover of the singleton is too long (I think it will be based on cluster size, oldest node becomes the singleton h

Re: Proposal on a future architecture of OpenWhisk

2018-08-17 Thread Tyson Norris
fect the cluster state). On Aug 16, 2018, at 11:01 AM, Tyson Norris mailto:tnor...@adobe.com.INVALID>> wrote: A couple comments on singleton: - use of cluster singleton will introduce a new single point of failure - from time of singleton node failure, to single resurrection on a diffe

Concurrency PR

2018-08-17 Thread Tyson Norris
Hi - I have been noodling with a few tests and the akka http client and gotten the concurrency PR [1] to a good place, I think, so if anyone can help review that would be appreciated. A couple of notes: - akka http client has some different notion of connection reuse than the apache client, to

Re: Proposal on a future architecture of OpenWhisk

2018-08-16 Thread Tyson Norris
. > On Aug 16, 2018, at 11:01 AM, Tyson Norris wrote: > > A couple comments on singleton: > - use of cluster singleton will introduce a new single point of failure - > from time of singleton node failure, to single resurrection on a different > instance, will be an outage from th

Re: Proposal on a future architecture of OpenWhisk

2018-08-16 Thread Tyson Norris
A couple comments on singleton: - use of cluster singleton will introduce a new single point of failure - from time of singleton node failure, to single resurrection on a different instance, will be an outage from the point of view of any ContainerRouter that does not already have a warm+free co

Re: logging baby step -- worth pursuing?

2018-08-15 Thread Tyson Norris
aut > wrote: > > "...we should be able to fully > process the logs offline and in a streaming manner and get the needed > activation id injected into every logline..." > > > +1 IIRC for concurrent activations Tyson Norris and Dan McWeeney were going > down

Re: System env vars in user containers

2018-08-06 Thread Tyson Norris
So what are the invoker changes that will leverage these runtime changes? I’m not sure that context was part of the thread yet, sorry if it was. > On Aug 6, 2018, at 10:52 AM, Carlos Santana wrote: > > To avoid scope creep. > > I implemented what Vadim proposed to make the runtime a bit more f

Re: System env vars in user containers

2018-08-06 Thread Tyson Norris
Other than parsing JSON (and compatibility), does setting env vars instead of propagating a JSON object into the /run handler give any benefit? Since any runtime that wants to support concurrency won’t leverage any (or at least most, except “action name”?) vars during /init, and won’t leverage e

Re: Proposal on a future architecture of OpenWhisk

2018-07-20 Thread Tyson Norris
On Logging, I think if you are considering enabling concurrent activation processing, you will encounter that the only approach to parsing logs to be associated with a specific activationId, is to force the log output to be structured, and always include the activationId with every log message.

Re: Concurrency in invoker http client

2018-07-14 Thread Tyson Norris
Right, got it. Got an initial impl working here: https://github.com/apache/incubator-openwhisk/pull/3812/files#diff-3fca67626fe5f92ce431902dd2318a10R170 So now in that PR, the entity is either consumed via Unmarshal().to[String], when within the allowed response size, or else via the truncated()

Re: Concurrency in invoker http client

2018-07-13 Thread Tyson Norris
ith r.size==1, rather r.size == response.entity.size I will try to fiddle with it some more(and ignore the tail) > On Jul 13, 2018, at 5:29 PM, Tyson Norris wrote: > > Thanks > > Currently HttpUtils does return (almost) the same response when it is > truncated.. so that the trunca

Re: Concurrency in invoker http client

2018-07-13 Thread Tyson Norris
Thanks Currently HttpUtils does return (almost) the same response when it is truncated.. so that the truncated version can later be included in the ErrorResponse… I would be OK with changing the error message to not include ANY of the response, since either way it is an error message. Then th

Re: Concurrency in invoker http client

2018-07-13 Thread Tyson Norris
characters, but I’m not sure how exactly to do this? Thanks Tyson > On Jun 25, 2018, at 10:32 AM, Rodric Rabbah wrote: > > the retires are only on failure to establish a connection - no other > retries should be happening iirc. > > -r > > On Mon, Jun 25, 2018 at 1:29

Re: ArtifactStore shutdown handling and shared resources

2018-06-26 Thread Tyson Norris
on ref counting 2. Do not close the pool on shutdown with assumption it should not cause much issue Chetan Mehrotra On Thu, Jun 21, 2018 at 9:06 PM, Tyson Norris mailto:tnor...@adobe.com.invalid>> wrote: I prefer not to introduce the ref counting - it is cryptic and if it is known to need

Re: Concurrency in invoker http client

2018-06-25 Thread Tyson Norris
Thanks Markus - one other question: Assuming retry is the current missing piece to using PoolingRestClient (or akka http directly), I’m also wondering if “retry” is the proper approach here? It may be worthwhile to initiate a port connection (with its own timeout/retry behavior) before the /ini

Concurrency in invoker http client

2018-06-25 Thread Tyson Norris
Hi - As part of the support for concurrency in action containers, I have this PR open in the nodejs action container: https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/41 I’ve been having real trouble getting concurrency tests to operate properly in travis env. The behavior app

Re: ArtifactStore shutdown handling and shared resources

2018-06-21 Thread Tyson Norris
I prefer not to introduce the ref counting - it is cryptic and if it is known to need replacement, these are signs that it should not go in. What is the exact affect treating shutdown the same as in couchdb store for now? (Meaning, shutdown is not really handled explicitly) Other than possibly c

  1   2   >