Hi Matt, I fully agree with your thoughts. We should definitely (stepwise) aim for an unified model for all runtimes.
Regards, Martin > On 3. Jul 2019, at 16:43, Matt Rutkowski <mrutk...@us.ibm.com> wrote: > > Hi Martin, > > It is my belief that Michele, now freshly returned from book editing > (congrats), was going to implement the same interface for pre/post > processing requests that we implemented for NodeJS. This would allow us a > path to support this across all language runtimes as well as formalize our > runtime contract that aligns with a Knative Service approach and then also > allows us to better explore some of the use cases being discussed on > another thread from Rodric. I believe that we should look at smaller > steps towards that alignment like removing the need for the "init" > entrypoint and allowing access to parameters either by env. vars. or > function arguments allowing both compat. with 12-factor app approach, as > well as attempting to encourage a functional programming model where you > should ideally be unaware of any system environment. > > Kind regards, > Matt > > > > > From: Martin Henke <martin.he...@web.de> > To: dev@openwhisk.apache.org > Date: 07/03/2019 09:29 AM > Subject: [EXTERNAL] Re: A plan to (re) implement OpenWhisk on top > of Knative > > > > Michele, > > FYI: Sugandha and myself published a Medium blog article which describes > to build Nodejs10 images using Tekton and run it on Knative > (based on the work from Priti). > It might be on interest in the context of your Actionloop related Knative > work. > > Link: > https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40sugandha.agrawal18_build-2Dknative-2Dservice-2Dwith-2Dtekton-2Dand-2Dapache-2Dopenwhisk-2Dnode-2Djs-2Druntime-2Df660bbc3a11e&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=6zQLM7Gc0Sv1iwayKOKa4_SFxRIxS478q2gZlAJj4Zw&m=bl7bwPaoiSE6fi0fyVHNC9bNUnh1ZUUlfl3lwUZbcH0&s=IQBGPfSTPiAjKhIlsqKDpAjoJBNd7By1YnjxOIx2454&e= > > > > Regards, > Martin > > > On 2019/05/20 15:00:02, "Michele Sciabarra" <m...@sciabarra.com> wrote: >> Ok great, I see the discussion is starting to bring ideas.> >> >> Yes my goal is basically to run existing actions in Knative, create and > invoke. And possibile retain the ability of an action to invoke another > action.> >> >> I understand the different way they expose services, so I am rethinking > the idea of using a "work-alike" path. > >> >> If it is needed we can add it with an ingress but it may be not > necessary in the initial implementation.> >> >> Also I checked a bit ML and discussions and I see this Tekton thing that > should be the preferred way.> >> >> Not sure if I understand the relation with the current Build API > documented in the website. Is Tekton "compatible" or it has a different > API?> >> >> >> -- > >> Michele Sciabarra> >> mich...@sciabarra.com> >> >> ----- Original message -----> >> From: "Markus Thömmes" <ma...@apache.org>> >> To: dev@openwhisk.apache.org> >> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative> >> Date: Monday, May 20, 2019 4:50 PM> >> >> Good discussion, thanks!> >> >> Can we try to define what the desired end-goal is here? I'm a bit > unclear> >> what resembling the OpenWhisk API actually buys us.> >> >> To me, the desired end-state would be to run OpenWhisk actions as-is on > a> >> Knative cluster (similar to OpenFaaS' and Azure's integration). There's > no> >> good way for us to provide the full API without spinning up a control > plane> >> and we can only handle so much via the CLI. So to me, the end-goal > looks> >> like:> >> >> 1. *wsk action create* actually doing all the pieces necessary to run a> > >> piece of code on Knative.> >> 2. *wsk action invoke* doing some HTTP call under the hood to "invoke" > that> >> action. The action should be reachable via a sensible URL. If we really> > >> want to keep the API surface (as I said, I'm dubious here) we can also > do> >> that via ingress level abstractions (like VirtualService).> >> >> Cheers,> >> Markus> >> >> Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke > <martin.he...@web.de> >>> :> >> >>>> >>>> On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>> >>> wrote:> >>>>> >>>>> Michele,> >>>>> >>>>> I like the idea to make the ActionLoop based runtimes to be > runnable on> >>> Knative.> >>>>>> >>>>> My thoughts on this:> >>>>> - I second Markus concern to implement the invocation API onto > Knative> >>> instead of just using Knative service syntax.> >>>> Can you elaborate this? I do not understand.> >>>> >>> Knative service syntax: https://<service(in our case => >>> action)>.<namespace>.<host>/> >>> OW invocation https://> >>> <api-host>/api/v1/namespaces/<namespace>/actions/<action>> >>>> >>> (I personally so no worth in inventing a distinct API for OW images, > but> >>> as said I would see that as a valid optional feature)> >>>> >>>>> >>>>> - I would have concerns to make it dependent on Gloo which is kind > of a> >>> minority choice for Knative load balancing> >>>> I do not think it will be hard to setup a test also using Istio, I > do> >>> not want to be limited to Gloo.> >>>> >>> I just wanted to prevent that Gloo gets a “official” prerequisite for > an> >>> “official” OW on Knative flow.> >>> It is of course free to you to use what ever you want to do in your> >>> prototype.> >>>> >>>> - In my opinion the goal should be to have some uniform behaviour > for> >>> ActionLoop based runtimes> >>>>> and other ones like the adapted NodeJS runtimes demonstrated by > Matt> >>> and Priti> >>>> As much as I can tell the current implementation is just the > building> >>> and exposing the "/init" and "/run" but I can be wrong.> >>>> The build can be of course reused, so it continues the effort. For > the> >>> frontend, from the documentation I think Matt wants to add a proxy, > while I> >>> would like to implemeent the "invocation" straight in the runtime. > This is> >>> open to discussion, but of course it is better to reach an agreement.> > >>>> >>> Also in the work of Priti and Matt the invocation goes directly to > the> >>> runtime. The action code is either passed with the call (not yet > tested by> >>> me) or set via environment variable in the docker build.> >>>> >>>>> >>>>> - As Knative Build seems be on a dead end I would propose to > target> >>> Tekton as the build system (which developed as kind of >successor out > of> >>> Knative)> >>>>> >>>> If Knative build is dead then it would be a bit unfair that they > change> >>> it as the scope of the Knative project!> >>>> It looks like the goal is to setup some standards! And I would be > very> >>> disappointed to know that.> >>>> >>> Tekton evolved out of Knative Build (or more correct out of Knative> >>> Pipelines) but is very similar to the Knative build.> >>> Flows can easily be ported from one to the other,> >>> If we target Tekton build we would target the platform were the > Knative> >>> build team is focusing on.> >>> But again feel free to use whatever platform for your prototype work.> > >>>> >>>> At this stage the build is the more interesting thing, and it could > be> >>> even imported in main openwhisk to speed up deployment.> >>>> I have already baked it in the ActionLoop runtimes (with > precompilation).> >>>> Also if we use Tekton, where is the Knative standard then? What is > the> >>> point? We can build our own system instead of "Knativizing" it...> >>>>> >>>>> Maybe it would be a good solution to tackle two things > independently.> >>>>> 1) Design and implement a common protocol of building, running and> > >>> calling OW runtimes on Knative> >>>>> 2) Implement the OW invocation API on top of Knative as an > additional> >>> option for those who have the need to expose it.> >>>>> >>>> On this, for my personal approach at building things, I want > something> >>> that works and it is complete and useful. A "MVP”.> >>>> >>> Cool. Just go on.> >>>> >>>> So I do not plan to split the effort. Version 0.1 must be a minimal> > >>> working subset of OpenWhisk on Knative.> >>>> Because otherwise there will be incomplete useless inusable pieces> >>> around (see for example kwsk).> >>>>> >>>> It does not mean that things cannot be modular, nor that everyone > must> >>> but to me "openwhisk-knative" must be a single repo with all the > pieces to> >>> make something where you can download is and deploy in a kubernetes > cluster> >>> and be able to deploy simple actions. When this works, we can improve> > >>> incrementally and split it but keeping it working.> >>>>> >>>>> I would looking forward to work with you on the first work item.> >>>> Great but I see now more details to discuss before we can start. > Most> >>> notably I need to understand how I can build on top of Mark and Priti > work> >>> and continue their work. ANd I can even probably recover some of the > code> >>> of kwsk as they implemented some openwhisk api, that I want now in > the> >>> runtime.> >>>>> >>>> >>> I do not want to stop you in any way. My hope is that the action loop> > >>> runtimes and the “other ones” do expose the same behaviour when being> > >>> called. So that the users is not surprised when calling different > actions> >>> in different languages.> >>> And behaving the same way might also mean to adapt the “other > languages”> >>> to the same behaviour as the action loop based ones.> >>> They just should be uniform to be used.> >>>> >>> When your prototype is accessible it would be a good point of time to> > >>> discuss this.> >>>> >>> As said I very much like your effort.> >>>> >>>>> >>>>> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>> > >>> wrote:> >>>>>> >>>>>> >>>>>>> I have an idea for implementing a prototype of OpenWhisk on top > of> >>> Knative.> >>>>>>> My basic ideas are: do not use any proxy, forwarding or adapter:> > >>> extend> >>>>>>> the runtime to support the REST call and expose them as ingress. > And> >>> use a> >>>>>>> wrapper on top of `kubectl` to generate all the needed > components.> >>>>>> >>>>>> Does this tie into the work that Matt was doing to the runtimes to > make> >>>>>> them runnable on Knative? Is this lined up with that at all?> >>>>> Actually yes. He suggested I can investigate how to migrate > ActionLoop> >>> to port many other languages to Knative.> >>>>> Also he recommended I add my idea and this is what I am doing. > Current> >>> code is, if I am not wrong, a Knative build of the nodejs runtime.> >>>>>> >>>>> There has been a number of attempts and proposal to move forward> >>> OpenWhisk. My idea that to succeed we need something small but that > just> >>> works. This is my idea to be able to implement in the shorter time > frame> >>> possible an actual subset of OpenWhisk that works and it is truly > built on> >>> top of Knative. So I am putting the thing a bit further than Matt > work.> >>>>>> >>>>>> >>>>>>> My goal is to have a functional work-alike of OpenWhisk built on > top> >>> of> >>>>>>> Knative, using ActionLoop as a foundation. I will extend > ActionLoop to> >>>>>>> support the required REST calls of OpenWhisk.> >>>>>>> >>>>>>> I also want to create tool, I will call `wskn`. This tool will> >>> initially> >>>>>>> just a python script, a wrapper on top of `kubectl` as it will> >>> generate> >>>>>>> kubernetes descriptors.> >>>>>> Why not build this into "wsk" itself? The Azure Functions CLI as > an> >>> example> >>>>>> supports multiple deployment types like this in one CLI.> >>>>>> >>>>> When it will works, yes, of course. But to start, what I really > need is> >>> a prototype that can generate kubernetes descripttors to feed to > kubectl,> >>> so a simplee, quick and ditry, separate tool (that I will keep > together> >>> the runtime) is all I need for now.> >>>>>> >>>>>>>> >>>>>>> It will support initially just the the action creation and> >>> invocation, and> >>>>>>> only synchronous (blocking) behaviour, as all the request will > go> >>> straight> >>>>>>> to the runtimes. Hopefully also a subset of `package` and> >>> `activation`.> >>>>>>> Again triggers, rules, asynchronous for later.> >>>>>>>> >>>>>>> The idea is that you will be able to create actions and web > actions> >>> that> >>>>>>> can run existing OpenWhisk actions, at least those with blocking> > >>> behaviour> >>>>>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust, > Ruby,> >>>>>>> Crystal...)> >>>>>>>> >>>>>>> Implementation.> >>>>>>> ==============> >>>>>>>> >>>>>>> This is how I plan to implement it.> >>>>>>>> >>>>>>> At this stage I want to use just Knative Serving and Knative > Build,> >>> using> >>>>>>> Gloo for the ingress part. I also plan to install a local Docker> > >>> registry> >>>>>>> Kubernetes registry, so we do not have to use DockerHub for> >>> everything. All> >>>>>>> of this can be done with existing command line tools in a few > minutes> >>> in> >>>>>>> any running Kubernetes deployment.> >>>>>>>> >>>>>> >>>>>> Why specifying Gloo here? Do you need anything specific from Gloo> > >>> itself?> >>>>>> If not I'd propose to just keep it on a Knative Serving API > surface> >>> level.> >>>>> I want to build it on top of Knative serving, full stop. > Currently,> >>> installing Gloo is pretty easy and is more lightweight than Istio so > I> >>> will use it for my first implementation.> >>>>>> >>>>>>>> >>>>>>> When I create an action, it will use Knative build that will > work> >>> roughly> >>>>>>> in this way:> >>>>>>>> >>>>>>> - create a configmap with the action code> >>>>>>> - build the actin using ActionLoop precompilation feature that > will> >>> return> >>>>>>> a zip file including all the needed to run the action> >>>>>>> - create a new docker image extending the runtime with the new > zip,> >>> using> >>>>>>> Kaanico> >>>>>>> - push the image in the local registry> >>>>>> This feels like a fairly heavyweight process, we should be able > to> >>> come up> >>>>>> with a way to circumvent zipping entirely. Maybe the runtime can > detect> >>>>>> that the unzipped content is already there and skip the unzip > step?> >>>>>> >>>>> Actually this is my first idea of how to use Knative build. And is > not> >>> complicated: when I create the action, a run a build that includes > Kanico.> >>> I generate a Dockerfile on the fly. The docker file uses the action > runtime> >>> that already know how to compile a script. And then I save an image. > I> >>> already implemented un "autoinit" so just launching the image will > give a> >>> runtime ready to run that execute an action already compiled.> >>>>>> >>>>>> >>>>>> I'm fairly hesitant on the usage of a ConfigMap for storing the > action> >>>>>> code. It's all stored in the in-cluster etcd instance and it has > a> >>> limit of> >>>>>> 1M. This is at most a stop-gap solution to provide a PoC I think. > Any> >>> ideas> >>>>>> on how to "productize" this?> >>>>>> >>>>> ConfigMap can be mounted as files, so it is an easy way to feed > an> >>> action to a build. It is just an easy way to feed the action code to > the> >>> Build.> >>>>>> >>>>> My initial constraint is that I want just to generate Kubernetes> >>> descriptors to feed to kubectl.> >>>>> Of course in the long run I can add some "file upload" storage.> >>>>>> >>>>> If I could to this file upload when invoking a build it could ideal > as> >>> I do not have to store anything anywhere, just process the code and> >>> generate a single layer to execute actions to be store in the > registry.> >>>>> I will investigate better this area, I understand your concern.> >>>>>> >>>>>>> >>>>>>> At this point you can run the action. ActionLoop will be extended > to> >>>>>>> support invocations in the format> >>>>>>> "/v1/namespaces/namespace/actions/package/action".> >>>>>> Why bother reimplementing this exact path? To obtain API > compatibility> >>> with> >>>>>> OpenWhisk as it is today?> >>>>>> >>>>> I want to implement a subset of the OpenWhisk API on top of > Knative> >>> serving.> >>>>> Knative serving already does the scaling and routing, so what we > need> >>> are the "endpoints" to invoke actions.> >>>>>> >>>>> Since I do not want to add additional components, not at the first> > >>> stage. Just knative serve and build, the runtime and a controller > script,> >>> the runtime is the natural place where to "handle" the API > invocations,> >>> since Knative only generates the URL but not anything else. If I> >>> understood well, Matt is adding a proxy. I do not want to add a proxy, > just> >>> add to the runtime the ability to respond to "API like" calls, at > least> >>> those regarding action invocation.> >>>>>> >>>>>>> It will do all the decoding required to invoke the action with > the> >>>>>>> expected paramenters (straight invocation thrhoug the actinloop> >>> protocol,> >>>>>>> not proxies).> >>>>>> Does this mean moving all of the Controller's "smartness" about> >>> incoming> >>>>>> and outgoing HTTP requests (see the whole WebActions for > example)?> >>>>>> >>>>> At least decoding web actions in the runtime, yes. Knative serving> > >>> already has routing and proxying.> >>>>> So a true implementation on top of Knative requires IHMO this> >>> sacrifice. Unless there is a way to keep the controller in a > "Knative"> >>> compatible way. Open to suggestions here.> >>>>>> >>>>>> Each action will then be exposed using an ingress with its > specific> >>>>>> invocation path.> >>>>>>> >>>>>> If the community agrees with this plan, I would create a repo> >>>>>> `incubator-openwhisk-knative` to work on it.> >>>>>>> >>>>>> Thoughts?> >>>> >>>> >> > > > >
