Cool!!!
Regards Sugandha Agrawal On Wed, Jul 3, 2019 at 4:37 PM Michele Sciabarra <mich...@sciabarra.com> wrote: > Thanks! Indeed it was my next step since I am now trying to build actions > with the actionloop based runtime with tekton. I thought to use Priti work > that has to be updated to use Tekton so your article is very welcome and > useful. > > -- > Michele Sciabarra > mich...@sciabarra.com > > ----- Original message ----- > From: Martin Henke <martin.he...@web.de> > To: dev@openwhisk.apache.org > Subject: Re: A plan to (re) implement OpenWhisk on top of Knative > Date: Wednesday, July 03, 2019 4:29 PM > > Michele, > > FYI: Sugandha and myself published a Medium blog article which describes > to build Nodejs10 images using Tekton and run it on Knative > (based on the work from Priti). > It might be on interest in the context of your Actionloop related Knative > work. > > Link: > > https://medium.com/@sugandha.agrawal18/build-knative-service-with-tekton-and-apache-openwhisk-node-js-runtime-f660bbc3a11e > > Regards, > Martin > > > On 2019/05/20 15:00:02, "Michele Sciabarra" <m...@sciabarra.com> wrote: > > Ok great, I see the discussion is starting to bring ideas.> > > > > Yes my goal is basically to run existing actions in Knative, create and > invoke. And possibile retain the ability of an action to invoke another > action.> > > > > I understand the different way they expose services, so I am rethinking > the idea of using a "work-alike" path. > > > > > If it is needed we can add it with an ingress but it may be not > necessary in the initial implementation.> > > > > Also I checked a bit ML and discussions and I see this Tekton thing that > should be the preferred way.> > > > > Not sure if I understand the relation with the current Build API > documented in the website. Is Tekton "compatible" or it has a different > API?> > > > > > > -- > > > Michele Sciabarra> > > mich...@sciabarra.com> > > > > ----- Original message -----> > > From: "Markus Thömmes" <ma...@apache.org>> > > To: dev@openwhisk.apache.org> > > Subject: Re: A plan to (re) implement OpenWhisk on top of Knative> > > Date: Monday, May 20, 2019 4:50 PM> > > > > Good discussion, thanks!> > > > > Can we try to define what the desired end-goal is here? I'm a bit > unclear> > > what resembling the OpenWhisk API actually buys us.> > > > > To me, the desired end-state would be to run OpenWhisk actions as-is on > a> > > Knative cluster (similar to OpenFaaS' and Azure's integration). There's > no> > > good way for us to provide the full API without spinning up a control > plane> > > and we can only handle so much via the CLI. So to me, the end-goal > looks> > > like:> > > > > 1. *wsk action create* actually doing all the pieces necessary to run a> > > piece of code on Knative.> > > 2. *wsk action invoke* doing some HTTP call under the hood to "invoke" > that> > > action. The action should be reachable via a sensible URL. If we really> > > want to keep the API surface (as I said, I'm dubious here) we can also > do> > > that via ingress level abstractions (like VirtualService).> > > > > Cheers,> > > Markus> > > > > Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke < > martin.he...@web.de> > > >:> > > > > >> > > > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>> > > > wrote:> > > > >> > > > >> Michele,> > > > >> > > > >> I like the idea to make the ActionLoop based runtimes to be > runnable on> > > > Knative.> > > > >>> > > > >> My thoughts on this:> > > > >> - I second Markus concern to implement the invocation API onto > Knative> > > > instead of just using Knative service syntax.> > > > > Can you elaborate this? I do not understand.> > > >> > > > Knative service syntax: https://<service(in our case => > > > action)>.<namespace>.<host>/> > > > OW invocation https://> > > > <api-host>/api/v1/namespaces/<namespace>/actions/<action>> > > >> > > > (I personally so no worth in inventing a distinct API for OW images, > but> > > > as said I would see that as a valid optional feature)> > > >> > > > >> > > > >> - I would have concerns to make it dependent on Gloo which is kind > of a> > > > minority choice for Knative load balancing> > > > > I do not think it will be hard to setup a test also using Istio, I > do> > > > not want to be limited to Gloo.> > > >> > > > I just wanted to prevent that Gloo gets a “official” prerequisite for > an> > > > “official” OW on Knative flow.> > > > It is of course free to you to use what ever you want to do in your> > > > prototype.> > > >> > > > > - In my opinion the goal should be to have some uniform behaviour > for> > > > ActionLoop based runtimes> > > > >> and other ones like the adapted NodeJS runtimes demonstrated by > Matt> > > > and Priti> > > > > As much as I can tell the current implementation is just the > building> > > > and exposing the "/init" and "/run" but I can be wrong.> > > > > The build can be of course reused, so it continues the effort. For > the> > > > frontend, from the documentation I think Matt wants to add a proxy, > while I> > > > would like to implemeent the "invocation" straight in the runtime. > This is> > > > open to discussion, but of course it is better to reach an agreement.> > > >> > > > Also in the work of Priti and Matt the invocation goes directly to > the> > > > runtime. The action code is either passed with the call (not yet > tested by> > > > me) or set via environment variable in the docker build.> > > >> > > > >> > > > >> - As Knative Build seems be on a dead end I would propose to > target> > > > Tekton as the build system (which developed as kind of >successor out > of> > > > Knative)> > > > >> > > > > If Knative build is dead then it would be a bit unfair that they > change> > > > it as the scope of the Knative project!> > > > > It looks like the goal is to setup some standards! And I would be > very> > > > disappointed to know that.> > > >> > > > Tekton evolved out of Knative Build (or more correct out of Knative> > > > Pipelines) but is very similar to the Knative build.> > > > Flows can easily be ported from one to the other,> > > > If we target Tekton build we would target the platform were the > Knative> > > > build team is focusing on.> > > > But again feel free to use whatever platform for your prototype work.> > > >> > > > > At this stage the build is the more interesting thing, and it could > be> > > > even imported in main openwhisk to speed up deployment.> > > > > I have already baked it in the ActionLoop runtimes (with > precompilation).> > > > > Also if we use Tekton, where is the Knative standard then? What is > the> > > > point? We can build our own system instead of "Knativizing" it...> > > > >> > > > >> Maybe it would be a good solution to tackle two things > independently.> > > > >> 1) Design and implement a common protocol of building, running and> > > > calling OW runtimes on Knative> > > > >> 2) Implement the OW invocation API on top of Knative as an > additional> > > > option for those who have the need to expose it.> > > > >> > > > > On this, for my personal approach at building things, I want > something> > > > that works and it is complete and useful. A "MVP”.> > > >> > > > Cool. Just go on.> > > >> > > > > So I do not plan to split the effort. Version 0.1 must be a minimal> > > > working subset of OpenWhisk on Knative.> > > > > Because otherwise there will be incomplete useless inusable pieces> > > > around (see for example kwsk).> > > > >> > > > > It does not mean that things cannot be modular, nor that everyone > must> > > > but to me "openwhisk-knative" must be a single repo with all the > pieces to> > > > make something where you can download is and deploy in a kubernetes > cluster> > > > and be able to deploy simple actions. When this works, we can improve> > > > incrementally and split it but keeping it working.> > > > >> > > > >> I would looking forward to work with you on the first work item.> > > > > Great but I see now more details to discuss before we can start. > Most> > > > notably I need to understand how I can build on top of Mark and Priti > work> > > > and continue their work. ANd I can even probably recover some of the > code> > > > of kwsk as they implemented some openwhisk api, that I want now in > the> > > > runtime.> > > > >> > > >> > > > I do not want to stop you in any way. My hope is that the action loop> > > > runtimes and the “other ones” do expose the same behaviour when being> > > > called. So that the users is not surprised when calling different > actions> > > > in different languages.> > > > And behaving the same way might also mean to adapt the “other > languages”> > > > to the same behaviour as the action loop based ones.> > > > They just should be uniform to be used.> > > >> > > > When your prototype is accessible it would be a good point of time to> > > > discuss this.> > > >> > > > As said I very much like your effort.> > > >> > > > >> > > > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>> > > > > wrote:> > > > >>> > > > >>> > > > >>>> I have an idea for implementing a prototype of OpenWhisk on top > of> > > > Knative.> > > > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:> > > > extend> > > > >>>> the runtime to support the REST call and expose them as ingress. > And> > > > use a> > > > >>>> wrapper on top of `kubectl` to generate all the needed > components.> > > > >>> > > > >>> Does this tie into the work that Matt was doing to the runtimes to > make> > > > >>> them runnable on Knative? Is this lined up with that at all?> > > > >> Actually yes. He suggested I can investigate how to migrate > ActionLoop> > > > to port many other languages to Knative.> > > > >> Also he recommended I add my idea and this is what I am doing. > Current> > > > code is, if I am not wrong, a Knative build of the nodejs runtime.> > > > >>> > > > >> There has been a number of attempts and proposal to move forward> > > > OpenWhisk. My idea that to succeed we need something small but that > just> > > > works. This is my idea to be able to implement in the shorter time > frame> > > > possible an actual subset of OpenWhisk that works and it is truly > built on> > > > top of Knative. So I am putting the thing a bit further than Matt > work.> > > > >>> > > > >>> > > > >>>> My goal is to have a functional work-alike of OpenWhisk built on > top> > > > of> > > > >>>> Knative, using ActionLoop as a foundation. I will extend > ActionLoop to> > > > >>>> support the required REST calls of OpenWhisk.> > > > >>>> > > > >>>> I also want to create tool, I will call `wskn`. This tool will> > > > initially> > > > >>>> just a python script, a wrapper on top of `kubectl` as it will> > > > generate> > > > >>>> kubernetes descriptors.> > > > >>> Why not build this into "wsk" itself? The Azure Functions CLI as > an> > > > example> > > > >>> supports multiple deployment types like this in one CLI.> > > > >>> > > > >> When it will works, yes, of course. But to start, what I really > need is> > > > a prototype that can generate kubernetes descripttors to feed to > kubectl,> > > > so a simplee, quick and ditry, separate tool (that I will keep > together> > > > the runtime) is all I need for now.> > > > >>> > > > >>>>> > > > >>>> It will support initially just the the action creation and> > > > invocation, and> > > > >>>> only synchronous (blocking) behaviour, as all the request will > go> > > > straight> > > > >>>> to the runtimes. Hopefully also a subset of `package` and> > > > `activation`.> > > > >>>> Again triggers, rules, asynchronous for later.> > > > >>>>> > > > >>>> The idea is that you will be able to create actions and web > actions> > > > that> > > > >>>> can run existing OpenWhisk actions, at least those with blocking> > > > behaviour> > > > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust, > Ruby,> > > > >>>> Crystal...)> > > > >>>>> > > > >>>> Implementation.> > > > >>>> ==============> > > > >>>>> > > > >>>> This is how I plan to implement it.> > > > >>>>> > > > >>>> At this stage I want to use just Knative Serving and Knative > Build,> > > > using> > > > >>>> Gloo for the ingress part. I also plan to install a local Docker> > > > registry> > > > >>>> Kubernetes registry, so we do not have to use DockerHub for> > > > everything. All> > > > >>>> of this can be done with existing command line tools in a few > minutes> > > > in> > > > >>>> any running Kubernetes deployment.> > > > >>>>> > > > >>> > > > >>> Why specifying Gloo here? Do you need anything specific from Gloo> > > > itself?> > > > >>> If not I'd propose to just keep it on a Knative Serving API > surface> > > > level.> > > > >> I want to build it on top of Knative serving, full stop. > Currently,> > > > installing Gloo is pretty easy and is more lightweight than Istio so > I> > > > will use it for my first implementation.> > > > >>> > > > >>>>> > > > >>>> When I create an action, it will use Knative build that will > work> > > > roughly> > > > >>>> in this way:> > > > >>>>> > > > >>>> - create a configmap with the action code> > > > >>>> - build the actin using ActionLoop precompilation feature that > will> > > > return> > > > >>>> a zip file including all the needed to run the action> > > > >>>> - create a new docker image extending the runtime with the new > zip,> > > > using> > > > >>>> Kaanico> > > > >>>> - push the image in the local registry> > > > >>> This feels like a fairly heavyweight process, we should be able > to> > > > come up> > > > >>> with a way to circumvent zipping entirely. Maybe the runtime can > detect> > > > >>> that the unzipped content is already there and skip the unzip > step?> > > > >>> > > > >> Actually this is my first idea of how to use Knative build. And is > not> > > > complicated: when I create the action, a run a build that includes > Kanico.> > > > I generate a Dockerfile on the fly. The docker file uses the action > runtime> > > > that already know how to compile a script. And then I save an image. > I> > > > already implemented un "autoinit" so just launching the image will > give a> > > > runtime ready to run that execute an action already compiled.> > > > >>> > > > >>> > > > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the > action> > > > >>> code. It's all stored in the in-cluster etcd instance and it has > a> > > > limit of> > > > >>> 1M. This is at most a stop-gap solution to provide a PoC I think. > Any> > > > ideas> > > > >>> on how to "productize" this?> > > > >>> > > > >> ConfigMap can be mounted as files, so it is an easy way to feed > an> > > > action to a build. It is just an easy way to feed the action code to > the> > > > Build.> > > > >>> > > > >> My initial constraint is that I want just to generate Kubernetes> > > > descriptors to feed to kubectl.> > > > >> Of course in the long run I can add some "file upload" storage.> > > > >>> > > > >> If I could to this file upload when invoking a build it could ideal > as> > > > I do not have to store anything anywhere, just process the code and> > > > generate a single layer to execute actions to be store in the > registry.> > > > >> I will investigate better this area, I understand your concern.> > > > >>> > > > >>>> > > > >>>> At this point you can run the action. ActionLoop will be extended > to> > > > >>>> support invocations in the format> > > > >>>> "/v1/namespaces/namespace/actions/package/action".> > > > >>> Why bother reimplementing this exact path? To obtain API > compatibility> > > > with> > > > >>> OpenWhisk as it is today?> > > > >>> > > > >> I want to implement a subset of the OpenWhisk API on top of > Knative> > > > serving.> > > > >> Knative serving already does the scaling and routing, so what we > need> > > > are the "endpoints" to invoke actions.> > > > >>> > > > >> Since I do not want to add additional components, not at the first> > > > stage. Just knative serve and build, the runtime and a controller > script,> > > > the runtime is the natural place where to "handle" the API > invocations,> > > > since Knative only generates the URL but not anything else. If I> > > > understood well, Matt is adding a proxy. I do not want to add a proxy, > just> > > > add to the runtime the ability to respond to "API like" calls, at > least> > > > those regarding action invocation.> > > > >>> > > > >>>> It will do all the decoding required to invoke the action with > the> > > > >>>> expected paramenters (straight invocation thrhoug the actinloop> > > > protocol,> > > > >>>> not proxies).> > > > >>> Does this mean moving all of the Controller's "smartness" about> > > > incoming> > > > >>> and outgoing HTTP requests (see the whole WebActions for > example)?> > > > >>> > > > >> At least decoding web actions in the runtime, yes. Knative serving> > > > already has routing and proxying.> > > > >> So a true implementation on top of Knative requires IHMO this> > > > sacrifice. Unless there is a way to keep the controller in a > "Knative"> > > > compatible way. Open to suggestions here.> > > > >>> > > > >>> Each action will then be exposed using an ingress with its > specific> > > > >>> invocation path.> > > > >>>> > > > >>> If the community agrees with this plan, I would create a repo> > > > >>> `incubator-openwhisk-knative` to work on it.> > > > >>>> > > > >>> Thoughts?> > > >> > > >> > > >
