Tekton is different but very very similar. You get a lot of DejaVus. No big learning Curve.
Martin > On 20. May 2019, at 17:00, Michele Sciabarra <mich...@sciabarra.com> wrote: > > Ok great, I see the discussion is starting to bring ideas. > > Yes my goal is basically to run existing actions in Knative, create and > invoke. And possibile retain the ability of an action to invoke another > action. > > I understand the different way they expose services, so I am rethinking the > idea of using a "work-alike" path. > > If it is needed we can add it with an ingress but it may be not necessary in > the initial implementation. > > Also I checked a bit ML and discussions and I see this Tekton thing that > should be the preferred way. > > Not sure if I understand the relation with the current Build API documented > in the website. Is Tekton "compatible" or it has a different API? > > > -- > Michele Sciabarra > mich...@sciabarra.com > > ----- Original message ----- > From: "Markus Thömmes" <markusthoem...@apache.org> > To: dev@openwhisk.apache.org > Subject: Re: A plan to (re) implement OpenWhisk on top of Knative > Date: Monday, May 20, 2019 4:50 PM > > Good discussion, thanks! > > Can we try to define what the desired end-goal is here? I'm a bit unclear > what resembling the OpenWhisk API actually buys us. > > To me, the desired end-state would be to run OpenWhisk actions as-is on a > Knative cluster (similar to OpenFaaS' and Azure's integration). There's no > good way for us to provide the full API without spinning up a control plane > and we can only handle so much via the CLI. So to me, the end-goal looks > like: > > 1. *wsk action create* actually doing all the pieces necessary to run a > piece of code on Knative. > 2. *wsk action invoke* doing some HTTP call under the hood to "invoke" that > action. The action should be reachable via a sensible URL. If we really > want to keep the API surface (as I said, I'm dubious here) we can also do > that via ingress level abstractions (like VirtualService). > > Cheers, > Markus > > Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke <martin.he...@web.de >> : > >> >>> On 20. May 2019, at 14:55, Michele Sciabarra <mich...@sciabarra.com> >> wrote: >>> >>>> Michele, >>> >>>> I like the idea to make the ActionLoop based runtimes to be runnable on >> Knative. >>>> >>>> My thoughts on this: >>>> - I second Markus concern to implement the invocation API onto Knative >> instead of just using Knative service syntax. >>> Can you elaborate this? I do not understand. >> >> Knative service syntax: https://<service(in our case = >> action)>.<namespace>.<host>/ >> OW invocation https:// >> <api-host>/api/v1/namespaces/<namespace>/actions/<action> >> >> (I personally so no worth in inventing a distinct API for OW images, but >> as said I would see that as a valid optional feature) >> >>> >>>> - I would have concerns to make it dependent on Gloo which is kind of a >> minority choice for Knative load balancing >>> I do not think it will be hard to setup a test also using Istio, I do >> not want to be limited to Gloo. >> >> I just wanted to prevent that Gloo gets a “official” prerequisite for an >> “official” OW on Knative flow. >> It is of course free to you to use what ever you want to do in your >> prototype. >> >>> - In my opinion the goal should be to have some uniform behaviour for >> ActionLoop based runtimes >>>> and other ones like the adapted NodeJS runtimes demonstrated by Matt >> and Priti >>> As much as I can tell the current implementation is just the building >> and exposing the "/init" and "/run" but I can be wrong. >>> The build can be of course reused, so it continues the effort. For the >> frontend, from the documentation I think Matt wants to add a proxy, while I >> would like to implemeent the "invocation" straight in the runtime. This is >> open to discussion, but of course it is better to reach an agreement. >> >> Also in the work of Priti and Matt the invocation goes directly to the >> runtime. The action code is either passed with the call (not yet tested by >> me) or set via environment variable in the docker build. >> >>> >>>> - As Knative Build seems be on a dead end I would propose to target >> Tekton as the build system (which developed as kind of >successor out of >> Knative) >>> >>> If Knative build is dead then it would be a bit unfair that they change >> it as the scope of the Knative project! >>> It looks like the goal is to setup some standards! And I would be very >> disappointed to know that. >> >> Tekton evolved out of Knative Build (or more correct out of Knative >> Pipelines) but is very similar to the Knative build. >> Flows can easily be ported from one to the other, >> If we target Tekton build we would target the platform were the Knative >> build team is focusing on. >> But again feel free to use whatever platform for your prototype work. >> >>> At this stage the build is the more interesting thing, and it could be >> even imported in main openwhisk to speed up deployment. >>> I have already baked it in the ActionLoop runtimes (with precompilation). >>> Also if we use Tekton, where is the Knative standard then? What is the >> point? We can build our own system instead of "Knativizing" it... >>> >>>> Maybe it would be a good solution to tackle two things independently. >>>> 1) Design and implement a common protocol of building, running and >> calling OW runtimes on Knative >>>> 2) Implement the OW invocation API on top of Knative as an additional >> option for those who have the need to expose it. >>> >>> On this, for my personal approach at building things, I want something >> that works and it is complete and useful. A "MVP”. >> >> Cool. Just go on. >> >>> So I do not plan to split the effort. Version 0.1 must be a minimal >> working subset of OpenWhisk on Knative. >>> Because otherwise there will be incomplete useless inusable pieces >> around (see for example kwsk). >>> >>> It does not mean that things cannot be modular, nor that everyone must >> but to me "openwhisk-knative" must be a single repo with all the pieces to >> make something where you can download is and deploy in a kubernetes cluster >> and be able to deploy simple actions. When this works, we can improve >> incrementally and split it but keeping it working. >>> >>>> I would looking forward to work with you on the first work item. >>> Great but I see now more details to discuss before we can start. Most >> notably I need to understand how I can build on top of Mark and Priti work >> and continue their work. ANd I can even probably recover some of the code >> of kwsk as they implemented some openwhisk api, that I want now in the >> runtime. >>> >> >> I do not want to stop you in any way. My hope is that the action loop >> runtimes and the “other ones” do expose the same behaviour when being >> called. So that the users is not surprised when calling different actions >> in different languages. >> And behaving the same way might also mean to adapt the “other languages” >> to the same behaviour as the action loop based ones. >> They just should be uniform to be used. >> >> When your prototype is accessible it would be a good point of time to >> discuss this. >> >> As said I very much like your effort. >> >>> >>>> On 20. May 2019, at 08:55, Michele Sciabarra <mich...@sciabarra.com> >> wrote: >>>> >>>> >>>>>> I have an idea for implementing a prototype of OpenWhisk on top of >> Knative. >>>>>> My basic ideas are: do not use any proxy, forwarding or adapter: >> extend >>>>>> the runtime to support the REST call and expose them as ingress. And >> use a >>>>>> wrapper on top of `kubectl` to generate all the needed components. >>>> >>>>> Does this tie into the work that Matt was doing to the runtimes to make >>>>> them runnable on Knative? Is this lined up with that at all? >>>> Actually yes. He suggested I can investigate how to migrate ActionLoop >> to port many other languages to Knative. >>>> Also he recommended I add my idea and this is what I am doing. Current >> code is, if I am not wrong, a Knative build of the nodejs runtime. >>>> >>>> There has been a number of attempts and proposal to move forward >> OpenWhisk. My idea that to succeed we need something small but that just >> works. This is my idea to be able to implement in the shorter time frame >> possible an actual subset of OpenWhisk that works and it is truly built on >> top of Knative. So I am putting the thing a bit further than Matt work. >>>> >>>> >>>>>> My goal is to have a functional work-alike of OpenWhisk built on top >> of >>>>>> Knative, using ActionLoop as a foundation. I will extend ActionLoop to >>>>>> support the required REST calls of OpenWhisk. >>>>> >>>>>> I also want to create tool, I will call `wskn`. This tool will >> initially >>>>>> just a python script, a wrapper on top of `kubectl` as it will >> generate >>>>>> kubernetes descriptors. >>>>> Why not build this into "wsk" itself? The Azure Functions CLI as an >> example >>>>> supports multiple deployment types like this in one CLI. >>>> >>>> When it will works, yes, of course. But to start, what I really need is >> a prototype that can generate kubernetes descripttors to feed to kubectl, >> so a simplee, quick and ditry, separate tool (that I will keep together >> the runtime) is all I need for now. >>>> >>>>>> >>>>>> It will support initially just the the action creation and >> invocation, and >>>>>> only synchronous (blocking) behaviour, as all the request will go >> straight >>>>>> to the runtimes. Hopefully also a subset of `package` and >> `activation`. >>>>>> Again triggers, rules, asynchronous for later. >>>>>> >>>>>> The idea is that you will be able to create actions and web actions >> that >>>>>> can run existing OpenWhisk actions, at least those with blocking >> behaviour >>>>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust, Ruby, >>>>>> Crystal...) >>>>>> >>>>>> Implementation. >>>>>> ============== >>>>>> >>>>>> This is how I plan to implement it. >>>>>> >>>>>> At this stage I want to use just Knative Serving and Knative Build, >> using >>>>>> Gloo for the ingress part. I also plan to install a local Docker >> registry >>>>>> Kubernetes registry, so we do not have to use DockerHub for >> everything. All >>>>>> of this can be done with existing command line tools in a few minutes >> in >>>>>> any running Kubernetes deployment. >>>>>> >>>> >>>>> Why specifying Gloo here? Do you need anything specific from Gloo >> itself? >>>>> If not I'd propose to just keep it on a Knative Serving API surface >> level. >>>> I want to build it on top of Knative serving, full stop. Currently, >> installing Gloo is pretty easy and is more lightweight than Istio so I >> will use it for my first implementation. >>>> >>>>>> >>>>>> When I create an action, it will use Knative build that will work >> roughly >>>>>> in this way: >>>>>> >>>>>> - create a configmap with the action code >>>>>> - build the actin using ActionLoop precompilation feature that will >> return >>>>>> a zip file including all the needed to run the action >>>>>> - create a new docker image extending the runtime with the new zip, >> using >>>>>> Kaanico >>>>>> - push the image in the local registry >>>>> This feels like a fairly heavyweight process, we should be able to >> come up >>>>> with a way to circumvent zipping entirely. Maybe the runtime can detect >>>>> that the unzipped content is already there and skip the unzip step? >>>> >>>> Actually this is my first idea of how to use Knative build. And is not >> complicated: when I create the action, a run a build that includes Kanico. >> I generate a Dockerfile on the fly. The docker file uses the action runtime >> that already know how to compile a script. And then I save an image. I >> already implemented un "autoinit" so just launching the image will give a >> runtime ready to run that execute an action already compiled. >>>> >>>> >>>>> I'm fairly hesitant on the usage of a ConfigMap for storing the action >>>>> code. It's all stored in the in-cluster etcd instance and it has a >> limit of >>>>> 1M. This is at most a stop-gap solution to provide a PoC I think. Any >> ideas >>>>> on how to "productize" this? >>>> >>>> ConfigMap can be mounted as files, so it is an easy way to feed an >> action to a build. It is just an easy way to feed the action code to the >> Build. >>>> >>>> My initial constraint is that I want just to generate Kubernetes >> descriptors to feed to kubectl. >>>> Of course in the long run I can add some "file upload" storage. >>>> >>>> If I could to this file upload when invoking a build it could ideal as >> I do not have to store anything anywhere, just process the code and >> generate a single layer to execute actions to be store in the registry. >>>> I will investigate better this area, I understand your concern. >>>> >>>>> >>>>>> At this point you can run the action. ActionLoop will be extended to >>>>>> support invocations in the format >>>>>> "/v1/namespaces/namespace/actions/package/action". >>>>> Why bother reimplementing this exact path? To obtain API compatibility >> with >>>>> OpenWhisk as it is today? >>>> >>>> I want to implement a subset of the OpenWhisk API on top of Knative >> serving. >>>> Knative serving already does the scaling and routing, so what we need >> are the "endpoints" to invoke actions. >>>> >>>> Since I do not want to add additional components, not at the first >> stage. Just knative serve and build, the runtime and a controller script, >> the runtime is the natural place where to "handle" the API invocations, >> since Knative only generates the URL but not anything else. If I >> understood well, Matt is adding a proxy. I do not want to add a proxy, just >> add to the runtime the ability to respond to "API like" calls, at least >> those regarding action invocation. >>>> >>>>>> It will do all the decoding required to invoke the action with the >>>>>> expected paramenters (straight invocation thrhoug the actinloop >> protocol, >>>>>> not proxies). >>>>> Does this mean moving all of the Controller's "smartness" about >> incoming >>>>> and outgoing HTTP requests (see the whole WebActions for example)? >>>> >>>> At least decoding web actions in the runtime, yes. Knative serving >> already has routing and proxying. >>>> So a true implementation on top of Knative requires IHMO this >> sacrifice. Unless there is a way to keep the controller in a "Knative" >> compatible way. Open to suggestions here. >>>> >>>>> Each action will then be exposed using an ingress with its specific >>>>> invocation path. >>>>> >>>>> If the community agrees with this plan, I would create a repo >>>>> `incubator-openwhisk-knative` to work on it. >>>>> >>>>> Thoughts? >> >>
