Hi Markus, Thanks! Yes, the design is based on you've put out a few weeks ago. The biggest difference is there's no ContainerRouter, instead, implement in LoadBalancer impl. The terminology I've changed for shorter Scala-based project naming convention, but basically follow up your idea. On the detailed protocols and design, I've tried my own way, and because there's a deadline in gsoc schedule, that I choose the way I'm more comfortable and can only afford subset of overall design. i.e. use akka message passing instead of http, no shared queues, no kube, reuse invoker-agent for pause/unpause, no error path, no log and may not race-free, etc. I don't expect to reuse anything in this implementation, hope this won't lead to misunderstanding around folks.
Thanks, Tzu-Chiao On Wed, Aug 29, 2018 at 5:00 PM Markus Thömmes <[email protected]> wrote: > Hi Tzu-Chiao, > > first of all: This looks amazing! As the terminology is similar: Is this a > separate design or is this based on what I've put out a few weeks ago? > > In general: Everybody is eligible to join the prototyping effort, I think > that's the whole point of having that discussion here. You're very welcome > to join there and it seems you'll even be more knowledgeable on some of the > things than anybody else, so it'd be awesome to get your experience in > there! > > Cheers, > Markus > > Am Mi., 29. Aug. 2018 um 10:48 Uhr schrieb TzuChiao Yeh < > [email protected]>: > > > Hi, > > > > I'm wondering if someone who pay interests aside from core teams (like > me) > > are eligible to join prototyping? I know it will increase the maintenance > > effort on managing all things being visible: opening and labeling issues, > > plans, assignees, etc. If yes, from my point of view, a standalone repo > > with issue tracker and pull requests system might be easier to achieve > the > > visibility and communication. Will it be possible to use submodule if > > there's a code-reuse or clarifying migration path concerned? > > > > For prototyping: > > I've drafted a small prototype [1] during last months. Bare me to > introduce > > some background about me: some folks might know that, I'm a master > student > > and have attended google summer of code this year with openwhisk, > mentored > > by Rodric and Carlos. The prototype is the submitted result for final > work > > during last few months, although it's buggy, ill-designed and > not-finished > > yet. However, if it's ready to start prototyping now, bare me to showoff > > this under a poor state, as the purpose of my past work, hope this help > to > > explore ideas before going to deeper and collaboration. (I've asked most > of > > issues I've found in previous threads and may be solved in mind between > > folks however.) > > > > [1] https://tz70s.github.io/posts/openwhisk-performance-improvement/ > > > > Thanks, > > Tzu-Chiao > > > > On Wed, Aug 29, 2018 at 11:31 AM Markus Thömmes < > [email protected] > > > > > wrote: > > > > > Hi, > > > > > > okay, let's separate concerns: > > > > > > ### Golang vs. Scala vs. something else entirely: ### > > > > > > I'm okay doing it in Scala although I don't see an issue in swapping > the > > > language for a component that needs to be rewritten from the ground up > > > anyway (which I'm suggesting at least the ContainerRouter is). > > > > > > The ContainerPool/ContainerProxy part of it comes to mind immediately. > It > > > was built with a scale of maybe 100 containers max in mind. There are a > > few > > > performance problems with it at scale, that I can point out off the > bat. > > > The changes needed to mitigate those are rather severe and while we > > **can** > > > implement those (and maybe even should if this proofs to make sense) I > > > believe it makes sense to at least experiment with a freshly written > > > component. In the new architecture, we can also greatly reduce the > state > > > model to just paused/unpaused for the ContainerRouter, further > > simplifying > > > its implementation. > > > > > > The ContainerRouter's speed requirements are more close to those of > > > nginx/envoy than they are to those of the current ContainerPool/Proxy > > > implementation. As Dave mentioned, for the ContainerRouter it will be > > very > > > critical to know how many containers it can handle in reality, hence > I'm > > > shooting for an as efficient implementation of its innerts as possible > to > > > flesh out how far we can push the boundaries here. > > > > > > In any case: Yes we can stick to Scala here, but at least the > > > ContainerRouter might be 100% new code and we might not even be able to > > > rely on things like akka-http for its implementation for the reasons > > > mentioned above. > > > > > > ### Seperate repo vs. branch ### > > > > > > Towards Michaels points: The prototype repo is supposed to be used for > > > experimenting/prototyping and for that purpose only. As Dave pointed > out > > > above, we'd want these experiments to happen with Apache visibility. > > There > > > will never be a completely working system available in that repository. > > The > > > goal of it is to find out if certain assumptions of a new design are > > > feasible or not. The goal is **not** to build productised components > that > > > we can take over to the mainline. > > > > > > I always thought of that rearch as an incremental step as well and it > > > shouldn't need major changes to the API (or at least none that we are > not > > > already introducing right now). I don't think we're going to see the > true > > > potential of this though if we try to take the system's components as > is > > > and reuse them in contexts they have never been built for in the first > > > place. > > > > > > As soon as we have clearance over which way we want to ultimately go > with > > > the architecture, I agree we should come up with a nice migration plan > > and > > > start building it into the main repository through that migration plan. > > The > > > prototyping repository should vanish at some point. > > > > > > ### Summary ### > > > > > > I don't feel super strong about a choice of language, I just thought > I'd > > > throw that in the ring as well. As sentiments seem quite strong against > > > doing that right now, which is perfectly fine, I'd be fine doing the > > > implementations in Scala. The disclaimer though is, that I'd like to > > write > > > critical components afresh to not have to fight with existing > > architectural > > > choices to much but to be able to think freely about the problems at > > hand. > > > > > > Which is why I do feel quite strongly about not doing this in a branch. > > The > > > structure of the project will be very different and I can easily see > > people > > > confused if we start having pull-requests against a "prototype" branch > > > mixed in with our usual pull-requests. > > > > > > I do however agree that we eventually will need to build this in an > > > incremental way, once we know what we want to build in the first place. > > For > > > me, it's off the table to attempt to rewrite the system from scratch, > > we'll > > > likely never get that done (I think that's version 2 syndrome > Michael?). > > I > > > agree that might end up in a dead end and we shouldn't do that. > > > > > > Does that make the intention more clear? I hope I addressed all the > > points > > > asked. Opinions welcome :) > > > > > > Cheers, > > > Markus > > > > > > Am Mi., 29. Aug. 2018 um 00:55 Uhr schrieb Dascalita Dragos < > > > [email protected]>: > > > > > > > I also click on the clean and lean upgrade path, with gradual > > > improvements > > > > to the existing system (which is production-proof), as Michael and > > Rodric > > > > are suggesting. > > > > > > > > At the same time I see how Go would make sense from the Kubernetes > and > > > > Knative impl POV, how the trends favor Go over Scala [1], [2], and > the > > > > potential benefits from addressing a larger dev audience. @Markus, > feel > > > > free to keep me honest if this is where you're coming from. > > > > > > > > Trying to keep our options open, I'm thinking: why not take Markus' > > > > challenge, and design these enhancements now to support polyglot > > > > implementations in the future ? This would be the important decision > we > > > can > > > > take. BTW, for the data plane itself, we could go at even lower > levels > > to > > > > Envoy or Nginx, to provide superior performance, should we choose to. > > > > > > > > [1] - > > > > > > > > > > > > > > https://trends.google.com/trends/explore?geo=US&q=%2Fm%2F091hdj,%2Fm%2F09gbxjr > > > > > > > > [2] - https://octoverse.github.com/ > > > > > > > > On Tue, Aug 28, 2018 at 2:53 PM Rodric Rabbah <[email protected]> > > wrote: > > > > > > > > > Thanks Michael for raising these points. I share the same opinion > and > > > > > sentiment and think a branch with a clean migration story is better > > and > > > > > makes more sense. I am not entirely convinced that the choice of > > > language > > > > > itself will make the difference vs the new architecture which is > > quite > > > > > different and should in itself be more efficient. > > > > > > > > > > -r > > > > > > > > > > On Tue, Aug 28, 2018 at 4:51 PM Michael Marth > > <[email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hi Markus, > > > > > > > > > > > > IMHO what you propose below is a rather severe change in scope of > > > this > > > > > > discussion and effort. > > > > > > Up until so far this was about _evolving_ the OW architecture. We > > > have > > > > > not > > > > > > explicitly discussed it, but one could assume that it is at least > > > > > feasible > > > > > > to gradually adopt the new architecture. So there would be a > smooth > > > > path > > > > > > between the current state of the code base and a future one. > > > > > > > > > > > > Your proposal below breaks this assumption somewhat (by > proposing a > > > new > > > > > > repo instead of a branch - which will inevitably make the 2 code > > > bases > > > > > > drift apart) as well as explicitly by suggesting a new > > implementation > > > > > > language. Especially the latter would create a schism between > > OW-now > > > > and > > > > > > OW-future. > > > > > > This schism has implications like the perception of OW-now being > > > > > > deprecated, the _possibility_ of no clean upgrade path, the > > immediate > > > > > split > > > > > > of the community between *-now and *-future and of course carries > > the > > > > > risk > > > > > > of the version 2 syndrome. > > > > > > > > > > > > I would propose to implement the future architecture in a branch > > and > > > in > > > > > > Scala first. If it turns out to be good, then subsequent > > experiments > > > > can > > > > > > show or not-show if a switch of language is of additional value. > > That > > > > > would > > > > > > allow to make a decision based on data rather than anything else. > > > > > > > > > > > > My2c > > > > > > Michael > > > > > > > > > > > > > > > > > > On 28.08.18, 14:26, "Markus Thömmes" <[email protected] > > > > > > wrote: > > > > > > > > > > > > Hi all, > > > > > > > > > > > > Am Mo., 27. Aug. 2018 um 20:04 Uhr schrieb David P Grove < > > > > > > [email protected] > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > "Markus Thömmes" <[email protected]> wrote on > > > 08/23/2018 > > > > > > 04:19:33 > > > > > > > PM: > > > > > > > > > > > > > > > > > > > > > > > Key point I want to make is: At some point we'll have to > > > start > > > > to > > > > > > > prototype > > > > > > > > things out and see if our assumptions actually hold > water. > > > For > > > > > > example, > > > > > > > my > > > > > > > > assumption on a work-stealing backend is pretty much in > the > > > > air. > > > > > > > > > > > > > > > > My proposal for going forward would be: > > > > > > > > 1. Create a playground for the implementation of some > parts > > > of > > > > > the > > > > > > system > > > > > > > > (a new repository?) > > > > > > > > 2. Let's build some of the things that are > uncontroversial > > > and > > > > > > absolutely > > > > > > > > needed in any case (ContainerRouter, ContainerManager). > > > > > > > > 3. Play around with them, hook them up in different ways, > > see > > > > > what > > > > > > works > > > > > > > > and what doesn't. > > > > > > > > > > > > > > > > Some things will need some testing out to see the scale > > that > > > > the > > > > > > > components > > > > > > > > can operate at. These things will narrow or widen the > > > solution > > > > > > space for > > > > > > > > the more controversial topics around how to distribute > > > > containers > > > > > > in the > > > > > > > > system, how to balance between the routers, work-stealing > > > > queue: > > > > > > yes/no > > > > > > > etc. > > > > > > > > > > > > > > > > Having some simple components fleshed out could encourage > > > > > > innovation and > > > > > > > > creates some facts that we need to focus things into a > good > > > > > > direction. > > > > > > > > > > > > > > > > What do you think? Too early to start with this and/or > the > > > > wrong > > > > > > way of > > > > > > > > doing it? > > > > > > > > > > > > > > > > > > > > > > +1 for starting to prototype. It's been a good discussion > > and > > > I > > > > > > think > > > > > > > we've identified some things that we know we don't know, so > > > time > > > > to > > > > > > > experiment and find out. > > > > > > > > > > > > > > > > > > > > > Not sure what the best logistics are for this. Would like > > the > > > > work > > > > > > to be > > > > > > > at Apache (community visibility). I'm not sure if the best > > way > > > > is > > > > > a > > > > > > new > > > > > > > repo or an experimental branch of the main repo (we could > > dial > > > > down > > > > > > the > > > > > > > level of testing on the branch to make it less > cumbersome?). > > > The > > > > > > branch is > > > > > > > attractive to me because it might make it easier to keep in > > > synch > > > > > > with the > > > > > > > components we aren't changing. > > > > > > > > > > > > > > > > > > > I actually think we should generate a new repository for > this. > > > > > Opening > > > > > > PRs > > > > > > etc. will then not clutter the "main" repository and we don't > > > need > > > > to > > > > > > worry > > > > > > about breaking anything. > > > > > > > > > > > > If nobody objects I'm going to get > > "incubator-openwhisk-protoype" > > > > > > generated > > > > > > and will create a rough outline in the repository (different > > > > folders > > > > > > for > > > > > > different parts of the system). > > > > > > > > > > > > One more basic question (and this is going to be > > controversial): > > > > Most > > > > > > of > > > > > > the componentry in the execution layer will have to be build > > > anew. > > > > > I'd > > > > > > like > > > > > > to switch the implementation language of at least the > > > > ContainerRouter > > > > > > to > > > > > > Golang. Why? Because scale/performance matters a lot for > them. > > As > > > > > Dave > > > > > > mentioned on multiple occasions, it will greatly matter how > > many > > > > > > containers > > > > > > such a Router can handle under load and that scale will > define > > > the > > > > > > implementation alternatives we will have at hand. I therefore > > > would > > > > > > like to > > > > > > optimise these Routers for minimal overhead. We could go even > > > lower > > > > > > level, > > > > > > but I guess that'd be at a kinda big maintenance cost. > > > > > > > > > > > > I can envision the ContainerManager to be simpler to > implement > > in > > > > > > Golang as > > > > > > well, at least for some of the deployment alternatives > > > (Kubernetes > > > > > > comes to > > > > > > mind). The Golang based clients seemed superior to me vs. > > clients > > > > in > > > > > > any > > > > > > other language. > > > > > > > > > > > > As all of the communication will probably be HTTP only > anyway, > > > > these > > > > > > implementations should be swappable anytime. > > > > > > > > > > > > Comments very welcome! Let me know your opinions. > > > > > > > > > > > > Cheers, > > > > > > Markus > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Tzu-Chiao Yeh (@tz70s) > > > -- Tzu-Chiao Yeh (@tz70s)
