Completely agree. It would be great to get more info from your experience.
To an extent what we have at the moment is very much about isolating the implementation of a model from the deployment and discovery mechanism, and to my mind we should very much keep that to enable any kind of model to plugin. The other thing worth discussing here would be how we can wrap around a model while maintaining the lose coupling, to provide things like generalised performance metrics for the models. Any thoughts on that front anyone? > On 18 Apr 2017, at 11:45, Otto Fowler <[email protected]> wrote: > > I will have to go back to my notes. There was a day or so when I went > through the code and was thinking of a couple of things, but that was a while > ago. > > Off the top of my head, I would want something factored enough, or loosely > coupled enough that it was not dependent on twill or maas or anything else. > This > would not impose implementation on the services. This would have to revolve > around a discovery/registration api and a rest interface contract. > > Does that make sense? > > > > > On April 18, 2017 at 14:34:28, Simon Elliston Ball > ([email protected] <mailto:[email protected]>) wrote: > >> Right, how about some sort of REST API? Or through Ambari? What would you >> say was the best way to start the service, and of course to submit model >> artefacts? >> >> Simon >> >>> On 18 Apr 2017, at 11:33, Otto Fowler <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> In my mind, I didn’t want to deploy the service as a bash script or wrapped >>> in one, if I recall correctly. >>> >>> >>> >>> On April 18, 2017 at 14:27:52, Simon Elliston Ball >>> ([email protected] <mailto:[email protected]>) wrote: >>> >>>> Any particular issues, or things that didn’t work Otto? >>>> >>>> Simon >>>> >>>> >>>> > On 18 Apr 2017, at 11:26, Otto Fowler <[email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > >>>> > I’ll try to take a look. There are a couple of things I wanted to do >>>> > with MaaS but could not >>>> > figure out because of a couple of limitations. I’d like to see if twill >>>> > offers more flexibility >>>> > >>>> > >>>> > >>>> > >>>> > On April 18, 2017 at 13:50:39, Nick Allen ([email protected] >>>> > <mailto:[email protected]>) wrote: >>>> > >>>> > I ran across the Apache Twill [1] project recently whose goal is to >>>> > reduce >>>> > the complexity of developing distributed applications that run on YARN. >>>> > My >>>> > first thought is that it might offer additional capabilities and/or >>>> > simplify our current MaaS implementation. >>>> > >>>> > Here are a list of features provided by Twill that I think might be >>>> > useful >>>> > for MaaS. >>>> > >>>> > - Service discovery >>>> > - Elastic scaling >>>> > - High Availability >>>> > - Placement policies - Which rack/host should the model run on? >>>> > - Security - Kerberos ticket refresh? >>>> > >>>> > Just wanted to float the thought in the community and see if anyone has >>>> > experience with Twill. I need to do some more research myself. >>>> > >>>> > [1] http://twill.apache.org/ <http://twill.apache.org/>
