> > Is the REST API interface planned to be be identical/close to the thrift > api interface?
The REST API is likely to be a replacement for the thrift API, so it will indeed require feature parity to reach that point. And for example, is AcquireLock scheduler method still used? AcquireLock is only used for client-side updates, and will likely be phased out with them. On the CreateJob, is a lock object required? No. It would be good to document these, so a generic client can be written. > Maybe that is what this ticket should do. There's some documentation in api.thrift [1], which is in turn rendered into HTML and served by the scheduler. However, expanding on the documentation would be great. [1] https://github.com/apache/aurora/blob/master/api/src/main/thrift/org/apache/aurora/gen/api.thrift -=Bill On Mon, Jul 27, 2015 at 12:34 AM, meghdoot bhattacharya < meghdoo...@yahoo.com.invalid> wrote: > Is the REST API interface planned to be be identical/close to the thrift > api interface? > And if we leave the client dsl modifications for custom executors, for now > we can build a generic client to use the thrift api directly for supporting > custom executors. Cursory look of the client api code looks like the > scheduler proxy object is calling the scheduler directly but would be nice > to have the thick client logic completely removed. The old style client > orchestrated updater code is very much present. And for example, is > AcquireLock scheduler method still used? Looks like new update method does > not require it but the old update method is the only one that needed it. > But if latter is deprecated, why not remove the scheduler method? On the > CreateJob, is a lock object required? It does not seem necessary. It would > be good to document these, so a generic client can be written. Maybe that > is what this ticket should do. > Thx > From: Bill Farner <wfar...@apache.org> > To: "dev@aurora.apache.org" <dev@aurora.apache.org>; meghdoot > bhattacharya <meghdoo...@yahoo.com> > Sent: Tuesday, July 21, 2015 9:53 PM > Subject: Re: Aurora-1288 custom executor support in client > > One of the biggest challenges i foresee if we embark on making the client > support custom executors is schema handling. By design, pystachio defines > schemas that configurations must match, which will likely make it difficult > to generalize to support swappable/arbitrary executors. I am not familiar > enough with pystachio to articulate how [in]feasible this is, so hopefully > someone else can expound on that. > > While i think this is possible, i wouldn't be disappointed if this use case > were used to apply pressure to getting a REST API in place, as i believe > that's one of the few major hurdles left necessitating a CLI. > > > -=Bill > > > > On Tue, Jul 21, 2015 at 2:42 PM, meghdoot bhattacharya < > meghdoo...@yahoo.com.invalid> wrote: > > > https://issues.apache.org/jira/browse/AURORA-1288 > > As the server side changes are getting wrapped up, Bill and I were > > discussing offline around custom executor support in aurora client, one > of > > the sub tasks in the ticket. So, we are bringing the discussion to the > > community to get feedback from you all. > > Meghdoot>Theidea should be to support the current DSL (to not break > > current integrations)but also introduce a new way of defining things > where > > even thermos or anycustom executor can have its own stuff defined in a > > generic way eventuallymarshalled into “data” blob supported as Brian had > > commented in the ticket > > > > Bill> Just so i follow - what drives the requirement ofdoing this in the > > existing client? Our current direction is to make theclient thinner over > > time, so hypothetically the surface area for a custom clientis getting > > pretty small. I'd also like to invest in a REST API quitesoon, which > > reduces the technology coupling as well. Do these detailschange the > > equation for you? > > > > > > Meghdoot>Generallyhaving a cli is powerful. Though we can rely on thrift > > api’s and future restapi’s directly to plug in with our web systems, > having > > a cli support is apowerful functionality to have especially if we can > make > > it generic to dealwith custom executors. And we are thinking of upgrading > > the aurora client andnot have some other cli. > > > > Inthe end any client should just send the job config and executor data > > buried inthe current field, but having a DSL where you can declare things > > can still bebeneficial. > > > > Butif we really are moving in a direction that the existing cli gets > used > > mostlyfor admin usage and we retire the DSL usage, then there is no point > > investingon it. Otherwise the question remains why DSL support only for > > thermos. > > Otherpoint to add if we really want to keep the DSL only for thermos, > then > > to havecustom executors thrive through clients of their own, we need a > > detailed writeup on how that will work. Meaning to create or update or > some > > other job relatedoperations what are the sequence of API’s to call (for > > example grab lock api,call main api, call release api), if behind the > > scenes the existing aurora client isdoing it today. Otherwisecustom > > executors will struggle in the absence of support from aurora client > andan > > integration guide. > > > > > > > > > > > > > > > > > > >
