Is the REST API interface planned to be be identical/close to the thrift api
interface?
And if we leave the client dsl modifications for custom executors, for now we
can build a generic client to use the thrift api directly for supporting custom
executors. Cursory look of the client api code looks like the scheduler proxy
object is calling the scheduler directly but would be nice to have the thick
client logic completely removed. The old style client orchestrated updater code
is very much present. And for example, is AcquireLock scheduler method still
used? Looks like new update method does not require it but the old update
method is the only one that needed it. But if latter is deprecated, why not
remove the scheduler method? On the CreateJob, is a lock object required? It
does not seem necessary. It would be good to document these, so a generic
client can be written. Maybe that is what this ticket should do.
Thx
From: Bill Farner <wfar...@apache.org>
To: "dev@aurora.apache.org" <dev@aurora.apache.org>; meghdoot bhattacharya
<meghdoo...@yahoo.com>
Sent: Tuesday, July 21, 2015 9:53 PM
Subject: Re: Aurora-1288 custom executor support in client
One of the biggest challenges i foresee if we embark on making the client
support custom executors is schema handling. By design, pystachio defines
schemas that configurations must match, which will likely make it difficult
to generalize to support swappable/arbitrary executors. I am not familiar
enough with pystachio to articulate how [in]feasible this is, so hopefully
someone else can expound on that.
While i think this is possible, i wouldn't be disappointed if this use case
were used to apply pressure to getting a REST API in place, as i believe
that's one of the few major hurdles left necessitating a CLI.
-=Bill
On Tue, Jul 21, 2015 at 2:42 PM, meghdoot bhattacharya <
meghdoo...@yahoo.com.invalid> wrote:
> https://issues.apache.org/jira/browse/AURORA-1288
> As the server side changes are getting wrapped up, Bill and I were
> discussing offline around custom executor support in aurora client, one of
> the sub tasks in the ticket. So, we are bringing the discussion to the
> community to get feedback from you all.
> Meghdoot>Theidea should be to support the current DSL (to not break
> current integrations)but also introduce a new way of defining things where
> even thermos or anycustom executor can have its own stuff defined in a
> generic way eventuallymarshalled into “data” blob supported as Brian had
> commented in the ticket
>
> Bill> Just so i follow - what drives the requirement ofdoing this in the
> existing client? Our current direction is to make theclient thinner over
> time, so hypothetically the surface area for a custom clientis getting
> pretty small. I'd also like to invest in a REST API quitesoon, which
> reduces the technology coupling as well. Do these detailschange the
> equation for you?
>
>
> Meghdoot>Generallyhaving a cli is powerful. Though we can rely on thrift
> api’s and future restapi’s directly to plug in with our web systems, having
> a cli support is apowerful functionality to have especially if we can make
> it generic to dealwith custom executors. And we are thinking of upgrading
> the aurora client andnot have some other cli.
>
> Inthe end any client should just send the job config and executor data
> buried inthe current field, but having a DSL where you can declare things
> can still bebeneficial.
>
> Butif we really are moving in a direction that the existing cli gets used
> mostlyfor admin usage and we retire the DSL usage, then there is no point
> investingon it. Otherwise the question remains why DSL support only for
> thermos.
> Otherpoint to add if we really want to keep the DSL only for thermos, then
> to havecustom executors thrive through clients of their own, we need a
> detailed writeup on how that will work. Meaning to create or update or some
> other job relatedoperations what are the sequence of API’s to call (for
> example grab lock api,call main api, call release api), if behind the
> scenes the existing aurora client isdoing it today. Otherwisecustom
> executors will struggle in the absence of support from aurora client andan
> integration guide.
>
>
>
>
>
>
>
>
