https://issues.apache.org/jira/browse/AURORA-1288 As the server side changes are getting wrapped up, Bill and I were discussing offline around custom executor support in aurora client, one of the sub tasks in the ticket. So, we are bringing the discussion to the community to get feedback from you all. Meghdoot>Theidea should be to support the current DSL (to not break current integrations)but also introduce a new way of defining things where even thermos or anycustom executor can have its own stuff defined in a generic way eventuallymarshalled into “data” blob supported as Brian had commented in the ticket
Bill> Just so i follow - what drives the requirement ofdoing this in the existing client? Our current direction is to make theclient thinner over time, so hypothetically the surface area for a custom clientis getting pretty small. I'd also like to invest in a REST API quitesoon, which reduces the technology coupling as well. Do these detailschange the equation for you? Meghdoot>Generallyhaving a cli is powerful. Though we can rely on thrift api’s and future restapi’s directly to plug in with our web systems, having a cli support is apowerful functionality to have especially if we can make it generic to dealwith custom executors. And we are thinking of upgrading the aurora client andnot have some other cli. Inthe end any client should just send the job config and executor data buried inthe current field, but having a DSL where you can declare things can still bebeneficial. Butif we really are moving in a direction that the existing cli gets used mostlyfor admin usage and we retire the DSL usage, then there is no point investingon it. Otherwise the question remains why DSL support only for thermos. Otherpoint to add if we really want to keep the DSL only for thermos, then to havecustom executors thrive through clients of their own, we need a detailed writeup on how that will work. Meaning to create or update or some other job relatedoperations what are the sequence of API’s to call (for example grab lock api,call main api, call release api), if behind the scenes the existing aurora client isdoing it today. Otherwisecustom executors will struggle in the absence of support from aurora client andan integration guide.
