+1 This is going to be really useful. Thanks for working on it!
On Wed, Sep 11, 2019 at 9:38 AM Felix Cheung <felixcheun...@hotmail.com> wrote: > +1 > > ------------------------------ > *From:* Thomas graves <tgra...@apache.org> > *Sent:* Wednesday, September 4, 2019 7:24:26 AM > *To:* dev <dev@spark.apache.org> > *Subject:* [VOTE] [SPARK-27495] SPIP: Support Stage level resource > configuration and scheduling > > Hey everyone, > > I'd like to call for a vote on SPARK-27495 SPIP: Support Stage level > resource configuration and scheduling > > This is for supporting stage level resource configuration and > scheduling. The basic idea is to allow the user to specify executor > and task resource requirements for each stage to allow the user to > control the resources required at a finer grain. One good example here > is doing some ETL to preprocess your data in one stage and then feed > that data into an ML algorithm (like tensorflow) that would run as a > separate stage. The ETL could need totally different resource > requirements for the executors/tasks than the ML stage does. > > The text for the SPIP is in the jira description: > > https://issues.apache.org/jira/browse/SPARK-27495 > > I split the API and Design parts into a google doc that is linked to > from the jira. > > This vote is open until next Fri (Sept 13th). > > [ ] +1: Accept the proposal as an official SPIP > [ ] +0 > [ ] -1: I don't think this is a good idea because ... > > I'll start with my +1 > > Thanks, > Tom > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > -- Ryan Blue Software Engineer Netflix
