+1 (non-binding), looks good! On Wed, Sep 11, 2019 at 10:05 AM Ryan Blue <rb...@netflix.com.invalid> wrote:
> +1 > > This is going to be really useful. Thanks for working on it! > > On Wed, Sep 11, 2019 at 9:38 AM Felix Cheung <felixcheun...@hotmail.com> > wrote: > >> +1 >> >> ------------------------------ >> *From:* Thomas graves <tgra...@apache.org> >> *Sent:* Wednesday, September 4, 2019 7:24:26 AM >> *To:* dev <dev@spark.apache.org> >> *Subject:* [VOTE] [SPARK-27495] SPIP: Support Stage level resource >> configuration and scheduling >> >> Hey everyone, >> >> I'd like to call for a vote on SPARK-27495 SPIP: Support Stage level >> resource configuration and scheduling >> >> This is for supporting stage level resource configuration and >> scheduling. The basic idea is to allow the user to specify executor >> and task resource requirements for each stage to allow the user to >> control the resources required at a finer grain. One good example here >> is doing some ETL to preprocess your data in one stage and then feed >> that data into an ML algorithm (like tensorflow) that would run as a >> separate stage. The ETL could need totally different resource >> requirements for the executors/tasks than the ML stage does. >> >> The text for the SPIP is in the jira description: >> >> https://issues.apache.org/jira/browse/SPARK-27495 >> >> I split the API and Design parts into a google doc that is linked to >> from the jira. >> >> This vote is open until next Fri (Sept 13th). >> >> [ ] +1: Accept the proposal as an official SPIP >> [ ] +0 >> [ ] -1: I don't think this is a good idea because ... >> >> I'll start with my +1 >> >> Thanks, >> Tom >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >> > > -- > Ryan Blue > Software Engineer > Netflix >
