+1 On Mon, Mar 4, 2019 at 12:52 PM Imran Rashid <im...@therashids.com> wrote:
> On Sun, Mar 3, 2019 at 6:51 PM Xiangrui Meng <men...@gmail.com> wrote: > >> On Sun, Mar 3, 2019 at 10:20 AM Felix Cheung <felixcheun...@hotmail.com> >> wrote: >> >>> IMO upfront allocation is less useful. Specifically too expensive for >>> large jobs. >>> >> >> This is also an API/design discussion. >> > > I agree with Felix -- this is more than just an API question. It has a > huge impact on the complexity of what you're proposing. You might be > proposing big changes to a core and brittle part of spark, which is already > short of experts. > > I don't see any value in having a vote on "does feature X sound cool?" We > have to evaluate the potential benefit against the risks the feature brings > and the continued maintenance cost. We don't need super low-level details, > but we have to a sketch of the design to be able to make that tradeoff. >
