+1 (Non-binding) The clustering approach covers most of my requirements on saving some shuffles. We kind of left the "should the user be allowed to provide a full partitioner" discussion on the table. I understand that would require exposing a lot of internals so this is perhaps a good compromise.
On Mon, Aug 28, 2017 at 12:20 PM Wenchen Fan <cloud0...@gmail.com> wrote: > Hi all, > > It has been almost 2 weeks since I proposed the data source V2 for > discussion, and we already got some feedbacks on the JIRA ticket and the > prototype PR, so I'd like to call for a vote. > > The full document of the Data Source API V2 is: > > https://docs.google.com/document/d/1n_vUVbF4KD3gxTmkNEon5qdQ-Z8qU5Frf6WMQZ6jJVM/edit > > Note that, this vote should focus on high-level design/framework, not > specified APIs, as we can always change/improve specified APIs during > development. > > The vote will be up for the next 72 hours. Please reply with your vote: > > +1: Yeah, let's go forward and implement the SPIP. > +0: Don't really care. > -1: I don't think this is a good idea because of the following technical > reasons. > > Thanks! >
