Design for continuous processing shuffle

Hi all, A few of us have been working on a design for how to do shuffling in continuous processing. Feel free to chip in if you have any comments or questions. doc: https://docs.google.com/document/d/1IL4kJoKrZWeyIhklKUJqsW-yEN7V7aL05MmM65AYOfE continuous processing SPIP: https://issues.apache.o

Re: AccumulatorV2 vs AccumulableParam (V1)

Hi Wenchen, Thanks a lot for clarification and help. Here is what I mean regarding the remaining points For 2: Should we update the documentation [1] regarding custom accumulators to be more clear and to highlight that a) custom accumulators should always override "copy" method to prevent unex