Re: Design for continuous processing shuffle

[Yuanjian Li]

Hi Joseph and devs,

Happy to see the discussion of CP shuffle, as comment in 
https://issues.apache.org/jira/browse/SPARK-20928?focusedCommentId=16245556&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16245556
 
,
 the team also do some design and demo work over CP shuffle, the docs show more 
detailed work:

https://docs.google.com/document/d/14cGJ75v9myznywtB35ytEqL9wHy9xfZRv06B6g2tUgI 




> 在 2018年5月5日，02:27，Joseph Torres  写道：
> 
> Hi all,
> 
> A few of us have been working on a design for how to do shuffling in 
> continuous processing. Feel free to chip in if you have any comments or 
> questions.
> 
> doc:
> https://docs.google.com/document/d/1IL4kJoKrZWeyIhklKUJqsW-yEN7V7aL05MmM65AYOfE
>  
> 
> 
> continuous processing SPIP: https://issues.apache.org/jira/browse/SPARK-20928 
> 
> 
> 
> Jose

Design for continuous processing shuffle

[Joseph Torres]

Hi all,

A few of us have been working on a design for how to do shuffling in
continuous processing. Feel free to chip in if you have any comments or
questions.

doc:
https://docs.google.com/document/d/1IL4kJoKrZWeyIhklKUJqsW-yEN7V7aL05MmM65AYOfE

continuous processing SPIP:
https://issues.apache.org/jira/browse/SPARK-20928


Jose