GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/21332
[SPARK-24236][SS] Continuous replacement for ShuffleExchangeExec
## What changes were proposed in this pull request?
1. New RDD named ContinuousShuffleRowRDD
2. New case class ContinuousShuffleExchangeExec
3. The rule named ReplaceShuffleExchange to replace original
ShuffleExchange to ContinuousShuffleExhcange
## How was this patch tested?
Existing UT. Add more UT later.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xuanyuanking/spark SPARK-24236
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21332.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21332
----
commit 09e5ec9dcf3b1822eee02d10f29eb3f02d806485
Author: Yuanjian Li <xyliyuanjian@...>
Date: 2018-05-15T13:20:59Z
Add ContinuousShuffledRowRDD for cp shuffle
commit 70c2a2b3016703950746142823fec147041b5158
Author: Yuanjian Li <xyliyuanjian@...>
Date: 2018-05-08T12:41:46Z
INF-SPARK-1388: Implementation of ContinuousShuffleExchange and
corresponding rule
commit bd05db1344e5d89136f9bda51f2c0cf292abe4cc
Author: Yuanjian Li <xyliyuanjian@...>
Date: 2018-05-09T10:04:01Z
INF-SPARK-1388: Refactor
commit 91d1bab0eeea612565b701166070d88b2e36882e
Author: Yuanjian Li <xyliyuanjian@...>
Date: 2018-05-15T13:41:59Z
ContinuouShuffleRowRDD support multi compute calls
commit 42c7b29ce49650c79f06d2269d4f2bff029ffe9e
Author: Yuanjian Li <xyliyuanjian@...>
Date: 2018-05-11T08:42:27Z
INF-SPARK-1392: fix mima test
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
