GitHub user sharkdtu opened a pull request:
https://github.com/apache/spark/pull/21658
[SPARK-24678][Spark-Streaming] Give priority in use of 'PROCESS_LOCAL' for
spark-streaming
## What changes were proposed in this pull request?
Currently, `BlockRDD.getPreferredLocations` only get hosts info of blocks,
which results in subsequent schedule level is not better than 'NODE_LOCAL'. We
can just make a small changes, the schedule level can be improved to
'PROCESS_LOCAL'
## How was this patch tested?
manual test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sharkdtu/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21658.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21658
----
commit 666fb4c5d343a1ea439ecc284d047810d6189c23
Author: sharkdtu <sharkdtu@...>
Date: 2018-06-28T07:35:52Z
give priority in use of 'PROCESS_LOCAL' for spark-streaming
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
