GitHub user dasahcc opened a pull request:
https://github.com/apache/helix/pull/169
Two Performance Improvement for Rebalance pipeline
Two improvements have been implemented :
1. Avoid to read data from ZK for ExternalViewStage
2. Support selective update for IdealState
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dasahcc/helix perf_improve
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/169.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #169
----
commit ead4d3760b36576d8be5cf0480c84139d989f676
Author: Junkai Xue <jxue@...>
Date: 2018-03-08T00:55:30Z
Avoid to read data from ZK for ExternalViewStage
Helix Controller is trying to external view back to compare with ZK data,
which is not necessary since we can cache the data and compare them in memory.
RB=1244812
BUG=HELIX-844
G=helix-reviewers
A=jjwang
commit c2378557be5f6e4d6be9f7c90088427fb2a98d43
Author: Junkai Xue <jxue@...>
Date: 2018-03-26T23:02:52Z
Support selective update for IdealState
To have better performance for Pinot, we need some improvements for
rebalance pipeline. One of the improvement is trying to read as less as
possible. So use version based IdealState read will be very helpful.
----
---
