Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14026 to look at the new patch set (#2). Change subject: IMPALA-8685,IMPALA-8677: Use consistent scheduling for small clusters ...................................................................... IMPALA-8685,IMPALA-8677: Use consistent scheduling for small clusters In the original change for consistent scheduling, if a cluster has fewer nodes than the number of remote executor candidates, then the scheduler falls back to using the old SelectRemoteExecutor(). SelectRemoteExecutor() considers all backends and picks the backend with the least assigned bytes; to break ties, it uses randomness. This means that clusters with fewer backends than num_remote_executor_candidates do not have consistent placement. For the file handle cache (the original user of consistent placement), this is not a major problem. However, for data caching, it can result in slower warm up of the data cache and greater duplication of the same data across different nodes. This changes the algorithm to use consistent placement even for small clusters (num nodes <= num_remote_executor_candidates). This also changes GetRemoteExecutorCandidates() to return the candidates in the order that they were selected. In code, this means looping over a vector to detect distinct backends rather than using a set. Testing: - Modify the scheduler-test backend test to verify that small clusters use consistent scheduling. Change-Id: Icfdb2cc53d7206e316ea8a1cc28ad443f246f741 --- M be/src/scheduling/scheduler-test.cc M be/src/scheduling/scheduler.cc 2 files changed, 55 insertions(+), 43 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/14026/2 -- To view, visit http://gerrit.cloudera.org:8080/14026 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icfdb2cc53d7206e316ea8a1cc28ad443f246f741 Gerrit-Change-Number: 14026 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>