> On Dec. 22, 2013, 8:41 p.m., Kishore Gopalakrishna wrote: > > what is the difference we are seeing in terms of performance. Will reading > > in different thread imply that we might read messages first and then > > currenst state or vice versa ? will there be any issues with the change in > > order? Keep in mind that participant first update CS and then deletes the > > message. Controller might read the stale copy of message/current state. we > > need to have a test case that tests all possible combinations > > > > Is this is most optimal way > > > > is it not better to pre construct all the paths that need to be read and > > read them all in one async call? Actually read them in two async calls, one > > call gets all the stats and other call gets the data for only the changed > > ones. > > > >
This varies, but it's cut down ReadClusterDataStage by about 100ms depending on what the cluster looks like (from ~1s to ~900ms when there are 100 nodes, 100 resources, 100 partitions per resource). That's a good call in aggregating the paths and splitting by stats, I didn't think of that. I'll change this up to try that approach and see what numbers I get. - Kanak ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16407/#review30804 ----------------------------------------------------------- On Dec. 19, 2013, 3:50 p.m., Kanak Biscuitwala wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/16407/ > ----------------------------------------------------------- > > (Updated Dec. 19, 2013, 3:50 p.m.) > > > Review request for helix, Zhen Zhang and Kishore Gopalakrishna. > > > Bugs: HELIX-345 > > > Repository: helix-git > > > Description > ------- > > commit 90e995b02a59622a0526897faaee3d6ae057b63e > Author: Kanak Biscuitwala <[email protected]> > Date: Thu Dec 19 15:44:48 2013 -0800 > > [HELIX-345] Speed up the controller pipeline for 0.6.x > > :100644 100644 5c0a94a... 5858958... M > helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java > :100644 100644 df893bb... 9e2452b... M > helix-core/src/test/java/org/apache/helix/Mocks.java > > > Diffs > ----- > > > helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java > 5c0a94a > helix-core/src/test/java/org/apache/helix/Mocks.java df893bb > > Diff: https://reviews.apache.org/r/16407/diff/ > > > Testing > ------- > > Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme > cases. > > > Thanks, > > Kanak Biscuitwala > >
