> On Dec. 22, 2013, 8:41 p.m., Kishore Gopalakrishna wrote: > > what is the difference we are seeing in terms of performance. Will reading > > in different thread imply that we might read messages first and then > > currenst state or vice versa ? will there be any issues with the change in > > order? Keep in mind that participant first update CS and then deletes the > > message. Controller might read the stale copy of message/current state. we > > need to have a test case that tests all possible combinations > > > > Is this is most optimal way > > > > is it not better to pre construct all the paths that need to be read and > > read them all in one async call? Actually read them in two async calls, one > > call gets all the stats and other call gets the data for only the changed > > ones. > > > > > > Kanak Biscuitwala wrote: > This varies, but it's cut down ReadClusterDataStage by about 100ms > depending on what the cluster looks like (from ~1s to ~900ms when there are > 100 nodes, 100 resources, 100 partitions per resource). > > That's a good call in aggregating the paths and splitting by stats, I > didn't think of that. I'll change this up to try that approach and see what > numbers I get.
By the way, the new thread reads messages and current state together and in the same order as before. However, by switching to the approach you suggested, multithreading probably isn't even necessary. - Kanak ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16407/#review30804 ----------------------------------------------------------- On Dec. 19, 2013, 3:50 p.m., Kanak Biscuitwala wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/16407/ > ----------------------------------------------------------- > > (Updated Dec. 19, 2013, 3:50 p.m.) > > > Review request for helix, Zhen Zhang and Kishore Gopalakrishna. > > > Bugs: HELIX-345 > > > Repository: helix-git > > > Description > ------- > > commit 90e995b02a59622a0526897faaee3d6ae057b63e > Author: Kanak Biscuitwala <[email protected]> > Date: Thu Dec 19 15:44:48 2013 -0800 > > [HELIX-345] Speed up the controller pipeline for 0.6.x > > :100644 100644 5c0a94a... 5858958... M > helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java > :100644 100644 df893bb... 9e2452b... M > helix-core/src/test/java/org/apache/helix/Mocks.java > > > Diffs > ----- > > > helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java > 5c0a94a > helix-core/src/test/java/org/apache/helix/Mocks.java df893bb > > Diff: https://reviews.apache.org/r/16407/diff/ > > > Testing > ------- > > Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme > cases. > > > Thanks, > > Kanak Biscuitwala > >
