----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22013/#review44251 -----------------------------------------------------------
exec/java-exec/src/test/java/org/apache/drill/exec/pop/TestFragmentChecker.java <https://reviews.apache.org/r/22013/#comment78599> Will add a test as part of HBase scan distribution that I am working atm. - Aditya Kishore On May 28, 2014, 11:15 p.m., Aditya Kishore wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/22013/ > ----------------------------------------------------------- > > (Updated May 28, 2014, 11:15 p.m.) > > > Review request for drill, Jacques Nadeau and Steven Phillips. > > > Repository: drill-git > > > Description > ------- > > Currently if only a small number of endpoints have data affinity, all the > work units get assigned to this small subset leaving other endpoints in the > cluster underutilized. > > It will lead to poor query performance since the level of parallelization is > reduced below what is otherwise possible. > > This JIRA intend to modify the assignment to do a slightly more equitable > distribution of work fragments among all available nodes while slightly > favoring the nodes with data affinity. > > This *favor* is controlled by the setting "drill.exec.work.affinity.factor" > (defaults to 1.2f). > > As an example, let's say that there are 2 endpoints of which only one is with > affinity and 10 work fragments. In the current scheme of things, all of these > work fragments will get assigned to the single node with affinity. With the > modified scheme and with default affinity factor of 1.2, it will get 20% more > assignment that it would get under equal distribution (i.e. 6:4 instead of > 5:5). > > > Diffs > ----- > > distribution/src/resources/drill-override.conf a9316a9 > exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java > d9e0833 > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/EndpointAffinity.java > f3059ae > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/SimpleExecPlanner.java > 4da6500 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java > 313a81d > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Wrapper.java > 8602bf0 > > exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java > eb1d738 > exec/java-exec/src/main/resources/drill-module.conf 26205bd > > exec/java-exec/src/test/java/org/apache/drill/exec/pop/TestFragmentChecker.java > 1b38dce > > Diff: https://reviews.apache.org/r/22013/diff/ > > > Testing > ------- > > > Thanks, > > Aditya Kishore > >
