Github user ppadma commented on the issue:
https://github.com/apache/drill/pull/639
Updated the JIRA with details on how current algorithm works, why remote
reads were happening and the new algorithm details.
https://issues.apache.org/jira/browse/DRILL-4706
---
If your project i
Github user sudheeshkatkam commented on the issue:
https://github.com/apache/drill/pull/639
@vkorukanti if you don't mind, can you review this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user ppadma commented on the issue:
https://github.com/apache/drill/pull/639
Updated with all review comments taken care of. Added
TestLocalAffinityFragmentParallelizer.java which has bunch of test cases with
examples.
---
If your project is set up for it, you can reply to
Github user ppadma commented on the issue:
https://github.com/apache/drill/pull/639
Some initial comments.
The issue is regarding assigning fragments based on strict locality. So why
is the parallelization logic affected, and not exclusively locality?
Parallelization
Github user sudheeshkatkam commented on the issue:
https://github.com/apache/drill/pull/639
Hmm the answer seems like a rephrasing of the question. Sorry, I misspoke.
Better asked:
The issue is regarding assigning **_work to_** fragments based on strict
locality (**_decide wh
Github user ppadma commented on the issue:
https://github.com/apache/drill/pull/639
Parallelization logic is affected for following reasons:
Depending upon how many rowGroups to scan on a node (based on locality
information) i.e. how much work the node has to do, we want to adjust
Github user ppadma commented on the issue:
https://github.com/apache/drill/pull/639
Merged with latest code. All review comments taken care of. All tests pass
with the option `store.parquet.use_local_affinity` = true and false, both.
---
If your project is set up for it, you can repl
