[GitHub] drill pull request #1129: DRILL-6180: Use System Option "output_batch_size" ...

2018-03-03 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/1129


---

[GitHub] drill pull request #1129: DRILL-6180: Use System Option "output_batch_size" ...

2018-02-28 Thread paul-rogers 
Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1129#discussion_r171314892
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/SortConfig.java
 ---
@@ -71,8 +72,8 @@
 
   private final int mSortBatchSize;
 
-  public SortConfig(DrillConfig config) {
-
+  public SortConfig(FragmentContext context) {
+DrillConfig config = context.getConfig();
--- End diff --

Suggestion: pass in the original `DrillConfig` plus an option manager 
rather than the fragment context. The suggestion minimizes undesired 
dependencies.


---

[GitHub] drill pull request #1129: DRILL-6180: Use System Option "output_batch_size" ...

2018-02-28 Thread paul-rogers 
Github user paul-rogers commented on a diff in the pull request:

https://github.com/apache/drill/pull/1129#discussion_r171315310
  
--- Diff: exec/java-exec/src/main/resources/drill-module.conf ---
@@ -421,7 +416,7 @@ drill.exec.options: {
 drill.exec.storage.implicit.fqn.column.label: "fqn",
 drill.exec.storage.implicit.suffix.column.label: "suffix",
 drill.exec.testing.controls: "{}",
-drill.exec.memory.operator.output_batch_size : 33554432, # 32 MB
+drill.exec.memory.operator.output_batch_size : 16777216, # 16 MB
--- End diff --

Thanks for making this adjustment.


---

[GitHub] drill pull request #1129: DRILL-6180: Use System Option "output_batch_size" ...

2018-02-22 Thread ppadma 
GitHub user ppadma opened a pull request:

https://github.com/apache/drill/pull/1129

DRILL-6180: Use System Option "output_batch_size" for External Sort

External Sort has boot time configuration for output batch size 
"drill.exec.sort.external.spill.merge_batch_size" which is defaulted to 16M.
To make batch sizing configuration uniform across all operators, change 
this to use new system option that is added 
"drill.exec.memory.operator.output_batch_size". 
This option has default value of 32M. Changed it to 16M.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ppadma/drill DRILL-6180

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1129.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1129


commit 5121663c1fac618d0374667c97c20570197b7455
Author: Padma Penumarthy 
Date:   2018-02-23T00:41:47Z

DRILL-6180: Use System Option "output_batch_size" for External Sort




---