Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/717#discussion_r99250103
--- Diff: exec/java-exec/src/main/resources/drill-module.conf ---
@@ -177,13 +177,47 @@ drill.exec: {
sort: {
purge.threshold : 1000,
external: {
- batch.size : 4000,
+ // Drill uses the managed External Sort Batch by default.
+ // Set this to true to use the legacy, unmanaged version.
+ // Disabled in the intial commit, to be enabled after
+ // tests are committed.
+ disable_managed: true
+ // Limit on the number of batches buffered in memory.
+ // Primarily for testing.
+ // 0 = unlimited
+ batch_limit: 0
+ // Limit on the amount of memory used for xsort. Overrides the
+ // value provided by Foreman. Primarily for testing.
+ // 0 = unlimited, Supports HOCON memory suffixes.
+ mem_limit: 0
+ // Limit on the number of spilled batches that can be merged in
+ // a single pass. Limits the number of open file handles.
+ // 0 = unlimited
+ merge_limit: 0
spill: {
- batch.size : 4000,
- group.size : 40000,
- threshold : 40000,
- directories : [ "/tmp/drill/spill" ],
- fs : "file:///"
+ // Deprecated for managed xsort; used only by legacy xsort
+ group.size: 40000,
+ // Deprecated for managed xsort; used only by legacy xsort
+ threshold: 40000,
+ // Minimum number of in-memory batches to spill per spill file
+ // Affects only spilling from memory to disk.
+ // Primarily for testing.
+ min_batches: 2,
+ // Maximum number of in-memory batches to spill per spill file
+ // Affects only spilling from memory to disk.
+ // Primarily for testing.
+ // 0 = unlimited
+ max_batches: 0,
+ // File system to use. Local file system by default.
+ fs: "file:///"
+ // List of directories to use. Directories are created
--- End diff --
Here that is implied by the JSON-like syntax.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
