There is one case I can think of where this is useful - for testing a
server we may want to modify these parameters without having to restart a
drillbit. With what I said above, that cannot be done.
Also, re security parameters: these can be exposed to show a user that they
are set, however their
There are some examples of using internal (undocumented) configuration
options in Drill. You simply don't put the option in drill-module.conf or
in the SystemOptionManager. Adding any (arbitrary) key name in
drill-override.conf automatically gives you an undocumented boot time
option name. Or provi
Hi Tim,
As you noted, it might be easier to have a single options system (that is, one
for runtime system/session) options rather than two. If we have two, we’d need
to create two system tables, two session tables and two bundles of options sent
along with each query, and two sets of SQL ALTER
Hello All,
Boaz has proposed the idea of internal options. Currently all the options are
visible to to the user when they do:
select * from sys.options.
We would like internal options to be not visible to the user by default.
Possibly requiring them to do:
select * from internal.options
in o
Timothy Farkas created DRILL-5723:
-
Summary: Add An Internal Options Table
Key: DRILL-5723
URL: https://issues.apache.org/jira/browse/DRILL-5723
Project: Apache Drill
Issue Type: New Feature
Here are the relevant lines of code (from
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractBase.java)
public static long MAX_ALLOCATION = 10_000_000_000L;
protected long maxAllocation = MAX_ALLOCATION;
So this applies to every (instance == minor fragment) of ever
I guess that's why someone recommended to specify a low limit, but then
specify a higher one for each query before execution, using the *ALTER
SESSION* command. I can't remember were did I read that.
Thanks,
Gelbana
On Tue, Aug 15, 2017 at 7:49 PM, Kunal Khatua wrote:
> The property *planner.me
The property *planner.memory.max_query_memory_per_node* is a cumulative limit
of all the operators' minor fragments' memory consumption.
However, like Boaz pointed out, the truly memory hungry operators like Hash,
Sort or Scan operators will take the lion's share of a query's memory. Since,
so
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/877
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133234012
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatConfig.java
---
@@ -24,9 +24,16 @@
import com.fasterxml.jackson
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133228018
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatConfig.java
---
@@ -24,9 +24,16 @@
import com.fasterxml
[
https://issues.apache.org/jira/browse/DRILL-5357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-5357.
-
Resolution: Fixed
Fix Version/s: 1.12.0
Fixed in the scope of DRILL-4735.
> Partit
[
https://issues.apache.org/jira/browse/DRILL-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-3407.
-
Resolution: Fixed
Fix Version/s: (was: Future)
1.12.0
Fixed
[
https://issues.apache.org/jira/browse/DRILL-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva resolved DRILL-1691.
-
Resolution: Fixed
Fixed in the scope of DRILL-4735.
> ConvertCountToDirectScan rule shou
Github user priteshm commented on the issue:
https://github.com/apache/drill/pull/905
@paul-rogers @jinfengni can you please review this one?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
By "instance", you mean minor fragments, correct ? And does the
*planner.memory.max_query_memory_per_node* limit apply to each *type* of
minor fragments individually ? Assuming the memory limit is set to *4 GB*,
and the running query involves external sort and hash aggregates, should I
expect the q
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133198372
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/MetadataContext.java
---
@@ -41,6 +42,8 @@
private PruneStatus pruneStat
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133198332
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/MetadataContext.java
---
@@ -41,6 +42,8 @@
private PruneStatus pruneStat
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133198419
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -543,78 +584,72 @@ private void readBlockMeta(String path,
Github user arina-ielchiieva commented on the issue:
https://github.com/apache/drill/pull/877
+1, LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
[
https://issues.apache.org/jira/browse/DRILL-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jinfeng Ni resolved DRILL-5685.
---
Resolution: Fixed
Fixed in ae123e166af2ddb22c79b8cb93278a66fccc3be4
> Provide a way to set common env
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/860
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/900
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/902
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/874
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/883
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/890
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/891
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133179047
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/MetadataContext.java
---
@@ -41,6 +42,8 @@
private PruneStatus p
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133177759
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -543,78 +584,72 @@ private void readBlockMeta(String
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133178381
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/MetadataContext.java
---
@@ -41,6 +42,8 @@
private PruneStatus p
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133159772
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -268,11 +276,15 @@ private FileSelection expandIfNec
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132515388
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/MetadataContext.java
---
@@ -41,6 +42,9 @@
private PruneStatus pruneStat
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133143486
--- Diff:
exec/java-exec/src/test/resources/parquet/unsupported_metadata/corrupted_metadata.requires_replace.txt
---
@@ -0,0 +1,41 @@
+{
+ "metadat
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132533745
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/MetadataVersion.java
---
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133141382
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -1123,4 +1163,22 @@ public GroupScan applyFilter(Log
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133143191
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetMetadataCache.java
---
@@ -452,33 +458,202 @@ public void testMoveCach
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133142647
--- Diff: exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java
---
@@ -639,14 +644,18 @@ protected static void
copyDirectoryIntoTempSpace(String
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132536020
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -268,11 +276,15 @@ private FileSelection expandIfNec
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132533379
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -519,17 +564,16 @@ private void writeFile(ParquetTableMetada
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133157793
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -633,13 +645,22 @@ public long getRowCount() {
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132652665
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -209,9 +211,10 @@ public ParquetGroupScan( //
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r13314
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -974,8 +1005,17 @@ private int updateRowGroupInfo(lo
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132517116
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -132,25 +140,62 @@ public static ParquetTableMetadata_v3
ge
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r133139901
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
---
@@ -974,8 +1005,17 @@ private int updateRowGroupInfo(lo
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132432541
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSelection.java
---
@@ -188,6 +188,10 @@ public void setExpandedFully() {
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r132534589
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java
---
@@ -222,15 +222,17 @@ public DrillTable isReadable(D
47 matches
Mail list logo