[jira] [Assigned] (DRILL-6086) TestSortImpl.testLargeBatch unit test fails randomly
[
https://issues.apache.org/jira/browse/DRILL-6086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers reassigned DRILL-6086:
--
Assignee: Paul Rogers
> TestSortImpl.testLargeBatch unit test fails randomly
>
>
> Key: DRILL-6086
> URL: https://issues.apache.org/jira/browse/DRILL-6086
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Paul Rogers
>
> {noformat}
> Failed tests:
>
> TestSortImpl.testLargeBatch:513->runJumboBatchTest:486->runLargeSortTest:455
> Value of 1:0 expected:<0> but was:<1>
> {noformat}
> The test fails due to memory corruption caused by a write out of the direct
> buffer allocated space. With bounds check enabled, the test fails reliably
> with
> {noformat}
> /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java
> -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:57731,suspend=y,server=n
> -Dvisualvm.id=131461133353377 -Ddrill.exec.rpc.user.timeout=0
> -Ddrill.exec.rpc.bit.timeout=0 -Dlog.path=${DRILL_LOG_DIR}/drill.log
> -Dlog.query.path=${DRILL_LOG_DIR}/query.log
> -Djava.io.tmpdir=/Users/vrozov/Projects/Apache/drill/exec/java-exec/target
> -Xms512m -Xmx4096m -Ddrill.exec.http.enabled=false
> -Ddrill.exec.sys.store.provider.local.write=false
> -Dorg.apache.drill.exec.server.Drillbit.system_options=org.apache.drill.exec.compile.ClassTransformer.scalar_replacement=on
> -Ddrill.test.query.printing.silent=true
> -Ddrill.catastrophic_to_standard_out=true -XX:MaxPermSize=512M
> -XX:MaxDirectMemorySize=3072M -Djava.net.preferIPv4Stack=true
> -Djava.awt.headless=true -XX:+CMSClassUnloadingEnabled -ea
> -Didea.test.cyclic.buffer.size=1048576
> -javaagent:/Users/vrozov/Library/Caches/IntelliJIdea2017.3/captureAgent/debugger-agent.jar=/private/var/folders/52/11m3mlk902g_wwp856y3sdvcgp/T/capture.props
> -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ
> IDEA.app/Contents/lib/idea_rt.jar:/Applications/IntelliJ
> IDEA.app/Contents/plugins/junit/lib/junit-rt.jar:/Applications/IntelliJ
> IDEA.app/Contents/plugins/junit/lib/junit5-rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/vrozov/Projects/Apache/drill/exec/java-exec/target/test-classes:/Users/vrozov/Projects/Apache/drill/exec/java-exec/target/classes:/Users/vrozov/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/vrozov/.m2/repository/org/apache/kerby/kerb-client/1.0.0-RC2/kerb-client-1.0.0-RC2.jar:/Users/vrozov/.m2/repository/o
[jira] [Commented] (DRILL-6080) Sort incorrectly limits batch size to 65535 records rather than 65536
[
https://issues.apache.org/jira/browse/DRILL-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324927#comment-16324927
]
ASF GitHub Bot commented on DRILL-6080:
---
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/1090
DRILL-6080: Sort incorrectly limits batch size to 65535 records
Sort incorrectly limits batch size to 65535 records rather than 65536.
This PR also includes a few code cleanup items.
Also fixes DRILL-6086: Overflow in offset vector in row set writer
@vrozov, please review.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/paul-rogers/drill DRILL-6080
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/1090.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1090
commit c1d3402a619f3355e47e845aae245fd0f96e2189
Author: Paul Rogers
Date: 2018-01-11T00:04:53Z
DRILL-6080: Sort incorrectly limits batch size to 65535 records
Sort incorrectly limits batch size to 65535 records rather than 65536.
This PR also includes a few code cleanup items.
Fix for overflow in offset vector in row set writer
> Sort incorrectly limits batch size to 65535 records rather than 65536
> -
>
> Key: DRILL-6080
> URL: https://issues.apache.org/jira/browse/DRILL-6080
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Minor
> Fix For: 1.13.0
>
>
> Drill places an upper limit on the number of rows in a batch of 64K. That is
> 65,536 decimal. When we index records, the indexes run from 0 to 64K-1 or 0
> to 65,535.
> The sort code incorrectly uses {{Character.MAX_VALUE}} as the maximum row
> count. So, if an incoming batch uses the full 64K size, sort ends up
> splitting batches unnecessarily.
> The fix is to instead use the correct constant `ValueVector.MAX_ROW_COUNT`.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5741) Automatically manage memory allocations during startup
[
https://issues.apache.org/jira/browse/DRILL-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5741:
Description:
Currently, during startup, a Drillbit can be assigned large values for the
following:
* Xmx (Heap)
* XX:MaxDirectMemorySize
* XX:ReservedCodeCacheSize
* XX:MaxPermSize
All of this, potentially, can exceed the available memory on a system when a
Drillbit is under heavy load. It would be good to have the Drillbit ensure
during startup itself that the cumulative value of these parameters does not
exceed a pre-defined upper limit for the Drill process.
This JIRA is a *proposal* to allow for automatic configuration (based on
configuration patterns observed in production Drill clusters). It leverages the
capability of providing default/distribution (and user-specific) checks during
Drill Startup from DRILL-6068.
The idea is to remove the need for a user to worry about managing the tuning
parameters, by providing the optimal values. In addition, it also allows for
the memory allocation to be implicitly managed by simply providing the Drill
process with a single dimensional of total process memory (either in absolute
values, or as a percentage of the total system memory), while {{auto-setup.sh}}
provides the individual allocations.
This allocation is then partitioned into allocations for Heap and Direct
Memory, with a small portion allocated for the Generated Java CodeCache as
well. If any of the individual allocations are also specified (via
{{distrib-env.sh}} or {{drill-env.sh}}), the remaining unspecified allocations
are adjusted to stay +within the limits+ of the total memory allocation.
The *details* of the proposal are here:
https://docs.google.com/spreadsheets/d/1N6VYlQFiPoTV4iD46XbkIrvEQesiGFUU9-GWXYsAPXs/edit#gid=0
For those unable to access the Google Document, PDFs are attached:
* [^Auto Mem Allocation Proposal - Computation Logic.pdf] - Provides the
equation used for computing the heap, direct and code cache allocations for a
given input
* [^Auto Mem Allocation Proposal - Scenarios.pdf] - Describes the various
inputs, and their expected allocations
The variables that are (_optionally_) defined (in memory, {{distrib-env.sh}} or
{{drill-env.sh}} ) are:
* {{DRILLBIT_MAX_PROC_MEM}} : Total Process Memory
* {{DRILL_HEAP}} : JVM Max Heap Size
* {{DRILL_MAX_DIRECT_MEMORY}} : JVM Max Direct Memory Size
* {{DRILLBIT_CODE_CACHE_SIZE}} : JVM Code Cache Size
Note: _With JDK8, MaxPermSize is no longer supported, so we do not account for
this any more, and will unset the variable if JDK8 or higher is detected._
was:
Currently, during startup, a Drillbit can be assigned large values for the
following:
* Xmx (Heap)
* XX:MaxDirectMemorySize
* XX:ReservedCodeCacheSize
* XX:MaxPermSize
All of this, potentially, can exceed the available memory on a system when a
Drillbit is under heavy load. It would be good to have the Drillbit ensure
during startup itself that the cumulative value of these parameters does not
exceed a pre-defined upper limit for the Drill process.
This JIRA is a *proposal* to allow for automatic configuration (based on
configuration patterns observed in production Drill clusters). It leverages the
capability of providing distribution (and user-specific) checks during Drill
Startup from DRILL-6068.
The idea is to remove the need for a user to worry about managing the tuning
parameters, by providing the optimal values. In addition, it also allows for
the memory allocation to be implicitly managed by simply providing the Drill
process with a single dimensional of total process memory (either in absolute
values, or as a percentage of the total system memory), while
{{distrib-auto.sh}} provides the individual allocations.
This allocation is then partitioned into allocations for Heap and Direct
Memory, with a small portion allocated for the Generated Java CodeCache as
well. If any of the individual allocations are also specified (via
{{distrib-env.sh}} or {{drill-env.sh}}), the remaining unspecified allocations
are adjusted to stay +within the limits+ of the total memory allocation.
The *details* of the proposal are here:
https://docs.google.com/spreadsheets/d/1N6VYlQFiPoTV4iD46XbkIrvEQesiGFUU9-GWXYsAPXs/edit#gid=0
For those unable to access the Google Document, PDFs are attached:
* [^Auto Mem Allocation Proposal - Computation Logic.pdf] - Provides the
equation used for computing the heap, direct and code cache allocations for a
given input
* [^Auto Mem Allocation Proposal - Scenarios.pdf] - Describes the various
inputs, and their expected allocations
The variables that are (_optionally_) defined (in memory, {{distrib-env.sh}} or
{{drill-env.sh}} ) are:
* {{DRILLBIT_MAX_PROC_MEM}} : Total Process Memory
* {{DRILL_HEAP}} : JVM Max Heap Size
* {{DRILL_MAX_DIRECT_MEMORY}} : JVM Max Direct Memory Size
* {{DRILLBIT_CODE_CACHE_SIZE}} : JVM Code Cac
[jira] [Commented] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324863#comment-16324863 ] ASF GitHub Bot commented on DRILL-6078: --- GitHub user chunhui-shi opened a pull request: https://github.com/apache/drill/pull/1089 DRILL-6078: support timestamp type to be pushed into MapRDB You can merge this pull request into a Git repository by running: $ git pull https://github.com/chunhui-shi/drill work3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/1089.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1089 commit 1c2a7c94fcdb56e7e430879b26f1b3e5a5144d11 Author: chunhui-shi Date: 2018-01-12T23:14:44Z DRILL-6078: support timestamp type to be pushed into MapRDB > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Chunhui Shi > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-6076) Reduce the default memory from a total of 13GB to 5GB
[ https://issues.apache.org/jira/browse/DRILL-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324757#comment-16324757 ] ASF GitHub Bot commented on DRILL-6076: --- Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1086 Ok. Looks like [DRILL-5926](https://issues.apache.org/jira/browse/DRILL-5926) (committed into Apache master) explicitly has a need for bumping up the memory to 4GB for heap alone. I guess we'll need to figure out something else to manage this. > Reduce the default memory from a total of 13GB to 5GB > - > > Key: DRILL-6076 > URL: https://issues.apache.org/jira/browse/DRILL-6076 > Project: Apache Drill > Issue Type: Task >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Fix For: 1.13.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, the default memory requirements for Drill are about 13GB, with the > following allocations: > * 4GB Heap > * 8GB Direct Memory > * 1GB CodeCache > * 512MB MaxPermSize > Also, with Drill 1.12.0, the recommendation is to move to JDK8, which makes > the MaxPermSize as irrelevant. > With that, the default requirements total to 13GB, which is rather high. This > is especially a problem for scenarios where people are trying out Drill and > might be using this in a development environment where 13GB is too high. > When using the public [test > framework|https://github.com/mapr/drill-test-framework/] for Apache Drill, it > was observed that the framework's functional and unit tests passed > successfully with memory as little as 5GB; based on the following allocation: > * 1GB Heap > * 3GB Direct Memory > * 512MB CodeCache > * 512MB MaxPermSize > Based on this finding, the proposal is to reduce the defaults from the > current settings to the values just mentioned above. The drill-env.sh file > already has details in the comments, along with the recommended values that > reflect the original 13GB defaults. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-6076) Reduce the default memory from a total of 13GB to 5GB
[ https://issues.apache.org/jira/browse/DRILL-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324758#comment-16324758 ] ASF GitHub Bot commented on DRILL-6076: --- Github user kkhatua closed the pull request at: https://github.com/apache/drill/pull/1086 > Reduce the default memory from a total of 13GB to 5GB > - > > Key: DRILL-6076 > URL: https://issues.apache.org/jira/browse/DRILL-6076 > Project: Apache Drill > Issue Type: Task >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Fix For: 1.13.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, the default memory requirements for Drill are about 13GB, with the > following allocations: > * 4GB Heap > * 8GB Direct Memory > * 1GB CodeCache > * 512MB MaxPermSize > Also, with Drill 1.12.0, the recommendation is to move to JDK8, which makes > the MaxPermSize as irrelevant. > With that, the default requirements total to 13GB, which is rather high. This > is especially a problem for scenarios where people are trying out Drill and > might be using this in a development environment where 13GB is too high. > When using the public [test > framework|https://github.com/mapr/drill-test-framework/] for Apache Drill, it > was observed that the framework's functional and unit tests passed > successfully with memory as little as 5GB; based on the following allocation: > * 1GB Heap > * 3GB Direct Memory > * 512MB CodeCache > * 512MB MaxPermSize > Based on this finding, the proposal is to reduce the defaults from the > current settings to the values just mentioned above. The drill-env.sh file > already has details in the comments, along with the recommended values that > reflect the original 13GB defaults. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (DRILL-3958) Improve error message when JDBC driver not found
[ https://issues.apache.org/jira/browse/DRILL-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khatua closed DRILL-3958. --- > Improve error message when JDBC driver not found > > > Key: DRILL-3958 > URL: https://issues.apache.org/jira/browse/DRILL-3958 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP >Affects Versions: 1.2.0 >Reporter: Uwe Geercken >Assignee: Kunal Khatua >Priority: Critical > Labels: ready-to-commit > Fix For: 1.13.0 > > > When setting up a storage definition for JDBC in the Drill web UI, the > appropriate driver has to be available in the 3rdparty folder before defining > the storage, otherwise an error is displayed. > The error message refers to a JSON mapping error which is completely > inappropriate in this case, because the error is the missing JDBC driver in > the 3rdparty folder and not the JSON mapping. > I request to change the error message to something appropriate that the > class/driver referred to could not be found (like for example: > com.mysql.jdbc.Driver) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324752#comment-16324752
]
ASF GitHub Bot commented on DRILL-5868:
---
Github user kkhatua commented on the issue:
https://github.com/apache/drill/pull/1084
@arina-ielchiieva updated the changes based on your review. Please have a
look.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
> !SnippetExample.png|Snippets in Action!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324750#comment-16324750
]
ASF GitHub Bot commented on DRILL-5868:
---
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/1084#discussion_r161350730
--- Diff:
exec/java-exec/src/main/resources/rest/static/js/ace-code-editor/snippets/sql.js
---
@@ -0,0 +1,46 @@
+/**
+ * Drill SQL Syntax Snippets
+ */
+
+ace.define("ace/snippets/sql",["require","exports","module"],
function(require, exports, module) {
+"use strict";
+
+exports.snippetText = "snippet info\n\
+ select * from INFORMATION_SCHEMA.${1:};\n\
+snippet sysmem\n\
+ select * from sys.mem;\n\
+snippet sysopt\n\
+ select * from sys.opt;\n\
+snippet sysbit\n\
+ select * from sys.bit;\n\
+snippet sysconn\n\
+ select * from sys.conn;\n\
+snippet sysprof\n\
+ select * from sys.prof;\n\
+snippet cview\n\
+ create view ${1:[workspace]}.${2:} ( ${3:} ) as
\n\
+ ${4:};\n\
+snippet ctas\n\
+ create table ${1:} ( ${2:} ) as \n\
+ ${3:};\n\
+snippet ctemp\n\
+ create temporary table ${1:} ( ${2:} ) as \n\
--- End diff --
Done.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
> !SnippetExample.png|Snippets in Action!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (DRILL-6086) TestSortImpl.testLargeBatch unit test fails randomly
Vlad Rozov created DRILL-6086:
-
Summary: TestSortImpl.testLargeBatch unit test fails randomly
Key: DRILL-6086
URL: https://issues.apache.org/jira/browse/DRILL-6086
Project: Apache Drill
Issue Type: Bug
Reporter: Vlad Rozov
{noformat}
Failed tests:
TestSortImpl.testLargeBatch:513->runJumboBatchTest:486->runLargeSortTest:455
Value of 1:0 expected:<0> but was:<1>
{noformat}
The test fails due to memory corruption caused by a write out of the direct
buffer allocated space. With bounds check enabled, the test fails reliably with
{noformat}
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java
-agentlib:jdwp=transport=dt_socket,address=127.0.0.1:57731,suspend=y,server=n
-Dvisualvm.id=131461133353377 -Ddrill.exec.rpc.user.timeout=0
-Ddrill.exec.rpc.bit.timeout=0 -Dlog.path=${DRILL_LOG_DIR}/drill.log
-Dlog.query.path=${DRILL_LOG_DIR}/query.log
-Djava.io.tmpdir=/Users/vrozov/Projects/Apache/drill/exec/java-exec/target
-Xms512m -Xmx4096m -Ddrill.exec.http.enabled=false
-Ddrill.exec.sys.store.provider.local.write=false
-Dorg.apache.drill.exec.server.Drillbit.system_options=org.apache.drill.exec.compile.ClassTransformer.scalar_replacement=on
-Ddrill.test.query.printing.silent=true
-Ddrill.catastrophic_to_standard_out=true -XX:MaxPermSize=512M
-XX:MaxDirectMemorySize=3072M -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -XX:+CMSClassUnloadingEnabled -ea
-Didea.test.cyclic.buffer.size=1048576
-javaagent:/Users/vrozov/Library/Caches/IntelliJIdea2017.3/captureAgent/debugger-agent.jar=/private/var/folders/52/11m3mlk902g_wwp856y3sdvcgp/T/capture.props
-Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ
IDEA.app/Contents/lib/idea_rt.jar:/Applications/IntelliJ
IDEA.app/Contents/plugins/junit/lib/junit-rt.jar:/Applications/IntelliJ
IDEA.app/Contents/plugins/junit/lib/junit5-rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/vrozov/Projects/Apache/drill/exec/java-exec/target/test-classes:/Users/vrozov/Projects/Apache/drill/exec/java-exec/target/classes:/Users/vrozov/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/vrozov/.m2/repository/org/apache/kerby/kerb-client/1.0.0-RC2/kerb-client-1.0.0-RC2.jar:/Users/vrozov/.m2/repository/org/apache/kerby/kerby-config/1.0.0-RC2/kerby-config-1.0.0-RC2.jar:/Users/vrozov/.m2/repository/org/apache/kerby/kerb-common/1.0.0-RC2/kerb-common-1.0.0-RC2.jar:/Users/vrozov/.m2/repository/org/apache/kerby/kerb-crypto/1.0.0-RC2/kerb-crypto-1.0.0-RC2.jar:/Users/vrozov/.m2/repository/org/apache/ke
[jira] [Updated] (DRILL-5967) Memory leak by HashPartitionSender
[
https://issues.apache.org/jira/browse/DRILL-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Farkas updated DRILL-5967:
--
Labels: ready-to-commit (was: )
> Memory leak by HashPartitionSender
> --
>
> Key: DRILL-5967
> URL: https://issues.apache.org/jira/browse/DRILL-5967
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
> Labels: ready-to-commit
>
> The error found by [~cch...@maprtech.com] and [~dechanggu]
> {code}
> 2017-10-25 15:43:28,658 [260eec84-7de3-03ec-300f-7fdbc111fb7c:frag:2:9] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException:
> Memory was leaked by query. Memory leaked: (9216)
> Allocator(op:2:9:0:HashPartitionSender) 100/9216/12831744/100
> (res/actual/peak/limit)
> Fragment 2:9
> [Error Id: 7eae6c2a-868c-49f8-aad8-b690243ffe9b on mperf113.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (9216)
> Allocator(op:2:9:0:HashPartitionSender) 100/9216/12831744/100
> (res/actual/peak/limit)
> Fragment 2:9
> [Error Id: 7eae6c2a-868c-49f8-aad8-b690243ffe9b on mperf113.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:586)
> ~[drill-common-1.11.0-mapr.jar:1.11.0-mapr]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:301)
> [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:267)
> [drill-java-exec-1.11.0-mapr.jar:1.11.0-mapr]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.11.0-mapr.jar:1.11.0-mapr]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (9216)
> Allocator(op:2:9:0:HashPartitionSender) 100/9216/12831744/100
> (res/actual/peak/limit)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324586#comment-16324586
]
Pritesh Maker commented on DRILL-5851:
--
added the ready-to-commit label since paul review and providing a +1
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
> Labels: ready-to-commit
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5851) Empty table during a join operation with a non empty table produces cast exception
[
https://issues.apache.org/jira/browse/DRILL-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-5851:
-
Labels: ready-to-commit (was: )
> Empty table during a join operation with a non empty table produces cast
> exception
> ---
>
> Key: DRILL-5851
> URL: https://issues.apache.org/jira/browse/DRILL-5851
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Hanumath Rao Maduri
>Assignee: Hanumath Rao Maduri
> Labels: ready-to-commit
>
> Hash Join operation on tables with one table empty and the other non empty
> throws an exception
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Join only supports implicit casts
> between 1. Numeric data
> 2. Varchar, Varbinary data 3. Date, Timestamp data Left type: VARCHAR, Right
> type: INT. Add explicit casts to avoid this error
> {code}
> Here is an example query with which it is reproducible.
> {code}
> select * from cp.`sample-data/nation.parquet` nation left outer join
> dfs.tmp.`2.csv` as two on two.a = nation.`N_COMMENT`;
> {code}
> the contents of 2.csv is empty (i.e not even header info).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6078) Query with INTERVAL in predicate does not return any rows
[ https://issues.apache.org/jira/browse/DRILL-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324583#comment-16324583 ] Pritesh Maker commented on DRILL-6078: -- [~cshi] I thought you had a PR ready for this one - perhaps it's not opened with a link to this JIRA? > Query with INTERVAL in predicate does not return any rows > - > > Key: DRILL-6078 > URL: https://issues.apache.org/jira/browse/DRILL-6078 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.12.0 >Reporter: Robert Hou >Assignee: Chunhui Shi > > This query does not return any rows when accessing MapR DB tables. > SELECT > C.C_CUSTKEY, > C.C_NAME, > SUM(L.L_EXTENDEDPRICE * (1 - L.L_DISCOUNT)) AS REVENUE, > C.C_ACCTBAL, > N.N_NAME, > C.C_ADDRESS, > C.C_PHONE, > C.C_COMMENT > FROM > customer C, > orders O, > lineitem L, > nation N > WHERE > C.C_CUSTKEY = O.O_CUSTKEY > AND L.L_ORDERKEY = O.O_ORDERKEY > AND O.O_ORDERDate >= DATE '1994-03-01' > AND O.O_ORDERDate < DATE '1994-03-01' + INTERVAL '3' MONTH > AND L.L_RETURNFLAG = 'R' > AND C.C_NATIONKEY = N.N_NATIONKEY > GROUP BY > C.C_CUSTKEY, > C.C_NAME, > C.C_ACCTBAL, > C.C_PHONE, > N.N_NAME, > C.C_ADDRESS, > C.C_COMMENT > ORDER BY > REVENUE DESC > LIMIT 20 > This query works against JSON tables. It should return 20 rows. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (DRILL-6085) Travis build sometimes fails becomes vm suddenly exits.
Timothy Farkas created DRILL-6085:
-
Summary: Travis build sometimes fails becomes vm suddenly exits.
Key: DRILL-6085
URL: https://issues.apache.org/jira/browse/DRILL-6085
Project: Apache Drill
Issue Type: Bug
Reporter: Timothy Farkas
Assignee: Timothy Farkas
Travis fails with the following error.
{code}
Failed to execute goal
[32morg.apache.maven.plugins:maven-surefire-plugin:2.17:test[m
[1m(default-test)[m on project [36mdrill-java-exec[m: [1;31mExecution
default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test
failed: The forked VM terminated without properly saying goodbye. VM crash or
System.exit called?[m
{code}
I believe this error is caused by the vm running out of memory. I will try
bumping up the memory slightly and doing several runs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6076) Reduce the default memory from a total of 13GB to 5GB
[ https://issues.apache.org/jira/browse/DRILL-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324461#comment-16324461 ] ASF GitHub Bot commented on DRILL-6076: --- Github user kkhatua commented on the issue: https://github.com/apache/drill/pull/1086 I think folks running unit tests can continue with the existing limits of 4GB heap+4GB Direct. The idea is to get Drill up and running for minimal use cases, and you are expected to bump up memory to higher limits if queries get OOM errors. Unit tests, if carrying any tests that require much more memory, would qualify under this. I've launched a [fork](https://github.com/kkhatua/drill/commits/LowerMemUnitTest) of this branch with reduced memory settings for the unit tests as well and letting TravisCI test it out: https://travis-ci.org/kkhatua/drill/builds/328250155 If it passes, we can either reduce those settings in the POM files as well, or leave it intact to ensure developers doing unit tests are able to run the tests in a reasonable amount of time. > Reduce the default memory from a total of 13GB to 5GB > - > > Key: DRILL-6076 > URL: https://issues.apache.org/jira/browse/DRILL-6076 > Project: Apache Drill > Issue Type: Task >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Fix For: 1.13.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, the default memory requirements for Drill are about 13GB, with the > following allocations: > * 4GB Heap > * 8GB Direct Memory > * 1GB CodeCache > * 512MB MaxPermSize > Also, with Drill 1.12.0, the recommendation is to move to JDK8, which makes > the MaxPermSize as irrelevant. > With that, the default requirements total to 13GB, which is rather high. This > is especially a problem for scenarios where people are trying out Drill and > might be using this in a development environment where 13GB is too high. > When using the public [test > framework|https://github.com/mapr/drill-test-framework/] for Apache Drill, it > was observed that the framework's functional and unit tests passed > successfully with memory as little as 5GB; based on the following allocation: > * 1GB Heap > * 3GB Direct Memory > * 512MB CodeCache > * 512MB MaxPermSize > Based on this finding, the proposal is to reduce the defaults from the > current settings to the values just mentioned above. The drill-env.sh file > already has details in the comments, along with the recommended values that > reflect the original 13GB defaults. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (DRILL-6084) Expose list of Drill functions for consumption by JavaScript libraries
Kunal Khatua created DRILL-6084:
---
Summary: Expose list of Drill functions for consumption by
JavaScript libraries
Key: DRILL-6084
URL: https://issues.apache.org/jira/browse/DRILL-6084
Project: Apache Drill
Issue Type: Improvement
Components: Web Server
Reporter: Kunal Khatua
Priority: Minor
Fix For: Future
DRILL-5868 introduces SQL highlighter and the auto-complete feature in the
web-UI's query editor.
For the backend Javascript to highlight the keywords correctly as SQL reserved
words or functions, it needs a list of all the Drill functions to be already
defined in a source JavaScript ( {{../mode-sql.js}} ) .
While this works well for standard Drill functions, it means that any new
function added to Drill needs to be explicitly hard-coded into the file, rather
than be programatically discovered and loaded for the library to highlight.
It would help if this can be done dynamically without contributors having to
explicitly alter the script file to add new function names.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Description:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
*Snapshot:*
was:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
*Snapshot:*
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Description:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
*Snapshot:*
!SnippetExample.png|Snippets in Action!
was:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
*Snapshot:*
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
> !SnippetExample.png|Snippets in Action!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Description:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
*Snapshot:*
was:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Attachment: SnippetExample.png
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
> Attachments: SnippetExample.png
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
> *Snapshot:*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Description:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
was:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this JIRA's commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Description:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFnJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
was:
It would be nice to have the Query Editor support syntax highlighting.
An autocomplete would be even better as new functions are introduced in Drill
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFnJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-5868:
Description:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFuncJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
was:
Based on the commit for DRILL-5981 ([PR
#1043|https://github.com/apache/drill/pull/1043]), this commit further
leverages the Ace JavaScript library with customizations specific to Drill.
This introduces the following to Drill:
# Syntax highlighting (This is also supported for submitted query profiles)
# Auto-complete supported in all SQL editors (including the Edit Query tab
within an existing profile to rerunning the query).
For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show a
drop down list for use. Arrow keys are used for navigating to the desired
option.
# Specifying Drill specific keywords and functions in visible autocomplete
# Key snippets (template SQLs) allowing for rapid writing of syntax:
i. Query System Tables
ii. CView, CTAS, CTempTAS and CFnJar
iii. Alter Session
iv. Explain and Select * queries
This is done by leveraging the *Snippets* feature in the Ace library, which
allows for writing SQL code from templates. Like auto-complete, this will
insert a template for you. The required/optional fields are then selected (in
order) for you to populate, as you complete each and navigate to the next using
{{Tab}}.
NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
added to Drill, these lists can be expanded.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Based on the commit for DRILL-5981 ([PR
> #1043|https://github.com/apache/drill/pull/1043]), this commit further
> leverages the Ace JavaScript library with customizations specific to Drill.
> This introduces the following to Drill:
> # Syntax highlighting (This is also supported for submitted query profiles)
> # Auto-complete supported in all SQL editors (including the Edit Query tab
> within an existing profile to rerunning the query).
> For browsers like Chrome, you should be able to type {{Ctrl+Space}} to show
> a drop down list for use. Arrow keys are used for navigating to the desired
> option.
> # Specifying Drill specific keywords and functions in visible autocomplete
> # Key snippets (template SQLs) allowing for rapid writing of syntax:
> i. Query System Tables
> ii. CView, CTAS, CTempTAS and CFuncJar
> iii. Alter Session
> iv. Explain and Select * queries
> This is done by leveraging the *Snippets* feature in the Ace library, which
> allows for writing SQL code from templates. Like auto-complete, this will
> insert a template for you. The required/optional fields are then selected (in
> order) for you to populate, as you complete each and navigate to the next
> using {{Tab}}.
> NOTE: The lists for items 3 and 4 are not exhaustive. As more features are
> added to Drill, these lists can be expanded.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6076) Reduce the default memory from a total of 13GB to 5GB
[ https://issues.apache.org/jira/browse/DRILL-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324388#comment-16324388 ] ASF GitHub Bot commented on DRILL-6076: --- Github user parthchandra commented on the issue: https://github.com/apache/drill/pull/1086 The settings for running the unit tests are (from the pom file : -Xms512m -Xmx4096m -XX:MaxPermSize=512M -XX:MaxDirectMemorySize=4096M -XX:+CMSClassUnloadingEnabled -ea At the very least, the default settings for Drill and the unit tests should match. > Reduce the default memory from a total of 13GB to 5GB > - > > Key: DRILL-6076 > URL: https://issues.apache.org/jira/browse/DRILL-6076 > Project: Apache Drill > Issue Type: Task >Reporter: Kunal Khatua >Assignee: Kunal Khatua >Priority: Critical > Fix For: 1.13.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, the default memory requirements for Drill are about 13GB, with the > following allocations: > * 4GB Heap > * 8GB Direct Memory > * 1GB CodeCache > * 512MB MaxPermSize > Also, with Drill 1.12.0, the recommendation is to move to JDK8, which makes > the MaxPermSize as irrelevant. > With that, the default requirements total to 13GB, which is rather high. This > is especially a problem for scenarios where people are trying out Drill and > might be using this in a development environment where 13GB is too high. > When using the public [test > framework|https://github.com/mapr/drill-test-framework/] for Apache Drill, it > was observed that the framework's functional and unit tests passed > successfully with memory as little as 5GB; based on the following allocation: > * 1GB Heap > * 3GB Direct Memory > * 512MB CodeCache > * 512MB MaxPermSize > Based on this finding, the proposal is to reduce the defaults from the > current settings to the values just mentioned above. The drill-env.sh file > already has details in the comments, along with the recommended values that > reflect the original 13GB defaults. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-6003) Unit test TestDynamicUDFSupport.testLazyInitWhenDynamicUdfSupportIsDisabled fails with FUNCTION ERROR: Failure reading Function class.
[
https://issues.apache.org/jira/browse/DRILL-6003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Farkas updated DRILL-6003:
--
Fix Version/s: 1.13.0
> Unit test TestDynamicUDFSupport.testLazyInitWhenDynamicUdfSupportIsDisabled
> fails with FUNCTION ERROR: Failure reading Function class.
> --
>
> Key: DRILL-6003
> URL: https://issues.apache.org/jira/browse/DRILL-6003
> Project: Apache Drill
> Issue Type: Bug
> Components: Tools, Build & Test
>Affects Versions: 1.12.0
>Reporter: Abhishek Girish
>Assignee: Timothy Farkas
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> {code}
> 14:05:23.170 [main] ERROR org.apache.drill.TestReporter - Test Failed (d: 0
> B(1 B), h: 229.7 MiB(1.1 GiB), nh: 187.0 KiB(73.2 MiB)):
> testLazyInitWhenDynamicUdfSupportIsDisabled(org.apache.drill.TestDynamicUDFSupport)
> org.apache.drill.exec.rpc.RpcException:
> org.apache.drill.common.exceptions.UserRemoteException: FUNCTION ERROR:
> Failure reading Function class.
> Function Class com.drill.udf.CustomLowerFunction
> Fragment 0:0
> [Error Id: 1d6ea0e5-fd65-4622-924d-d196defaedc8 on 10.10.104.57:31010]
> at
> org.apache.drill.exec.rpc.RpcException.mapException(RpcException.java:60)
> ~[drill-rpc-1.12.0.jar:1.12.0]
> at
> org.apache.drill.exec.client.DrillClient$ListHoldingResultsListener.getResults(DrillClient.java:865)
> ~[classes/:na]
> at
> org.apache.drill.exec.client.DrillClient.runQuery(DrillClient.java:567)
> ~[classes/:na]
> at
> org.apache.drill.test.BaseTestQuery.testRunAndReturn(BaseTestQuery.java:338)
> ~[test-classes/:na]
> at
> org.apache.drill.test.BaseTestQuery$ClassicTestServices.testRunAndReturn(BaseTestQuery.java:276)
> ~[test-classes/:na]
> at
> org.apache.drill.test.DrillTestWrapper.testRunAndReturn(DrillTestWrapper.java:830)
> ~[test-classes/:na]
> at
> org.apache.drill.test.DrillTestWrapper.compareUnorderedResults(DrillTestWrapper.java:484)
> ~[test-classes/:na]
> at
> org.apache.drill.test.DrillTestWrapper.run(DrillTestWrapper.java:147)
> ~[test-classes/:na]
> at org.apache.drill.test.TestBuilder.go(TestBuilder.java:139)
> ~[test-classes/:na]
> at
> org.apache.drill.TestDynamicUDFSupport.testLazyInitWhenDynamicUdfSupportIsDisabled(TestDynamicUDFSupport.java:506)
> ~[test-classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.7.0_131]
> at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_131]
> at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_131]
> org.apache.drill.common.exceptions.UserRemoteException: FUNCTION ERROR:
> Failure reading Function class.
> Function Class com.drill.udf.CustomLowerFunction
> Fragment 0:0
> [Error Id: 1d6ea0e5-fd65-4622-924d-d196defaedc8 on 10.10.104.57:31010]
> at
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
> ~[classes/:na]
> at
> org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:468)
> ~[classes/:na]
> at
> org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:102)
> ~[classes/:na]
> at
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:274)
> ~[drill-rpc-1.12.0.jar:1.12.0]
> at
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:244)
> ~[drill-rpc-1.12.0.jar:1.12.0]
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
> ~[netty-codec-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
> ~[netty-transport-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
> ~[netty-transport-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
> ~[netty-transport-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
> ~[netty-handler-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
> ~[netty-transport-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
> ~[netty-transport-4.0.48.Final.jar:4.0.48.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
> ~[netty-transport-4.0.48.Final.jar:4.0.48
[jira] [Resolved] (DRILL-6047) Update doc to include instructions for libpam4j
[ https://issues.apache.org/jira/browse/DRILL-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens resolved DRILL-6047. --- Resolution: Fixed > Update doc to include instructions for libpam4j > --- > > Key: DRILL-6047 > URL: https://issues.apache.org/jira/browse/DRILL-6047 > Project: Apache Drill > Issue Type: Sub-task > Components: Documentation >Affects Versions: 1.12.0 >Reporter: Bridget Bevens >Assignee: Bridget Bevens >Priority: Minor > Fix For: 1.12.0 > > > Update Apache Drill docs to include JPAM and libpam4j PAM authenticator > instructions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-4606) Create DrillClient.Builder class
[ https://issues.apache.org/jira/browse/DRILL-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-4606: - Labels: (was: ready-to-commit) > Create DrillClient.Builder class > > > Key: DRILL-4606 > URL: https://issues.apache.org/jira/browse/DRILL-4606 > Project: Apache Drill > Issue Type: Improvement >Reporter: Sudheesh Katkam >Assignee: Sudheesh Katkam > > + Create a helper class to build DrillClient instances, and deprecate > DrillClient constructors > + Allow DrillClient to specify an event loop group (so user event loop can be > used for queries from Web API calls) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5919) Add non-numeric support for JSON processing
[
https://issues.apache.org/jira/browse/DRILL-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324331#comment-16324331
]
ASF GitHub Bot commented on DRILL-5919:
---
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/1026
> Add non-numeric support for JSON processing
> ---
>
> Key: DRILL-5919
> URL: https://issues.apache.org/jira/browse/DRILL-5919
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - JSON
>Affects Versions: 1.11.0
>Reporter: Volodymyr Tkach
>Assignee: Volodymyr Tkach
> Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Add session options to allow drill working with non standard json strings
> number literals like: NaN, Infinity, -Infinity. By default these options will
> be switched off, the user will be able to toggle them during working session.
> *For documentation*
> 1. Added two session options {{store.json.reader.allow_nan_inf}} and
> {{store.json.writer.allow_nan_inf}} that allow to read/write NaN and Infinity
> as numbers. By default these options are set to true.
> 2. Extended signature of {{convert_toJSON}} and {{convert_fromJSON}}
> functions by adding second optional parameter that enables read/write NaN and
> Infinity.
> For example:
> {noformat}
> select convert_fromJSON('{"key": NaN}') from (values(1)); will result with
> JsonParseException, but
> select convert_fromJSON('{"key": NaN}', true) from (values(1)); will parse
> NaN as a number.
> {noformat}
> 3. Added unit tests, including tests for math functions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6025) Execution time of a running query shown as 'NOT AVAILABLE'
[ https://issues.apache.org/jira/browse/DRILL-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324336#comment-16324336 ] ASF GitHub Bot commented on DRILL-6025: --- Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1074 > Execution time of a running query shown as 'NOT AVAILABLE' > -- > > Key: DRILL-6025 > URL: https://issues.apache.org/jira/browse/DRILL-6025 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP >Affects Versions: 1.11.0 >Reporter: Prasad Nagaraj Subramanya >Assignee: Prasad Nagaraj Subramanya >Priority: Minor > Labels: ready-to-commit > Fix For: 1.13.0 > > > When a query is in 'RUNNING' state, the execution time is shown as 'NOT > AVAILABLE' > We could show the execution duration till the current time -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-3958) Improve error message when JDBC driver not found
[ https://issues.apache.org/jira/browse/DRILL-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324330#comment-16324330 ] ASF GitHub Bot commented on DRILL-3958: --- Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1067 > Improve error message when JDBC driver not found > > > Key: DRILL-3958 > URL: https://issues.apache.org/jira/browse/DRILL-3958 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP >Affects Versions: 1.2.0 >Reporter: Uwe Geercken >Assignee: Kunal Khatua >Priority: Critical > Labels: ready-to-commit > Fix For: 1.13.0 > > > When setting up a storage definition for JDBC in the Drill web UI, the > appropriate driver has to be available in the 3rdparty folder before defining > the storage, otherwise an error is displayed. > The error message refers to a JSON mapping error which is completely > inappropriate in this case, because the error is the missing JDBC driver in > the 3rdparty folder and not the JSON mapping. > I request to change the error message to something appropriate that the > class/driver referred to could not be found (like for example: > com.mysql.jdbc.Driver) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5971) Fix INT64, INT32 logical types in complex parquet reader
[ https://issues.apache.org/jira/browse/DRILL-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324334#comment-16324334 ] ASF GitHub Bot commented on DRILL-5971: --- Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1049 > Fix INT64, INT32 logical types in complex parquet reader > > > Key: DRILL-5971 > URL: https://issues.apache.org/jira/browse/DRILL-5971 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0 >Reporter: Parth Chandra >Assignee: Parth Chandra > Labels: ready-to-commit > Fix For: 1.13.0 > > > The 'complex' Parquet reader does not recognize the Parquet logical types > INT64, and INT32. > Should be a simple change to add these logical types. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-6054) Issues in FindPartitionConditions
[ https://issues.apache.org/jira/browse/DRILL-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324338#comment-16324338 ] ASF GitHub Bot commented on DRILL-6054: --- Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1078 > Issues in FindPartitionConditions > - > > Key: DRILL-6054 > URL: https://issues.apache.org/jira/browse/DRILL-6054 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.12.0 >Reporter: Chunhui Shi >Assignee: Chunhui Shi > Labels: ready-to-commit > Fix For: 1.13.0 > > > When the condition is these cases, partition is not done correctly: > b = 3 OR (dir0 = 1 and a = 2) > not (dir0 = 1 AND b = 2) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5922) Intermittent Memory Leaks in the ROOT allocator
[
https://issues.apache.org/jira/browse/DRILL-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324332#comment-16324332
]
ASF GitHub Bot commented on DRILL-5922:
---
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/1023
> Intermittent Memory Leaks in the ROOT allocator
> -
>
> Key: DRILL-5922
> URL: https://issues.apache.org/jira/browse/DRILL-5922
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Timothy Farkas
>Assignee: Timothy Farkas
>Priority: Minor
> Labels: ready-to-commit
>
> This issue was originall found by [~ben-zvi]. I am able to consistently
> reproduce the error on my laptop by running the following unit test:
> org.apache.drill.exec.DrillSeparatePlanningTest#testMultiMinorFragmentComplexQuery
> {code}
> java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding
> child allocators.
> Allocator(ROOT) 0/1048576/10113536/3221225472 (res/actual/peak/limit)
> child allocators: 1
> Allocator(query:26049b50-0cec-0a92-437c-bbe486e1fcbf)
> 1048576/0/0/268435456 (res/actual/peak/limit)
> child allocators: 0
> ledgers: 0
> reservations: 0
> ledgers: 0
> reservations: 0
> at
> org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:496)
> ~[classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76)
> [classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64)
> [classes/:na]
> at
> org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:256)
> ~[classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76)
> [classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64)
> [classes/:na]
> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:205)
> [classes/:na]
> at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:315)
> [test-classes/:na]
> at
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:157)
> [test-classes/:na]
> at
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:148)
> [test-classes/:na]
> at
> org.apache.drill.exec.DrillSeparatePlanningTest.getFragmentsHelper(DrillSeparatePlanningTest.java:185)
> [test-classes/:na]
> at
> org.apache.drill.exec.DrillSeparatePlanningTest.testMultiMinorFragmentComplexQuery(DrillSeparatePlanningTest.java:108)
> [test-classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_144]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_144]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_144]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> [junit-4.11.jar:na]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.11.jar:na]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> [junit-4.11.jar:na]
> at
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
> [jmockit-1.3.jar:na]
> at
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
> [jmockit-1.3.jar:na]
> at
> mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
> [jmockit-1.3.jar:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_144]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_144]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_144]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
> at
> mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
> [jmockit-1.3.jar:na]
> at
> mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
> [jmockit-1.3.jar:na]
> at
> mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
> [jmockit-1.3.jar:na]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
> [junit-4.11.jar:na]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.11.jar:na]
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementT
[jira] [Commented] (DRILL-6004) Direct buffer bounds checking should be disabled by default
[
https://issues.apache.org/jira/browse/DRILL-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324335#comment-16324335
]
ASF GitHub Bot commented on DRILL-6004:
---
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/1070
> Direct buffer bounds checking should be disabled by default
> ---
>
> Key: DRILL-6004
> URL: https://issues.apache.org/jira/browse/DRILL-6004
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
>Priority: Minor
> Labels: ready-to-commit
>
> Direct buffer bounds checking is enabled either when assertions are enabled
> (see DRILL-6001) or when {{drill.enable_unsafe_memory_access}} property is
> not set to true, so it is enabled in production as by default
> {{drill.enable_unsafe_memory_access}} is not set.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6030) Managed sort should minimize number of batches in a k-way merge
[ https://issues.apache.org/jira/browse/DRILL-6030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324337#comment-16324337 ] ASF GitHub Bot commented on DRILL-6030: --- Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/1075 > Managed sort should minimize number of batches in a k-way merge > --- > > Key: DRILL-6030 > URL: https://issues.apache.org/jira/browse/DRILL-6030 > Project: Apache Drill > Issue Type: Improvement >Reporter: Vlad Rozov >Assignee: Vlad Rozov > Labels: ready-to-commit > > The time complexity of the algorithm is O(n*k*log(k)) where k is a number of > batches to merge and n is a number of records in each batch (assuming equal > size batches). As n*k is the total number of record to merge and it can be > quite large, minimizing k should give better results. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5961) For long running queries (> 10 min) Drill may raise FragmentSetupException for completed/cancelled fragments
[
https://issues.apache.org/jira/browse/DRILL-5961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324333#comment-16324333
]
ASF GitHub Bot commented on DRILL-5961:
---
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/1041
> For long running queries (> 10 min) Drill may raise FragmentSetupException
> for completed/cancelled fragments
>
>
> Key: DRILL-5961
> URL: https://issues.apache.org/jira/browse/DRILL-5961
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> {{WorkEventBus}} uses {{recentlyFinishedFragments}} cache to check for
> completed or cancelled fragments. Such check is not reliable as entries in
> {{recentlyFinishedFragments}} expire after 10 minutes, so
> {{FragmentSetupException}} is raised even for completed or cancelled queries.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5868) Support SQL syntax highlighting of queries
[
https://issues.apache.org/jira/browse/DRILL-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324317#comment-16324317
]
ASF GitHub Bot commented on DRILL-5868:
---
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/1084#discussion_r161288148
--- Diff:
exec/java-exec/src/main/resources/rest/static/js/ace-code-editor/snippets/sql.js
---
@@ -0,0 +1,46 @@
+/**
+ * Drill SQL Syntax Snippets
+ */
+
+ace.define("ace/snippets/sql",["require","exports","module"],
function(require, exports, module) {
+"use strict";
+
+exports.snippetText = "snippet info\n\
+ select * from INFORMATION_SCHEMA.${1:};\n\
+snippet sysmem\n\
+ select * from sys.mem;\n\
+snippet sysopt\n\
+ select * from sys.opt;\n\
+snippet sysbit\n\
+ select * from sys.bit;\n\
+snippet sysconn\n\
+ select * from sys.conn;\n\
+snippet sysprof\n\
+ select * from sys.prof;\n\
+snippet cview\n\
--- End diff --
Good catch.
I was generating the code for the snippets script, so I missed these (they
went into a recent commit). I'll fix these.
> Support SQL syntax highlighting of queries
> --
>
> Key: DRILL-5868
> URL: https://issues.apache.org/jira/browse/DRILL-5868
> Project: Apache Drill
> Issue Type: New Feature
> Components: Web Server
>Affects Versions: 1.12.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.13.0
>
>
> It would be nice to have the Query Editor support syntax highlighting.
> An autocomplete would be even better as new functions are introduced in Drill
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (DRILL-5822) The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order
[
https://issues.apache.org/jira/browse/DRILL-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex closed DRILL-5822.
---
> The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1
> doesn't preserve column order
> ---
>
> Key: DRILL-5822
> URL: https://issues.apache.org/jira/browse/DRILL-5822
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Vitalii Diravka
> Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> Columns ordering doesn't preserve for the star query with sorting when this
> is planned into multiple fragments.
> Repro steps:
> 1) {code}alter session set `planner.slice_target`=1;{code}
> 2) ORDER BY clause in the query.
> Scenarios:
> {code}
> 0: jdbc:drill:zk=local> alter session reset `planner.slice_target`;
> +---++
> | ok |summary |
> +---++
> | true | planner.slice_target updated. |
> +---++
> 1 row selected (0.082 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by
> n_name limit 1;
> +--+--+--+--+
> | n_nationkey | n_name | n_regionkey | n_comment
> |
> +--+--+--+--+
> | 0| ALGERIA | 0| haggle. carefully final deposits
> detect slyly agai |
> +--+--+--+--+
> 1 row selected (0.141 seconds)
> 0: jdbc:drill:zk=local> alter session set `planner.slice_target`=1;
> +---++
> | ok |summary |
> +---++
> | true | planner.slice_target updated. |
> +---++
> 1 row selected (0.091 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by
> n_name limit 1;
> +--+--+--+--+
> | n_comment | n_name |
> n_nationkey | n_regionkey |
> +--+--+--+--+
> | haggle. carefully final deposits detect slyly agai | ALGERIA | 0
>| 0|
> +--+--+--+--+
> 1 row selected (0.201 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5822) The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order
[
https://issues.apache.org/jira/browse/DRILL-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324095#comment-16324095
]
Alex commented on DRILL-5822:
-
The fix was re-tested in v.1.13.0-SNAPSHOT/1.12.0, status - PASSED:
- Setup planner.slice_target=1
- Ran SELECT queries (with order by/order by desc clauses) from different data
sources (from parquet/json/csv files, database table)
> The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1
> doesn't preserve column order
> ---
>
> Key: DRILL-5822
> URL: https://issues.apache.org/jira/browse/DRILL-5822
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Prasad Nagaraj Subramanya
>Assignee: Vitalii Diravka
> Labels: ready-to-commit
> Fix For: 1.12.0
>
>
> Columns ordering doesn't preserve for the star query with sorting when this
> is planned into multiple fragments.
> Repro steps:
> 1) {code}alter session set `planner.slice_target`=1;{code}
> 2) ORDER BY clause in the query.
> Scenarios:
> {code}
> 0: jdbc:drill:zk=local> alter session reset `planner.slice_target`;
> +---++
> | ok |summary |
> +---++
> | true | planner.slice_target updated. |
> +---++
> 1 row selected (0.082 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by
> n_name limit 1;
> +--+--+--+--+
> | n_nationkey | n_name | n_regionkey | n_comment
> |
> +--+--+--+--+
> | 0| ALGERIA | 0| haggle. carefully final deposits
> detect slyly agai |
> +--+--+--+--+
> 1 row selected (0.141 seconds)
> 0: jdbc:drill:zk=local> alter session set `planner.slice_target`=1;
> +---++
> | ok |summary |
> +---++
> | true | planner.slice_target updated. |
> +---++
> 1 row selected (0.091 seconds)
> 0: jdbc:drill:zk=local> select * from cp.`tpch/nation.parquet` order by
> n_name limit 1;
> +--+--+--+--+
> | n_comment | n_name |
> n_nationkey | n_regionkey |
> +--+--+--+--+
> | haggle. carefully final deposits detect slyly agai | ALGERIA | 0
>| 0|
> +--+--+--+--+
> 1 row selected (0.201 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5846) Improve Parquet Reader Performance for Flat Data types
[ https://issues.apache.org/jira/browse/DRILL-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324035#comment-16324035 ] ASF GitHub Bot commented on DRILL-5846: --- Github user sachouche commented on the issue: https://github.com/apache/drill/pull/1060 @paul-rogers with regard to the design aspects that you brought up: ** Corrections about the proposed Design ** Your analysis somehow assumes the Vector is the one driving the loading operation. This is not the case, a) it is the reader which invokes the bulk save API b) the Vector save method runs a loop requesting the rest of the data. The reader can any time stop the loading phase; currently the reader uses the batch size to make this decision. The plan is to enhance this logic by having a feedback from the Vector save method about memory usage (e.g., save current row set with the condition the memory usage doesn't get X bytes). I believe I have raised this design decision early on and the agreement was to implement the memory batch restrictions in a next Drill release. ** Agile Development ** Through the years I came to appreciate the value of agile development. One needs to prioritize design activities; it is completely valid to start with a design (which is not guaranteed to be perfect) and then refine it overtime as long as the underlying implementation doesn't spill to other modules (encapsulation). This has the merit of showcasing incremental improvements and allowing the developer to get new insight as they have a better understanding. > Improve Parquet Reader Performance for Flat Data types > --- > > Key: DRILL-5846 > URL: https://issues.apache.org/jira/browse/DRILL-5846 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Parquet >Affects Versions: 1.11.0 >Reporter: salim achouche >Assignee: salim achouche > Labels: performance > Fix For: 1.13.0 > > > The Parquet Reader is a key use-case for Drill. This JIRA is an attempt to > further improve the Parquet Reader performance as several users reported that > Parquet parsing represents the lion share of the overall query execution. It > tracks Flat Data types only as Nested DTs might involve functional and > processing enhancements (e.g., a nested column can be seen as a Document; > user might want to perform operations scoped at the document level that is no > need to span all rows). Another JIRA will be created to handle the nested > columns use-case. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[ https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323927#comment-16323927 ] ASF GitHub Bot commented on DRILL-6049: --- Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1085#discussion_r161208998 --- Diff: common/src/test/java/org/apache/drill/test/DrillTest.java --- @@ -54,8 +56,7 @@ static MemWatcher memWatcher; static String className; - @Rule public final TestRule TIMEOUT = TestTools.getTimeoutRule(10); - + @Rule public final TestRule TIMEOUT = new DisableOnDebug(TestTools.getTimeoutRule(100_000)); --- End diff -- Great enhancement! > Rollup of hygiene changes from "batch size" project > --- > > Key: DRILL-6049 > URL: https://issues.apache.org/jira/browse/DRILL-6049 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.12.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > Fix For: 1.13.0 > > > PR to include all non-core changes related to the batch size project to allow > simpler review of the core changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[
https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323928#comment-16323928
]
ASF GitHub Bot commented on DRILL-6049:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1085#discussion_r161208584
--- Diff: common/src/main/java/org/apache/drill/common/types/Types.java ---
@@ -728,4 +733,49 @@ public static boolean isLaterType(MajorType type) {
return type.getMinorType() == MinorType.LATE;
}
+ public static boolean isEquivalent(MajorType type1, MajorType type2) {
+
+// Requires full type equality, including fields such as precision and
scale.
+// But, unset fields are equivalent to 0. Can't use the
protobuf-provided
+// isEquals() which treats set and unset fields as different.
+
+if (type1.getMinorType() != type2.getMinorType() ||
+type1.getMode() != type2.getMode() ||
+type1.getScale() != type2.getScale() ||
+type1.getPrecision() != type2.getPrecision()) {
+ return false;
+}
+
+// Subtypes are only for unions and are seldom used.
+
+if (type1.getMinorType() != MinorType.UNION) {
+ return true;
+}
+
+List subtypes1 = type1.getSubTypeList();
+List subtypes2 = type2.getSubTypeList();
+if (subtypes1 == subtypes2) { // Only occurs if both are null
+ return true;
+}
+if (subtypes1 == null || subtypes2 == null) {
+ return false;
+}
+if (subtypes1.size() != subtypes2.size()) {
+ return false;
+}
+
+// Now it gets slow because subtype lists are not ordered.
+
+List copy1 = new ArrayList<>();
+List copy2 = new ArrayList<>();
+copy1.addAll(subtypes1);
+copy2.addAll(subtypes2);
+Collections.sort(copy1);
+Collections.sort(copy2);
+return copy1.equals(copy2);
+ }
+
+ public static String typeKey(MinorType type) {
--- End diff --
Why we need this method? It looks like it is never used? If it's intended
for future use please add java doc then.
> Rollup of hygiene changes from "batch size" project
> ---
>
> Key: DRILL-6049
> URL: https://issues.apache.org/jira/browse/DRILL-6049
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.13.0
>
>
> PR to include all non-core changes related to the batch size project to allow
> simpler review of the core changes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[
https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323926#comment-16323926
]
ASF GitHub Bot commented on DRILL-6049:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1085#discussion_r161211784
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java ---
@@ -278,22 +273,62 @@ public void addDoubleMetrics(OperatorProfile.Builder
builder) {
}
}
- @Override
+ /**
+ * Set a stat to the specified long value. Creates the stat
+ * if the stat does not yet exist.
+ *
+ * @param metric the metric to update
+ * @param value the value to set
+ */
+
public void addLongStat(MetricDef metric, long value){
longMetrics.putOrAdd(metric.metricId(), value, value);
}
- @Override
+ @VisibleForTesting
+ public long getLongStat(MetricDef metric) {
+return longMetrics.get(metric.metricId());
--- End diff --
If metric is absent, will it return 0 or fail?
> Rollup of hygiene changes from "batch size" project
> ---
>
> Key: DRILL-6049
> URL: https://issues.apache.org/jira/browse/DRILL-6049
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.13.0
>
>
> PR to include all non-core changes related to the batch size project to allow
> simpler review of the core changes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[
https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323925#comment-16323925
]
ASF GitHub Bot commented on DRILL-6049:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1085#discussion_r161207650
--- Diff:
common/src/main/java/org/apache/drill/common/exceptions/UserException.java ---
@@ -83,23 +83,17 @@ public static Builder memoryError() {
* The cause message will be used unless {@link
Builder#message(String, Object...)} is called.
* If the wrapped exception is, or wraps, a user exception it will be
returned by {@link Builder#build(Logger)}
* instead of creating a new exception. Any added context will be added
to the user exception as well.
- *
- * This exception, previously deprecated, has been repurposed to
indicate unspecified
- * errors. In particular, the case in which a lower level bit of code
throws an
- * exception other than UserException. The catching code then only knows
"something went
- * wrong", but not enough information to categorize the error.
- *
- * System errors also indicate illegal internal states, missing
functionality, and other
- * code-related errors -- all of which "should never occur."
*
* @see
org.apache.drill.exec.proto.UserBitShared.DrillPBError.ErrorType#SYSTEM
*
* @param cause exception we want the user exception to wrap. If cause
is, or wrap, a user exception it will be
* returned by the builder instead of creating a new user
exception
* @return user exception builder
*
+ * @deprecated This method should never need to be used explicitly,
unless you are passing the exception to the
+ * Rpc layer or UserResultListener.submitFailed()
*/
-
+ @Deprecated
--- End diff --
Deprecated means that we might remove it in future. But since you mention
that it is still ok to use in certain cases, I am not sure that such annotation
is appropriate.
> Rollup of hygiene changes from "batch size" project
> ---
>
> Key: DRILL-6049
> URL: https://issues.apache.org/jira/browse/DRILL-6049
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.13.0
>
>
> PR to include all non-core changes related to the batch size project to allow
> simpler review of the core changes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-6049) Rollup of hygiene changes from "batch size" project
[ https://issues.apache.org/jira/browse/DRILL-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323924#comment-16323924 ] ASF GitHub Bot commented on DRILL-6049: --- Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/1085#discussion_r161215163 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/ExecTest.java --- @@ -38,20 +38,26 @@ import org.apache.drill.exec.server.options.SystemOptionManager; import org.apache.drill.exec.store.sys.store.provider.LocalPersistentStoreProvider; import org.apache.drill.exec.util.GuavaPatcher; +import org.apache.drill.test.BaseDirTestWatcher; --- End diff -- Starting from this class, github does not highlight Java code. I remember @vdiravka had similar problem, it turned out that he copied code from text editor with some hidden symbols. Please check and fix. > Rollup of hygiene changes from "batch size" project > --- > > Key: DRILL-6049 > URL: https://issues.apache.org/jira/browse/DRILL-6049 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.12.0 >Reporter: Paul Rogers >Assignee: Paul Rogers > Fix For: 1.13.0 > > > PR to include all non-core changes related to the batch size project to allow > simpler review of the core changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (DRILL-5690) RepeatedDecimal18Vector does not pass scale, precision to data vector
[
https://issues.apache.org/jira/browse/DRILL-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Volodymyr Vysotskyi resolved DRILL-5690.
Resolution: Fixed
Fixed in
https://github.com/apache/drill/commit/40de8ca4f47533fa6593d1266403868ae1a2119f
> RepeatedDecimal18Vector does not pass scale, precision to data vector
> -
>
> Key: DRILL-5690
> URL: https://issues.apache.org/jira/browse/DRILL-5690
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.13.0
>
>
> Decimal types require not just the type (Decimal9, Decimal18, etc.) but also
> a precision and scale. The triple of (minor type, precision, scale) appears
> in the {{MaterializedField}} for the nullable or required vectors.
> A repeated vector has three parts: the {{RepeatedDecimal18Vector}} which is
> composed of a {{UInt4Vector}} offset vector and a {{Decimal18Vector}} that
> holds values.
> When {{RepeatedDecimal18Vector}} creates the {{Decimal18Vector}} to hold the
> values, it clones the {{MaterializedField}}. But, it *does not* clone the
> scale and precision, resulting in the loss of critical information.
> {code}
> public RepeatedDecimal18Vector(MaterializedField field, BufferAllocator
> allocator) {
> super(field, allocator);
>
> addOrGetVector(VectorDescriptor.create(Types.required(field.getType().getMinorType(;
> }
> {code}
> This is normally not a problem because most code access the data via the
> repeated vector. But, for code that needs to work with the values, the
> results are wrong given that the types are wrong. (Values stored with one
> scale, 123.45, (scale 2) will be retrieved with 0 scale (123, say).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-4185) UNION ALL involving empty directory on any side of union all results in Failed query
[
https://issues.apache.org/jira/browse/DRILL-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323874#comment-16323874
]
ASF GitHub Bot commented on DRILL-4185:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/1083#discussion_r161205848
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/TestJoinNullable.java ---
@@ -568,6 +570,22 @@ public void nullMixedComparatorEqualJoinHelper(final
String query) throws Except
.go();
}
+ /** InnerJoin with empty dir table on nullable cols, MergeJoin */
+ // TODO: the same tests should be added for HashJoin operator, DRILL-6070
+ @Test
--- End diff --
Please add test for merge join with ignore annotation. Also please check
that NLJ also works fine. If not, please fix.
> UNION ALL involving empty directory on any side of union all results in
> Failed query
>
>
> Key: DRILL-4185
> URL: https://issues.apache.org/jira/browse/DRILL-4185
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.4.0
>Reporter: Khurram Faraaz
>Assignee: Vitalii Diravka
>
> UNION ALL query that involves an empty directory on either side of UNION ALL
> operator results in FAILED query. We should return the results for the
> non-empty side (input) of UNION ALL.
> Note that empty_DIR is an empty directory, the directory exists, but it has
> no files in it.
> Drill 1.4 git.commit.id=b9068117
> 4 node cluster on CentOS
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select columns[0] from empty_DIR UNION ALL
> select cast(columns[0] as int) c1 from `testWindow.csv`;
> Error: VALIDATION ERROR: From line 1, column 24 to line 1, column 32: Table
> 'empty_DIR' not found
> [Error Id: 5c024786-6703-4107-8a4a-16c96097be08 on centos-01.qa.lab:31010]
> (state=,code=0)
> 0: jdbc:drill:schema=dfs.tmp> select cast(columns[0] as int) c1 from
> `testWindow.csv` UNION ALL select columns[0] from empty_DIR;
> Error: VALIDATION ERROR: From line 1, column 90 to line 1, column 98: Table
> 'empty_DIR' not found
> [Error Id: 58c98bc4-99df-425c-aa07-c8c5faec4748 on centos-01.qa.lab:31010]
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-3993) Rebase Drill on Calcite master branch
[
https://issues.apache.org/jira/browse/DRILL-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323793#comment-16323793
]
ASF GitHub Bot commented on DRILL-3993:
---
Github user vvysotskyi commented on a diff in the pull request:
https://github.com/apache/drill/pull/1066#discussion_r161185413
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/AggPruleBase.java
---
@@ -82,4 +83,19 @@ protected boolean create2PhasePlan(RelOptRuleCall call,
DrillAggregateRel aggreg
}
return true;
}
+
+ /**
+ * Returns group-by keys with the remapped arguments for specified
aggregate.
+ *
+ * @param groupSet ImmutableBitSet of aggregate rel node, whose group-by
keys should be remapped.
+ * @return {@link ImmutableBitSet} instance with remapped keys.
+ */
+ public static ImmutableBitSet remapGroupSet(ImmutableBitSet groupSet) {
--- End diff --
After the changes in CALCITE-1930 in the class
`AggregateExpandDistinctAggregatesRule`, this rule started applying more
correctly, since in the older version there were checks like this:
```
aggCall.getAggregation() instanceof SqlCountAggFunction
```
but they were replaced by checks like this:
```
final SqlKind aggCallKind = aggCall.getAggregation().getKind();
switch (aggCallKind)
```
So for the cases when instead of Calcite s `SqlCountAggFunction` were used
`DrillCalciteSqlAggFunctionWrapper`s this rule changed its behaviour and
instead of returning joins of distinct and non-distinct aggregate rel nodes, it
returns distinct aggregate which has an input with non-distinct aggregates
grouped by column, which is distinct in outer aggregate.
These Drill rules were able to work correctly only for the cases when
aggregate rel node does not contain ` aggCalls` and contains only the single
field in `rowType` (in this case `groupSet` is always `{0}`, and it still
correct for outer aggregates which are created in `*AggPrule`s)
With the new version of `AggregateExpandDistinctAggregatesRule` these Drill
rules received aggregate rel nodes with the non-empty lists of ` aggCalls`,
therefore its `rowType` contains more than one field. But before my changes,
the same `groupSet` was passed to the constructor of outer aggregate and row
type of aggregate differed from the row type of its input. So it was incorrect
`groupSet`.
Aggregate rel nodes always specify the group by columns in the first
positions of the list, so correct `groupSet` for outer aggregate should be
`ImmutableBitSet` with the same size as the `groupSet` of nested aggregate, but
the ordinals of columns should start from 0.
As for the point of iterating through the group set,
`ImmutableBitSet.size()`, `ImmutableBitSet.length()` and
`ImmutableBitSet.cardinality()` does not return desired "size" of the
`groupSet`. `AggregateExpandDistinctAggregatesRule` also contains similar code
which iterating through the `groupSet` for similar purposes.
> Rebase Drill on Calcite master branch
> -
>
> Key: DRILL-3993
> URL: https://issues.apache.org/jira/browse/DRILL-3993
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0
>Reporter: Sudheesh Katkam
>Assignee: Roman Kulyk
>
> Calcite keeps moving, and now we need to catch up to Calcite 1.5, and ensure
> there are no regressions.
> Also, how do we resolve this 'catching up' issue in the long term?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-6081) Duration of completed queries is continuously increasing
[
https://issues.apache.org/jira/browse/DRILL-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6081:
Reviewer: Paul Rogers
> Duration of completed queries is continuously increasing
>
>
> Key: DRILL-6081
> URL: https://issues.apache.org/jira/browse/DRILL-6081
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Arina Ielchiieva
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Steps:
> 1. Execute some query (for example "select * from sys.version")
> 2. Go to the profiles page: http://node1:8047/profiles
> 3. Open the details page for the query
> 4. Expand the duration section
> Expected result:
> The duration should be adequate and shouldn't be changed after the page reload
> Actual result:
> The duration is continuously increasing (You should reload page to notice
> that)
> ||Planning||Queued||Execution||Total||
> |0.092 sec|0.012 sec|59 min 36.487 sec|59 min 36.591 sec|
> Workaround: none
> Note:
> The issue introduced after the following fix:
> https://issues.apache.org/jira/browse/DRILL-5963
> *ROOT CAUSE*
> After refactoring made in DRILL-5963 making query start / queue / end time
> was moved to another abstraction {{queryStateProcessor}}. On Foreman close
> {{queryStateProcessor}} was closed after final query profile was stored thus
> query end time was never updated. To fix this issue we need to close
> {{queryStateProcessor}} before writing query final profile.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-6081) Duration of completed queries is continuously increasing
[
https://issues.apache.org/jira/browse/DRILL-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6081:
Labels: ready-to-commit (was: )
> Duration of completed queries is continuously increasing
>
>
> Key: DRILL-6081
> URL: https://issues.apache.org/jira/browse/DRILL-6081
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Anton Gozhiy
>Assignee: Arina Ielchiieva
> Labels: ready-to-commit
> Fix For: 1.13.0
>
>
> Steps:
> 1. Execute some query (for example "select * from sys.version")
> 2. Go to the profiles page: http://node1:8047/profiles
> 3. Open the details page for the query
> 4. Expand the duration section
> Expected result:
> The duration should be adequate and shouldn't be changed after the page reload
> Actual result:
> The duration is continuously increasing (You should reload page to notice
> that)
> ||Planning||Queued||Execution||Total||
> |0.092 sec|0.012 sec|59 min 36.487 sec|59 min 36.591 sec|
> Workaround: none
> Note:
> The issue introduced after the following fix:
> https://issues.apache.org/jira/browse/DRILL-5963
> *ROOT CAUSE*
> After refactoring made in DRILL-5963 making query start / queue / end time
> was moved to another abstraction {{queryStateProcessor}}. On Foreman close
> {{queryStateProcessor}} was closed after final query profile was stored thus
> query end time was never updated. To fix this issue we need to close
> {{queryStateProcessor}} before writing query final profile.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5919) Add non-numeric support for JSON processing
[
https://issues.apache.org/jira/browse/DRILL-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323741#comment-16323741
]
Arina Ielchiieva commented on DRILL-5919:
-
[~parthc] it does not. DRILL-6018 was created for future enhancement after this
Jira will be merged.
> Add non-numeric support for JSON processing
> ---
>
> Key: DRILL-5919
> URL: https://issues.apache.org/jira/browse/DRILL-5919
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - JSON
>Affects Versions: 1.11.0
>Reporter: Volodymyr Tkach
>Assignee: Volodymyr Tkach
> Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Add session options to allow drill working with non standard json strings
> number literals like: NaN, Infinity, -Infinity. By default these options will
> be switched off, the user will be able to toggle them during working session.
> *For documentation*
> 1. Added two session options {{store.json.reader.allow_nan_inf}} and
> {{store.json.writer.allow_nan_inf}} that allow to read/write NaN and Infinity
> as numbers. By default these options are set to true.
> 2. Extended signature of {{convert_toJSON}} and {{convert_fromJSON}}
> functions by adding second optional parameter that enables read/write NaN and
> Infinity.
> For example:
> {noformat}
> select convert_fromJSON('{"key": NaN}') from (values(1)); will result with
> JsonParseException, but
> select convert_fromJSON('{"key": NaN}', true) from (values(1)); will parse
> NaN as a number.
> {noformat}
> 3. Added unit tests, including tests for math functions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
