[jira] [Commented] (DRILL-5721) Query with only root fragment and no non-root fragment hangs when Drillbit to Drillbit Control Connection has network issues
[ https://issues.apache.org/jira/browse/DRILL-5721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144620#comment-16144620 ] ASF GitHub Bot commented on DRILL-5721: --- Github user sohami commented on the issue: https://github.com/apache/drill/pull/919 Rebased the branch to latest master and updated test's included in DRILL-5701 > Query with only root fragment and no non-root fragment hangs when Drillbit to > Drillbit Control Connection has network issues > > > Key: DRILL-5721 > URL: https://issues.apache.org/jira/browse/DRILL-5721 > Project: Apache Drill > Issue Type: Bug >Reporter: Sorabh Hamirwasia >Assignee: Sorabh Hamirwasia > Fix For: 1.12.0 > > > Recently I found an issue (Thanks to [~knguyen] to create this scenario) > related to Fragment Status reporting and would like some feedback on it. > When a client submits a query to Foreman, then it is planned by Foreman and > later fragments are scheduled to root and non-root nodes. Foreman creates a > DriilbitStatusListener and FragmentStatusListener to know about the health of > Drillbit node and a fragment respectively. The way root and non-root > fragments are setup by Foreman are different: > Root fragments are setup without any communication over control channel > (since it is executed locally on Foreman) > Non-root fragments are setup by sending control message > (REQ_INITIALIZE_FRAGMENTS_VALUE) over wire. If there is failure in sending > any such control message (like due to network hiccup's) during query setup > then the query is failed and client is notified. > Each fragment is executed on it's node with the help Fragment Executor which > has an instance for FragmentStatusReporter. FragmentStatusReporter helps to > update the status of a fragment to Foreman node over a control tunnel or > connection using RPC message (REQ_FRAGMENT_STATUS) both for root and non-root > fragments. > Based on above when root fragment is submitted for setup then it is done > locally without any RPC communication whereas when status for that fragment > is reported by fragment executor that happens over control connection by > sending a RPC message. But for non-root fragment setup and status update both > happens using RPC message over control connection. > *Issue 1:* > What was observed is if for a simple query which has only 1 root fragment > running on Foreman node then setup will work fine. But as part of status > update when the fragment tries to create a control connection and fails to > establish that, then the query hangs. This is because the root fragment will > complete execution but will fail to update Foreman about it and Foreman think > that the query is running for ever. > *Proposed Solution:* > For root fragment the setup of fragment is happening locally without RPC > message, so we can do the same for status update of root fragments. This will > avoid RPC communication for status update of fragments running locally on the > foreman and hence will resolve issue 1. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5723) Support System/Session Internal Options
[ https://issues.apache.org/jira/browse/DRILL-5723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144612#comment-16144612 ] ASF GitHub Bot commented on DRILL-5723: --- Github user ilooner commented on the issue: https://github.com/apache/drill/pull/923 @paul-rogers Applied your comments and cleaned up the PR, it should be ready for review now. If additional work needs to be done to make this easier to review please let me know. Thanks, Tim > Support System/Session Internal Options > --- > > Key: DRILL-5723 > URL: https://issues.apache.org/jira/browse/DRILL-5723 > Project: Apache Drill > Issue Type: New Feature >Reporter: Timothy Farkas >Assignee: Timothy Farkas > > This is a feature proposed by [~ben-zvi]. > Currently all the options are accessible by the user in sys.options. We would > like to add internal options which can be altered, but are not visible in the > sys.options table. These internal options could be seen by another alias > select * from internal.options. The intention would be to put new options we > weren't comfortable with exposing to the end user in this table. > After the options and their corresponding features are considered stable they > could be changed to appear in the sys.option table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5729) Fix Travis Checks
[ https://issues.apache.org/jira/browse/DRILL-5729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144606#comment-16144606 ] ASF GitHub Bot commented on DRILL-5729: --- Github user ilooner commented on the issue: https://github.com/apache/drill/pull/913 @parthchandra I think the PR still needs a +1 from a committer. If Vlad's review is sufficient, could you give this a +1? Thanks, Tim > Fix Travis Checks > - > > Key: DRILL-5729 > URL: https://issues.apache.org/jira/browse/DRILL-5729 > Project: Apache Drill > Issue Type: Bug >Reporter: Timothy Farkas >Assignee: Timothy Farkas > Fix For: 1.12.0 > > > Currently the Travis Checks are failing. The failures are happening because > Travis recently switched their default build containers from Ubuntu precise > to Ubuntu Trusty and we do not explicitly define the dist we build on in our > travis.yml -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-5748) Web UI displays JSON message for a missing query profile
[
https://issues.apache.org/jira/browse/DRILL-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-5748:
---
Description:
Launched a local Drillbit, using ZK. Entered the following query:
Ran a simple query. Clicked on the Profiles tab. Clicked on a query. Got the
following JSON error message:
{code}
{
"errorMessage" : "HTTP 404 Not Found"
}
{code}
Not sure if this is due to a misconfiguration error in my branch, or due to
some real issue. In either case, it would seem that the UI should display the
error as HTML, not JSON.
was:
Launched a local Drillbit, using ZK. Entered the following query:
{code}
ALTER SYSTEM SET `exec.queue.enable` = true
{code}
Result:
{code}
Query Failed: An Error Occurred
java.lang.NullPointerException
{code}
Turns out, the problem is more general. The following produces the same error:
{code}
SELECT name, type FROM sys.options WHERE type in ('SYSTEM','SESSION') order by
name;
{code}
> Web UI displays JSON message for a missing query profile
>
>
> Key: DRILL-5748
> URL: https://issues.apache.org/jira/browse/DRILL-5748
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Priority: Minor
>
> Launched a local Drillbit, using ZK. Entered the following query:
> Ran a simple query. Clicked on the Profiles tab. Clicked on a query. Got the
> following JSON error message:
> {code}
> {
> "errorMessage" : "HTTP 404 Not Found"
> }
> {code}
> Not sure if this is due to a misconfiguration error in my branch, or due to
> some real issue. In either case, it would seem that the UI should display the
> error as HTML, not JSON.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5748) Web UI displays JSON message for a missing query profile
[
https://issues.apache.org/jira/browse/DRILL-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-5748:
---
Summary: Web UI displays JSON message for a missing query profile (was:
NPE with invalid ALTER SYSTEM)
> Web UI displays JSON message for a missing query profile
>
>
> Key: DRILL-5748
> URL: https://issues.apache.org/jira/browse/DRILL-5748
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Priority: Minor
>
> Launched a local Drillbit, using ZK. Entered the following query:
> {code}
> ALTER SYSTEM SET `exec.queue.enable` = true
> {code}
> Result:
> {code}
> Query Failed: An Error Occurred
> java.lang.NullPointerException
> {code}
> Turns out, the problem is more general. The following produces the same error:
> {code}
> SELECT name, type FROM sys.options WHERE type in ('SYSTEM','SESSION') order
> by name;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5748) NPE with invalid ALTER SYSTEM
[
https://issues.apache.org/jira/browse/DRILL-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-5748:
---
Description:
Launched a local Drillbit, using ZK. Entered the following query:
{code}
ALTER SYSTEM SET `exec.queue.enable` = true
{code}
Result:
{code}
Query Failed: An Error Occurred
java.lang.NullPointerException
{code}
Turns out, the problem is more general. The following produces the same error:
{code}
SELECT name, type FROM sys.options WHERE type in ('SYSTEM','SESSION') order by
name;
{code}
was:
Launched a local Drillbit, using ZK. Entered the following query:
{code}
ALTER SYSTEM SET `exec.queue.enable` = true
{code}
Result:
{code}
Query Failed: An Error Occurred
java.lang.NullPointerException
{code}
> NPE with invalid ALTER SYSTEM
> -
>
> Key: DRILL-5748
> URL: https://issues.apache.org/jira/browse/DRILL-5748
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.11.0
>Reporter: Paul Rogers
>Priority: Minor
>
> Launched a local Drillbit, using ZK. Entered the following query:
> {code}
> ALTER SYSTEM SET `exec.queue.enable` = true
> {code}
> Result:
> {code}
> Query Failed: An Error Occurred
> java.lang.NullPointerException
> {code}
> Turns out, the problem is more general. The following produces the same error:
> {code}
> SELECT name, type FROM sys.options WHERE type in ('SYSTEM','SESSION') order
> by name;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (DRILL-5748) NPE with invalid ALTER SYSTEM
Paul Rogers created DRILL-5748:
--
Summary: NPE with invalid ALTER SYSTEM
Key: DRILL-5748
URL: https://issues.apache.org/jira/browse/DRILL-5748
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.11.0
Reporter: Paul Rogers
Priority: Minor
Launched a local Drillbit, using ZK. Entered the following query:
{code}
ALTER SYSTEM SET `exec.queue.enable` = true
{code}
Result:
{code}
Query Failed: An Error Occurred
java.lang.NullPointerException
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (DRILL-5746) Pcap PR manually edited Protobuf files, values lost on next build
[
https://issues.apache.org/jira/browse/DRILL-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-5746:
---
Description:
Drill recently accepted the pcap format plugin. As part of that work, the
author added a new operator type, {{PCAP_SUB_SCAN_VALUE}}.
But, apparently this was done by editing the generated Protobuf files to add
the values, rather than modifying the protobuf definitions and rebuilding the
generated files. The result is, on the next build of the Protobuf sources, the
following compile error appears:
{code}
[ERROR]
/Users/paulrogers/git/drill/exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapFormatPlugin.java:[80,41]
error: cannot find symbol
[ERROR] symbol: variable PCAP_SUB_SCAN_VALUE
[ERROR] location: class CoreOperatorType
{code}
The solution is to properly edit the Protobuf definitions with the required
symbol.
was:
Drill recently accepted the pcap format plugin. As part of that work, the
author added a new operator type, {{PCAP_SUB_SCAN_VALUE}}.
But, apparently this was done by editing the generated Protobuf files to add
the values, rather than modifying the protobuf definitions and rebuilding the
generated files. The result is, on the next build of the Protobuf sources, the
following compile error appears:
{code}
[ERROR]
/Users/paulrogers/git/drill/exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapFormatPlugin.java:[80,41]
error: cannot find symbol
[ERROR] symbol: variable PCAP_SUB_SCAN_VALUE
[ERROR] location: class CoreOperatorType
{code}
The solution is to properly edit the Protobuf definitions with the required
symbol.
In addition, the name {{PCAP_SUB_SCAN_VALUE}} does not follow Drill's naming
conventions. It should be: {{PCAP_SUB_SCAN}}.
> Pcap PR manually edited Protobuf files, values lost on next build
> -
>
> Key: DRILL-5746
> URL: https://issues.apache.org/jira/browse/DRILL-5746
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
> Fix For: 1.12.0
>
>
> Drill recently accepted the pcap format plugin. As part of that work, the
> author added a new operator type, {{PCAP_SUB_SCAN_VALUE}}.
> But, apparently this was done by editing the generated Protobuf files to add
> the values, rather than modifying the protobuf definitions and rebuilding the
> generated files. The result is, on the next build of the Protobuf sources,
> the following compile error appears:
> {code}
> [ERROR]
> /Users/paulrogers/git/drill/exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapFormatPlugin.java:[80,41]
> error: cannot find symbol
> [ERROR] symbol: variable PCAP_SUB_SCAN_VALUE
> [ERROR] location: class CoreOperatorType
> {code}
> The solution is to properly edit the Protobuf definitions with the required
> symbol.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5747) Drill should put directory name field in same sequence w.r.t regular column for select * query
[
https://issues.apache.org/jira/browse/DRILL-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144475#comment-16144475
]
Paul Rogers commented on DRILL-5747:
This change is already being done as part of the revised {{ScanBatch}} for the
project to limit Drill batch sizes.
> Drill should put directory name field in same sequence w.r.t regular column
> for select * query
> --
>
> Key: DRILL-5747
> URL: https://issues.apache.org/jira/browse/DRILL-5747
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> Today, star column * in Drill would expand into a list of regular columns,
> and the directory name field such as dir0, dir1. However, Drill does not put
> the directory name field with respect to regular field in a consistent way.
> For instance, for parquet files, dir0 is put behind the list of regular
> columns.
> {code}
> select * from dfs.tmp.parquetTbl where dir0 = 1990;
> +--+--+--+--+---+
> | N_NATIONKEY |N_NAME| N_REGIONKEY | N_COMMENT | dir0 |
> +--+--+--+--+---+
> | 0| [B@5527446 | 0| [B@684fa264 | 1990 |
> | 1| [B@442e88bc | 1| [B@4b13119c | 1990 |
> | 2| [B@50e93f45 | 1| [B@138f483 | 1990 |
> | 3| [B@423cc515 | 1| [B@23af07ac | 1990 |
> | 4| [B@3820bf81 | 4| [B@6dfccaf0 | 1990 |
> | 5| [B@6f6f8af9 | 0| [B@40d1a97 | 1990 |
> | 6| [B@784cb194 | 3| [B@731ea93f | 1990 |
> | 7| [B@61f9a224 | 3| [B@4c041bbc | 1990 |
> | 8| [B@21b8faa1 | 2| [B@774e7152 | 1990 |
> | 9| [B@3ef1fbaf | 2| [B@c2be72| 1990 |
> | 10 | [B@71652ec1 | 4| [B@29e0bb10 | 1990 |
> | 11 | [B@61192cea | 4| [B@3bd3e873 | 1990 |
> | 12 | [B@5541f4b4 | 2| [B@5d288126 | 1990 |
> | 13 | [B@e371592 | 4| [B@42692b88 | 1990 |
> | 14 | [B@6a90fc8 | 0| [B@454b16e2 | 1990 |
> | 15 | [B@44cb72f8 | 0| [B@8e91b11 | 1990 |
> | 16 | [B@7feffda8 | 0| [B@64f66236 | 1990 |
> | 17 | [B@6ba9fb02 | 1| [B@649e7786 | 1990 |
> | 18 | [B@5fb93205 | 2| [B@7783175b | 1990 |
> | 19 | [B@3f7294a9 | 3| [B@7b7e03c9 | 1990 |
> | 20 | [B@e2ac076 | 4| [B@18c18a3e | 1990 |
> | 21 | [B@4a5af924 | 2| [B@1a9ad09f | 1990 |
> | 22 | [B@29f6845e | 3| [B@776c4cd7 | 1990 |
> | 23 | [B@6728f481 | 3| [B@31cc7610 | 1990 |
> | 24 | [B@665b2dfa | 1| [B@6c27ac95 | 1990 |
> +--+--+--+--+---+
> {code}
> Notice in the above output, dir0 = 1990 is the last column.
> However, for JSON, dir0 is put in front of the list of regular columns.
> {code}
> select * from dfs.tmp.jsonTbl where dir0 = 1990;
> +---+--+
> | dir0 | a |
> +---+--+
> | 1990 | 100 |
> | 1990 | 200 |
> +---+--+
> {code}
> It would be good to present the directory name field in the same sequence
> regardless of file format, storage plugin. IMHO, it makes sense to put the
> directory name field in front of the list of regular columns ( the behavior
> that JSON format present today).
> This ticket is opened to modify Drill's ScanBatch code for the above
> explained purpose.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (DRILL-5747) Drill should put directory name field in same sequence w.r.t regular column for select * query
Jinfeng Ni created DRILL-5747:
-
Summary: Drill should put directory name field in same sequence
w.r.t regular column for select * query
Key: DRILL-5747
URL: https://issues.apache.org/jira/browse/DRILL-5747
Project: Apache Drill
Issue Type: Bug
Reporter: Jinfeng Ni
Assignee: Jinfeng Ni
Today, star column * in Drill would expand into a list of regular columns, and
the directory name field such as dir0, dir1. However, Drill does not put the
directory name field with respect to regular field in a consistent way.
For instance, for parquet files, dir0 is put behind the list of regular columns.
{code}
select * from dfs.tmp.parquetTbl where dir0 = 1990;
+--+--+--+--+---+
| N_NATIONKEY |N_NAME| N_REGIONKEY | N_COMMENT | dir0 |
+--+--+--+--+---+
| 0| [B@5527446 | 0| [B@684fa264 | 1990 |
| 1| [B@442e88bc | 1| [B@4b13119c | 1990 |
| 2| [B@50e93f45 | 1| [B@138f483 | 1990 |
| 3| [B@423cc515 | 1| [B@23af07ac | 1990 |
| 4| [B@3820bf81 | 4| [B@6dfccaf0 | 1990 |
| 5| [B@6f6f8af9 | 0| [B@40d1a97 | 1990 |
| 6| [B@784cb194 | 3| [B@731ea93f | 1990 |
| 7| [B@61f9a224 | 3| [B@4c041bbc | 1990 |
| 8| [B@21b8faa1 | 2| [B@774e7152 | 1990 |
| 9| [B@3ef1fbaf | 2| [B@c2be72| 1990 |
| 10 | [B@71652ec1 | 4| [B@29e0bb10 | 1990 |
| 11 | [B@61192cea | 4| [B@3bd3e873 | 1990 |
| 12 | [B@5541f4b4 | 2| [B@5d288126 | 1990 |
| 13 | [B@e371592 | 4| [B@42692b88 | 1990 |
| 14 | [B@6a90fc8 | 0| [B@454b16e2 | 1990 |
| 15 | [B@44cb72f8 | 0| [B@8e91b11 | 1990 |
| 16 | [B@7feffda8 | 0| [B@64f66236 | 1990 |
| 17 | [B@6ba9fb02 | 1| [B@649e7786 | 1990 |
| 18 | [B@5fb93205 | 2| [B@7783175b | 1990 |
| 19 | [B@3f7294a9 | 3| [B@7b7e03c9 | 1990 |
| 20 | [B@e2ac076 | 4| [B@18c18a3e | 1990 |
| 21 | [B@4a5af924 | 2| [B@1a9ad09f | 1990 |
| 22 | [B@29f6845e | 3| [B@776c4cd7 | 1990 |
| 23 | [B@6728f481 | 3| [B@31cc7610 | 1990 |
| 24 | [B@665b2dfa | 1| [B@6c27ac95 | 1990 |
+--+--+--+--+---+
{code}
Notice in the above output, dir0 = 1990 is the last column.
However, for JSON, dir0 is put in front of the list of regular columns.
{code}
select * from dfs.tmp.jsonTbl where dir0 = 1990;
+---+--+
| dir0 | a |
+---+--+
| 1990 | 100 |
| 1990 | 200 |
+---+--+
{code}
It would be good to present the directory name field in the same sequence
regardless of file format, storage plugin. IMHO, it makes sense to put the
directory name field in front of the list of regular columns ( the behavior
that JSON format present today).
This ticket is opened to modify Drill's ScanBatch code for the above explained
purpose.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (DRILL-5746) Pcap PR manually edited Protobuf files, values lost on next build
Paul Rogers created DRILL-5746:
--
Summary: Pcap PR manually edited Protobuf files, values lost on
next build
Key: DRILL-5746
URL: https://issues.apache.org/jira/browse/DRILL-5746
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.10.0
Reporter: Paul Rogers
Assignee: Paul Rogers
Fix For: 1.12.0
Drill recently accepted the pcap format plugin. As part of that work, the
author added a new operator type, {{PCAP_SUB_SCAN_VALUE}}.
But, apparently this was done by editing the generated Protobuf files to add
the values, rather than modifying the protobuf definitions and rebuilding the
generated files. The result is, on the next build of the Protobuf sources, the
following compile error appears:
{code}
[ERROR]
/Users/paulrogers/git/drill/exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapFormatPlugin.java:[80,41]
error: cannot find symbol
[ERROR] symbol: variable PCAP_SUB_SCAN_VALUE
[ERROR] location: class CoreOperatorType
{code}
The solution is to properly edit the Protobuf definitions with the required
symbol.
In addition, the name {{PCAP_SUB_SCAN_VALUE}} does not follow Drill's naming
conventions. It should be: {{PCAP_SUB_SCAN}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (DRILL-5745) Invalid "location" information in Drill web server
Paul Rogers created DRILL-5745:
--
Summary: Invalid "location" information in Drill web server
Key: DRILL-5745
URL: https://issues.apache.org/jira/browse/DRILL-5745
Project: Apache Drill
Issue Type: Bug
Components: Web Server
Affects Versions: 1.11.0
Reporter: Paul Rogers
Priority: Minor
The file {{ProfileResources.java}} has the following incorrect code line:
{code}
this.location = "http://localhost:8047/profile/"; + queryId + ".json";
{code}
This code makes three errors.
1. The "http" prefix ignores the fact that the Drillbit can have SSL enabled
for the web server.
2. In a browser, "localhost" refers to the the machine running the browser.
This is valid only if the browser runs on the same machine as the Drillbit,
which is not, in general, true.
3. The port number is hardcoded to 8047, but it can be customized in the config
file.
Therefore, most of the time, the link won't work on a production server.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5741) During startup Drill should not exceed the available memory
[
https://issues.apache.org/jira/browse/DRILL-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144315#comment-16144315
]
ASF GitHub Bot commented on DRILL-5741:
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/922
@kkhatua, thanks for the note. No doubt the proposed model helps in _some_
cases. The problem is, it adds complexity for other cases.
Perhaps offer the change as an additional script that can be called from
`drill_env.sh` for those sites that need it, rather than making it part of the
core, common script. That way, this solution is orthogonal to, rather than part
of, a solution driven by a resource manager such as YARN, Mesos or Warden.
> During startup Drill should not exceed the available memory
> ---
>
> Key: DRILL-5741
> URL: https://issues.apache.org/jira/browse/DRILL-5741
> Project: Apache Drill
> Issue Type: Improvement
> Components: Server
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.12.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Currently, during startup, a Drillbit can be assigned large values for the
> following:
> * Xmx (Heap)
> * XX:MaxDirectMemorySize
> * XX:ReservedCodeCacheSize
> * XX:MaxPermSize
> All of this, potentially, can exceed the available memory on a system when a
> Drillbit is under heavy load. It would be good to have the Drillbit ensure
> during startup itself that the cumulative value of these parameters does not
> exceed a pre-defined upper limit for the Drill process.
> The proposal is to have the
> [runbit|https://github.com/apache/drill/blob/master/distribution/src/resources/runbit]
> script look for an additional environment variable:
> {{DRILLBIT_MAX_PROC_MEM}}
> The parameter can specify the maximum in GB/MB (similar in syntax to how the
> Java's MaxHeap is defined), or in terms of percentage of available memory
> (not to exceed 95%).
> The
> [runbit|https://github.com/apache/drill/blob/master/distribution/src/resources/runbit]
> script will perform the calculation of the sum of memory required by the
> memory spaces (heap, direct, etc) and ensure that it is within the limit
> defined by the {{DRILLBIT_MAX_PROC_MEM}} env variable.
> In the absence of this parameter, there will be no restriction. A node admin
> can then define this variable in the default terminal's environment (e.g.
> {{/root/.bashrc}} ) files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO
[ https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144283#comment-16144283 ] Sindhuri Ramanarayan Rayavaram edited comment on DRILL-5425 at 8/28/17 8:28 PM: [~parthc], [~cshi], [~shamirwasia], [~kbotzum], [~sudheeshkatkam], [~laurentgo] Please review the document. was (Author: sindhurir): [~parthc], [~cshi], [~shamirwasia], [~sudheeshkatkam] Please review the document. > Support HTTP Kerberos auth using SPNEGO > --- > > Key: DRILL-5425 > URL: https://issues.apache.org/jira/browse/DRILL-5425 > Project: Apache Drill > Issue Type: New Feature > Components: Web Server >Reporter: Sudheesh Katkam >Assignee: Sindhuri Ramanarayan Rayavaram > > DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests > to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for > HTTP connections. > This requires creating "direct" web sessions; currently web sessions are > sessions over Java client sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO
[ https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144283#comment-16144283 ] Sindhuri Ramanarayan Rayavaram edited comment on DRILL-5425 at 8/28/17 8:23 PM: [~parthc], [~cshi], [~shamirwasia], [~sudheeshkatkam] Please review the document. was (Author: sindhurir): [~parthc], [~cshi], [~shamirwasia] Please review the document. > Support HTTP Kerberos auth using SPNEGO > --- > > Key: DRILL-5425 > URL: https://issues.apache.org/jira/browse/DRILL-5425 > Project: Apache Drill > Issue Type: New Feature > Components: Web Server >Reporter: Sudheesh Katkam >Assignee: Sindhuri Ramanarayan Rayavaram > > DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests > to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for > HTTP connections. > This requires creating "direct" web sessions; currently web sessions are > sessions over Java client sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO
[ https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144283#comment-16144283 ] Sindhuri Ramanarayan Rayavaram commented on DRILL-5425: --- [~parthc], [~cshi], [~shamirwasia] Please review the document. > Support HTTP Kerberos auth using SPNEGO > --- > > Key: DRILL-5425 > URL: https://issues.apache.org/jira/browse/DRILL-5425 > Project: Apache Drill > Issue Type: New Feature > Components: Web Server >Reporter: Sudheesh Katkam >Assignee: Sindhuri Ramanarayan Rayavaram > > DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests > to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for > HTTP connections. > This requires creating "direct" web sessions; currently web sessions are > sessions over Java client sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO
[ https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144273#comment-16144273 ] Sindhuri Ramanarayan Rayavaram commented on DRILL-5425: --- The Design document for SPNEGO: https://docs.google.com/document/d/1Z7DHJTFItFbljG9dBT-vYnrGkPht0MXkJ2VzCuCZjjg/edit?usp=sharing > Support HTTP Kerberos auth using SPNEGO > --- > > Key: DRILL-5425 > URL: https://issues.apache.org/jira/browse/DRILL-5425 > Project: Apache Drill > Issue Type: New Feature > Components: Web Server >Reporter: Sudheesh Katkam >Assignee: Sindhuri Ramanarayan Rayavaram > > DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests > to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for > HTTP connections. > This requires creating "direct" web sessions; currently web sessions are > sessions over Java client sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-5447) Managed External Sort : Unable to allocate sv2 vector
[
https://issues.apache.org/jira/browse/DRILL-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Hou updated DRILL-5447:
--
Attachment: 26617a7e-b953-7ac3-556d-43fd88e51b19.sys.drill
drillbit.log
> Managed External Sort : Unable to allocate sv2 vector
> -
>
> Key: DRILL-5447
> URL: https://issues.apache.org/jira/browse/DRILL-5447
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.10.0
>Reporter: Rahul Challapalli
>Assignee: Paul Rogers
> Fix For: 1.12.0
>
> Attachments: 26617a7e-b953-7ac3-556d-43fd88e51b19.sys.drill,
> 26fee988-ed18-a86a-7164-3e75118c0ffc.sys.drill, drillbit.log, drillbit.log
>
>
> git.commit.id.abbrev=3e8b01d
> Dataset :
> {code}
> Every records contains a repeated type with 2000 elements.
> The repeated type contains varchars of length 250 for the first 2000 records
> and single character strings for the next 2000 records
> The above pattern is repeated a few types
> {code}
> The below query fails
> {code}
> ALTER SESSION SET `exec.sort.disable_managed` = false;
> alter session set `planner.width.max_per_node` = 1;
> alter session set `planner.disable_exchanges` = true;
> alter session set `planner.width.max_per_query` = 1;
> select count(*) from (select * from (select id, flatten(str_list) str from
> dfs.`/drill/testdata/resource-manager/flatten-large-small.json`) d order by
> d.str) d1 where d1.id=0;
> Error: RESOURCE ERROR: Unable to allocate sv2 buffer
> Fragment 0:0
> [Error Id: 9e45c293-ab26-489d-a90e-25da96004f15 on qa-node190.qa.lab:31010]
> (state=,code=0)
> {code}
> Exception from the logs
> {code}
> [Error Id: 9e45c293-ab26-489d-a90e-25da96004f15 ]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544)
> ~[drill-common-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.newSV2(ExternalSortBatch.java:1463)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.makeSelectionVector(ExternalSortBatch.java:799)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.processBatch(ExternalSortBatch.java:856)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.loadBatch(ExternalSortBatch.java:618)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.load(ExternalSortBatch.java:660)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.innerNext(ExternalSortBatch.java:559)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:93)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.1
[jira] [Reopened] (DRILL-5447) Managed External Sort : Unable to allocate sv2 vector
[
https://issues.apache.org/jira/browse/DRILL-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Hou reopened DRILL-5447:
---
This is not passing on Jenkins.
Here is the query:
{noformat}
ALTER SESSION SET `exec.sort.disable_managed` = false;
alter session set `planner.width.max_per_node` = 1;
alter session set `planner.disable_exchanges` = true;
alter session set `planner.width.max_per_query` = 1;
select count(*) from (select * from (select id, flatten(str_list) str from
dfs.`/drill/testdata/resource-manager/flatten-large-small.json`) d order by
d.str) d1 where d1.id=0;
ALTER SESSION SET `exec.sort.disable_managed` = true;
alter session set `planner.width.max_per_node` = 17;
alter session set `planner.disable_exchanges` = false;
alter session set `planner.width.max_per_query` = 1000;
alter session set `planner.memory.max_query_memory_per_node` = 268435456;
{noformat}
Stack trace is:
{noformat}
2017-08-24 00:51:38,821 [26617a7e-b953-7ac3-556d-43fd88e51b19:frag:0:0] ERROR
o.a.d.e.p.i.x.m.ExternalSortBatch - Insufficient memory to merge two batches
. Incoming batch size: 1073819648, available memory: 268435456
2017-08-24 00:51:38,832 [26617a7e-b953-7ac3-556d-43fd88e51b19:frag:0:0] INFO
o.a.d.e.p.i.x.m.BufferedBatches - User Error Occurred: Unable to allocate sv
2 buffer (Unable to allocate sv2 buffer)
org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Unable to
allocate sv2 buffer
[Error Id: 17cdce48-2ff3-44cc-916d-136ab409e896 ]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:550)
~[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.BufferedBatches.newSV2(BufferedBatches.java:157)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-
SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.BufferedBatches.makeSelectionVector(BufferedBatches.java:142)
[drill-java-exec-1.12.0-SNAPSHO
T.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.BufferedBatches.add(BufferedBatches.java:97)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAP
SHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.SortImpl.addBatch(SortImpl.java:265)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.loadBatch(ExternalSortBatch.java:422)
[drill-java-exec-1.12.0-SNAPSHOT.jar:
1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.load(ExternalSortBatch.java:358)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.
0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.innerNext(ExternalSortBatch.java:303)
[drill-java-exec-1.12.0-SNAPSHOT.jar:
1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0
-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:93)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1
.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0
-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0
-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNex
[jira] [Commented] (DRILL-5447) Managed External Sort : Unable to allocate sv2 vector
[
https://issues.apache.org/jira/browse/DRILL-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144229#comment-16144229
]
Robert Hou commented on DRILL-5447:
---
The memory is not being constricted for this test. E.g.
`planner.memory.max_query_memory_per_node` is not being set.
> Managed External Sort : Unable to allocate sv2 vector
> -
>
> Key: DRILL-5447
> URL: https://issues.apache.org/jira/browse/DRILL-5447
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.10.0
>Reporter: Rahul Challapalli
>Assignee: Paul Rogers
> Fix For: 1.12.0
>
> Attachments: 26fee988-ed18-a86a-7164-3e75118c0ffc.sys.drill,
> drillbit.log
>
>
> git.commit.id.abbrev=3e8b01d
> Dataset :
> {code}
> Every records contains a repeated type with 2000 elements.
> The repeated type contains varchars of length 250 for the first 2000 records
> and single character strings for the next 2000 records
> The above pattern is repeated a few types
> {code}
> The below query fails
> {code}
> ALTER SESSION SET `exec.sort.disable_managed` = false;
> alter session set `planner.width.max_per_node` = 1;
> alter session set `planner.disable_exchanges` = true;
> alter session set `planner.width.max_per_query` = 1;
> select count(*) from (select * from (select id, flatten(str_list) str from
> dfs.`/drill/testdata/resource-manager/flatten-large-small.json`) d order by
> d.str) d1 where d1.id=0;
> Error: RESOURCE ERROR: Unable to allocate sv2 buffer
> Fragment 0:0
> [Error Id: 9e45c293-ab26-489d-a90e-25da96004f15 on qa-node190.qa.lab:31010]
> (state=,code=0)
> {code}
> Exception from the logs
> {code}
> [Error Id: 9e45c293-ab26-489d-a90e-25da96004f15 ]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544)
> ~[drill-common-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.newSV2(ExternalSortBatch.java:1463)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.makeSelectionVector(ExternalSortBatch.java:799)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.processBatch(ExternalSortBatch.java:856)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.loadBatch(ExternalSortBatch.java:618)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.load(ExternalSortBatch.java:660)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.innerNext(ExternalSortBatch.java:559)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:93)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:215)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> [drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> [drill-java-exec-
[jira] [Commented] (DRILL-5741) During startup Drill should not exceed the available memory
[
https://issues.apache.org/jira/browse/DRILL-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144197#comment-16144197
]
ASF GitHub Bot commented on DRILL-5741:
---
Github user kkhatua commented on the issue:
https://github.com/apache/drill/pull/922
@paul-rogers
The reason this would be needed is because the system admin might not be
the one managing the specifics of the Drill memory allocations. This allows a
node admin to define the limits within which Drill will work, while the
specifications can be delegated to the drill-env.sh script which is managed by
a power user.
There isn't a conflict with the way a service like Warden would allocate
memory, because it does not need to manage the internal allocations, only the
overall. The Drill service will continue to own its configuration's specifics.
Your solution is pretty much in line with what the PR proposes. The
exception being that, in the presence of the env variable defined, the
specifications **cannot** oversubscribe the available (or max permissible)
memory. In case of oversubscribing, Drill will not startup.
If there are uber resource managers (for e.g. YARN), they have the option
of either
- providing this limit and relying on files like ```drill-env.sh``` to
specify the limits...
or
- provide the explicit parameters for Heap, DirectMemory, etc to the JVM
and **not** have to specify ```DRILLBIT_MAX_PROC_MEM``` .
> During startup Drill should not exceed the available memory
> ---
>
> Key: DRILL-5741
> URL: https://issues.apache.org/jira/browse/DRILL-5741
> Project: Apache Drill
> Issue Type: Improvement
> Components: Server
>Affects Versions: 1.11.0
>Reporter: Kunal Khatua
>Assignee: Kunal Khatua
> Fix For: 1.12.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Currently, during startup, a Drillbit can be assigned large values for the
> following:
> * Xmx (Heap)
> * XX:MaxDirectMemorySize
> * XX:ReservedCodeCacheSize
> * XX:MaxPermSize
> All of this, potentially, can exceed the available memory on a system when a
> Drillbit is under heavy load. It would be good to have the Drillbit ensure
> during startup itself that the cumulative value of these parameters does not
> exceed a pre-defined upper limit for the Drill process.
> The proposal is to have the
> [runbit|https://github.com/apache/drill/blob/master/distribution/src/resources/runbit]
> script look for an additional environment variable:
> {{DRILLBIT_MAX_PROC_MEM}}
> The parameter can specify the maximum in GB/MB (similar in syntax to how the
> Java's MaxHeap is defined), or in terms of percentage of available memory
> (not to exceed 95%).
> The
> [runbit|https://github.com/apache/drill/blob/master/distribution/src/resources/runbit]
> script will perform the calculation of the sum of memory required by the
> memory spaces (heap, direct, etc) and ensure that it is within the limit
> defined by the {{DRILLBIT_MAX_PROC_MEM}} env variable.
> In the absence of this parameter, there will be no restriction. A node admin
> can then define this variable in the default terminal's environment (e.g.
> {{/root/.bashrc}} ) files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (DRILL-5425) Support HTTP Kerberos auth using SPNEGO
[ https://issues.apache.org/jira/browse/DRILL-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sindhuri Ramanarayan Rayavaram reassigned DRILL-5425: - Assignee: Sindhuri Ramanarayan Rayavaram > Support HTTP Kerberos auth using SPNEGO > --- > > Key: DRILL-5425 > URL: https://issues.apache.org/jira/browse/DRILL-5425 > Project: Apache Drill > Issue Type: New Feature > Components: Web Server >Reporter: Sudheesh Katkam >Assignee: Sindhuri Ramanarayan Rayavaram > > DRILL-4280 supports Kerberos through JDBC and ODBC API. This ticket requests > to add Kerberos (using [SPENGO|https://en.wikipedia.org/wiki/SPNEGO]) for > HTTP connections. > This requires creating "direct" web sessions; currently web sessions are > sessions over Java client sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (DRILL-5744) External sort fails with OOM error
[
https://issues.apache.org/jira/browse/DRILL-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Hou updated DRILL-5744:
--
Description:
Query is:
{noformat}
ALTER SESSION SET `exec.sort.disable_managed` = false;
alter session set `planner.width.max_per_node` = 1;
alter session set `planner.disable_exchanges` = true;
alter session set `planner.width.max_per_query` = 1;
alter session set `planner.memory.max_query_memory_per_node` = 152428800;
select count(*) from (
select * from (
select s1.type type, flatten(s1.rms.rptd) rptds, s1.rms, s1.uid
from (
select d.type type, d.uid uid, flatten(d.map.rm) rms from
dfs.`/drill/testdata/resource-manager/nested-large.json` d order by d.uid
) s1
) s2
order by s2.rms.mapid
);
ALTER SESSION SET `exec.sort.disable_managed` = true;
alter session set `planner.width.max_per_node` = 17;
alter session set `planner.disable_exchanges` = false;
alter session set `planner.width.max_per_query` = 1000;
alter session set `planner.memory.max_query_memory_per_node` = 2147483648;
{noformat}
Stack trace is:
{noformat}
2017-08-23 06:59:42,763 [266275e5-ebdb-14ae-d52d-00fa3a154f6d:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - User Error Occurred: One or more nodes
ran out of memory while executing the query. (Unable to allocate buffer of
size 4194304 (rounded from 3276750) due to memory limit. Current allocation: 7
9986944)
org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: One or more
nodes ran out of memory while executing the query.
Unable to allocate buffer of size 4194304 (rounded from 3276750) due to memory
limit. Current allocation: 79986944
[Error Id: 4f4959df-0921-4a50-b75e-56488469ab10 ]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:550)
~[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:244)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Unable to
allocate buffer of size 4194304 (rounded from 3276750) due to memory limit. Cur
rent allocation: 79986944
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:238)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:213)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:402)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:236)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.AllocationHelper.allocatePrecomputedChildCount(AllocationHelper.java:33)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPS
HOT]
at
org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:46)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:113)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT
]
at
org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:95)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateMap(VectorInitializer.java:130)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:93)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateBatch(VectorInitializer.java:85)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.PriorityQueueCopierWrapper$BatchMerger.next(PriorityQueueCopierWrapper.java:262)
~[drill-java
-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.load(ExternalSortBatch.java:374)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12
.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.innerNext(ExternalSortBatch.java:303)
~[drill-java-exec-1.12.0-SNAPSHOT.jar
:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:164)
~[drill-java-ex
[jira] [Updated] (DRILL-5744) External sort fails with OOM error
[
https://issues.apache.org/jira/browse/DRILL-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Hou updated DRILL-5744:
--
Attachment: 266275e5-ebdb-14ae-d52d-00fa3a154f6d.sys.drill
drillbit.log
> External sort fails with OOM error
> --
>
> Key: DRILL-5744
> URL: https://issues.apache.org/jira/browse/DRILL-5744
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.10.0
>Reporter: Robert Hou
>Assignee: Paul Rogers
> Fix For: 1.12.0
>
> Attachments: 266275e5-ebdb-14ae-d52d-00fa3a154f6d.sys.drill,
> drillbit.log
>
>
> Query is:
> {noformat}
> ALTER SESSION SET `exec.sort.disable_managed` = false;
> alter session set `planner.width.max_per_node` = 1;
> alter session set `planner.disable_exchanges` = true;
> alter session set `planner.width.max_per_query` = 1;
> alter session set `planner.memory.max_query_memory_per_node` = 152428800;
> select count(*) from (
> select * from (
> select s1.type type, flatten(s1.rms.rptd) rptds, s1.rms, s1.uid
> from (
> select d.type type, d.uid uid, flatten(d.map.rm) rms from
> dfs.`/drill/testdata/resource-manager/nested-large.json` d order by d.uid
> ) s1
> ) s2
> order by s2.rms.mapid
> );
> ALTER SESSION SET `exec.sort.disable_managed` = true;
> alter session set `planner.width.max_per_node` = 17;
> alter session set `planner.disable_exchanges` = false;
> alter session set `planner.width.max_per_query` = 1000;
> alter session set `planner.memory.max_query_memory_per_node` = 2147483648;
> {noformat}
> Stack trace is:
> {noformat}
> 2017-08-23 06:59:42,763 [266275e5-ebdb-14ae-d52d-00fa3a154f6d:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor - User Error Occurred: One or more nodes
> ran out of memory while executing the query. (Unable to allocate buffer of
> size 4194304 (rounded from 3276750) due to memory limit. Current allocation: 7
> 9986944)
> org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: One or more
> nodes ran out of memory while executing the query.
> Unable to allocate buffer of size 4194304 (rounded from 3276750) due to
> memory limit. Current allocation: 79986944
> [Error Id: 4f4959df-0921-4a50-b75e-56488469ab10 ]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:550)
> ~[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:244)
> [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Unable to
> allocate buffer of size 4194304 (rounded from 3276750) due to memory limit.
> Cur
> rent allocation: 79986944
> at
> org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:238)
> ~[drill-memory-base-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:213)
> ~[drill-memory-base-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:402)
> ~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:236)
> ~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.AllocationHelper.allocatePrecomputedChildCount(AllocationHelper.java:33)
> ~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPS
> HOT]
> at
> org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:46)
> ~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:113)
> ~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT
> ]
> at
> org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:95)
> ~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.VectorInitializer.allocateMap(VectorInitializer.java:130)
> ~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:93)
> ~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.VectorInitializer.allocateBatch(
[jira] [Created] (DRILL-5744) External sort fails with OOM error
Robert Hou created DRILL-5744:
-
Summary: External sort fails with OOM error
Key: DRILL-5744
URL: https://issues.apache.org/jira/browse/DRILL-5744
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Affects Versions: 1.10.0
Reporter: Robert Hou
Assignee: Paul Rogers
Fix For: 1.12.0
Query is:
{noformat}
ALTER SESSION SET `exec.sort.disable_managed` = false;
alter session set `planner.width.max_per_node` = 1;
alter session set `planner.disable_exchanges` = true;
alter session set `planner.width.max_per_query` = 1;
alter session set `planner.memory.max_query_memory_per_node` = 152428800;
select count(*) from (
select * from (
select s1.type type, flatten(s1.rms.rptd) rptds, s1.rms, s1.uid
from (
select d.type type, d.uid uid, flatten(d.map.rm) rms from
dfs.`/drill/testdata/resource-manager/nested-large.json` d order by d.uid
) s1
) s2
order by s2.rms.mapid
);
ALTER SESSION SET `exec.sort.disable_managed` = true;
alter session set `planner.width.max_per_node` = 17;
alter session set `planner.disable_exchanges` = false;
alter session set `planner.width.max_per_query` = 1000;
alter session set `planner.memory.max_query_memory_per_node` = 2147483648;
{noformat}
Stack trace is:
{noformat}
2017-08-23 06:59:42,763 [266275e5-ebdb-14ae-d52d-00fa3a154f6d:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - User Error Occurred: One or more nodes
ran out of memory while executing the query. (Unable to allocate buffer of
size 4194304 (rounded from 3276750) due to memory limit. Current allocation: 7
9986944)
org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: One or more
nodes ran out of memory while executing the query.
Unable to allocate buffer of size 4194304 (rounded from 3276750) due to memory
limit. Current allocation: 79986944
[Error Id: 4f4959df-0921-4a50-b75e-56488469ab10 ]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:550)
~[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:244)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Unable to
allocate buffer of size 4194304 (rounded from 3276750) due to memory limit. Cur
rent allocation: 79986944
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:238)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:213)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:402)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:236)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.AllocationHelper.allocatePrecomputedChildCount(AllocationHelper.java:33)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPS
HOT]
at
org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:46)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:113)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT
]
at
org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:95)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateMap(VectorInitializer.java:130)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateVector(VectorInitializer.java:93)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.VectorInitializer.allocateBatch(VectorInitializer.java:85)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.PriorityQueueCopierWrapper$BatchMerger.next(PriorityQueueCopierWrapper.java:262)
~[drill-java
-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.managed.ExternalSortBatch.load(ExternalSortBatch.java:374)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12
.0-SNAPSHOT]
at
org.apache.dr
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[
https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144172#comment-16144172
]
ASF GitHub Bot commented on DRILL-5726:
---
Github user sohami commented on a diff in the pull request:
https://github.com/apache/drill/pull/910#discussion_r135599220
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DrillRestServer.java
---
@@ -230,6 +230,27 @@ public WebUserConnection provide() {
public void dispose(WebUserConnection instance) {
}
+
+/**
+ * Creates session user principal. If impersonation is enabled without
authentication and User-Name header is present and valid,
+ * will create session user principal with provided user name,
otherwise anonymous user name will be used.
+ * In both cases session user principal will have admin rights.
+ *
+ * @param config drill config
+ * @param request client request
+ * @return session user principal
+ */
+private Principal createSessionUserPrincipal(DrillConfig config,
HttpServletRequest request) {
+ final boolean checkForUserName =
!config.getBoolean(ExecConstants.USER_AUTHENTICATION_ENABLED) &&
config.getBoolean(ExecConstants.IMPERSONATION_ENABLED);
+ if (checkForUserName) {
--- End diff --
Makes sense. Thank You for explanation
> Support Impersonation without authentication for REST API
> -
>
> Key: DRILL-5726
> URL: https://issues.apache.org/jira/browse/DRILL-5726
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.12.0
>
> Attachments: login_page.JPG, query_page_with_user_name.JPG
>
>
> Today if a user is not authenticated via REST API then there is no way to
> provide a user name for executing queries. It will by default be executed as
> "anonymous" user. This doesn't work when impersonation without authentication
> is enabled on Drill server side, since anonymous user doesn't exist the query
> will fail. We need a way to provide a user name when impersonation is enabled
> on Drill side and query is executed from REST API.
> There are two approaches to achieve that:
> *1. Use form-based authentication*
> On Web UI user will be prompted to enter only login, then session for that
> user will be created, user will be treated as admin. Form-based
> authentication will cache user information, so user won't need to set user
> name each time he / she wants to execute the query. Log in / out options will
> be also available. Example screenshot of login page is attached
> (login_page.JPG).
> From the programmatic perspective, user would need first to authenticate and
> use cookie to get query result.
> *2. Use {{User-Name}} header in request*
> On Web UI on Query page additional input field will appear. User would need
> to enter user name before issuing the query. Example screenshot of query page
> is attached (query_page_with_user_name.JPG). Under the hood with user name
> would be added to client request as request header. On server side this
> header would be used to create user session principal. From the programmatic
> perspective, user would need to add header when issuing the request.
> *_From the two above options second was chosen as it would ease REST API
> usage from the programmatic perspective, plus using form-based authentication
> may lead to false assumption that user is authenticated which is in reality
> is not true._*
> *Implementation details of the second approach:*
> _Note: the below implementation will take affect only if authentication is
> disabled and impersonation is enabled. By means of freemarker page won't
> include js lib and script if condition is not met._
> On the client side additional input field was added to the query page. When
> client is submitting the query, request would be changed using ajax to add
> {{User-Name}} header which would be taken from the new input field. On the
> server side, this header would be used to create session principal with
> provided user name and admin rights. If user name header was not provided
> (null or empty), the default anonymous principal will be used.
> *Adding user name header approaches:*
> _Web UI_
> enter user name in the User Name input field on Query page before submiiting
> the query (query_page_with_user_name.JPG)
> _sqlline_
> {code}./drill-localhost -n user1{code}
> _curl_
> {code} curl -v -H "Content-Type: application/json" -H "User-Name: user1" -d
> '{"queryType":"SQL", "query": "select * from sys.version"}'
> http://localhost:8047/query.json {code}
> _Java way_
> {code}
> String url = "http://
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[
https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144174#comment-16144174
]
ASF GitHub Bot commented on DRILL-5726:
---
Github user sohami commented on the issue:
https://github.com/apache/drill/pull/910
+1 LGTM
> Support Impersonation without authentication for REST API
> -
>
> Key: DRILL-5726
> URL: https://issues.apache.org/jira/browse/DRILL-5726
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.12.0
>
> Attachments: login_page.JPG, query_page_with_user_name.JPG
>
>
> Today if a user is not authenticated via REST API then there is no way to
> provide a user name for executing queries. It will by default be executed as
> "anonymous" user. This doesn't work when impersonation without authentication
> is enabled on Drill server side, since anonymous user doesn't exist the query
> will fail. We need a way to provide a user name when impersonation is enabled
> on Drill side and query is executed from REST API.
> There are two approaches to achieve that:
> *1. Use form-based authentication*
> On Web UI user will be prompted to enter only login, then session for that
> user will be created, user will be treated as admin. Form-based
> authentication will cache user information, so user won't need to set user
> name each time he / she wants to execute the query. Log in / out options will
> be also available. Example screenshot of login page is attached
> (login_page.JPG).
> From the programmatic perspective, user would need first to authenticate and
> use cookie to get query result.
> *2. Use {{User-Name}} header in request*
> On Web UI on Query page additional input field will appear. User would need
> to enter user name before issuing the query. Example screenshot of query page
> is attached (query_page_with_user_name.JPG). Under the hood with user name
> would be added to client request as request header. On server side this
> header would be used to create user session principal. From the programmatic
> perspective, user would need to add header when issuing the request.
> *_From the two above options second was chosen as it would ease REST API
> usage from the programmatic perspective, plus using form-based authentication
> may lead to false assumption that user is authenticated which is in reality
> is not true._*
> *Implementation details of the second approach:*
> _Note: the below implementation will take affect only if authentication is
> disabled and impersonation is enabled. By means of freemarker page won't
> include js lib and script if condition is not met._
> On the client side additional input field was added to the query page. When
> client is submitting the query, request would be changed using ajax to add
> {{User-Name}} header which would be taken from the new input field. On the
> server side, this header would be used to create session principal with
> provided user name and admin rights. If user name header was not provided
> (null or empty), the default anonymous principal will be used.
> *Adding user name header approaches:*
> _Web UI_
> enter user name in the User Name input field on Query page before submiiting
> the query (query_page_with_user_name.JPG)
> _sqlline_
> {code}./drill-localhost -n user1{code}
> _curl_
> {code} curl -v -H "Content-Type: application/json" -H "User-Name: user1" -d
> '{"queryType":"SQL", "query": "select * from sys.version"}'
> http://localhost:8047/query.json {code}
> _Java way_
> {code}
> String url = "http://localhost:8047/query.json";;
> URLConnection connection = new URL(url).openConnection();
> connection.setDoOutput(true); // Triggers POST.
> connection.addRequestProperty("User-Name", "user1");
> connection.setRequestProperty("Content-Type", "application/json");
> String data = "{\"queryType\":\"SQL\", \"query\": \"select * from
> sys.version\"}";
> try (OutputStream output = connection.getOutputStream()) {
> output.write(data.getBytes(StandardCharsets.UTF_8.name()));
> }
> try (InputStream response = connection.getInputStream()) {
> String result = IOUtils.toString(response);
> System.out.println(result);
> }
> {code}
> Note: {{Apache HttpClient}} can be used as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5670) Varchar vector throws an assertion error when allocating a new vector
[
https://issues.apache.org/jira/browse/DRILL-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144160#comment-16144160
]
Robert Hou commented on DRILL-5670:
---
This bug has been fixed.
> Varchar vector throws an assertion error when allocating a new vector
> -
>
> Key: DRILL-5670
> URL: https://issues.apache.org/jira/browse/DRILL-5670
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Rahul Challapalli
>Assignee: Paul Rogers
> Fix For: 1.12.0
>
> Attachments: 269969ca-8d4d-073a-d916-9031e3d3fbf0.sys.drill,
> drillbit.log, drillbit.out, drill-override.conf
>
>
> I am running this test on a private branch of [paul's
> repository|https://github.com/paul-rogers/drill]. Below is the commit info
> {code}
> git.commit.id.abbrev=d86e16c
> git.commit.user.email=prog...@maprtech.com
> git.commit.message.full=DRILL-5601\: Rollup of external sort fixes an
> improvements\n\n- DRILL-5513\: Managed External Sort \: OOM error during the
> merge phase\n- DRILL-5519\: Sort fails to spill and results in an OOM\n-
> DRILL-5522\: OOM during the merge and spill process of the managed external
> sort\n- DRILL-5594\: Excessive buffer reallocations during merge phase of
> external sort\n- DRILL-5597\: Incorrect "bits" vector allocation in nullable
> vectors allocateNew()\n- DRILL-5602\: Repeated List Vector fails to
> initialize the offset vector\n\nAll of the bugs have to do with handling
> low-memory conditions, and with\ncorrectly estimating the sizes of vectors,
> even when those vectors come\nfrom the spill file or from an exchange. Hence,
> the changes for all of\nthe above issues are interrelated.\n
> git.commit.id=d86e16c551e7d3553f2cde748a739b1c5a7a7659
> git.commit.message.short=DRILL-5601\: Rollup of external sort fixes an
> improvements
> git.commit.user.name=Paul Rogers
> git.build.user.name=Rahul Challapalli
> git.commit.id.describe=0.9.0-1078-gd86e16c
> git.build.user.email=challapallira...@gmail.com
> git.branch=d86e16c551e7d3553f2cde748a739b1c5a7a7659
> git.commit.time=05.07.2017 @ 20\:34\:39 PDT
> git.build.time=12.07.2017 @ 14\:27\:03 PDT
> git.remote.origin.url=g...@github.com\:paul-rogers/drill.git
> {code}
> Below query fails with an Assertion Error
> {code}
> 0: jdbc:drill:zk=10.10.100.190:5181> ALTER SESSION SET
> `exec.sort.disable_managed` = false;
> +---+-+
> | ok | summary |
> +---+-+
> | true | exec.sort.disable_managed updated. |
> +---+-+
> 1 row selected (1.044 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> alter session set
> `planner.memory.max_query_memory_per_node` = 482344960;
> +---++
> | ok | summary |
> +---++
> | true | planner.memory.max_query_memory_per_node updated. |
> +---++
> 1 row selected (0.372 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> alter session set
> `planner.width.max_per_node` = 1;
> +---+--+
> | ok | summary|
> +---+--+
> | true | planner.width.max_per_node updated. |
> +---+--+
> 1 row selected (0.292 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> alter session set
> `planner.width.max_per_query` = 1;
> +---+---+
> | ok |summary|
> +---+---+
> | true | planner.width.max_per_query updated. |
> +---+---+
> 1 row selected (0.25 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> select count(*) from (select * from
> dfs.`/drill/testdata/resource-manager/3500cols.tbl` order by
> columns[450],columns[330],columns[230],columns[220],columns[110],columns[90],columns[80],columns[70],columns[40],columns[10],columns[20],columns[30],columns[40],columns[50],
>
> columns[454],columns[413],columns[940],columns[834],columns[73],columns[140],columns[104],columns[],columns[30],columns[2420],columns[1520],
> columns[1410],
> columns[1110],columns[1290],columns[2380],columns[705],columns[45],columns[1054],columns[2430],columns[420],columns[404],columns[3350],
>
> columns[],columns[153],columns[356],columns[84],columns[745],columns[1450],columns[103],columns[2065],columns[343],columns[3420],columns[530],
> columns[3210] ) d where d.col433 = 'sjka skjf';
[jira] [Closed] (DRILL-5670) Varchar vector throws an assertion error when allocating a new vector
[
https://issues.apache.org/jira/browse/DRILL-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Hou closed DRILL-5670.
-
This bug has been fixed and verified.
> Varchar vector throws an assertion error when allocating a new vector
> -
>
> Key: DRILL-5670
> URL: https://issues.apache.org/jira/browse/DRILL-5670
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.11.0
>Reporter: Rahul Challapalli
>Assignee: Paul Rogers
> Fix For: 1.12.0
>
> Attachments: 269969ca-8d4d-073a-d916-9031e3d3fbf0.sys.drill,
> drillbit.log, drillbit.out, drill-override.conf
>
>
> I am running this test on a private branch of [paul's
> repository|https://github.com/paul-rogers/drill]. Below is the commit info
> {code}
> git.commit.id.abbrev=d86e16c
> git.commit.user.email=prog...@maprtech.com
> git.commit.message.full=DRILL-5601\: Rollup of external sort fixes an
> improvements\n\n- DRILL-5513\: Managed External Sort \: OOM error during the
> merge phase\n- DRILL-5519\: Sort fails to spill and results in an OOM\n-
> DRILL-5522\: OOM during the merge and spill process of the managed external
> sort\n- DRILL-5594\: Excessive buffer reallocations during merge phase of
> external sort\n- DRILL-5597\: Incorrect "bits" vector allocation in nullable
> vectors allocateNew()\n- DRILL-5602\: Repeated List Vector fails to
> initialize the offset vector\n\nAll of the bugs have to do with handling
> low-memory conditions, and with\ncorrectly estimating the sizes of vectors,
> even when those vectors come\nfrom the spill file or from an exchange. Hence,
> the changes for all of\nthe above issues are interrelated.\n
> git.commit.id=d86e16c551e7d3553f2cde748a739b1c5a7a7659
> git.commit.message.short=DRILL-5601\: Rollup of external sort fixes an
> improvements
> git.commit.user.name=Paul Rogers
> git.build.user.name=Rahul Challapalli
> git.commit.id.describe=0.9.0-1078-gd86e16c
> git.build.user.email=challapallira...@gmail.com
> git.branch=d86e16c551e7d3553f2cde748a739b1c5a7a7659
> git.commit.time=05.07.2017 @ 20\:34\:39 PDT
> git.build.time=12.07.2017 @ 14\:27\:03 PDT
> git.remote.origin.url=g...@github.com\:paul-rogers/drill.git
> {code}
> Below query fails with an Assertion Error
> {code}
> 0: jdbc:drill:zk=10.10.100.190:5181> ALTER SESSION SET
> `exec.sort.disable_managed` = false;
> +---+-+
> | ok | summary |
> +---+-+
> | true | exec.sort.disable_managed updated. |
> +---+-+
> 1 row selected (1.044 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> alter session set
> `planner.memory.max_query_memory_per_node` = 482344960;
> +---++
> | ok | summary |
> +---++
> | true | planner.memory.max_query_memory_per_node updated. |
> +---++
> 1 row selected (0.372 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> alter session set
> `planner.width.max_per_node` = 1;
> +---+--+
> | ok | summary|
> +---+--+
> | true | planner.width.max_per_node updated. |
> +---+--+
> 1 row selected (0.292 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> alter session set
> `planner.width.max_per_query` = 1;
> +---+---+
> | ok |summary|
> +---+---+
> | true | planner.width.max_per_query updated. |
> +---+---+
> 1 row selected (0.25 seconds)
> 0: jdbc:drill:zk=10.10.100.190:5181> select count(*) from (select * from
> dfs.`/drill/testdata/resource-manager/3500cols.tbl` order by
> columns[450],columns[330],columns[230],columns[220],columns[110],columns[90],columns[80],columns[70],columns[40],columns[10],columns[20],columns[30],columns[40],columns[50],
>
> columns[454],columns[413],columns[940],columns[834],columns[73],columns[140],columns[104],columns[],columns[30],columns[2420],columns[1520],
> columns[1410],
> columns[1110],columns[1290],columns[2380],columns[705],columns[45],columns[1054],columns[2430],columns[420],columns[404],columns[3350],
>
> columns[],columns[153],columns[356],columns[84],columns[745],columns[1450],columns[103],columns[2065],columns[343],columns[3420],columns[530],
> columns[3210] ) d where d.col433 = 'sjka skjf';
> Error: RESOURCE ERROR: External Sort encoun
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144073#comment-16144073
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135578936
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullHolder.java
---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.vector;
+
+import org.apache.drill.common.types.TypeProtos;
+import org.apache.drill.common.types.Types;
+import org.apache.drill.exec.expr.holders.ValueHolder;
+
+public class UntypedNullHolder implements ValueHolder {
+ public static final TypeProtos.MajorType TYPE =
Types.optional(TypeProtos.MinorType.NULL);
+
+ public TypeProtos.MajorType getType() {return TYPE;}
+
+ public static final int WIDTH = 0;
+
+ public int isSet = 1;
+
+ @Deprecated
+ public int hashCode(){
+throw new UnsupportedOperationException();
+ }
--- End diff --
See note for `toString()`. This has the same issues.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144076#comment-16144076
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135579157
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullVector.java
---
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+
**/
+
+package org.apache.drill.exec.vector;
+
+
+import com.google.common.base.Preconditions;
+import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.memory.BufferAllocator;
+import org.apache.drill.exec.proto.UserBitShared.SerializedField;
+import org.apache.drill.exec.record.MaterializedField;
+import org.apache.drill.exec.record.TransferPair;
+import org.apache.drill.exec.vector.complex.reader.FieldReader;
+
+/** UntypedNullVector is to represent a value vector with {@link
org.apache.drill.common.types.MinorType.NULL}
+ * All values in the vector represent two semantic implications: 1) the
value is unknown, 2) the type is unknown.
+ * Because of this, we only have to keep track of the number of values in
value vector,
+ * and there is no allocated buffer to back up this value vector.
Therefore, the majority of
+ * methods in this class is either no-op, or throws {@link
UnsupportedOperationException}.
+ *
+ */
+public final class UntypedNullVector extends BaseDataValueVector
implements FixedWidthVector {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(UntypedNullVector.class);
+
+ /**
+ * Width of each fixed-width value.
+ */
+ public static final int VALUE_WIDTH = 0;
+
+ private final Accessor accessor = new Accessor();
+ private final Mutator mutator = new Mutator();
+ private int valueCount ;
--- End diff --
Remove space before ;
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144075#comment-16144075
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135578735
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullHolder.java
---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.vector;
+
+import org.apache.drill.common.types.TypeProtos;
+import org.apache.drill.common.types.Types;
+import org.apache.drill.exec.expr.holders.ValueHolder;
+
+public class UntypedNullHolder implements ValueHolder {
+ public static final TypeProtos.MajorType TYPE =
Types.optional(TypeProtos.MinorType.NULL);
+
+ public TypeProtos.MajorType getType() {return TYPE;}
--- End diff --
The typical Java convention is to list fields first, then methods, so maybe
move this after the fields.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144067#comment-16144067
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135579027
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullHolder.java
---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.vector;
+
+import org.apache.drill.common.types.TypeProtos;
+import org.apache.drill.common.types.Types;
+import org.apache.drill.exec.expr.holders.ValueHolder;
+
+public class UntypedNullHolder implements ValueHolder {
+ public static final TypeProtos.MajorType TYPE =
Types.optional(TypeProtos.MinorType.NULL);
+
+ public TypeProtos.MajorType getType() {return TYPE;}
+
+ public static final int WIDTH = 0;
+
+ public int isSet = 1;
+
+ @Deprecated
+ public int hashCode(){
+throw new UnsupportedOperationException();
+ }
+
+ /*
+ * Reason for deprecation is that ValueHolders are potential scalar
replacements
+ * and hence we don't want any methods to be invoked on them.
+ */
+ @Deprecated
+ public String toString(){
+throw new UnsupportedOperationException();
+ }
+
+
+
--- End diff --
Remove extra blank lines.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144072#comment-16144072
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135578594
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullHolder.java
---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.vector;
+
+import org.apache.drill.common.types.TypeProtos;
+import org.apache.drill.common.types.Types;
+import org.apache.drill.exec.expr.holders.ValueHolder;
+
+public class UntypedNullHolder implements ValueHolder {
+ public static final TypeProtos.MajorType TYPE =
Types.optional(TypeProtos.MinorType.NULL);
+
+ public TypeProtos.MajorType getType() {return TYPE;}
+
+ public static final int WIDTH = 0;
+
+ public int isSet = 1;
+
+ @Deprecated
+ public int hashCode(){
+throw new UnsupportedOperationException();
+ }
+
+ /*
+ * Reason for deprecation is that ValueHolders are potential scalar
replacements
+ * and hence we don't want any methods to be invoked on them.
+ */
+ @Deprecated
+ public String toString(){
+throw new UnsupportedOperationException();
--- End diff --
Why unsupported? According to Java, all classes should implement toString.
Else, if we are in the debugger, and try to look at an instance of this class,
the debugger will encounter an exception.
Also, it is not a good idea to deprecate a standard Java method.
Either leave the default implementation, or simply return
`getClass().getSimpleName()`.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144069#comment-16144069
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135574741
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillScanRel.java
---
@@ -160,12 +161,7 @@ public RelOptCost computeSelfCost(final RelOptPlanner
planner, RelMetadataQuery
final ScanStats stats = groupScan.getScanStats(settings);
int columnCount = getRowType().getFieldCount();
double ioCost = 0;
-boolean isStarQuery = Iterables.tryFind(getRowType().getFieldNames(),
new Predicate() {
- @Override
- public boolean apply(String input) {
-return Preconditions.checkNotNull(input).equals("*");
- }
-}).isPresent();
+boolean isStarQuery = AbstractRecordReader.isStarQuery(columns);
--- End diff --
Should this planning-time class depend on a specific execution-time class?
Or, would it be better to move the method from the execution class onto
something common between plan and execution?
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144070#comment-16144070
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135575060
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ProjectPrel.java
---
@@ -35,18 +35,43 @@
import org.apache.calcite.rex.RexNode;
import org.apache.calcite.sql.SqlKind;
+/**
+ * A physical Prel node for Project operator.
+ */
public class ProjectPrel extends DrillProjectRelBase implements Prel{
static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(ProjectPrel.class);
+ private final boolean outputProj;
public ProjectPrel(RelOptCluster cluster, RelTraitSet traits, RelNode
child, List exps,
RelDataType rowType) {
+this(cluster, traits, child, exps, rowType, false);
+ }
+
+ /**
+ * Constructor for ProjectPrel.
+ * @param cluster
+ * @param traits traits of ProjectPrel node
+ * @param child input
+ * @param exps list of RexNode, representing expressions of projection.
+ * @param rowType output rowType of projection expression.
+ * @param outputProj true if ProjectPrel is inserted by {@link
org.apache.drill.exec.planner.physical.visitor.TopProjectVisitor}
+ * Such top Project operator does the following
processing, before the result was presented to Screen/Writer
+ * 1) ensure final output field names are preserved,
+ * 2) handle cases where input does not return any
batch (a fast NONE) (see ProjectRecordBatch.handleFastNone() method)
+ * 3) handle cases where expressions in upstream
operator were evaluated to NULL type
--- End diff --
This is Javadoc, so rather than a pre-formatted numeric list, consider
\\...\\.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144068#comment-16144068
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135577415
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractSingleRecordBatch.java
---
@@ -125,4 +131,19 @@ public BatchSchema getSchema() {
protected abstract boolean setupNewSchema() throws SchemaChangeException;
protected abstract IterOutcome doWork();
+
+ /**
+ * Default behavior to handle fast NONE (incoming's first next() return
NONE, in stead of OK_NEW_SCHEMA):
+ * FAST NONE could happen when the underneath Scan operators do not
produce any batch with schema.
+ *
+ * This behavior could be override in each individual operator, if the
operator's semantics is to
+ * inject a batch with schema.
+ *
+ * @return IterOutcome.NONE.
+ */
+ protected IterOutcome handleFastNone() {
+container.buildSchema(SelectionVectorMode.NONE);
+return IterOutcome.NONE;
+ };
--- End diff --
Delete the semi-colon.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144077#comment-16144077
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135577968
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/TestEmptyBatchSql.java ---
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill;
+
+import com.google.common.collect.Lists;
+import org.apache.commons.lang3.tuple.Pair;
+import org.apache.drill.common.expression.SchemaPath;
+import org.apache.drill.common.types.TypeProtos;
+import org.apache.drill.common.util.FileUtils;
+import org.junit.Test;
+
+import java.util.List;
+
+public class TestEmptyBatchSql extends BaseTestQuery {
--- End diff --
Nice set of tests for JSON. Thanks!
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144074#comment-16144074
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135580061
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullVector.java
---
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+
**/
+
+package org.apache.drill.exec.vector;
+
+
+import com.google.common.base.Preconditions;
+import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.memory.BufferAllocator;
+import org.apache.drill.exec.proto.UserBitShared.SerializedField;
+import org.apache.drill.exec.record.MaterializedField;
+import org.apache.drill.exec.record.TransferPair;
+import org.apache.drill.exec.vector.complex.reader.FieldReader;
+
+/** UntypedNullVector is to represent a value vector with {@link
org.apache.drill.common.types.MinorType.NULL}
+ * All values in the vector represent two semantic implications: 1) the
value is unknown, 2) the type is unknown.
+ * Because of this, we only have to keep track of the number of values in
value vector,
+ * and there is no allocated buffer to back up this value vector.
Therefore, the majority of
+ * methods in this class is either no-op, or throws {@link
UnsupportedOperationException}.
+ *
+ */
+public final class UntypedNullVector extends BaseDataValueVector
implements FixedWidthVector {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(UntypedNullVector.class);
+
+ /**
+ * Width of each fixed-width value.
+ */
+ public static final int VALUE_WIDTH = 0;
+
+ private final Accessor accessor = new Accessor();
+ private final Mutator mutator = new Mutator();
+ private int valueCount ;
+
+ public UntypedNullVector(MaterializedField field, BufferAllocator
allocator) {
+super(field, allocator);
+valueCount = 0;
+ }
+
+ @Override
+ public FieldReader getReader() { throw new
UnsupportedOperationException(); }
+
+ @Override
+ public int getBufferSizeFor(final int valueCount) {
+return 0;
+ }
+
+ @Override
+ public int getValueCapacity(){
+return Character.MAX_VALUE;
+ }
+
+ @Override
+ public Accessor getAccessor() { return accessor; }
+
+ @Override
+ public Mutator getMutator() { return mutator; }
+
+ @Override
+ public void setInitialCapacity(final int valueCount) {
+ }
+
+ @Override
+ public void allocateNew() {
+ }
+
+ @Override
+ public boolean allocateNewSafe() {
+return true;
+ }
+
+ @Override
+ public void allocateNew(final int valueCount) {
+ }
+
+ @Override
+ public void reset() {
+ }
+
+ /**
+ * {@inheritDoc}
+ */
+ @Override
+ public void zeroVector() {
+ }
+
+ @Override
+ public void load(SerializedField metadata, DrillBuf buffer) {
+
Preconditions.checkArgument(this.field.getPath().equals(metadata.getNamePart().getName()),
+"The field %s doesn't match the provided metadata %s.",
this.field, metadata);
+final int actualLength = metadata.getBufferLength();
+final int valueCount = metadata.getValueCount();
+final int expectedLength = valueCount * VALUE_WIDTH;
+assert actualLength == expectedLength : String.format("Expected to
load %d bytes but actually loaded %d bytes", expectedLength, actualLength);
+
+this.valueCount = valueCount;
+ }
+
+ @Override
+ public TransferPair getTransferPair(BufferAllocator allocator){
+return new TransferImpl(getField(), allocator);
+ }
+
+ @Override
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144078#comment-16144078
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135579971
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullVector.java
---
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+
**/
+
+package org.apache.drill.exec.vector;
+
+
+import com.google.common.base.Preconditions;
+import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.memory.BufferAllocator;
+import org.apache.drill.exec.proto.UserBitShared.SerializedField;
+import org.apache.drill.exec.record.MaterializedField;
+import org.apache.drill.exec.record.TransferPair;
+import org.apache.drill.exec.vector.complex.reader.FieldReader;
+
+/** UntypedNullVector is to represent a value vector with {@link
org.apache.drill.common.types.MinorType.NULL}
+ * All values in the vector represent two semantic implications: 1) the
value is unknown, 2) the type is unknown.
+ * Because of this, we only have to keep track of the number of values in
value vector,
+ * and there is no allocated buffer to back up this value vector.
Therefore, the majority of
+ * methods in this class is either no-op, or throws {@link
UnsupportedOperationException}.
+ *
+ */
+public final class UntypedNullVector extends BaseDataValueVector
implements FixedWidthVector {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(UntypedNullVector.class);
+
+ /**
+ * Width of each fixed-width value.
+ */
+ public static final int VALUE_WIDTH = 0;
+
+ private final Accessor accessor = new Accessor();
+ private final Mutator mutator = new Mutator();
+ private int valueCount ;
+
+ public UntypedNullVector(MaterializedField field, BufferAllocator
allocator) {
+super(field, allocator);
+valueCount = 0;
+ }
+
+ @Override
+ public FieldReader getReader() { throw new
UnsupportedOperationException(); }
+
+ @Override
+ public int getBufferSizeFor(final int valueCount) {
+return 0;
+ }
+
+ @Override
+ public int getValueCapacity(){
+return Character.MAX_VALUE;
+ }
+
+ @Override
+ public Accessor getAccessor() { return accessor; }
+
+ @Override
+ public Mutator getMutator() { return mutator; }
+
+ @Override
+ public void setInitialCapacity(final int valueCount) {
+ }
+
+ @Override
+ public void allocateNew() {
+ }
+
+ @Override
+ public boolean allocateNewSafe() {
+return true;
+ }
+
+ @Override
+ public void allocateNew(final int valueCount) {
+ }
+
+ @Override
+ public void reset() {
+ }
+
+ /**
+ * {@inheritDoc}
+ */
+ @Override
+ public void zeroVector() {
+ }
+
+ @Override
+ public void load(SerializedField metadata, DrillBuf buffer) {
+
Preconditions.checkArgument(this.field.getPath().equals(metadata.getNamePart().getName()),
+"The field %s doesn't match the provided metadata %s.",
this.field, metadata);
+final int actualLength = metadata.getBufferLength();
+final int valueCount = metadata.getValueCount();
+final int expectedLength = valueCount * VALUE_WIDTH;
+assert actualLength == expectedLength : String.format("Expected to
load %d bytes but actually loaded %d bytes", expectedLength, actualLength);
+
+this.valueCount = valueCount;
+ }
+
+ @Override
+ public TransferPair getTransferPair(BufferAllocator allocator){
+return new TransferImpl(getField(), allocator);
+ }
+
+ @Override
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144071#comment-16144071
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135579508
--- Diff:
exec/vector/src/main/java/org/apache/drill/exec/vector/UntypedNullVector.java
---
@@ -0,0 +1,267 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+
**/
+
+package org.apache.drill.exec.vector;
+
+
+import com.google.common.base.Preconditions;
+import io.netty.buffer.DrillBuf;
+import org.apache.drill.exec.memory.BufferAllocator;
+import org.apache.drill.exec.proto.UserBitShared.SerializedField;
+import org.apache.drill.exec.record.MaterializedField;
+import org.apache.drill.exec.record.TransferPair;
+import org.apache.drill.exec.vector.complex.reader.FieldReader;
+
+/** UntypedNullVector is to represent a value vector with {@link
org.apache.drill.common.types.MinorType.NULL}
+ * All values in the vector represent two semantic implications: 1) the
value is unknown, 2) the type is unknown.
+ * Because of this, we only have to keep track of the number of values in
value vector,
+ * and there is no allocated buffer to back up this value vector.
Therefore, the majority of
+ * methods in this class is either no-op, or throws {@link
UnsupportedOperationException}.
+ *
+ */
+public final class UntypedNullVector extends BaseDataValueVector
implements FixedWidthVector {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(UntypedNullVector.class);
+
+ /**
+ * Width of each fixed-width value.
+ */
+ public static final int VALUE_WIDTH = 0;
+
+ private final Accessor accessor = new Accessor();
+ private final Mutator mutator = new Mutator();
+ private int valueCount ;
+
+ public UntypedNullVector(MaterializedField field, BufferAllocator
allocator) {
+super(field, allocator);
+valueCount = 0;
+ }
+
+ @Override
+ public FieldReader getReader() { throw new
UnsupportedOperationException(); }
+
+ @Override
+ public int getBufferSizeFor(final int valueCount) {
+return 0;
+ }
+
+ @Override
+ public int getValueCapacity(){
+return Character.MAX_VALUE;
--- End diff --
Please use `ValueVector.MAX_ROW_COUNT`. It is a coincidence that the
maximum row count happens to be (one greater than) `Character.MAX_VALUE`.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144065#comment-16144065
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135576938
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractBinaryRecordBatch.java
---
@@ -0,0 +1,75 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.record;
+
+import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.ops.FragmentContext;
+import org.apache.drill.exec.physical.base.PhysicalOperator;
+
+public abstract class AbstractBinaryRecordBatch extends AbstractRecordBatch {
+ protected final RecordBatch left;
+ protected final RecordBatch right;
+
+ // state (IterOutcome) of the left input
+ protected IterOutcome leftUpstream = IterOutcome.NONE;
+
+ // state (IterOutcome) of the right input
+ protected IterOutcome rightUpstream = IterOutcome.NONE;
--- End diff --
Nice improvement!
Consider wrapping the two sides in a new class, say, BatchIterator, that
holds the batch and iter outcome. Then, the code here is just a bit cleaner
because we don't have to repeat variables for left and right.
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144064#comment-16144064
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135577493
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractSingleRecordBatch.java
---
@@ -125,4 +131,19 @@ public BatchSchema getSchema() {
protected abstract boolean setupNewSchema() throws SchemaChangeException;
protected abstract IterOutcome doWork();
+
+ /**
+ * Default behavior to handle fast NONE (incoming's first next() return
NONE, in stead of OK_NEW_SCHEMA):
+ * FAST NONE could happen when the underneath Scan operators do not
produce any batch with schema.
+ *
--- End diff --
Javadoc is HTML, so insert \
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5546) Schema change problems caused by empty batch
[
https://issues.apache.org/jira/browse/DRILL-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144066#comment-16144066
]
ASF GitHub Bot commented on DRILL-5546:
---
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/906#discussion_r135577279
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractSingleRecordBatch.java
---
@@ -61,6 +63,10 @@ public IterOutcome innerNext() {
}
switch (upstream) {
case NONE:
+ if (state == BatchState.FIRST) {
+return handleFastNone();
--- End diff --
Again, consider a more descriptive name, such as `handleEmptyInput()`.
Otherwise, everyone has to remember that "fast none" == "empty input".
> Schema change problems caused by empty batch
>
>
> Key: DRILL-5546
> URL: https://issues.apache.org/jira/browse/DRILL-5546
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> There have been a few JIRAs opened related to schema change failure caused by
> empty batch. This JIRA is opened as an umbrella for all those related JIRAS (
> such as DRILL-4686, DRILL-4734, DRILL4476, DRILL-4255, etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[
https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143990#comment-16143990
]
ASF GitHub Bot commented on DRILL-5726:
---
Github user arina-ielchiieva commented on the issue:
https://github.com/apache/drill/pull/910
@sohami addressed code review comments. Please review when possible.
> Support Impersonation without authentication for REST API
> -
>
> Key: DRILL-5726
> URL: https://issues.apache.org/jira/browse/DRILL-5726
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.12.0
>
> Attachments: login_page.JPG, query_page_with_user_name.JPG
>
>
> Today if a user is not authenticated via REST API then there is no way to
> provide a user name for executing queries. It will by default be executed as
> "anonymous" user. This doesn't work when impersonation without authentication
> is enabled on Drill server side, since anonymous user doesn't exist the query
> will fail. We need a way to provide a user name when impersonation is enabled
> on Drill side and query is executed from REST API.
> There are two approaches to achieve that:
> *1. Use form-based authentication*
> On Web UI user will be prompted to enter only login, then session for that
> user will be created, user will be treated as admin. Form-based
> authentication will cache user information, so user won't need to set user
> name each time he / she wants to execute the query. Log in / out options will
> be also available. Example screenshot of login page is attached
> (login_page.JPG).
> From the programmatic perspective, user would need first to authenticate and
> use cookie to get query result.
> *2. Use {{User-Name}} header in request*
> On Web UI on Query page additional input field will appear. User would need
> to enter user name before issuing the query. Example screenshot of query page
> is attached (query_page_with_user_name.JPG). Under the hood with user name
> would be added to client request as request header. On server side this
> header would be used to create user session principal. From the programmatic
> perspective, user would need to add header when issuing the request.
> *_From the two above options second was chosen as it would ease REST API
> usage from the programmatic perspective, plus using form-based authentication
> may lead to false assumption that user is authenticated which is in reality
> is not true._*
> *Implementation details of the second approach:*
> _Note: the below implementation will take affect only if authentication is
> disabled and impersonation is enabled. By means of freemarker page won't
> include js lib and script if condition is not met._
> On the client side additional input field was added to the query page. When
> client is submitting the query, request would be changed using ajax to add
> {{User-Name}} header which would be taken from the new input field. On the
> server side, this header would be used to create session principal with
> provided user name and admin rights. If user name header was not provided
> (null or empty), the default anonymous principal will be used.
> *Adding user name header approaches:*
> _Web UI_
> enter user name in the User Name input field on Query page before submiiting
> the query (query_page_with_user_name.JPG)
> _sqlline_
> {code}./drill-localhost -n user1{code}
> _curl_
> {code} curl -v -H "Content-Type: application/json" -H "User-Name: user1" -d
> '{"queryType":"SQL", "query": "select * from sys.version"}'
> http://localhost:8047/query.json {code}
> _Java way_
> {code}
> String url = "http://localhost:8047/query.json";;
> URLConnection connection = new URL(url).openConnection();
> connection.setDoOutput(true); // Triggers POST.
> connection.addRequestProperty("User-Name", "user1");
> connection.setRequestProperty("Content-Type", "application/json");
> String data = "{\"queryType\":\"SQL\", \"query\": \"select * from
> sys.version\"}";
> try (OutputStream output = connection.getOutputStream()) {
> output.write(data.getBytes(StandardCharsets.UTF_8.name()));
> }
> try (InputStream response = connection.getInputStream()) {
> String result = IOUtils.toString(response);
> System.out.println(result);
> }
> {code}
> Note: {{Apache HttpClient}} can be used as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[
https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143982#comment-16143982
]
ASF GitHub Bot commented on DRILL-5726:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/910#discussion_r135572514
--- Diff: exec/java-exec/src/main/resources/rest/query/query.ftl ---
@@ -21,8 +24,16 @@
×
Sample SQL query: SELECT * FROM cp.`employee.json` LIMIT
20
-
-
+
+ <#if model?? && model>
+
+ User Name
+
--- End diff --
Done. Did not use `required` attribute since it's not working with
`onclick`, instead check is using js.
> Support Impersonation without authentication for REST API
> -
>
> Key: DRILL-5726
> URL: https://issues.apache.org/jira/browse/DRILL-5726
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.12.0
>
> Attachments: login_page.JPG, query_page_with_user_name.JPG
>
>
> Today if a user is not authenticated via REST API then there is no way to
> provide a user name for executing queries. It will by default be executed as
> "anonymous" user. This doesn't work when impersonation without authentication
> is enabled on Drill server side, since anonymous user doesn't exist the query
> will fail. We need a way to provide a user name when impersonation is enabled
> on Drill side and query is executed from REST API.
> There are two approaches to achieve that:
> *1. Use form-based authentication*
> On Web UI user will be prompted to enter only login, then session for that
> user will be created, user will be treated as admin. Form-based
> authentication will cache user information, so user won't need to set user
> name each time he / she wants to execute the query. Log in / out options will
> be also available. Example screenshot of login page is attached
> (login_page.JPG).
> From the programmatic perspective, user would need first to authenticate and
> use cookie to get query result.
> *2. Use {{User-Name}} header in request*
> On Web UI on Query page additional input field will appear. User would need
> to enter user name before issuing the query. Example screenshot of query page
> is attached (query_page_with_user_name.JPG). Under the hood with user name
> would be added to client request as request header. On server side this
> header would be used to create user session principal. From the programmatic
> perspective, user would need to add header when issuing the request.
> *_From the two above options second was chosen as it would ease REST API
> usage from the programmatic perspective, plus using form-based authentication
> may lead to false assumption that user is authenticated which is in reality
> is not true._*
> *Implementation details of the second approach:*
> _Note: the below implementation will take affect only if authentication is
> disabled and impersonation is enabled. By means of freemarker page won't
> include js lib and script if condition is not met._
> On the client side additional input field was added to the query page. When
> client is submitting the query, request would be changed using ajax to add
> {{User-Name}} header which would be taken from the new input field. On the
> server side, this header would be used to create session principal with
> provided user name and admin rights. If user name header was not provided
> (null or empty), the default anonymous principal will be used.
> *Adding user name header approaches:*
> _Web UI_
> enter user name in the User Name input field on Query page before submiiting
> the query (query_page_with_user_name.JPG)
> _sqlline_
> {code}./drill-localhost -n user1{code}
> _curl_
> {code} curl -v -H "Content-Type: application/json" -H "User-Name: user1" -d
> '{"queryType":"SQL", "query": "select * from sys.version"}'
> http://localhost:8047/query.json {code}
> _Java way_
> {code}
> String url = "http://localhost:8047/query.json";;
> URLConnection connection = new URL(url).openConnection();
> connection.setDoOutput(true); // Triggers POST.
> connection.addRequestProperty("User-Name", "user1");
> connection.setRequestProperty("Content-Type", "application/json");
> String data = "{\"queryType\":\"SQL\", \"query\": \"select * from
> sys.version\"}";
> try (OutputStream output = connection.getOutputStream()) {
> output.write(data.getBytes(StandardCharsets.UTF_8.name()));
> }
> try (InputStream response = connection.getInputStream()) {
> String result = IOUtils.toString(response);
> System.out.println(result);
> }
> {code}
>
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[
https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143980#comment-16143980
]
ASF GitHub Bot commented on DRILL-5726:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/910#discussion_r135492293
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryResources.java
---
@@ -54,7 +56,10 @@
@Path("/query")
@Produces(MediaType.TEXT_HTML)
public Viewable getQuery() {
-return ViewableWithPermissions.create(authEnabled.get(),
"/rest/query/query.ftl", sc);
+final DrillConfig config = work.getContext().getConfig();
+// if impersonation is enabled without impersonation, will provide
mechanism to add user name to request header from Web UI
--- End diff --
Fixed.
> Support Impersonation without authentication for REST API
> -
>
> Key: DRILL-5726
> URL: https://issues.apache.org/jira/browse/DRILL-5726
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.12.0
>
> Attachments: login_page.JPG, query_page_with_user_name.JPG
>
>
> Today if a user is not authenticated via REST API then there is no way to
> provide a user name for executing queries. It will by default be executed as
> "anonymous" user. This doesn't work when impersonation without authentication
> is enabled on Drill server side, since anonymous user doesn't exist the query
> will fail. We need a way to provide a user name when impersonation is enabled
> on Drill side and query is executed from REST API.
> There are two approaches to achieve that:
> *1. Use form-based authentication*
> On Web UI user will be prompted to enter only login, then session for that
> user will be created, user will be treated as admin. Form-based
> authentication will cache user information, so user won't need to set user
> name each time he / she wants to execute the query. Log in / out options will
> be also available. Example screenshot of login page is attached
> (login_page.JPG).
> From the programmatic perspective, user would need first to authenticate and
> use cookie to get query result.
> *2. Use {{User-Name}} header in request*
> On Web UI on Query page additional input field will appear. User would need
> to enter user name before issuing the query. Example screenshot of query page
> is attached (query_page_with_user_name.JPG). Under the hood with user name
> would be added to client request as request header. On server side this
> header would be used to create user session principal. From the programmatic
> perspective, user would need to add header when issuing the request.
> *_From the two above options second was chosen as it would ease REST API
> usage from the programmatic perspective, plus using form-based authentication
> may lead to false assumption that user is authenticated which is in reality
> is not true._*
> *Implementation details of the second approach:*
> _Note: the below implementation will take affect only if authentication is
> disabled and impersonation is enabled. By means of freemarker page won't
> include js lib and script if condition is not met._
> On the client side additional input field was added to the query page. When
> client is submitting the query, request would be changed using ajax to add
> {{User-Name}} header which would be taken from the new input field. On the
> server side, this header would be used to create session principal with
> provided user name and admin rights. If user name header was not provided
> (null or empty), the default anonymous principal will be used.
> *Adding user name header approaches:*
> _Web UI_
> enter user name in the User Name input field on Query page before submiiting
> the query (query_page_with_user_name.JPG)
> _sqlline_
> {code}./drill-localhost -n user1{code}
> _curl_
> {code} curl -v -H "Content-Type: application/json" -H "User-Name: user1" -d
> '{"queryType":"SQL", "query": "select * from sys.version"}'
> http://localhost:8047/query.json {code}
> _Java way_
> {code}
> String url = "http://localhost:8047/query.json";;
> URLConnection connection = new URL(url).openConnection();
> connection.setDoOutput(true); // Triggers POST.
> connection.addRequestProperty("User-Name", "user1");
> connection.setRequestProperty("Content-Type", "application/json");
> String data = "{\"queryType\":\"SQL\", \"query\": \"select * from
> sys.version\"}";
> try (OutputStream output = connection.getOutputStream()) {
> output.write(data.getBytes(StandardCharsets.UTF_8.name()));
> }
> try (InputStream response = c
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[
https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143983#comment-16143983
]
ASF GitHub Bot commented on DRILL-5726:
---
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/910#discussion_r135571777
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/DrillRestServer.java
---
@@ -230,6 +230,27 @@ public WebUserConnection provide() {
public void dispose(WebUserConnection instance) {
}
+
+/**
+ * Creates session user principal. If impersonation is enabled without
authentication and User-Name header is present and valid,
+ * will create session user principal with provided user name,
otherwise anonymous user name will be used.
+ * In both cases session user principal will have admin rights.
+ *
+ * @param config drill config
+ * @param request client request
+ * @return session user principal
+ */
+private Principal createSessionUserPrincipal(DrillConfig config,
HttpServletRequest request) {
+ final boolean checkForUserName =
!config.getBoolean(ExecConstants.USER_AUTHENTICATION_ENABLED) &&
config.getBoolean(ExecConstants.IMPERSONATION_ENABLED);
+ if (checkForUserName) {
--- End diff --
We should not require `User-Name` for each request since it is needed only
when we need to query the data. I will leave this method but also add
`UserNameFilter` that would be applied to all `/query` and `/query.json`
requests and check if header is provided, i.e. we would allow using `anonymous`
user if `User-Name` is not provided since we'll know that `UserNameFilter` will
filter out request where `User-Name` is required but not set.
> Support Impersonation without authentication for REST API
> -
>
> Key: DRILL-5726
> URL: https://issues.apache.org/jira/browse/DRILL-5726
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.11.0
>Reporter: Arina Ielchiieva
>Assignee: Arina Ielchiieva
> Fix For: 1.12.0
>
> Attachments: login_page.JPG, query_page_with_user_name.JPG
>
>
> Today if a user is not authenticated via REST API then there is no way to
> provide a user name for executing queries. It will by default be executed as
> "anonymous" user. This doesn't work when impersonation without authentication
> is enabled on Drill server side, since anonymous user doesn't exist the query
> will fail. We need a way to provide a user name when impersonation is enabled
> on Drill side and query is executed from REST API.
> There are two approaches to achieve that:
> *1. Use form-based authentication*
> On Web UI user will be prompted to enter only login, then session for that
> user will be created, user will be treated as admin. Form-based
> authentication will cache user information, so user won't need to set user
> name each time he / she wants to execute the query. Log in / out options will
> be also available. Example screenshot of login page is attached
> (login_page.JPG).
> From the programmatic perspective, user would need first to authenticate and
> use cookie to get query result.
> *2. Use {{User-Name}} header in request*
> On Web UI on Query page additional input field will appear. User would need
> to enter user name before issuing the query. Example screenshot of query page
> is attached (query_page_with_user_name.JPG). Under the hood with user name
> would be added to client request as request header. On server side this
> header would be used to create user session principal. From the programmatic
> perspective, user would need to add header when issuing the request.
> *_From the two above options second was chosen as it would ease REST API
> usage from the programmatic perspective, plus using form-based authentication
> may lead to false assumption that user is authenticated which is in reality
> is not true._*
> *Implementation details of the second approach:*
> _Note: the below implementation will take affect only if authentication is
> disabled and impersonation is enabled. By means of freemarker page won't
> include js lib and script if condition is not met._
> On the client side additional input field was added to the query page. When
> client is submitting the query, request would be changed using ajax to add
> {{User-Name}} header which would be taken from the new input field. On the
> server side, this header would be used to create session principal with
> provided user name and admin rights. If user name header was not provided
> (null or empty), the default anonymous principal will be used.
> *Adding user name header approaches:*
> _Web UI_
> en
[jira] [Commented] (DRILL-5726) Support Impersonation without authentication for REST API
[ https://issues.apache.org/jira/browse/DRILL-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143981#comment-16143981 ] ASF GitHub Bot commented on DRILL-5726: --- Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/910#discussion_r135571864 --- Diff: exec/java-exec/src/main/resources/rest/query/query.ftl --- @@ -47,8 +58,32 @@ Query -Submit + +"button" onclick="doSubmit()"<#else>"submit"> + Submit + + +<#if model?? && model> +
