This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 1e45c19 add web ui features to doc for 1.16
1e45c19 is described below
commit 1e45c195cf4d7b4025c7e83c1674480c4900249c
Author: Bridget Bevens <bbev...@maprtech.com>
AuthorDate: Thu Apr 18 12:16:39 2019 -0700
add web ui features to doc for 1.16
---
.../030-planning-and-exececution-options.md | 64 ++++++++++++++--
.../036-configuring-storage-plugins.md | 10 ++-
.../rest-api/010-rest-api-introduction.md | 7 +-
...on-is-enabled-and-authentication-is-disabled.md | 4 +-
_docs/img/ui-export-all.png | Bin 0 -> 39876 bytes
_docs/img/ui-limit-results-warn.png | Bin 0 -> 13718 bytes
_docs/img/ui-options.PNG | Bin 0 -> 41028 bytes
_docs/img/ui-order-duration.png | Bin 0 -> 56440 bytes
_docs/img/ui-results-page.png | Bin 0 -> 101904 bytes
_docs/img/ui-row-limit.png | Bin 0 -> 34559 bytes
_docs/img/ui-spilled-data.png | Bin 0 -> 318 bytes
_docs/img/ui-submit-popup.png | Bin 0 -> 25778 bytes
_docs/img/ui-turtle-wait.png | Bin 0 -> 677 bytes
_docs/img/ui-wait-op.png | Bin 0 -> 369 bytes
_docs/install/060-starting-the-web-ui.md | 27 +++++--
.../020-query-profiles.md | 82 ++++++++++++++++-----
_docs/query-data/010-query-data-introduction.md | 52 ++++++++++---
17 files changed, 195 insertions(+), 51 deletions(-)
diff --git
a/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
b/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
index f486435..b10816b 100644
---
a/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
+++
b/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
@@ -1,6 +1,6 @@
---
title: "Planning and Execution Options"
-date: 2018-12-28
+date: 2019-04-18
parent: "Configuration Options"
---
You can set Drill query planning and execution options per cluster, at the
@@ -10,7 +10,6 @@ system level affect the entire system and persist between
restarts. Session
level settings override system level settings for the duration of the session.
Planning options are prepended by planner, for example
`planner.enable_hashjoin`. Execution options are prepended by drill.exe, for
example `drill.exec.functions.cast_empty_string_to_null`.
-
## Setting Planning and Execution Options
You can set planning and execution options for Drill from the Drill shell
(SQLLine) or the Drill Web UI. Options set in the Drill Web UI are set at the
system level. You can override system level options set in the Drill Web UI by
setting the options at the session level from the Drill shell.
@@ -39,14 +38,14 @@ See [Querying the Options
Table]({{site.baseurl}}/docs/querying-system-tables/#q
### Setting Options from the Drill Web UI
-When Drill is running, you can access the Drill Web UI at
`http://<drill-hostname>:8047/`. The Drill Web UI has an Options button located
in the upper right corner that you can click to display an Options page. The
Options page lists all the Drill options that you can set. When you set options
from this page, you are setting the options at the system level. To override an
option for the duration of a session, set the option from the Drill shell using
the SET command.
+When Drill is running, you can access the Drill Web UI at
`http://<drill-hostname>:8047/`. The Drill Web UI has an Options button located
in the upper right corner that you can click to display an Options page. The
Options page lists all the Drill options that you can set. When you set options
from this page, you are setting the options at the system level. To override an
option for the duration of a session, set the option from the Drill shell using
the SET command.
-
+
Starting in Drill 1.15, the Options page includes the following enhancements:
**Search Field**
-The search field enables you to quickly search across all available Drill
configuration options to quickly find options you are interested in. For
example, you can search on the keyword "memory" for a list of all options
related to configuring Drill memory.
+The search field enables you to quickly search across all available Drill
configuration options to quickly find options you are interested in. For
example, you can search on the keyword "memory" for a list of all options
related to configuring Drill memory. Starting in Drill 1.16, the Search field
contains an x that clears the search field when clicked.
**Quick Filters**
Next to the search field are quick filter buttons that filter options by
important keywords in one click. For example, the "planner" quick filter button
quickly lists all options related to the query planner. The query planning
options are prepended by the keyword "planner." Quick filters return options
that include the keyword in the option name and/or description.
@@ -90,7 +89,60 @@ Issuing ` select timestamp '2008-2-23 12:23:34' from
(values(1))` returns the fo
Issuing the query after setting the `web.display_format.timestamp` option to
the format `yyyy-MM-dd HH:mm:ss.SS` returns the following results in the Web
UI:
- 2008-02-23 12:23:34.00
+ 2008-02-23 12:23:34.00
+
+##Limiting the Number of Rows Returned for Result Sets
+The `exec.query.max_rows` option, introduced in Drill 1.16, automatically
limits the number of rows that Drill returns from a result set. When you
configure a limit, the limit is automatically applied by Drill when you run a
query. You do not need to include the LIMIT clause. Configuring a limit on
results reduces the amount of processing and streaming work for the web server
when using the Drill WEb UI.
+
+The default setting for `exec.query.max_rows` is 0, which indicates no limit;
queries will return all results. An administrator can modify the
exec.query.max_rows setting at the system level (setting persists across
restarts) from the Options page in the Drill Web UI or using the ALTER SYSTEM
command. Alternatively, the option can be set at the session level (valid for a
user session only) using the SET command or from SQLLine.
+
+**Session Level**
+To set the `exec.query.max_rows` option per user session, use the SET command
or the SQLLine line command, as shown:
+
+ SET `exec.query.max_rows` = <value>;
+ !set rowlimit <value>
+
+**System Level**
+To set the `exec.query.max_rows` option such that it persists across users,
change the setting in the Drill Web UI (`http://<drill-hostname>:8047/options`)
or use the ALTER SYSTEM SET command, as shown:
+
+ ALTER SYSTEM SET `exec.query.max_rows` = <value>;
+
+***
+**NOTE**
+The `exec.query.max_rows` option overrides row limit settings configured in
the Drill Web UI and `drill-override.conf` file, which are described below.
+***
+
+### Drill Web UI Row Limit Settings
+In addition to the `exec.query.max_rows` option, the Query page in the Drill
Web UI provides a field where you can enter a limit on the number of rows
returned from a result set when you run a query.
+
+**Note:** You can also configure the row limit for a REST client through the
[autoLimit parameter in the REST
API]({{site.baseurl}}/docs/rest-api-introduction/#post-query-json).
+
+
+When you submit the query, results display across one or multiple pages. You
can set the number of results that display per page. When results span multiple
pages, scroll through the pages using the links in the bottom right corner of
the screen.
+
+Clicking the Query Profile tab at the top of the results page displays the
query profile with a warning stating that the results were automatically
limited.
+
+Note that the `exec.query.max_rows` setting overrides the setting in the Web
UI. For example, if the `exec.query.max_rows` option is set to 10 and you limit
results to 20 on the Query page in the Web UI, Drill only returns 10 rows from
the result set.
+
+###Drill-override.conf Row Limit Settings
+The `drill-override.conf` file contains the same row limit setting as the
Drill Web UI; however, the setting in the `drill-override.conf` file overrides
the limit set in the Drill Web UI. Note that the `exec.query.max_rows` option
overrides the setting in `drill-override.conf` and the Drill Web UI. You must
restart the Drillbit after you modify `drill-override.conf`.
+
+Add the following configuration to `drill-override.conf`, changing the row
limit setting and rows per page setting, as needed:
+
+ drill.exec.http: {
+ web.client.resultset: {
+ autolimit {
+ enable: true,
+ rows: 1000
+ },
+ defaultRowsPerPage: 10
+ }
+ }
+
+
+
+
+
diff --git a/_docs/connect-a-data-source/036-configuring-storage-plugins.md
b/_docs/connect-a-data-source/036-configuring-storage-plugins.md
index 6e470df..658b137 100644
--- a/_docs/connect-a-data-source/036-configuring-storage-plugins.md
+++ b/_docs/connect-a-data-source/036-configuring-storage-plugins.md
@@ -1,6 +1,6 @@
---
title: "Configuring Storage Plugins"
-date: 2018-12-08
+date: 2019-04-18
parent: "Storage Plugin Configuration"
---
@@ -88,11 +88,15 @@ The file is removed after Drill uses the file for the first
time.
## Exporting Storage Plugin Configurations
-Starting in Drill 1.14, you can export and save your storage plugin
configurations from the Storage page in the Drill Web UI. The Export option
next to each enabled storage plugin provides an easy way to export your storage
plugin configurations to a JSON file when you want to save and reuse the
configurations. For example, before an upgrade.
+Starting in Drill 1.14, you can export and save your storage plugin
configurations from the Storage page in the Drill Web UI. The Export option
next to each enabled storage plugin provides an easy way to export each storage
plugin configuration to a JSON file when you want to save and reuse the
configurations. For example, before an upgrade.
-You can also access the Export option on the Update page, by selecting the
Update option next to an enabled or disabled storage plugin.
+You can also access the Export option on the Update page, by selecting the
Update option next to an enabled or disabled storage plugin.
+
+Starting in Drill 1.16, you can export all storage plugin configurations at
once, by clicking the Export All button. You have the option to export all
storage plugin configurations to a HOCON format (.conf file) or a JSON file.
+
+
When you need to configure the storage plugin, copy and paste the contents of
the JSON file into the Configuration field. You can access the Configuration
field for a storage plugin by selecting Update next to the storage plugin.
diff --git a/_docs/developer-information/rest-api/010-rest-api-introduction.md
b/_docs/developer-information/rest-api/010-rest-api-introduction.md
index 7ef22bc..be66bbc 100644
--- a/_docs/developer-information/rest-api/010-rest-api-introduction.md
+++ b/_docs/developer-information/rest-api/010-rest-api-introduction.md
@@ -1,6 +1,6 @@
---
title: "REST API Introduction"
-date: 2018-12-08
+date: 2019-04-18
parent: "REST API"
---
@@ -62,7 +62,8 @@ Submit a query and return results.
**Parameters**
* queryType--SQL, PHYSICAL, or LOGICAL are valid types. Use only "SQL". Other
types are for internal use only.
-* query--A SQL query that runs in Drill.
+* query--A SQL query that runs in Drill.
+* autoLimit--Limits the number of rows returned from the result set. (Drill
1.16+)
**Request Body**
@@ -73,7 +74,7 @@ Submit a query and return results.
**Example**
- curl -X POST -H "Content-Type: application/json" -d '{"queryType":"SQL",
"query": "select * from
dfs.`/Users/joe-user/apache-drill-1.4.0/sample-data/donuts.json` where name=
\u0027Cake\u0027"}' http://localhost:8047/query.json
+ curl -X POST -H "Content-Type: application/json" -d '{"queryType":"SQL",
"query": "select * from
dfs.`/Users/joe-user/apache-drill-1.4.0/sample-data/donuts.json` where name=
\u0027Cake\u0027", "autoLimit":1}' http://localhost:8047/query.json
**Response Body**
diff --git
a/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
b/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
index d3984bf..e18941e 100644
---
a/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
+++
b/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
@@ -1,6 +1,6 @@
---
title: "Submitting Queries from the REST API when Impersonation is Enabled and
Authentication is Disabled"
-date: 2018-12-08
+date: 2019-04-18
parent: "REST API"
---
@@ -14,7 +14,7 @@ This feature only works when impersonation is enabled and
authentication is disa
A user can issue queries through the Drill Web UI, SQLLine, using curl
commands, or Java code, as follows:
##Drill Web UI
-You can submit a query through the Query page in the Drill Web UI after
entering a valid username in the Username field. The Drill Web UI is accessible
through the URL http(s)://<ip-address>:8047.
+You can submit a query through the Query page in the Drill Web UI after
entering a valid username in the Username field. The Drill Web UI is accessible
through the URL `http(s)://<drill-hostname>:8047`.
To re-run a query from the Profiles page in the Drill Web UI, you must submit
a username prior to re-running the query.
diff --git a/_docs/img/ui-export-all.png b/_docs/img/ui-export-all.png
new file mode 100644
index 0000000..cfb932c
Binary files /dev/null and b/_docs/img/ui-export-all.png differ
diff --git a/_docs/img/ui-limit-results-warn.png
b/_docs/img/ui-limit-results-warn.png
new file mode 100644
index 0000000..1a10b7a
Binary files /dev/null and b/_docs/img/ui-limit-results-warn.png differ
diff --git a/_docs/img/ui-options.PNG b/_docs/img/ui-options.PNG
new file mode 100644
index 0000000..aea171e
Binary files /dev/null and b/_docs/img/ui-options.PNG differ
diff --git a/_docs/img/ui-order-duration.png b/_docs/img/ui-order-duration.png
new file mode 100644
index 0000000..acfc146
Binary files /dev/null and b/_docs/img/ui-order-duration.png differ
diff --git a/_docs/img/ui-results-page.png b/_docs/img/ui-results-page.png
new file mode 100644
index 0000000..ff7cc18
Binary files /dev/null and b/_docs/img/ui-results-page.png differ
diff --git a/_docs/img/ui-row-limit.png b/_docs/img/ui-row-limit.png
new file mode 100644
index 0000000..063c93b
Binary files /dev/null and b/_docs/img/ui-row-limit.png differ
diff --git a/_docs/img/ui-spilled-data.png b/_docs/img/ui-spilled-data.png
new file mode 100644
index 0000000..4e5a75d
Binary files /dev/null and b/_docs/img/ui-spilled-data.png differ
diff --git a/_docs/img/ui-submit-popup.png b/_docs/img/ui-submit-popup.png
new file mode 100644
index 0000000..50bb865
Binary files /dev/null and b/_docs/img/ui-submit-popup.png differ
diff --git a/_docs/img/ui-turtle-wait.png b/_docs/img/ui-turtle-wait.png
new file mode 100644
index 0000000..f83f60d
Binary files /dev/null and b/_docs/img/ui-turtle-wait.png differ
diff --git a/_docs/img/ui-wait-op.png b/_docs/img/ui-wait-op.png
new file mode 100644
index 0000000..49b813c
Binary files /dev/null and b/_docs/img/ui-wait-op.png differ
diff --git a/_docs/install/060-starting-the-web-ui.md
b/_docs/install/060-starting-the-web-ui.md
index 54d6992..8876721 100644
--- a/_docs/install/060-starting-the-web-ui.md
+++ b/_docs/install/060-starting-the-web-ui.md
@@ -1,6 +1,6 @@
---
title: "Starting the Web UI"
-date: 2018-12-28
+date: 2019-04-18
parent: Install Drill
---
@@ -11,19 +11,30 @@ The Drill Web UI is one of several [client
interfaces](/docs/architecture-introd
- [View query profiles]({{site.baseurl}}/docs/query-profiles/)
- [Update and configure storage
plugins]({{site.baseurl}}/docs/storage-plugin-registration/)
- View [logs]({{site.baseurl}}/docs/log-and-debug-introduction/) and
[metrics]({{site.baseurl}}/docs/monitoring-metrics/)
-- [Set configuration
options]({{site.baseurl}}/docs/planning-and-execution-options/#setting-options-from-the-drill-web-ui)
-
-Starting in Drill 1.15, you can use a Meta+Enter key combination instead of
clicking the Submit button to submit queries through the query editor (Query
page) in Drill Web UI. On Mac keyboards, the combination is Ctrl+Meta or
Ctrl+Enter. On Windows and Linux, the combination is Ctrl+Enter; however, on
Linux, if you mapped the meta key to another physical key on the keyboard, use
that key + Enter.
+- [Set configuration
options]({{site.baseurl}}/docs/planning-and-execution-options/#setting-options-from-the-drill-web-ui)
## Accessing the Web UI
-To access the Drill Web UI, enter the URL appropriate for your Drill
configuration. The following list describes the URLs for various Drill
configurations:
+To access the Drill Web UI, enter
`http(s)://<drill-hostname-or-ip-address>:8047` in your browser's address bar.
Note that Drill must be running on the node in order to access the Drill Web
UI.
+
+The following list describes the URLs for various Drill configurations:
-* `http://<IP address or host name>:8047`
+* `http://<drill-hostname-or-ip-address>:8047`
Use this URL when [HTTPS
support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/#https-support)
is disabled (the default).
-* `https://<IP address or host name>:8047`
+* `https://<drill-hostname-or-ip-address>:8047`
Use this URL when HTTPS support is enabled.
* `http://localhost:8047`
- Use this URL when running Drill in embedded mode (./drill-embedded).
+ Use this URL when running Drill in embedded mode (./drill-embedded).
+
+## Running Queries from the Web UI
+
+To run a query in the Web UI, click Query in the toolbar to access the Query
page. Enter your query in the query editor and click Submit or press Ctrl+Enter
to submit the query.
+
+Starting in Drill 1.15, you can use a Meta+Enter key combination to submit
queries through the query editor. On Mac keyboards, the combination is
Ctrl+Meta or Ctrl+Enter. On Windows and Linux, the combination is Ctrl+Enter;
however, on Linux, if you mapped the meta key to another physical key on the
keyboard, use that key + Enter.
+
+Starting in Drill 1.16, a Query Submitted pop-up window appears when you
submit a query.
+
+The pop-up serves as an indicator that a query is in progress and prevents any
further action until the query returns results. The pop-up includes a Check
Status button that links to the Profiles page. If needed, you can [cancel the
query from the Profiles
page]({{site.baseurl}}/docs/monitoring-and-canceling-queries-in-the-drill-web-ui/).
+
## Web UI Security
diff --git
a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
index 53da190..4d70091 100644
---
a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
+++
b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
@@ -1,41 +1,89 @@
---
title: "Query Profiles"
-date: 2018-12-08
+date: 2019-04-18
parent: "Identifying Performance Issues"
---
-A profile is a summary of metrics collected for each query that Drill
executes. Query profiles provide information that you can use to monitor and
analyze query performance. When Drill executes a query, Drill writes the
profile of each query to disk, which is either the local filesystem or a
distributed file system, such as HDFS. As of Drill 1.11, Drill can [store
profiles in
memory]({{site.baseurl}}/docs/start-up-options/#configuring-start-up-options)
instead of writing them to disk. Yo [...]
+A profile is a summary of metrics collected for each query that Drill
executes. Query profiles provide information that you can use to monitor and
analyze query performance. When Drill executes a query, Drill writes the
profile of each query to disk, which is either the local filesystem or a
distributed file system, such as HDFS. As of Drill 1.11, Drill can [store
profiles in
memory]({{site.baseurl}}/docs/start-up-options/#configuring-start-up-options)
instead of writing them to disk. Yo [...]
-##Query Profiles in the Drill Web UI
-The Drill Web UI provides aggregate statistics across profile lists. Profile
lists consist of data from major and minor fragments, operators, and input
streams. You can use profiles in conjunction with Drill logs for debugging
purposes. In addition to viewing query profiles, you can modify, resubmit, or
cancel queries from the Drill Web UI.
+The Drill Web UI provides aggregate statistics across profile lists. Profile
lists consist of data from major and minor fragments, operators, and input
streams. Use query profiles in conjunction with Drill logs to debug issues.
-###Query, Fragment, and Operator Identifiers
-
-Metrics in a query profile are associated with a coordinate system of
identifiers. Drill uses a coordinate system comprised of query, fragment, and
operator identifiers to track query execution activities and resources. Drill
assigns a unique identifier, the QueryID, to each query received and then
assigns an identifier to each fragment and operator that executes the query. An
example of a QueryID is 2aa98add-15b3-e155-5669-603c03bfde86. The following
images shows an example of fragment [...]
+## Viewing a Query Profile
-
+You can view query profiles in the Profiles tab of the Drill Web UI. When you
select the Profiles tab, you see a list of queries than ran or are currently
running in the cluster.
-### Viewing a Query Profile
+
-You can view query profiles in the Profiles tab of the Drill Web UI. When you
select the Profiles tab, you see a list of the last 100 queries than ran or are
currently running in the cluster.
+Starting in Drill 1.16, you can sort the columns on the Profiles page. For
example, if you want to quickly locate queries with the longest running times,
you can sort the Duration column in descending order such that the longest
running queries appear at the top of the list. You can also use the Search
Profiles field to filter the list of query profiles and then use the sort
feature.
-
+For example, the following image shows the query profiles filtered by the mapr
user on a specific date with the Duration column sorted in descending order:
-You must click on a query to see its profile.
+
-
+To see the profile for a specific query, click on the query in the profiles
list.
-When you select a profile, notice that the URL in the address bar contains the
QueryID, as shown in the following URL:
+
- http://<drill_node>:8047/profiles/2aa98add-15b3-e155-5669-603c03bfde86
-
The Query Profile section summarizes a few key details about the query,
including:
* The state of the query, either running, completed, or failed.
* The node operating as the Foreman; the Drillbit that receives a query from
the client or application becomes the Foreman and drives the entire query.
* The total number of minor fragments required to execute the query.
-Further down you can see the Fragment Profiles and Operator Profiles sections.
+When you select a profile, notice that the URL in the address bar contains the
QueryID, as shown in the following URL:
+
+ http://<drill_node>:8047/profiles/2aa98add-15b3-e155-5669-603c03bfde86
+
+##Query, Fragment, and Operator Identifiers
+
+Metrics in a query profile are associated with a coordinate system of
identifiers. Drill uses a coordinate system comprised of query, fragment, and
operator identifiers to track query execution activities and resources. Drill
assigns a unique identifier, the QueryID, to each query received and then
assigns an identifier to each fragment and operator that executes the query. An
example of a QueryID is 2aa98add-15b3-e155-5669-603c03bfde86. The following
image shows an example of fragment a [...]
+
+
+
+
+## Query Profile Warnings
+When Drill executes a query, the workload should be uniformly distributed
across the fragments and operators processing the data. When you evaluate a
query profile in the Drill Web UI, seeing a disproportionate distribution of
work across fragments (in terms of time) or excess memory use typically
indicates performance issues and requires some performance tuning.
+
+Starting in Drill 1.16, the Drill Web UI displays warning messages in the
query profile for the following issues:
+
+- No query progress within a certain amount of time
+- Operators spilling data to disk (operators do not have enough memory to
complete operations completely in-memory)
+- Operators spending significantly more time waiting for data than processing
it
+
+In addition to the warnings, the Operator Profiles section of the query
profile also displays an icon in the column to indicate which operator is
askew.
+
+The following table lists the warnings, icons, configurable options related to
the warnings, and option descriptions:
+
+| Warning Message
| Icon
| Related Option(s)
| Description
[...]
+|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|----------------------------------------------------------------------------
[...]
+| !WARNING: No fragments have made any progress in the last 300 seconds. (See
Last Progress below.)
| None
| drill.exec.http.profile.warning.progress.threshold
| If none of the fragments make any progress within the set
time, a warning [...]
+| !WARNING: Some operators have data spilled to disk. This will result in
performance loss. (See Avg Peak memory and Max Peak Memory below)
|  Look for
the icon in the Avg Peak Memory and Max Peak Memory columns to find the
operators that spilled data to disk. | See [Sort-Based and
Hash-Based Memory Constrained
Operators](https://drill.apache.org/docs/sort-based-and-hash-based-memory-constrai
[...]
+| !WARNING: Some of the operators spent more time waiting for data than
processing it. (See AVG WAIT TIME as compared to Average Process Time for the
operators below.) |  Look
for the icon in the Max Process Time or Max Wait Time columns to locate the
operators that waited or processed too long. |
drill.exec.http.profile.warning.time.skew.min
| Sets the minimum threshold for operators wi [...]
+| See warning for drill.exec.http.profile.warning.time.skew.min
|
| drill.exec.http.profile.warning.time.skew.ratio.process
| Defines the threshold ratio for processing. When the
maxProcessing: [...]
+| See warning for drill.exec.http.profile.warning.time.skew.min
|
| drill.exec.http.profile.warning.time.skew.ratio.wait
| Defines the threshold ratio for waiting. When the
maxWait:avgWait r [...]
+| !WARNING: Some of the SCAN operators spent more time waiting for data than
processing it. (See AVG WAIT TIME as compared to Average Process Time for the
scan operators below.) |  Look for the
icon in the Avg Wait Time or Avg Process Time columns to locate the scan
operators that waited too long. |
drill.exec.http.profile.warning.scan.wait.min
| Sets a minimum threshold ratio for the scan [...]
+
+### Configuring Warning Threshold Options
+You can configure thresholds for the warnings through configuration options in
the drill-override.conf file, as shown:
+
+ http: {
+ profile.warning: {
+ progress.threshold: 300,
+ time.skew: {
+ min: 2,
+ ratio: {
+ process: 2
+ wait: 2
+ }
+ },
+ scan.wait.min: 60
+ },
+ ...
+ }
+
+You must restart Drillbits after you modify drill-override.conf.
+
+The following sections describe the Fragment Profiles and Operator Profiles
sections of the Drill Web UI in more detail.
## Fragment Profiles
diff --git a/_docs/query-data/010-query-data-introduction.md
b/_docs/query-data/010-query-data-introduction.md
index dbe81d5..36afe2a 100644
--- a/_docs/query-data/010-query-data-introduction.md
+++ b/_docs/query-data/010-query-data-introduction.md
@@ -1,16 +1,47 @@
---
title: "Query Data Introduction"
-date: 2018-12-08
+date: 2019-04-18
parent: "Query Data"
---
-You can query local and distributed file systems, Hive, HBase data, complex
data, INFORMATION SCHEMA, and system tables as described in the subtopics of
this section.
-
-The query specifies the data source location and includes data casting.
+You can submit SQL queries against various data sources from the [Drill shell
(SQLLine)]({{site.baseurl}}/docs/configuring-the-drill-shell/), [Drill Web
UI]({{site.baseurl}}/docs/starting-the-web-ui/), [REST
API]({{site.baseurl}}/docs/rest-api/), and tools that connect to Drill via
[ODBC or JDBC]({{site.baseurl}}/docs/odbc-jdbc-interfaces/). Drill has [several
storage and format
plugins]({{site.baseurl}}/docs/connect-a-data-source-introduction/) that enable
queries against multiple data [...]
+
+The following sections provide some general information about Drill queries.
## Specifying the Data Source Location
-The optional [USE statement]({{site.baseurl}}/docs/use) runs subsequent
queries against a particular [storage
plugin]({{site.baseurl}}/docs/connect-a-data-source-introduction/). The USE
statement typically saves typing some of the storage plugin information in the
FROM statement. If you omit the USE statement, specify a storage plugin, such
as dfs, and optionally a workspace, such as default, and a path to the data
source using dot notation and back ticks. For example:
+The optional [USE command]({{site.baseurl}}/docs/use) runs subsequent queries
against a particular [storage plugin or
schema]({{site.baseurl}}/docs/connect-a-data-source-introduction/). When you
run the USE command to switch to a particular storage plugin or schema, you do
not have to include the full path to the data in the FROM clause, for example:
+
+The following query was run before switching to the dfs.schema. A workspace
named "samples" was configured in the dfs storage plugin, creating a schema
named `dfs.samples`. Notice that you have to use dot notation for the schema
and back ticks around the table name. In some cases you may point to a
directory or file in the schema, in which case you would put back ticks around
the entire path, for example ```dfs.samples.`/nation/data/nation.parquet/````.
+
+
+ apache drill> select * from dfs.samples.`nation1`;
+ +-------------+----------------+-------------+----------------------+
+ | N_NATIONKEY | N_NAME | N_REGIONKEY | N_COMMENT |
+ +-------------+----------------+-------------+----------------------+
+ | 0 | ALGERIA | 0 | haggle. carefully f |
+ | 1 | ARGENTINA | 1 | al foxes promise sly |
+ ...
+ +-------------+----------------+-------------+----------------------+
+
+Running USE to switch to the `dfs.samples` schema:
+
+ apache drill> use dfs.samples;
+ +------+-----------------------------------------+
+ | ok | summary |
+ +------+-----------------------------------------+
+ | true | Default schema changed to [dfs.samples] |
+ +------+-----------------------------------------+
+
+Query written without identifying the schema (without dot notation or back
ticks):
+
+ apache drill (dfs.samples)> select * from nation1;
+ +-------------+----------------+-------------+----------------------+
+ | N_NATIONKEY | N_NAME | N_REGIONKEY | N_COMMENT |
+ +-------------+----------------+-------------+----------------------+
+ | 0 | ALGERIA | 0 | haggle. carefully f |
+ | 1 | ARGENTINA | 1 | al foxes promise sly |
+ ...
+ +-------------+----------------+-------------+----------------------+
-``dfs.`default`.`/Users/drill-user/apache-drill-1.1.0/log/sqlline_queries.json```;
## Casting Data
In some cases, Drill converts schema-less data to correctly-typed data
implicitly. In this case, you do not need to [cast the
data]({{site.baseurl}}/docs/supported-data-types/#casting-and-converting-data-types)
to another type. The file format of the data and the nature of your query
determines the requirement for casting or converting. Differences in casting
depend on the data source.
@@ -22,15 +53,12 @@ To query HBase data using Drill, convert every column of an
HBase table to/from
## Troubleshooting Queries
-In addition to testing queries interactively in the Drill shell, and examining
error messages, use the [EXPLAIN command]({{site.baseurl}}/docs/explain/) to
analyze errors and troubleshoot queries
-that do not run. For example, if you run into a casting error, the query plan
-text may help you isolate the problem.
+In addition to analyzing error messages printed by the Drill shell, you can
troubleshoot queries from the [Profiles page]({{ site.baseurl
}}/docs/identifying-performance-issues/) in the Drill Web UI or run the
[EXPLAIN command]({{site.baseurl}}/docs/explain/) to review the query plan for
issues. For example, if you run into a casting error, the query plan text may
help you isolate the problem.
0: jdbc:drill:zk=local> !set maxwidth 10000
- 0: jdbc:drill:zk=local> explain plan for select ... ;
+ 0: jdbc:drill:zk=local> explain plan for <query>;
-[Drill shell commands]({{site.baseurl}}/docs/configuring-the-drill-shell/)
include the `!set <set variable> <value>` to increase the default text display
(number of characters). By
-default, most of the plan output is hidden.
+[Drill shell commands]({{site.baseurl}}/docs/configuring-the-drill-shell/)
include the `!set <set variable> <value>` to increase the default text display
(number of characters). By default, most of the plan output is hidden.
## Query Syntax Tips
