Doc updates for the Drill 1.11 Release
Project: http://git-wip-us.apache.org/repos/asf/drill/repo Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/ec54d586 Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/ec54d586 Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/ec54d586 Branch: refs/heads/gh-pages Commit: ec54d58624f5376db47aa67e94d77b9b4a3ed199 Parents: eaf8c4f Author: Bridget Bevens <bbev...@maprtech.com> Authored: Sun Jul 30 21:43:22 2017 -0700 Committer: Bridget Bevens <bbev...@maprtech.com> Committed: Sun Jul 30 21:43:22 2017 -0700 ---------------------------------------------------------------------- _data/version.json | 10 +- _docs/getting-started/010-drill-introduction.md | 17 +- .../047-installing-drill-on-the-cluster.md | 8 +- ...20-installing-drill-on-linux-and-mac-os-x.md | 8 +- .../040-installing-drill-on-windows.md | 4 +- _docs/rn/001-1.10.0-rn.md | 290 ----------- _docs/rn/001-1.11.0-rn.md | 297 +++++++++++ _docs/rn/002-1.10.0-rn.md | 290 +++++++++++ _docs/rn/002-1.9.0-rn.md | 180 ------- _docs/rn/003-1.8.0-rn.md | 173 ------- _docs/rn/003-1.9.0-rn.md | 180 +++++++ _docs/rn/004-1.7.0-rn.md | 159 ------ _docs/rn/004-1.8.0-rn.md | 173 +++++++ _docs/rn/005-1.6.0-rn.md | 126 ----- _docs/rn/005-1.7.0-rn.md | 159 ++++++ _docs/rn/006-1.5.0-rn.md | 162 ------ _docs/rn/006-1.6.0-rn.md | 126 +++++ _docs/rn/007-1.4.0-rn.md | 105 ---- _docs/rn/007-1.5.0-rn.md | 162 ++++++ _docs/rn/008-1.3.0-rn.md | 159 ------ _docs/rn/008-1.4.0-rn.md | 105 ++++ _docs/rn/009-1.2.0-rn.md | 509 ------------------- _docs/rn/009-1.3.0-rn.md | 159 ++++++ _docs/rn/010-1.1.0-rn.md | 367 ------------- _docs/rn/010-1.2.0-rn.md | 509 +++++++++++++++++++ _docs/rn/020-1.0.0-rn.md | 505 ------------------ _docs/rn/020-1.1.0-rn.md | 367 +++++++++++++ _docs/rn/030-0.9.0-rn.md | 29 -- _docs/rn/030-1.0.0-rn.md | 505 ++++++++++++++++++ _docs/rn/040-0.9.0-rn.md | 29 ++ _docs/rn/071-0.5.0rn.md | 29 ++ _docs/rn/072-0.4.0rn.md | 42 ++ _docs/rn/073-alpha-rn.md | 39 ++ _docs/rn/074-m1-alpha-rn.md | 39 ++ _docs/rn/080-m1-alpha-rn.md | 39 -- _docs/rn/090-alpha-rn.md | 39 -- _docs/rn/100-0.5.0rn.md | 29 -- _docs/rn/110-0.4.0rn.md | 42 -- _docs/tutorials/020-drill-in-10-minutes.md | 10 +- blog/_posts/2017-07-31-drill-1.11-released.md | 55 ++ 40 files changed, 3301 insertions(+), 2934 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_data/version.json ---------------------------------------------------------------------- diff --git a/_data/version.json b/_data/version.json index c5ea3a9..dee77c9 100644 --- a/_data/version.json +++ b/_data/version.json @@ -1,7 +1,7 @@ { - "display_version": "1.10", - "full_version": "1.10.0", - "release_date": "March 15, 2017", - "blog_post":"/blog/2017/03/15/drill-1.10-released", - "release_notes": "https://drill.apache.org/docs/apache-drill-1-10-0-release-notes/" + "display_version": "1.11", + "full_version": "1.11.0", + "release_date": "July 31, 2017", + "blog_post":"/blog/2017/07/31/drill-1.11-released", + "release_notes": "https://drill.apache.org/docs/apache-drill-1-11-0-release-notes/" } http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/getting-started/010-drill-introduction.md ---------------------------------------------------------------------- diff --git a/_docs/getting-started/010-drill-introduction.md b/_docs/getting-started/010-drill-introduction.md index 83d9bb0..b59b07a 100644 --- a/_docs/getting-started/010-drill-introduction.md +++ b/_docs/getting-started/010-drill-introduction.md @@ -1,6 +1,6 @@ --- title: "Drill Introduction" -date: 2017-05-09 01:40:51 UTC +date: 2017-07-31 04:43:33 UTC parent: "Getting Started" --- Drill is an Apache open-source SQL query engine for Big Data exploration. @@ -10,6 +10,21 @@ applications, while still providing the familiarity and ecosystem of ANSI SQL, the industry-standard query language. Drill provides plug-and-play integration with existing Apache Hive and Apache HBase deployments. +## What's New in Apache Drill 1.11 + +Drill 1.11 provides the following new features and improvements: + +- Cryptography-related functions. (DRILL-5634) +- Spill to disk for the hash aggregate operator. (DRILL-5457) +- Format plugin support for PCAP files. (DRILL-5432) +- Ability to change the HDFS block Size for Parquet files. (DRILL-5379) +- Ability to store query profiles in memory. (DRILL-5481) +- Configurable CTAS directory and file permissions option. (DRILL-5391) +- Support for network encryption. (DRILL-4335) +- Relative paths stored in the metadata file. (DRILL-3867) +- Support for ANSI_QUOTES. (DRILL-3510) + + ## What's New in Apache Drill 1.10 Drill 1.10 provides the following new features and improvements: http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/install/047-installing-drill-on-the-cluster.md ---------------------------------------------------------------------- diff --git a/_docs/install/047-installing-drill-on-the-cluster.md b/_docs/install/047-installing-drill-on-the-cluster.md index 815b29c..73c49b8 100644 --- a/_docs/install/047-installing-drill-on-the-cluster.md +++ b/_docs/install/047-installing-drill-on-the-cluster.md @@ -1,13 +1,13 @@ --- title: "Installing Drill on the Cluster" -date: 2017-03-15 21:28:32 UTC +date: 2017-07-31 04:43:41 UTC parent: "Installing Drill in Distributed Mode" --- You install Drill on nodes in the cluster, configure a cluster ID, and add Zookeeper information, as described in the following steps: - 1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) with the command appropriate for your system: - * `wget http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz` - * `curl -o apache-drill-1.10.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz` + 1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) with the command appropriate for your system: + * `wget http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz` + * `curl -o apache-drill-1.11.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz` 2. Extract the tarball to the directory of your choice, such as `/opt`: `tar -xzvf apache-drill-<version>.tar.gz` 3. In `drill-override.conf,` use the Drill `cluster ID`, and provide ZooKeeper host names and port numbers to configure a connection to your ZooKeeper quorum. http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md ---------------------------------------------------------------------- diff --git a/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md b/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md index 4d97073..c5a7412 100644 --- a/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md +++ b/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md @@ -1,6 +1,6 @@ --- title: "Installing Drill on Linux and Mac OS X" -date: 2017-03-15 21:28:34 UTC +date: 2017-07-31 04:43:45 UTC parent: "Installing Drill in Embedded Mode" --- First, check that you [meet the prerequisites]({{site.baseurl}}/docs/embedded-mode-prerequisites), and then install Apache Drill on Linux or Mac OS X: @@ -8,9 +8,9 @@ First, check that you [meet the prerequisites]({{site.baseurl}}/docs/embedded-mo Complete the following steps to install Drill: 1. In a terminal window, change to the directory where you want to install Drill. -2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) with the command appropriate for your system: - * `wget http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz` - * `curl -o apache-drill-1.10.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz` +2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) with the command appropriate for your system: + * `wget http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz` + * `curl -o apache-drill-1.11.0.tar.gz http://www.apache.org/dyn/closer.cgi/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz` 3. Copy the downloaded file to the directory where you want to install Drill. 4. Extract the contents of the Drill `.tar.gz` file. Use sudo only if necessary: `tar -xvzf <.tar.gz file name>` http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md ---------------------------------------------------------------------- diff --git a/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md b/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md index ed67ad0..5c814c9 100644 --- a/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md +++ b/_docs/install/installing-drill-in-embedded-mode/040-installing-drill-on-windows.md @@ -1,11 +1,11 @@ --- title: "Installing Drill on Windows" -date: 2017-03-15 21:28:35 UTC +date: 2017-07-31 04:43:51 UTC parent: "Installing Drill in Embedded Mode" --- First, check that you [meet the prerequisites]({{site.baseurl}}/docs/embedded-mode-prerequisites), including setting the JAVA_HOME environment variable, and then install Drill. Currently, Drill supports 64-bit Windows only. Complete the following steps to install Drill: -1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz). +1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz). 2. Move the downloaded file to the directory where you want to install Drill. 3. Unzip the GZ file using a third-party tool. If the tool you use does not unzip the underlying TAR file as well as the GZ file, perform a second unzip to extract the Drill software. The extraction process creates the installation directory containing the Drill software. http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/001-1.10.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/001-1.10.0-rn.md b/_docs/rn/001-1.10.0-rn.md deleted file mode 100644 index 0f83cf3..0000000 --- a/_docs/rn/001-1.10.0-rn.md +++ /dev/null @@ -1,290 +0,0 @@ ---- -title: "Apache Drill 1.10.0 Release Notes" -parent: "Release Notes" ---- - -**Release date:** March 15, 2017 - -Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/). - -## New Features and Improvements -This release of Drill provides the following new features and improvements: - -- Support for the CREATE TEMPORARY TABLE AS (CTTAS) command. -- A JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client. -- The Web Console displays the Drill version and additional query profile statistics. -- Drill implicitly interprets the INT96 timestamp data type in Parquet files. -- Support for Kerberos authentication between the client and drillbit. - -The following sections list additional bug fixes and improvements: - -<h2> Sub-task -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4272'>DRILL-4272</a>] - When sort runs out of memory and query fails, resources are seemingly not freed -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4301'>DRILL-4301</a>] - OOM : Unable to allocate sv2 for 1000 records, and not enough batchGroups to spill. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4730'>DRILL-4730</a>] - Update JDBC DatabaseMetaData implementation to use new Metadata APIs -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5008'>DRILL-5008</a>] - Refactor, document and simplify ExternalSortBatch -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5011'>DRILL-5011</a>] - External Sort Batch memory use depends on record width -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5014'>DRILL-5014</a>] - ExternalSortBatch cache size, spill count differs from config setting -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5017'>DRILL-5017</a>] - Config param drill.exec.sort.external.batch.size is not used -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5019'>DRILL-5019</a>] - ExternalSortBatch spills all batches to disk even if even one spills -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5020'>DRILL-5020</a>] - ExternalSortBatch has inconsistent notions of the memory limit -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5022'>DRILL-5022</a>] - ExternalSortBatch sets two different limits for "copier" memory -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5023'>DRILL-5023</a>] - ExternalSortBatch does not spill fully, throws off spill calculations -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5025'>DRILL-5025</a>] - ExternalSortBatch provides weak control over spill file size -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5026'>DRILL-5026</a>] - ExternalSortBatch uses two memory allocators; one will do -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5027'>DRILL-5027</a>] - ExternalSortBatch is inefficient: rewrites data unnecessarily -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5055'>DRILL-5055</a>] - External Sort does not delete spill file if error occurs during close -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5062'>DRILL-5062</a>] - External sort refers to the deprecated HDFS fs.default.name param -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5066'>DRILL-5066</a>] - External sort attempts to retry sv2 memory alloc, even if can never succeed -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5210'>DRILL-5210</a>] - External Sort BatchGroup leaks memory if an OOM occurs during read -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5262'>DRILL-5262</a>] - NPE in managed external sort while spilling to disk -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5264'>DRILL-5264</a>] - Managed External Sort fails with OOM -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5267'>DRILL-5267</a>] - Managed external sort spills too often with Parquet data -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5285'>DRILL-5285</a>] - Provide detailed, accurate estimate of size consumed by a record batch -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5294'>DRILL-5294</a>] - Managed External Sort throws an OOM during the merge and spill phase -</li> -</ul> - -<h2> Bug -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1808'>DRILL-1808</a>] - Large compilation unit tests fails due to high memory allocation -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2293'>DRILL-2293</a>] - CTAS does not clean up when it fails -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3562'>DRILL-3562</a>] - Query fails when using flatten on JSON data where some documents have an empty array -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4578'>DRILL-4578</a>] - "children" missing from results of full scan over JSON data -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4764'>DRILL-4764</a>] - Parquet file with INT_16, etc. logical types not supported by simple SELECT -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4812'>DRILL-4812</a>] - Wildcard queries fail on Windows -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4850'>DRILL-4850</a>] - TPCDS Query 33 failed in the second and 3rd runs, but succeeded in the 1st run -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4872'>DRILL-4872</a>] - NPE from CTAS partitioned by a projected casted null -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4919'>DRILL-4919</a>] - Fix select count(1) / count(*) on csv with header -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4938'>DRILL-4938</a>] - Report UserException when constant expression reduction fails -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4963'>DRILL-4963</a>] - Issues when overloading Drill native functions with dynamic UDFs -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4982'>DRILL-4982</a>] - Hive Queries degrade when queries switch between different formats -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4994'>DRILL-4994</a>] - Prepared statement stopped working between 1.8.0 client and < 1.8.0 server -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4995'>DRILL-4995</a>] - Allow lazy init when dynamic UDF support is disabled -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4996'>DRILL-4996</a>] - Parquet Date auto-correction is not working in auto-partitioned parquet files generated by drill-1.6 -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5005'>DRILL-5005</a>] - Potential issues with external sort info in query profile -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5015'>DRILL-5015</a>] - As per documentation, when issuing a list of drillbits in the connection string, we always attempt to connect only to the first one -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5032'>DRILL-5032</a>] - Drill query on hive parquet table failed with OutOfMemoryError: Java heap space -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5034'>DRILL-5034</a>] - Select timestamp from hive generated parquet always return in UTC -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5039'>DRILL-5039</a>] - NPE - CTAS PARTITION BY (<char-type-column>) -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5040'>DRILL-5040</a>] - Interrupted CTAS should not succeed & should not create physical file on disk -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5044'>DRILL-5044</a>] - Fix retry logic to handle VersionMismatchException by not deleting jars in remote UDFs area -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5048'>DRILL-5048</a>] - Fix type mismatch error in case statement with null timestamp -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5050'>DRILL-5050</a>] - C++ client library has symbol resolution issues when loaded by a process that already uses boost::asio -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5051'>DRILL-5051</a>] - DRILL-5051: Fix incorrect result returned in nest query with offset specified -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5070'>DRILL-5070</a>] - Code gen: create methods in fixed order to allow test verification -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5081'>DRILL-5081</a>] - Excessive info level logging introduced in DRILL-4203 -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5086'>DRILL-5086</a>] - ClassCastException when filter pushdown is used with a bigint or float column and metadata caching. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5088'>DRILL-5088</a>] - Error when reading DBRef column -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5091'>DRILL-5091</a>] - JDBC unit test fail on Java 8 -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5094'>DRILL-5094</a>] - Assure Comparator to be transitive -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5097'>DRILL-5097</a>] - Using store.parquet.reader.int96_as_timestamp gives IOOB whereas convert_from works -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5104'>DRILL-5104</a>] - Foreman sets external sort memory allocation even for a physical plan -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5112'>DRILL-5112</a>] - Unit tests derived from PopUnitTestBase fail in IDE due to config errors -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5113'>DRILL-5113</a>] - Upgrade Maven RAT plugin to avoid annoying XML errors -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5117'>DRILL-5117</a>] - Compile error when query a json file with 1000+columns -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5119'>DRILL-5119</a>] - Update MapR version to 5.2.0.40963-mapr -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5121'>DRILL-5121</a>] - A memory leak is observed when exact case is not specified for a column in a filter condition -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5127'>DRILL-5127</a>] - Revert the fix for DRILL-4831 -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5157'>DRILL-5157</a>] - Multiple Snappy versions on class path; causes unit test failures -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5159'>DRILL-5159</a>] - ProjectMergeRule in Drill should operate on RelNodes with same convention trait. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5164'>DRILL-5164</a>] - Equi-join query results in CompileException when inputs have large number of columns -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5167'>DRILL-5167</a>] - C++ connector does not set escape string for metadata search pattern -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5190'>DRILL-5190</a>] - Display planning and queued time for a query in its profile page -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5196'>DRILL-5196</a>] - Could not run a single MongoDB unit test case through command line or IDE -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5207'>DRILL-5207</a>] - Improve Parquet scan pipelining -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5208'>DRILL-5208</a>] - Finding path to java executable should be deterministic -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5218'>DRILL-5218</a>] - Support Disabling Heartbeats in C++ Client -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5224'>DRILL-5224</a>] - CTTAS: fix errors connected with system path delimiters (Windows) -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5230'>DRILL-5230</a>] - Translation of millisecond duration into hours is incorrect -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5238'>DRILL-5238</a>] - CTTAS: unable to resolve temporary table if workspace is indicated without schema -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5242'>DRILL-5242</a>] - The UI breaks when trying to render profiles having unknown metrics -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5243'>DRILL-5243</a>] - Fix TestContextFunctions.sessionIdUDFWithinSameSession unit test -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5252'>DRILL-5252</a>] - A condition returns always true -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5263'>DRILL-5263</a>] - Prevent left NLJoin with non scalar subqueries -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5266'>DRILL-5266</a>] - Parquet Reader produces "low density" record batches - bits vs. bytes -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5273'>DRILL-5273</a>] - CompliantTextReader exhausts 4 GB memory when reading 5000 small files -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5274'>DRILL-5274</a>] - Exception thrown in Drillbit shutdown in UDF cleanup code -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5275'>DRILL-5275</a>] - Sort spill serialization is slow due to repeated buffer allocations -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5284'>DRILL-5284</a>] - Roll-up of final fixes for managed sort -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5287'>DRILL-5287</a>] - Provide option to skip updates of ephemeral state changes in Zookeeper -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5293'>DRILL-5293</a>] - Poor performance of Hash Table due to same hash value as distribution below -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5304'>DRILL-5304</a>] - Queries fail intermittently when there is skew in data distribution -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5313'>DRILL-5313</a>] - C++ client build failure on linux -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5326'>DRILL-5326</a>] - Unit tests failures related to the SERVER_METADTA -</li> -</ul> - -<h2> Improvement -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4217'>DRILL-4217</a>] - Query parquet file treat INT_16 & INT_8 as INT32 -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4280'>DRILL-4280</a>] - Kerberos Authentication -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4373'>DRILL-4373</a>] - Drill and Hive have incompatible timestamp representations in parquet -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4604'>DRILL-4604</a>] - Generate warning on Web UI if drillbits version mismatch is detected -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4864'>DRILL-4864</a>] - Add ANSI format for date/time functions -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4956'>DRILL-4956</a>] - Temporary tables support -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4980'>DRILL-4980</a>] - Upgrading of the approach of parquet date correctness status detection -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4987'>DRILL-4987</a>] - Use ImpersonationUtil in RemoteFunctionRegistry -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5043'>DRILL-5043</a>] - Function that returns a unique id per session/connection similar to MySQL's CONNECTION_ID() -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5052'>DRILL-5052</a>] - Option to debug generated Java code using an IDE -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5056'>DRILL-5056</a>] - UserException does not write full message to log -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5065'>DRILL-5065</a>] - Optimize count(*) queries on MapR-DB JSON Tables -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5080'>DRILL-5080</a>] - Create a memory-managed version of the External Sort operator -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5085'>DRILL-5085</a>] - Add / update description for dynamic UDFs directories in drill-env.sh and drill-module.conf -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5098'>DRILL-5098</a>] - Improving fault tolerance for connection between client and foreman node. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5108'>DRILL-5108</a>] - Reduce output from Maven git-commit-id-plugin -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5116'>DRILL-5116</a>] - Enable generated code debugging in each Drill operator -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5123'>DRILL-5123</a>] - Write query profile after sending final response to client to improve latency -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5126'>DRILL-5126</a>] - Provide simplified, unified "cluster fixture" for tests -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5172'>DRILL-5172</a>] - Display elapsed time for queries in the UI -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5195'>DRILL-5195</a>] - Publish Operator and MajorFragment Stats in Profile page -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5215'>DRILL-5215</a>] - CTTAS: disallow temp tables in view expansion logic -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5221'>DRILL-5221</a>] - cancel message is delayed until queryid or data is received -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5254'>DRILL-5254</a>] - Enhance default reduction factors in optimizer -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5255'>DRILL-5255</a>] - Unit tests fail due to CTTAS temporary name space checks -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5257'>DRILL-5257</a>] - Provide option to save query profiles sync, async or not at all -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5258'>DRILL-5258</a>] - Allow "extended" mock tables access from SQL queries -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5259'>DRILL-5259</a>] - Allow listing a user-defined number of profiles -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5260'>DRILL-5260</a>] - Refinements to new "Cluster Fixture" test framework -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5290'>DRILL-5290</a>] - Provide an option to build operator table once for built-in static functions and reuse it across queries. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5301'>DRILL-5301</a>] - Add server metadata API -</li> -</ul> - -<h2> New Feature -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4935'>DRILL-4935</a>] - Allow drillbits to advertise a configurable host address to Zookeeper -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4979'>DRILL-4979</a>] - Make dataport configurable -</li> -</ul> - -<h2> Task -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5354'>DRILL-5354</a>] - Create CTTAS Documentaion -</li> -</ul> - - \ No newline at end of file http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/001-1.11.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/001-1.11.0-rn.md b/_docs/rn/001-1.11.0-rn.md new file mode 100644 index 0000000..378cb47 --- /dev/null +++ b/_docs/rn/001-1.11.0-rn.md @@ -0,0 +1,297 @@ +--- +title: "Apache Drill 1.11.0 Release Notes" +parent: "Release Notes" +--- + +**Release date:** July 31, 2017 + +Today, we're happy to announce the availability of Drill 1.11.0. You can download it [here](https://drill.apache.org/download/). + +## New Features and Improvements +This release of Drill provides the following new features and improvements: + +- Cryptography-related functions. (DRILL-5634) +- Spill to disk for the hash aggregate operator. (DRILL-5457) +- Format plugin support for PCAP files. (DRILL-5432) +- Ability to change the HDFS block Size for Parquet files. (DRILL-5379) +- Ability to store query profiles in memory. (DRILL-5481) +- Configurable CTAS directory and file permissions option. (DRILL-5391) +- Support for network encryption. (DRILL-4335) +- Relative paths stored in the metadata file. (DRILL-3867) +- Support for ANSI_QUOTES. (DRILL-3510) + + +The following sections list additional bug fixes and improvements: + +<h2> Sub-task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3250'>DRILL-3250</a>] - Drill fails to compare multi-byte characters from hive table +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4301'>DRILL-4301</a>] - OOM : Unable to allocate sv2 for 1000 records, and not enough batchGroups to spill. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5008'>DRILL-5008</a>] - Refactor, document and simplify ExternalSortBatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5011'>DRILL-5011</a>] - External Sort Batch memory use depends on record width +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5014'>DRILL-5014</a>] - ExternalSortBatch cache size, spill count differs from config setting +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5019'>DRILL-5019</a>] - ExternalSortBatch spills all batches to disk even if even one spills +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5020'>DRILL-5020</a>] - ExternalSortBatch has inconsistent notions of the memory limit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5022'>DRILL-5022</a>] - ExternalSortBatch sets two different limits for "copier" memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5023'>DRILL-5023</a>] - ExternalSortBatch does not spill fully, throws off spill calculations +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5025'>DRILL-5025</a>] - ExternalSortBatch provides weak control over spill file size +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5026'>DRILL-5026</a>] - ExternalSortBatch uses two memory allocators; one will do +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5027'>DRILL-5027</a>] - ExternalSortBatch is inefficient: rewrites data unnecessarily +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5055'>DRILL-5055</a>] - External Sort does not delete spill file if error occurs during close +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5062'>DRILL-5062</a>] - External sort refers to the deprecated HDFS fs.default.name param +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5066'>DRILL-5066</a>] - External sort attempts to retry sv2 memory alloc, even if can never succeed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5210'>DRILL-5210</a>] - External Sort BatchGroup leaks memory if an OOM occurs during read +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5285'>DRILL-5285</a>] - Provide detailed, accurate estimate of size consumed by a record batch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5312'>DRILL-5312</a>] - "Record batch sizer" does not include overhead for variable-sized vectors +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5319'>DRILL-5319</a>] - Refactor FragmentContext and OptionManager for unit testing +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5320'>DRILL-5320</a>] - Refactor OptionManager for unit testing +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5321'>DRILL-5321</a>] - Refactor FragmentContext for unit testing +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5322'>DRILL-5322</a>] - Provide an OperatorFixture for sub-operator unit testing setup +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5323'>DRILL-5323</a>] - Provide test tools to create, populate and compare row sets +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5324'>DRILL-5324</a>] - Provide simplified column reader/writer for use in tests +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5331'>DRILL-5331</a>] - NPE in FunctionImplementationRegistry.findDrillFunction() if dynamic UDFs disabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5342'>DRILL-5342</a>] - Refactor "managed" external sort for unit tests +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5567'>DRILL-5567</a>] - Review changes for DRILL 5514 +</li> +</ul> + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3867'>DRILL-3867</a>] - Store relative paths in metadata file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4039'>DRILL-4039</a>] - Query fails when non-ascii characters are used in string literals +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4347'>DRILL-4347</a>] - Planning time for query64 from TPCDS test suite has increased 10 times compared to 1.4 release +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4511'>DRILL-4511</a>] - refresh over empty folder results in error, we need a better error message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4678'>DRILL-4678</a>] - Tune metadata by generating a dispatcher at runtime +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4720'>DRILL-4720</a>] - MINDIR() and IMINDIR() functions return no results with metadata cache +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4722'>DRILL-4722</a>] - Fix EqualityVisitor for interval day expressions with millis +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4755'>DRILL-4755</a>] - StringIndexOutOfBoundsException seen with CONVERT_FROM function +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4903'>DRILL-4903</a>] - Implicit columns are shown when Jdbc plugin is enabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4970'>DRILL-4970</a>] - Wrong results when casting double to bigint or int +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4971'>DRILL-4971</a>] - Query encounters system error, when there aren't eval subexpressions of any function in boolean and/or expressions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5005'>DRILL-5005</a>] - Potential issues with external sort info in query profile +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5083'>DRILL-5083</a>] - RecordIterator can sometimes restart a query on close +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5130'>DRILL-5130</a>] - UNION ALL difference in results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5140'>DRILL-5140</a>] - Fix CompileException in run-time generated code when record batch has large number of fields. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5160'>DRILL-5160</a>] - Memory leak in Parquet async reader when Snappy fails +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5164'>DRILL-5164</a>] - Equi-join query results in CompileException when inputs have large number of columns +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5165'>DRILL-5165</a>] - wrong results - LIMIT ALL and OFFSET clause in same query +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5213'>DRILL-5213</a>] - Prepared statement for actual query is missing the query text +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5226'>DRILL-5226</a>] - External Sort encountered an error while spilling to disk +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5229'>DRILL-5229</a>] - Upgrade kudu client to org.apache.kudu:kudu-client:1.2.0 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5234'>DRILL-5234</a>] - External sort's spilling functionality does not work when the spilled columns contains a map type column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5284'>DRILL-5284</a>] - Roll-up of final fixes for managed sort +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5297'>DRILL-5297</a>] - Print the plan text when plan pattern check fails in unit tests +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5311'>DRILL-5311</a>] - C++ connector connect doesn't check handshake result for timeout +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5316'>DRILL-5316</a>] - C++ Client Crashes When drillbitsVector.count is 0 after zoo_get_children completed with ZOK +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5330'>DRILL-5330</a>] - NPE in FunctionImplementationRegistry.functionReplacement() +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5344'>DRILL-5344</a>] - External sort priority queue copier fails with an empty batch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5349'>DRILL-5349</a>] - TestParquetWriter unit tests fail with synchronous parquet reader +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5359'>DRILL-5359</a>] - ClassCastException when push down filter on the output of flatten into parquet scan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5368'>DRILL-5368</a>] - Memory leak in C++ server metadata handler +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5369'>DRILL-5369</a>] - Missing initialization for ServerMetaContext +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5373'>DRILL-5373</a>] - Drill JDBC error in the process of connection via SQuirrel: java.lang.NoClassDefFoundError: javax/validation/constraints/NotNull +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5375'>DRILL-5375</a>] - Nested loop join: return correct result for left join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5378'>DRILL-5378</a>] - Put more information into SchemaChangeException when HashJoin hit SchemaChangeException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5385'>DRILL-5385</a>] - Vector serializer fails to read saved SV2 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5387'>DRILL-5387</a>] - TestBitBitKerberos and TestUserBitKerberos cause sporadic unit test failures +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5395'>DRILL-5395</a>] - Query on MapR-DB table fails with NPE due to an issue with assignment logic +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5397'>DRILL-5397</a>] - Random Error : Unable to get holder type for minor type [LATE] and mode [OPTIONAL] +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5399'>DRILL-5399</a>] - Fix race condition in DrillComplexWriterFuncHolder +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5409'>DRILL-5409</a>] - Update MapR version to 5.2.1-mapr +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5413'>DRILL-5413</a>] - DrillConnectionImpl.isReadOnly() throws NullPointerException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5419'>DRILL-5419</a>] - Calculate return string length for literals & some string functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5420'>DRILL-5420</a>] - all cores at 100% of all servers +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5424'>DRILL-5424</a>] - Fix IOBE for reverse function +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5428'>DRILL-5428</a>] - submit_plan fails after Drill 1.8 script revisions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5429'>DRILL-5429</a>] - Improve query performance for MapR DB JSON Tables +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5450'>DRILL-5450</a>] - Fix initcap function to convert upper case characters correctly +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5496'>DRILL-5496</a>] - Must restart drillbits whenever a secure Hive metastore is restarted +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5498'>DRILL-5498</a>] - CSV text reader does not handle duplicate header names +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5523'>DRILL-5523</a>] - Revert if condition in UnionAllRecordBatch changed in DRILL-5419 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5533'>DRILL-5533</a>] - Fix flag assignment in FunctionInitializer.checkInit() method +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5537'>DRILL-5537</a>] - Display columns alias for queries with sum() when RDBMS storage plugin is enabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5538'>DRILL-5538</a>] - Create TopProject with validatedNodeType after PHYSICAL phase +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5541'>DRILL-5541</a>] - C++ Client Crashes During Simple "Man in the Middle" Attack Test with Exploitable Write AV +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5544'>DRILL-5544</a>] - Out of heap running CTAS against text delimited +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5560'>DRILL-5560</a>] - Create configuration file for distribution specific configuration +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5577'>DRILL-5577</a>] - Column alias are ignored when Storage Plugin is enabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5587'>DRILL-5587</a>] - Validate Parquet blockSize and pageSize configured with SYSTEM/SESSION option +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5589'>DRILL-5589</a>] - JDBC client crashes after successful authentication if trace logging is enabled. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5590'>DRILL-5590</a>] - Drill return IndexOutOfBoundsException when a (Text) file > 4096 rows +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5599'>DRILL-5599</a>] - Notify StatusHandlerListener that batch sending has failed even if channel is still open +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5616'>DRILL-5616</a>] - Hash Agg Spill: OOM while reading irregular varchar data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5659'>DRILL-5659</a>] - C++ Client (master) behavior is unstable resulting incorrect result or exception in API calls +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5665'>DRILL-5665</a>] - planner.force_2phase.aggr Set to TRUE for HashAgg may cause wrong results for VARIANCE and STD_DEV +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5668'>DRILL-5668</a>] - C++ connector crash when query error message is too long +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5669'>DRILL-5669</a>] - Multiple TPCH queries failed due to OOM +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5678'>DRILL-5678</a>] - Undefined behavior due to un-initialized values in ServerMetaContext +</li> +</ul> + +<h2> Improvement +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2974'>DRILL-2974</a>] - Make OutOfMemoryException an unchecked exception and remove OutOfMemoryRuntimeException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3510'>DRILL-3510</a>] - Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5056'>DRILL-5056</a>] - UserException does not write full message to log +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5080'>DRILL-5080</a>] - Create a memory-managed version of the External Sort operator +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5163'>DRILL-5163</a>] - External sort on Mac creates a separate child process per spill via HDFS FS +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5315'>DRILL-5315</a>] - Small Comment Typo in drillClient.hpp +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5318'>DRILL-5318</a>] - Create a sub-operator test framework +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5325'>DRILL-5325</a>] - Implement sub-operator unit tests for managed external sort +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5351'>DRILL-5351</a>] - Excessive bounds checking in the Parquet reader +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5352'>DRILL-5352</a>] - Extend test framework profile parser printer for multi-fragment queries +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5355'>DRILL-5355</a>] - Misc. code cleanup +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5356'>DRILL-5356</a>] - Refactor Parquet Record Reader +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5379'>DRILL-5379</a>] - Set Hdfs Block Size based on Parquet Block Size +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5391'>DRILL-5391</a>] - CTAS: make folder and file permission configurable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5394'>DRILL-5394</a>] - Optimize query planning for MapR-DB tables by caching row counts +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5415'>DRILL-5415</a>] - Improve Fixture Builder to configure client properties and keep collection type properties for server +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5423'>DRILL-5423</a>] - Refactor ScanBatch to allow unit testing record readers +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5457'>DRILL-5457</a>] - Support Spill to Disk for the Hash Aggregate Operator +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5481'>DRILL-5481</a>] - Allow Drill to persist profiles in-memory only with a max capacity +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5485'>DRILL-5485</a>] - Remove WebServer dependency on DrillClient +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5504'>DRILL-5504</a>] - Vector validator to diagnose offset vector issues +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5512'>DRILL-5512</a>] - Standardize error handling in ScanBatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5514'>DRILL-5514</a>] - Enhance VectorContainer to merge two row sets +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5516'>DRILL-5516</a>] - Limit memory usage for Hbase reader +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5517'>DRILL-5517</a>] - Provide size-aware set operations in value vectors +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5518'>DRILL-5518</a>] - Roll-up of a number of test framework enhancements +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5545'>DRILL-5545</a>] - Add findbugs to build +</li> +</ul> + +<h2> New Feature +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-291'>DRILL-291</a>] - Add SASL support for Drill +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4335'>DRILL-4335</a>] - Apache Drill should support network encryption - SASL encryption between Drill Client to Drillbit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5432'>DRILL-5432</a>] - Added pcap-format support +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5634'>DRILL-5634</a>] - Add Crypto and Hash Functions +</li> +</ul> + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/002-1.10.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/002-1.10.0-rn.md b/_docs/rn/002-1.10.0-rn.md new file mode 100644 index 0000000..0f83cf3 --- /dev/null +++ b/_docs/rn/002-1.10.0-rn.md @@ -0,0 +1,290 @@ +--- +title: "Apache Drill 1.10.0 Release Notes" +parent: "Release Notes" +--- + +**Release date:** March 15, 2017 + +Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/). + +## New Features and Improvements +This release of Drill provides the following new features and improvements: + +- Support for the CREATE TEMPORARY TABLE AS (CTTAS) command. +- A JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client. +- The Web Console displays the Drill version and additional query profile statistics. +- Drill implicitly interprets the INT96 timestamp data type in Parquet files. +- Support for Kerberos authentication between the client and drillbit. + +The following sections list additional bug fixes and improvements: + +<h2> Sub-task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4272'>DRILL-4272</a>] - When sort runs out of memory and query fails, resources are seemingly not freed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4301'>DRILL-4301</a>] - OOM : Unable to allocate sv2 for 1000 records, and not enough batchGroups to spill. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4730'>DRILL-4730</a>] - Update JDBC DatabaseMetaData implementation to use new Metadata APIs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5008'>DRILL-5008</a>] - Refactor, document and simplify ExternalSortBatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5011'>DRILL-5011</a>] - External Sort Batch memory use depends on record width +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5014'>DRILL-5014</a>] - ExternalSortBatch cache size, spill count differs from config setting +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5017'>DRILL-5017</a>] - Config param drill.exec.sort.external.batch.size is not used +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5019'>DRILL-5019</a>] - ExternalSortBatch spills all batches to disk even if even one spills +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5020'>DRILL-5020</a>] - ExternalSortBatch has inconsistent notions of the memory limit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5022'>DRILL-5022</a>] - ExternalSortBatch sets two different limits for "copier" memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5023'>DRILL-5023</a>] - ExternalSortBatch does not spill fully, throws off spill calculations +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5025'>DRILL-5025</a>] - ExternalSortBatch provides weak control over spill file size +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5026'>DRILL-5026</a>] - ExternalSortBatch uses two memory allocators; one will do +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5027'>DRILL-5027</a>] - ExternalSortBatch is inefficient: rewrites data unnecessarily +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5055'>DRILL-5055</a>] - External Sort does not delete spill file if error occurs during close +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5062'>DRILL-5062</a>] - External sort refers to the deprecated HDFS fs.default.name param +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5066'>DRILL-5066</a>] - External sort attempts to retry sv2 memory alloc, even if can never succeed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5210'>DRILL-5210</a>] - External Sort BatchGroup leaks memory if an OOM occurs during read +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5262'>DRILL-5262</a>] - NPE in managed external sort while spilling to disk +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5264'>DRILL-5264</a>] - Managed External Sort fails with OOM +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5267'>DRILL-5267</a>] - Managed external sort spills too often with Parquet data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5285'>DRILL-5285</a>] - Provide detailed, accurate estimate of size consumed by a record batch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5294'>DRILL-5294</a>] - Managed External Sort throws an OOM during the merge and spill phase +</li> +</ul> + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1808'>DRILL-1808</a>] - Large compilation unit tests fails due to high memory allocation +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2293'>DRILL-2293</a>] - CTAS does not clean up when it fails +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3562'>DRILL-3562</a>] - Query fails when using flatten on JSON data where some documents have an empty array +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4578'>DRILL-4578</a>] - "children" missing from results of full scan over JSON data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4764'>DRILL-4764</a>] - Parquet file with INT_16, etc. logical types not supported by simple SELECT +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4812'>DRILL-4812</a>] - Wildcard queries fail on Windows +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4850'>DRILL-4850</a>] - TPCDS Query 33 failed in the second and 3rd runs, but succeeded in the 1st run +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4872'>DRILL-4872</a>] - NPE from CTAS partitioned by a projected casted null +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4919'>DRILL-4919</a>] - Fix select count(1) / count(*) on csv with header +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4938'>DRILL-4938</a>] - Report UserException when constant expression reduction fails +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4963'>DRILL-4963</a>] - Issues when overloading Drill native functions with dynamic UDFs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4982'>DRILL-4982</a>] - Hive Queries degrade when queries switch between different formats +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4994'>DRILL-4994</a>] - Prepared statement stopped working between 1.8.0 client and < 1.8.0 server +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4995'>DRILL-4995</a>] - Allow lazy init when dynamic UDF support is disabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4996'>DRILL-4996</a>] - Parquet Date auto-correction is not working in auto-partitioned parquet files generated by drill-1.6 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5005'>DRILL-5005</a>] - Potential issues with external sort info in query profile +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5015'>DRILL-5015</a>] - As per documentation, when issuing a list of drillbits in the connection string, we always attempt to connect only to the first one +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5032'>DRILL-5032</a>] - Drill query on hive parquet table failed with OutOfMemoryError: Java heap space +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5034'>DRILL-5034</a>] - Select timestamp from hive generated parquet always return in UTC +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5039'>DRILL-5039</a>] - NPE - CTAS PARTITION BY (<char-type-column>) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5040'>DRILL-5040</a>] - Interrupted CTAS should not succeed & should not create physical file on disk +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5044'>DRILL-5044</a>] - Fix retry logic to handle VersionMismatchException by not deleting jars in remote UDFs area +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5048'>DRILL-5048</a>] - Fix type mismatch error in case statement with null timestamp +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5050'>DRILL-5050</a>] - C++ client library has symbol resolution issues when loaded by a process that already uses boost::asio +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5051'>DRILL-5051</a>] - DRILL-5051: Fix incorrect result returned in nest query with offset specified +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5070'>DRILL-5070</a>] - Code gen: create methods in fixed order to allow test verification +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5081'>DRILL-5081</a>] - Excessive info level logging introduced in DRILL-4203 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5086'>DRILL-5086</a>] - ClassCastException when filter pushdown is used with a bigint or float column and metadata caching. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5088'>DRILL-5088</a>] - Error when reading DBRef column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5091'>DRILL-5091</a>] - JDBC unit test fail on Java 8 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5094'>DRILL-5094</a>] - Assure Comparator to be transitive +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5097'>DRILL-5097</a>] - Using store.parquet.reader.int96_as_timestamp gives IOOB whereas convert_from works +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5104'>DRILL-5104</a>] - Foreman sets external sort memory allocation even for a physical plan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5112'>DRILL-5112</a>] - Unit tests derived from PopUnitTestBase fail in IDE due to config errors +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5113'>DRILL-5113</a>] - Upgrade Maven RAT plugin to avoid annoying XML errors +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5117'>DRILL-5117</a>] - Compile error when query a json file with 1000+columns +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5119'>DRILL-5119</a>] - Update MapR version to 5.2.0.40963-mapr +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5121'>DRILL-5121</a>] - A memory leak is observed when exact case is not specified for a column in a filter condition +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5127'>DRILL-5127</a>] - Revert the fix for DRILL-4831 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5157'>DRILL-5157</a>] - Multiple Snappy versions on class path; causes unit test failures +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5159'>DRILL-5159</a>] - ProjectMergeRule in Drill should operate on RelNodes with same convention trait. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5164'>DRILL-5164</a>] - Equi-join query results in CompileException when inputs have large number of columns +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5167'>DRILL-5167</a>] - C++ connector does not set escape string for metadata search pattern +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5190'>DRILL-5190</a>] - Display planning and queued time for a query in its profile page +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5196'>DRILL-5196</a>] - Could not run a single MongoDB unit test case through command line or IDE +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5207'>DRILL-5207</a>] - Improve Parquet scan pipelining +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5208'>DRILL-5208</a>] - Finding path to java executable should be deterministic +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5218'>DRILL-5218</a>] - Support Disabling Heartbeats in C++ Client +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5224'>DRILL-5224</a>] - CTTAS: fix errors connected with system path delimiters (Windows) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5230'>DRILL-5230</a>] - Translation of millisecond duration into hours is incorrect +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5238'>DRILL-5238</a>] - CTTAS: unable to resolve temporary table if workspace is indicated without schema +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5242'>DRILL-5242</a>] - The UI breaks when trying to render profiles having unknown metrics +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5243'>DRILL-5243</a>] - Fix TestContextFunctions.sessionIdUDFWithinSameSession unit test +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5252'>DRILL-5252</a>] - A condition returns always true +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5263'>DRILL-5263</a>] - Prevent left NLJoin with non scalar subqueries +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5266'>DRILL-5266</a>] - Parquet Reader produces "low density" record batches - bits vs. bytes +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5273'>DRILL-5273</a>] - CompliantTextReader exhausts 4 GB memory when reading 5000 small files +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5274'>DRILL-5274</a>] - Exception thrown in Drillbit shutdown in UDF cleanup code +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5275'>DRILL-5275</a>] - Sort spill serialization is slow due to repeated buffer allocations +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5284'>DRILL-5284</a>] - Roll-up of final fixes for managed sort +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5287'>DRILL-5287</a>] - Provide option to skip updates of ephemeral state changes in Zookeeper +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5293'>DRILL-5293</a>] - Poor performance of Hash Table due to same hash value as distribution below +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5304'>DRILL-5304</a>] - Queries fail intermittently when there is skew in data distribution +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5313'>DRILL-5313</a>] - C++ client build failure on linux +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5326'>DRILL-5326</a>] - Unit tests failures related to the SERVER_METADTA +</li> +</ul> + +<h2> Improvement +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4217'>DRILL-4217</a>] - Query parquet file treat INT_16 & INT_8 as INT32 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4280'>DRILL-4280</a>] - Kerberos Authentication +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4373'>DRILL-4373</a>] - Drill and Hive have incompatible timestamp representations in parquet +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4604'>DRILL-4604</a>] - Generate warning on Web UI if drillbits version mismatch is detected +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4864'>DRILL-4864</a>] - Add ANSI format for date/time functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4956'>DRILL-4956</a>] - Temporary tables support +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4980'>DRILL-4980</a>] - Upgrading of the approach of parquet date correctness status detection +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4987'>DRILL-4987</a>] - Use ImpersonationUtil in RemoteFunctionRegistry +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5043'>DRILL-5043</a>] - Function that returns a unique id per session/connection similar to MySQL's CONNECTION_ID() +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5052'>DRILL-5052</a>] - Option to debug generated Java code using an IDE +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5056'>DRILL-5056</a>] - UserException does not write full message to log +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5065'>DRILL-5065</a>] - Optimize count(*) queries on MapR-DB JSON Tables +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5080'>DRILL-5080</a>] - Create a memory-managed version of the External Sort operator +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5085'>DRILL-5085</a>] - Add / update description for dynamic UDFs directories in drill-env.sh and drill-module.conf +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5098'>DRILL-5098</a>] - Improving fault tolerance for connection between client and foreman node. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5108'>DRILL-5108</a>] - Reduce output from Maven git-commit-id-plugin +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5116'>DRILL-5116</a>] - Enable generated code debugging in each Drill operator +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5123'>DRILL-5123</a>] - Write query profile after sending final response to client to improve latency +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5126'>DRILL-5126</a>] - Provide simplified, unified "cluster fixture" for tests +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5172'>DRILL-5172</a>] - Display elapsed time for queries in the UI +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5195'>DRILL-5195</a>] - Publish Operator and MajorFragment Stats in Profile page +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5215'>DRILL-5215</a>] - CTTAS: disallow temp tables in view expansion logic +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5221'>DRILL-5221</a>] - cancel message is delayed until queryid or data is received +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5254'>DRILL-5254</a>] - Enhance default reduction factors in optimizer +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5255'>DRILL-5255</a>] - Unit tests fail due to CTTAS temporary name space checks +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5257'>DRILL-5257</a>] - Provide option to save query profiles sync, async or not at all +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5258'>DRILL-5258</a>] - Allow "extended" mock tables access from SQL queries +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5259'>DRILL-5259</a>] - Allow listing a user-defined number of profiles +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5260'>DRILL-5260</a>] - Refinements to new "Cluster Fixture" test framework +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5290'>DRILL-5290</a>] - Provide an option to build operator table once for built-in static functions and reuse it across queries. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5301'>DRILL-5301</a>] - Add server metadata API +</li> +</ul> + +<h2> New Feature +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4935'>DRILL-4935</a>] - Allow drillbits to advertise a configurable host address to Zookeeper +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4979'>DRILL-4979</a>] - Make dataport configurable +</li> +</ul> + +<h2> Task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5354'>DRILL-5354</a>] - Create CTTAS Documentaion +</li> +</ul> + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/002-1.9.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/002-1.9.0-rn.md b/_docs/rn/002-1.9.0-rn.md deleted file mode 100644 index 93710db..0000000 --- a/_docs/rn/002-1.9.0-rn.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -title: "Apache Drill 1.9.0 Release Notes" -parent: "Release Notes" ---- - -**Release date:** November 29, 2016 - -Today, we're happy to announce the availability of Drill 1.9.0. You can download it [here](https://drill.apache.org/download/). - -## New Features -This release of Drill provides the following new features: - -- Asynchronous Parquet reader -- Parquet filter pushdown -- Dynamic UDF support -- HTTPD format plugin - -The following sections list additional bug fixes and improvements: - -<h2> Sub-task -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4420'>DRILL-4420</a>] - C client and ODBC driver should move to using the new metadata methods provided by DRILL-4385 -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4452'>DRILL-4452</a>] - Update avatica version for Drill jdbc -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4560'>DRILL-4560</a>] - ZKClusterCoordinator does not call DrillbitStatusListener.drillbitRegistered for new bits -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4968'>DRILL-4968</a>] - Add column size information to ColumnMetadata -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4969'>DRILL-4969</a>] - Basic implementation for displaySize -</li> -</ul> - -<h2> Bug -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1996'>DRILL-1996</a>] - C++ Client: Make Cancel API Public -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3898'>DRILL-3898</a>] - No space error during external sort does not cancel the query -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4203'>DRILL-4203</a>] - Parquet File : Date is stored wrongly -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4369'>DRILL-4369</a>] - Database driver fails to report any major or minor version information -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4370'>DRILL-4370</a>] - DatabaseMetadata returning <Properties resource apache-drill-jdbc.properties not loaded> -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4525'>DRILL-4525</a>] - Query with BETWEEN clause on Date and Timestamp values fails with Validation Error -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4542'>DRILL-4542</a>] - if external sort fails to spill to disk, memory is leaked and wrong error message is displayed -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4618'>DRILL-4618</a>] - random numbers generator function broken -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4763'>DRILL-4763</a>] - Parquet file with DATE logical type produces wrong results for simple SELECT -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4767'>DRILL-4767</a>] - Parquet reader throw IllegalArgumentException for int32 type with GZIP compression -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4769'>DRILL-4769</a>] - forman spins query int32 data with snappy compression -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4770'>DRILL-4770</a>] - ParquetRecordReader throws NPE querying a single int64 column file -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4823'>DRILL-4823</a>] - Fix OOM while trying to prune partitions with reasonable data size -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4824'>DRILL-4824</a>] - JSON with complex nested data produces incorrect output with missing fields -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4826'>DRILL-4826</a>] - Query against INFORMATION_SCHEMA.TABLES degrades as the number of views increases -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4862'>DRILL-4862</a>] - wrong results - use of convert_from(binary_string(key),'UTF8') in filter results in wrong results -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4870'>DRILL-4870</a>] - drill-config.sh sets JAVA_HOME incorrectly for the Mac -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4874'>DRILL-4874</a>] - "No UserGroupInformation while generating ORC splits" - hive known issue in 1.2.0-mapr-1607 release. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4877'>DRILL-4877</a>] - max(dir0), max(dir1) query against parquet data slower by 2X -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4880'>DRILL-4880</a>] - Support JDBC driver registration using ServiceLoader -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4884'>DRILL-4884</a>] - Fix IOB exception in limit n query when n is beyond 65535. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4888'>DRILL-4888</a>] - putIfAbsent for ZK stores is not atomic -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4894'>DRILL-4894</a>] - Fix unit test failure in 'storage-hive/core' module -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4905'>DRILL-4905</a>] - Push down the LIMIT to the parquet reader scan to limit the numbers of records read -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4906'>DRILL-4906</a>] - CASE Expression with constant generates class exception -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4911'>DRILL-4911</a>] - SimpleParallelizer should avoid plan serialization for logging purpose when debug logging is not enabled. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4921'>DRILL-4921</a>] - Scripts drill_config.sh, drillbit.sh, and drill-embedded fail when accessed via a symbolic link -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4925'>DRILL-4925</a>] - Add types filter to getTables metadata API -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4930'>DRILL-4930</a>] - Metadata results are not sorted -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4934'>DRILL-4934</a>] - ServiceEngine does not use property useIP for DrillbitStartup -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4941'>DRILL-4941</a>] - UnsupportedOperationException : CASE WHEN true or null then 1 else 0 end -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4945'>DRILL-4945</a>] - Missing subtype information in metadata returned by prepared statement -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4950'>DRILL-4950</a>] - Consume Spurious Empty Batches in JDBC -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4954'>DRILL-4954</a>] - allTextMode in the MapRDB plugin always return nulls -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4964'>DRILL-4964</a>] - Drill fails to connect to hive metastore after hive metastore is restarted unless drillbits are restarted also -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4972'>DRILL-4972</a>] - Drillbit shuts down immediately after starting if embedded web server is disabled -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4974'>DRILL-4974</a>] - NPE in FindPartitionConditions.analyzeCall() for 'holistic' expressions -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4989'>DRILL-4989</a>] - Fix TestParquetWriter.testImpalaParquetBinaryAsTimeStamp_DictChange -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4990'>DRILL-4990</a>] - Use new HDFS API access instead of listStatus to check if users have permissions to access workspace. -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4993'>DRILL-4993</a>] - Documentation: Wrong output displayed for convert_from() with a map -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5004'>DRILL-5004</a>] - Parquet date correction gives null pointer exception if there is no createdBy entry in the metadata -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5007'>DRILL-5007</a>] - Dynamic UDF lazy-init does not work correctly in multi-node cluster -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5009'>DRILL-5009</a>] - Query with a simple join fails on Hive generated parquet -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5047'>DRILL-5047</a>] - When session option is string, query profile is displayed incorrectly on Web UI -</li> -</ul> - -<h2> Improvement -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1950'>DRILL-1950</a>] - Implement filter pushdown for Parquet -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3178'>DRILL-3178</a>] - csv reader should allow newlines inside quotes -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4653'>DRILL-4653</a>] - Malformed JSON should not stop the entire query from progressing -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4674'>DRILL-4674</a>] - Allow casting to boolean the same literals as in Postgre -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4752'>DRILL-4752</a>] - Remove submit_plan script from Drill distribution -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4771'>DRILL-4771</a>] - Drill should avoid doing the same join twice if count(distinct) exists -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4792'>DRILL-4792</a>] - Include session options used for a query as part of the profile -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4800'>DRILL-4800</a>] - Improve parquet reader performance -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4927'>DRILL-4927</a>] - Add support for Null Equality Joins -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4967'>DRILL-4967</a>] - Adding template_name to source code generated using freemarker template -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4986'>DRILL-4986</a>] - Allow users to customize the Drill log file name -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4987'>DRILL-4987</a>] - Use ImpersonationUtil in RemoteFunctionRegistry -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5031'>DRILL-5031</a>] - Documentation for HTTPD Parser -</li> -</ul> - -<h2> New Feature -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1268'>DRILL-1268</a>] - C++ Client. Write Unit Test for Drill Client -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3423'>DRILL-3423</a>] - Add New HTTPD format plugin -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4714'>DRILL-4714</a>] - Add metadata and prepared statement APIs to DrillClient<->Drillbit interface -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4726'>DRILL-4726</a>] - Dynamic UDFs support -</li> -</ul> - -<h2> Task -</h2> -<ul> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4853'>DRILL-4853</a>] - Update C++ protobuf source files -</li> -<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4886'>DRILL-4886</a>] - Merge maprdb format plugin source code -</li> -</ul> - \ No newline at end of file