Repository: drill Updated Branches: refs/heads/gh-pages eaf8c4ff7 -> ec54d5862
http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/030-1.0.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/030-1.0.0-rn.md b/_docs/rn/030-1.0.0-rn.md new file mode 100644 index 0000000..6ccea14 --- /dev/null +++ b/_docs/rn/030-1.0.0-rn.md @@ -0,0 +1,505 @@ +--- +title: "Apache Drill 1.0.0 Release Notes" +parent: "Release Notes" +--- +**Release date:** May 19, 2015 + +Today we're happy to announce the availability of Drill 1.0.0, providing additional enhancements and bug fixes. This release includes the following new features, enhancements, unresolved issues, and bug fixes: + +<h2> Sub-task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2150'>DRILL-2150</a>] - Create an abstraction for repeated value vectors. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2358'>DRILL-2358</a>] - Ensure DrillScanRel differentiates skip-all, scan-all & scan-some in a backward compatible fashion +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2893'>DRILL-2893</a>] - ScanBatch throws a NullPointerException instead of returning OUT_OF_MEMORY +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2895'>DRILL-2895</a>] - AbstractRecordBatch.buildSchema() should properly handle OUT_OF_MEMORY outcome +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2902'>DRILL-2902</a>] - Add support for context UDFs: user (and its synonyms session_user, system_user) and current_schema +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2905'>DRILL-2905</a>] - RootExec implementations should properly handle IterOutcome.OUT_OF_MEMORY +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2920'>DRILL-2920</a>] - properly handle OutOfMemoryException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2947'>DRILL-2947</a>] - AllocationHelper.allocateNew() doesn't have a consistent behavior when it can't allocate +</li> +</ul> + +<h2> Unresolved Issues +</h2> +<ul> + <li>[<a href='https://issues.apache.org/jira/browse/DRILL-1868'>DRILL-1868</a>] Aliases are not allowed in WHERE, HAVING and GROUP BY clauses. Drill should return an error when such aliases are encountered, but instead Drill returns an incorrect result. + </li> + <li>[<a href='https://issues.apache.org/jira/browse/DRILL-2015'>DRILL-2015</a>] Casting a numeric value that does not fit the data type of the value and causes overflow returns an incorrect result. + </li> + <li>[<a href='https://issues.apache.org/jira/browse/DRILL-2355'>DRILL-2355</a>] Drill output from the TRUNC function in some cases includes an extra .0 in the result. Drill binds TRUNC functions having two input parameters to the function holder. The output type of TRUNC functions is FLOAT8 when the input is FLOAT8, resulting in the extra .0. + </li> +</ul> +<h2> Bug Fixes +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-148'>DRILL-148</a>] - Remove sandbox directory from source control, it is no longer utilized +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-625'>DRILL-625</a>] - Server does not release resources even after client connection is closed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-708'>DRILL-708</a>] - TRUNC(n1) function returns a decimal instead of int +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-994'>DRILL-994</a>] - Reduce hbase timeout when it is not reachable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1245'>DRILL-1245</a>] - Drill should pinpoint to the "Problem Record" when it fails to parse a json file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1440'>DRILL-1440</a>] - Allow delimited files to have customizable quote characters +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1460'>DRILL-1460</a>] - JsonReader fails reading files with decimal numbers and integers in the same field +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1502'>DRILL-1502</a>] - Can't connect to mongo when requiring auth +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1503'>DRILL-1503</a>] - CTAS does not work against mongo plugin +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1542'>DRILL-1542</a>] - Early fragment termination causes non running intermediate fragments to error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1545'>DRILL-1545</a>] - Json files can only be read when they have a .json extension +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1727'>DRILL-1727</a>] - REPEATED_CONTAINS sometimes doesn't work +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1827'>DRILL-1827</a>] - Unit test framework reports expected and actual values backwards in unordered comparison +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1832'>DRILL-1832</a>] - Select * from json file failed with java.lang.IllegalArgumentException: null +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1866'>DRILL-1866</a>] - Tests that include limit sporadically fail when run as part of entire test suite on Linux +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1891'>DRILL-1891</a>] - Error message does not get propagated correctly when reading from JSON file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1973'>DRILL-1973</a>] - Tableau query causes parsing error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1980'>DRILL-1980</a>] - Create table with a Cast to interval day results in a file which cannot be read +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2005'>DRILL-2005</a>] - Create table fails to write out a parquet file created from hive- read works fine +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2006'>DRILL-2006</a>] - Implement text reader with advanced capabilities +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2036'>DRILL-2036</a>] - select * query returns wrong result when column name in json file changes case +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2073'>DRILL-2073</a>] - Filter on a field in a nested repeated type throws an exception +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2074'>DRILL-2074</a>] - Queries fail with OutOfMemory Exception when Hash Join & Agg are turned off +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2085'>DRILL-2085</a>] - Failed to propagate error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2091'>DRILL-2091</a>] - NPE in AbstractSqlAccessor +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2093'>DRILL-2093</a>] - Columns of time and timestamp data type are not stored correctly in json format on CTAS +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2140'>DRILL-2140</a>] - RPC Error querying JSON with empty nested maps +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2141'>DRILL-2141</a>] - Data type error in group by and order by for JSON +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2158'>DRILL-2158</a>] - Failure while attempting to start Drillbit in embedded mode. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2179'>DRILL-2179</a>] - better handle column called 'Timestamp' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2181'>DRILL-2181</a>] - Throw proper error message when flatten is used within an 'order by' or 'group by' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2201'>DRILL-2201</a>] - clear error message on join on complex type +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2208'>DRILL-2208</a>] - Error message must be updated when query contains operations on a flattened column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2219'>DRILL-2219</a>] - Concurrent modification exception in TopLevelAllocator if a child allocator is added during loop in close() +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2221'>DRILL-2221</a>] - CTAS (JSON) creates unreadable files when writing empty arrays +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2228'>DRILL-2228</a>] - Projecting '*' returns all nulls when we have flatten in a filter and order by +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2229'>DRILL-2229</a>] - SQL syntax errors should use SQLSyntaxErrorException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2232'>DRILL-2232</a>] - Flatten functionality not well defined when we use flatten in an order by without projecting it +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2264'>DRILL-2264</a>] - Incorrect data when we use aggregate functions with flatten +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2277'>DRILL-2277</a>] - COUNT(*) should return 0 instead of an empty result set when there are no records +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2281'>DRILL-2281</a>] - Drill never returns when we use aggregate functions after a join with an order by +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2292'>DRILL-2292</a>] - CTAS broken when we have repeated maps +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2301'>DRILL-2301</a>] - Query fails when multiple table aliases are provided for CTEs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2340'>DRILL-2340</a>] - count(*) fails with subquery not containing limit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2350'>DRILL-2350</a>] - Star query failed with exception on JSON data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2376'>DRILL-2376</a>] - UNION ALL on Aggregates with GROUP BY returns incomplete results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2404'>DRILL-2404</a>] - After we cancel a query, DRILL sometimes hangs for the next query +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2408'>DRILL-2408</a>] - CTAS should not create empty folders when underlying query returns no results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2411'>DRILL-2411</a>] - Scalar SUM/AVG over empty result set returns no rows instead of NULL +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2423'>DRILL-2423</a>] - DROP VIEW against non-existent views fails with ZK error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2425'>DRILL-2425</a>] - Wrong results when identifier change cases within the same data file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2437'>DRILL-2437</a>] - enhance exception injection to support session level injections +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2452'>DRILL-2452</a>] - ResultSet.getDouble should not throw an exception when the underlying type is a FLOAT +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2476'>DRILL-2476</a>] - Handle IterOutcome.STOP in buildSchema() +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2506'>DRILL-2506</a>] - IOOB with order by and limit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2511'>DRILL-2511</a>] - Assert with full outer join when one of the join predicates is of a required type (nullabe parquet) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2528'>DRILL-2528</a>] - Drill-JDBC-All Jar uses outdated classes +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2532'>DRILL-2532</a>] - Glob not always fired for DFS storage engine +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2533'>DRILL-2533</a>] - Metrics displayed in the profile UI should be rounded off instead of being truncated +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2535'>DRILL-2535</a>] - Column labels on drill profile page are incorrect +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2536'>DRILL-2536</a>] - Peak Mem column in the profile page displays 0 when value is less than 1MB +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2545'>DRILL-2545</a>] - Killing a JDBC client program does not kill the query on drillbits +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2548'>DRILL-2548</a>] - JDBC driver prints misleading SQL exception on getting record batches with no data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2552'>DRILL-2552</a>] - ZK disconnect to foreman node results in hung query on client +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2554'>DRILL-2554</a>] - Incorrect results for repeated values when using jdbc +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2569'>DRILL-2569</a>] - Minor fragmentId in Profile UI gets truncated to the last 2 digits +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2570'>DRILL-2570</a>] - Broken JDBC-All Jar packaging can cause missing XML classes +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2589'>DRILL-2589</a>] - Creating a view with duplicate column names should fail or give a warning to the user +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2598'>DRILL-2598</a>] - Order by with limit on complex type throw IllegalStateException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2617'>DRILL-2617</a>] - Errors in the execution stack will cause DeferredException to throw an IllegalStateException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2624'>DRILL-2624</a>] - org.apache.drill.common.StackTrace prints garbage for line numbers +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2662'>DRILL-2662</a>] - Exception type not being included when propagating exception message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2723'>DRILL-2723</a>] - Inaccurate row count estimate for text files results in BroadcastExchange +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2750'>DRILL-2750</a>] - Running 1 or more queries against Drillbits having insufficient DirectMem renders the Drillbits in an unusable state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2753'>DRILL-2753</a>] - Implicit cast fails when comparing a double column and a varchar literal +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2754'>DRILL-2754</a>] - Allocation bug in splitAndTransfer method causing some flatten queries to fail +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2755'>DRILL-2755</a>] - Use and handle InterruptedException during query processing +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2757'>DRILL-2757</a>] - Verify operators correctly handle low memory conditions and cancellations +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2776'>DRILL-2776</a>] - querying a .json file that contains a repeated type returns the wrong results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2778'>DRILL-2778</a>] - Killing the drillbit which is the foreman results in hung sqlline +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2780'>DRILL-2780</a>] - java.lang.IllegalStateException files open exceptions in drillbit.out +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2793'>DRILL-2793</a>] - Killing a non foreman node results in direct memory being held on +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2801'>DRILL-2801</a>] - ORDER BY produces extra records +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2806'>DRILL-2806</a>] - Querying data from compressed csv file returns nulls and unreadable data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2809'>DRILL-2809</a>] - Increase the default value of partitioner_sender_threads_factor +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2811'>DRILL-2811</a>] - Need option to specify Drillbit in the connection URI to connect to that specific Drillbit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2816'>DRILL-2816</a>] - system error does not display the original Exception message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2823'>DRILL-2823</a>] - Merge join should use implicit cast +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2824'>DRILL-2824</a>] - Function resolution should be deterministic +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2826'>DRILL-2826</a>] - Improve resilience to memory leaks and unclosed allocators +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2841'>DRILL-2841</a>] - Web UI very slow in a multi node machine +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2847'>DRILL-2847</a>] - DrillBufs from the RPC layer are being leaked +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2848'>DRILL-2848</a>] - Disable decimal data type by default +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2849'>DRILL-2849</a>] - Difference in query results over CSV file created by CTAS, compared to results over original CSV file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2865'>DRILL-2865</a>] - Drillbit runs out of memory on multiple consecutive CTAS +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2870'>DRILL-2870</a>] - Fix return type of aggregate functions to be nullable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2871'>DRILL-2871</a>] - Plan for TPC-H 20 changed with DRILL-1384 (or DRILL-2761) causing performance degradation +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2872'>DRILL-2872</a>] - Result from json file returns data from map type fields as "null" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2875'>DRILL-2875</a>] - IllegalStateException when querying the public yelp json dataset +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2878'>DRILL-2878</a>] - FragmentExecutor.closeOutResources() is not called if an exception happens in the Foreman before the fragment executor starts running +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2884'>DRILL-2884</a>] - Have cancel() cause "query canceled" rather than just "ResultSet closed" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2886'>DRILL-2886</a>] - JDBC driver doesn't detect lost connection +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2887'>DRILL-2887</a>] - Fix bad applications of JdbcTest.connect() +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2889'>DRILL-2889</a>] - Rename JdbcTest to JdbcTestBase. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2894'>DRILL-2894</a>] - FixedValueVectors shouldn't set it's data buffer to null when it fails to allocate it +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2897'>DRILL-2897</a>] - Update Limit 0 to avoid parallelization +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2904'>DRILL-2904</a>] - Fix wrong "before rows" message to "after rows" message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2907'>DRILL-2907</a>] - Drill performance degrades significantly over time - resource leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2914'>DRILL-2914</a>] - regression: Mondrian query534.q, drill give wrong result +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2921'>DRILL-2921</a>] - Query with a mix of distinct and not distinct scalar aggregates runs out of memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2927'>DRILL-2927</a>] - Pending query in resource queue starts after timeout +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2928'>DRILL-2928</a>] - C++ Client - io_service needs to be reset if it runs out of work +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2932'>DRILL-2932</a>] - Error text reported via System.out.println rather than thrown SQLException's message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2934'>DRILL-2934</a>] - Exception when distinct aggregate is compared to numeric literal with decimal point +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2936'>DRILL-2936</a>] - TPCH 4 and 18 SF100 hangs when hash agg is turned off +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2940'>DRILL-2940</a>] - Large allocations are not released until GC kicks in +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2942'>DRILL-2942</a>] - Allow Use of epoll RPC layer on Linux +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2943'>DRILL-2943</a>] - Drill parsing error during deserialization for an Order-By +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2944'>DRILL-2944</a>] - Switch to G1GC to reduce GC cpu overhead. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2951'>DRILL-2951</a>] - Tables are not visible when Drillbit is specified in the connection URL +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2953'>DRILL-2953</a>] - Group By + Order By query results are not ordered. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2957'>DRILL-2957</a>] - Netty Memory Manager doesn't move empty chunks between lists +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2959'>DRILL-2959</a>] - Compression codecs are leaking or slow to recapture memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2960'>DRILL-2960</a>] - Default hive storage plugin missing from fresh drill install +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2961'>DRILL-2961</a>] - Statement.setQueryTimeout() should throw a SQLException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2962'>DRILL-2962</a>] - Correlated subquery with scalar aggregate or scalar aggregate with expression throws and error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2963'>DRILL-2963</a>] - Exists with empty left batch causes IllegalStateException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2966'>DRILL-2966</a>] - HAVING clause with CASE statement with IN predicate causes assertion +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2968'>DRILL-2968</a>] - crash on parquet file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2971'>DRILL-2971</a>] - If Bit<>Bit connection is unexpectedly closed and we were already blocked on writing to socket, we'll stay forever in ResettableBarrier.await() +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2973'>DRILL-2973</a>] - Error messages not showing up in sqlline +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2976'>DRILL-2976</a>] - Set default of extended JSON support for output to false until issues are resolved +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2977'>DRILL-2977</a>] - In WorkManager, startFragmentPendingRemote() and addFragmentRunner() need to be permuted +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2978'>DRILL-2978</a>] - FragmentManager is not removed from the WorkEventBus if it's FragmentExecutor is cancelled before it starts running +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2979'>DRILL-2979</a>] - Storage HBase doesn't support customized hbase property zookeeper.znode.parent +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2989'>DRILL-2989</a>] - TPCDS Query corrupts Drillbits and causing subsequent unrelated queries to hang (and timeout) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2993'>DRILL-2993</a>] - SQLLine hangs when we cancel a query in the middle of displaying results +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2994'>DRILL-2994</a>] - Incorrect error message when disconnecting from server (using direct connection to drillbit) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2998'>DRILL-2998</a>] - Update C++ client to send/receive heartbeat message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3000'>DRILL-3000</a>] - I got JIRA report #3000. Now ... to use it for good or evil? +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3001'>DRILL-3001</a>] - Some functional tests fail when new text reader is disabled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3005'>DRILL-3005</a>] - Spurious Error messages when using PrintingResultsListener +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3006'>DRILL-3006</a>] - CTAS with interval data type creates invalid parquet file +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3007'>DRILL-3007</a>] - Update Drill configuration settings to avoid mmap threshold increases on Linux +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3009'>DRILL-3009</a>] - Reduce the IN list threshold to take advantage of Values operator +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3010'>DRILL-3010</a>] - Convert bad command error messages into UserExceptions in SqlHandlers +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3012'>DRILL-3012</a>] - Values Operator doesn't propagate operator id +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3017'>DRILL-3017</a>] - NPE when cleaning up some RecordReader implementations +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3018'>DRILL-3018</a>] - Queries with scalar aggregate and non equality (non correlated) fail to plan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3020'>DRILL-3020</a>] - Some exception message text not displayed in SQLLine, etc.; copy to thrown SQLException's message +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3022'>DRILL-3022</a>] - Ensure sequential shutdown of Drillbits +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3033'>DRILL-3033</a>] - Add memory leak fixes found so far in DRILL-1942 to 1.0 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3037'>DRILL-3037</a>] - Unable to query on hdfs after moving to 0.9.0 version +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3046'>DRILL-3046</a>] - Memory Leak after cancelling a query +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3047'>DRILL-3047</a>] - Command failed while establishing connection +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3048'>DRILL-3048</a>] - Disable assertions by default +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3049'>DRILL-3049</a>] - Increase sort spooling threshold +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3050'>DRILL-3050</a>] - Increase query context max memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3051'>DRILL-3051</a>] - Integer overflow in TimedRunnable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3052'>DRILL-3052</a>] - canceling a fragment executor before it starts running will cause the Foreman to wait indefinitely for a terminal message from that fragment +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3057'>DRILL-3057</a>] - A query that used to work before now fails in the optimizer +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3058'>DRILL-3058</a>] - RemoteConnection of RPC double closes the connection +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3061'>DRILL-3061</a>] - Fix memory leaks in TestDrillbitResilience +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3062'>DRILL-3062</a>] - regression: Mondrian query447.q - lots of rows missing in result set +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3063'>DRILL-3063</a>] - TestQueriesOnLargeFile leaks memory with 16M limit +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3065'>DRILL-3065</a>] - Memory Leak at ExternalSortBatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3066'>DRILL-3066</a>] - AtomicRemainder - Tried to close remainder, but it has already been closed. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3069'>DRILL-3069</a>] - Wrong result for aggregate query with filter on SF100 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3070'>DRILL-3070</a>] - Memory Leak when we run out of memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3071'>DRILL-3071</a>] - RecordBatchLoader#load leaks memory if an exception is thrown while loading the batch. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3072'>DRILL-3072</a>] - Profile UI fails to load when there is an empty json profile +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3074'>DRILL-3074</a>] - ReconnectingClient.waitAndRun can stuck in infinite loop if it fails to establish the connection +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3077'>DRILL-3077</a>] - sqlline's return code is 0 even when it force exits due to failed sql command +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3079'>DRILL-3079</a>] - Move JSON Execution Plan parsing to FragmentExecutor +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3080'>DRILL-3080</a>] - Error message is invalid if workload queue times out +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3081'>DRILL-3081</a>] - Fix situation where Drill reports null <--> null in connection error +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3085'>DRILL-3085</a>] - In ExternalSortBatch, Memory Leak in Runtime Generation Code +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3087'>DRILL-3087</a>] - Union All returns incorrect results. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3088'>DRILL-3088</a>] - IllegalStateException: Cleanup before finished. 0 out of 1 strams have finished +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3089'>DRILL-3089</a>] - Revert to 4 forked test and allow override from command line +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3092'>DRILL-3092</a>] - Memory leak when an allocation fails near the creation of a RecordBatchData object +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3093'>DRILL-3093</a>] - Leaking RawBatchBuffer +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3098'>DRILL-3098</a>] - Set Unix style "line.separator" for tests +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3099'>DRILL-3099</a>] - FileSelection's selectionRoot does not include the scheme and authority +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3100'>DRILL-3100</a>] - TestImpersonationDisabledWithMiniDFS fails on Windows +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3101'>DRILL-3101</a>] - Setting "slice_target" to 1 changes the order of the columns in a "select *" query with order by +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3103'>DRILL-3103</a>] - EncoderException: RpcEncoder must produce at least one message. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3105'>DRILL-3105</a>] - OutOfMemoryError: GC overhead limit exceeded +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3107'>DRILL-3107</a>] - Dynamic partition pruning fails on Windows (TestDirectoryExplorerUDFs) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3109'>DRILL-3109</a>] - Cancellation from sqlline is broken with the updated version +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3110'>DRILL-3110</a>] - OutOfMemoryError causes memory accounting leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3112'>DRILL-3112</a>] - Drill UI profile page shows exceptions where a long running query is submitted via the UI +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3114'>DRILL-3114</a>] - Sqlline throws exception at launch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3115'>DRILL-3115</a>] - SQLLine colors do not work well with CYGWIN +</li> +</ul> + +<h2> Improvement +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1662'>DRILL-1662</a>] - drillbit.sh stop should timeout +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2433'>DRILL-2433</a>] - Implicit cast between date and timestamp is missing in joins +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2508'>DRILL-2508</a>] - Add new column to sys.options table that exposes whether or not the current system value is the default +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2602'>DRILL-2602</a>] - Throw an error on schema change during streaming aggregation +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2697'>DRILL-2697</a>] - Pause injections should pause indefinitely until signalled +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2725'>DRILL-2725</a>] - Faster work assignment logic +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2772'>DRILL-2772</a>] - Display status of query when viewing the query's profile page +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2946'>DRILL-2946</a>] - Tableau 9.0 Desktop Enablement Document +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2955'>DRILL-2955</a>] - Enable color in sqlline for exceptions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2969'>DRILL-2969</a>] - Readers don't report number of records read in profile +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2981'>DRILL-2981</a>] - Add simplified activity log +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2982'>DRILL-2982</a>] - Tableau 9.0 Server Enablement Documentation +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2984'>DRILL-2984</a>] - UserException is logging through its parent class logger +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3027'>DRILL-3027</a>] - Add convenience methods to test builder for creating nested baseline values +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3053'>DRILL-3053</a>] - add unchecked exception injection site in ChildAllocator +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3084'>DRILL-3084</a>] - Add drill-* convenience methods for common cli startup commands +</li> +</ul> + +<h2> New Feature +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1573'>DRILL-1573</a>] - Add configuration to skip header row in text files +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2382'>DRILL-2382</a>] - enhance exception injection to support node-specific injections +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2383'>DRILL-2383</a>] - add exception and pause injections for testing drillbit stability +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2658'>DRILL-2658</a>] - Add ilike and regex substring functions +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2958'>DRILL-2958</a>] - Move Drill to alternative cost-based planner for Join planning +</li> +</ul> + +<h2> Task +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2316'>DRILL-2316</a>] - Docs Enhancement: Data Sources and File Formats, Basics Tutorial +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2336'>DRILL-2336</a>] - configuration storage plugin docs update +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2364'>DRILL-2364</a>] - JSON Data Model Reference 2nd draft +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2381'>DRILL-2381</a>] - write lexical structures section, JSON/Parquet reference fixes, updates to data types +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2397'>DRILL-2397</a>] - Enhance SQL Ref Data Types docs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2736'>DRILL-2736</a>] - review feedback on multitenancy and user auth +</li> +</ul> + +You can now [download Drill 1.0.0]({{ site.baseurl }}/download/). + + + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/040-0.9.0-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/040-0.9.0-rn.md b/_docs/rn/040-0.9.0-rn.md new file mode 100644 index 0000000..fcc5b8e --- /dev/null +++ b/_docs/rn/040-0.9.0-rn.md @@ -0,0 +1,29 @@ +--- +title: "Apache Drill 0.9.0 Release Notes" +parent: "Release Notes" +--- +It has been about a month since the release of Drill 0.8, which included [more than 240 improvements]({{ site.baseurl }}/blog/2016/08/30/drill-1.8-released/). Today we're happy to announce the availability of Drill 0.9, providing additional enhancements and bug fixes. In fact, this release includes [200 resolved JIRAs](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&version=12328813). Some of the noteworthy features in Drill 0.9 are: + +* **Authentication** ([DRILL-2674](https://issues.apache.org/jira/browse/DRILL-2674)). Drill now supports username/password authentication through the Java and C++ clients, as well as JDBC and ODBC. On the server-side, Drill leverages Linux PAM to securely validate the credentials. Users can choose to use an external user directory such as Active Directory or LDAP. To enable authentication, set the `security.user.auth` option in `drill-override.conf`. +* **Impersonation** ([DRILL-2363](https://issues.apache.org/jira/browse/DRILL-2363)). Queries now execute and access resources using the identity of the user who submitted the query. Previously, all queries would run as the same user (eg, `drill`). With the new impersonation capability, the query will fail if the submitting user does not have permission to read the requested file(s) in the distributed file system. To enable impersonation, set the `drill.exec.impersonation` option in `drill-override.conf`. +* **Ownership chaining**. Drill now allows views with different owners to be chained. This represents a very flexible access control solution. For example, an administrator with access to raw, sensitive data could create a view called `masked` which would expose only a subset of the data to other users. The administrator would enable users to read the `masked` view but not the raw data. Note that Drill provides an option `max_chained_user_hops` that specifies how many ownership changed are allowed in a chain, thereby providing administrators (or data stewards) more control over sharing of data. +* **MongoDB authentication** ([DRILL-1502](https://issues.apache.org/jira/browse/DRILL-1502)). Drill can now connect to a MongoDB cluster that requires authentication. +* **Extended JSON datatypes**. Our friends at MongoDB invented [extended JSON](http://docs.mongodb.org/manual/reference/mongodb-extended-json/) - a set of extensions to the JSON format for supporting additional data types. We decided to embrace extended JSON in Drill. For example, standard JSON doesn't have a time type, so a time could be represented as either a string or a number: `{"foo": "19:20:30.450Z"}` is just a string. With extended JSON, the `$time` qualifier can be used to specify that `foo` is a time `{"foo": {"$time": "19:20:30.450Z"}}`. + We now support a number of qualifiers including `$bin`, `$date`, `$time`, `$interval`, `$numberLong` and `$dateDay` (see [the example](https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources/vector/complex/extended.json)). We're in the process of adding some additional qualifiers to make sure that all of MongoDB's extended types are supported (this is particularly important when querying data in MongoDB). +* **Avro support** ([DRILL-1512](https://issues.apache.org/jira/browse/DRILL-1512)). Drill can now read Avro files. This patch was contributed by Andrew Selden at Elastic.co (formerly known as Elasticsearch). +* **Improved error messages** ([DRILL-2675](https://issues.apache.org/jira/browse/DRILL-2675) and more). It can be challenging for a complex distributed system like Drill to translate low-level internal conditions into actionable messages to the user. This release includes several enhancements that enable Drill to accomplish just that in a variety of cases. +* **Parquet and Calcite enhancements** ([DRILL-1410](https://issues.apache.org/jira/browse/DRILL-1410) and [DRILL-1384](https://issues.apache.org/jira/browse/DRILL-1384)). Drill isn't a traditional query engine - it's the first analytical query engine with a JSON data model. This has required us to enhance Parquet (our columnar format) and Calcite (our SQL parser). These enhancements have now been contributed back to those projects, and Drill is using the latest versions which include these enhancements. +* **New sys tables for memory and thread information** ([DRILL-2275](https://issues.apache.org/jira/browse/DRILL-2275)). Drill includes two new `sys` tables that provide real-time metrics about memory utilization and threads on each of the nodes in the cluster. You can run a simple `SELECT *` to see what information is available: + + ```sql + SELECT * FROM sys.drillmemory; + SELECT * FROM sys.drillbitthreads; + ``` + +* **Support for very wide tables** ([DRILL-2739](https://issues.apache.org/jira/browse/DRILL-2739)). Drill previously had some issues with tables that had more than 4095 colums. This limitation has been addressed. + +You can now [download Drill 0.9]({{ site.baseurl }}/download/). As always, you can check out the official [release notes]({{ site.baseurl }}/docs/release-notes/) for more details. + +We're gearing up for Drill's 1.0 release later this month. Stay tuned! + + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/071-0.5.0rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/071-0.5.0rn.md b/_docs/rn/071-0.5.0rn.md new file mode 100644 index 0000000..2dff978 --- /dev/null +++ b/_docs/rn/071-0.5.0rn.md @@ -0,0 +1,29 @@ +--- +title: "Apache Drill 0.5.0 Release Notes" +parent: "Release Notes" +--- + +Apache Drill 0.5.0, the first beta release for Drill, is designed to help +enthusiasts start working and experimenting with Drill. It also continues the +Drill monthly release cycle as we drive towards general availability. + +The 0.5.0 release is primarily a bug fix release, with [more than 100 JIRAs](h +ttps://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&versi +on=12324880) closed, but there are some notable features. For information +about the features, see the [Apache Drill Blog for the 0.5.0 +release](https://blogs.apache.org/drill/entry/apache_drill_beta_release_see). + +This release is available as [binary](http://www.apache.org/dyn/closer.cgi/inc +ubator/drill/drill-0.5.0-incubating/apache-drill-0.5.0-incubating.tar.gz) and +[source](http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-0.5.0-incu +bating/apache-drill-0.5.0-incubating-src.tar.gz) tarballs that are compiled +against Apache Hadoop. Drill has been tested against MapR, Cloudera, and +Hortonworks Hadoop distributions. There are associated build profiles and +JIRAs that can help you run Drill against your preferred distribution. + +Apache Drill 0.5.0 Key Notes and Limitations + + * The current release supports in memory and beyond memory execution. However, you must disable memory-intensive hash aggregate and hash join operations to leverage this functionality. + * While the Drill execution engine supports dynamic schema changes during the course of a query, some operators have yet to implement support for this behavior, such as Sort. Others operations, such as streaming aggregate, may have partial support that leads to unexpected results. + * There are known issues with joining text files without using an intervening view. See [DRILL-1401](https://issues.apache.org/jira/browse/DRILL-1401) for more information. + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/072-0.4.0rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/072-0.4.0rn.md b/_docs/rn/072-0.4.0rn.md new file mode 100644 index 0000000..d0dac30 --- /dev/null +++ b/_docs/rn/072-0.4.0rn.md @@ -0,0 +1,42 @@ +--- +title: "Apache Drill 0.4.0 Release Notes" +parent: "Release Notes" +--- +The 0.4.0 release is a developer preview release, designed to help enthusiasts +start to work with and experiment with Drill. It is the first Drill release +that provides distributed query execution. + +This release is built upon [more than 800 +JIRAs](https://issues.apache.org/jira/browse/DRILL/fixforversion/12324963/). +It is a pre-beta release on the way towards Drill. As a developer snapshot, +the release contains a large number of outstanding bugs that will make some +use cases challenging. Feel free to consult outstanding issues [targeted for +the 0.5.0 +release](https://issues.apache.org/jira/browse/DRILL/fixforversion/12324880/) +to see whether your use case is affected. + +To read more about this release and new features introduced, please view the +[0.4.0 announcement blog +entry](https://blogs.apache.org/drill/entry/announcing_apache_drill_0_4). + +The release is available as both [binary](http://www.apache.org/dyn/closer.cgi +/incubator/drill/drill-0.4.0-incubating/apache-drill-0.4.0-incubating.tar.gz) +and [source](http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-0.4.0- +incubating/apache-drill-0.4.0-incubating-src.tar.gz) tarballs. In both cases, +these are compiled against Apache Hadoop. Drill has also been tested against +MapR, Cloudera and Hortonworks Hadoop distributions and there are associated +build profiles or JIRAs that can help you run against your preferred +distribution. + +Some Key Notes & Limitations + + * The current release supports in memory and beyond memory execution. However, users must disable memory-intensive hash aggregate and hash join operations to leverage this functionality. + * In many cases,merge join operations return incorrect results. + * Use of a local filter in a join âonâ clause when using left, right or full outer joins may result in incorrect results. + * Because of known memory leaks and memory overrun issues you may need more memory and you may need to restart the system in some cases. + * Some types of complex expressions, especially those involving empty arrays may fail or return incorrect results. + * While the Drill execution engine supports dynamic schema changes during the course of a query, some operators have yet to implement support for this behavior (such as Sort). Others operations (such as streaming aggregate) may have partial support that leads to unexpected results. + * Protobuf, UDF, query plan interfaces and all interfaces are subject to change in incompatible ways. + * Multiplication of some types of DECIMAL(28+,*) will return incorrect result. + + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/073-alpha-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/073-alpha-rn.md b/_docs/rn/073-alpha-rn.md new file mode 100644 index 0000000..256300a --- /dev/null +++ b/_docs/rn/073-alpha-rn.md @@ -0,0 +1,39 @@ +--- +title: "Apache Drill M1 Release Notes (Apache Drill Alpha)" +parent: "Release Notes" +--- +### Milestone 1 Goals + +The first release of Apache Drill is designed as a technology preview for +people to better understand the architecture and vision. It is a functional +release tying to piece together the key components of a next generation MPP +query engine. It is designed to allow milestone 2 (M2) to focus on +architectural analysis and performance optimization. + + * Provide a new optimistic DAG execution engine for data analysis + * Build a new columnar shredded in-memory format and execution model that minimizes data serialization/deserialization costs and operator complexity + * Provide a model for runtime generated functions and relational operators that minimizes complexity and maximizes performance + * Support queries against columnar on disk format (Parquet) and JSON + * Support the most common set of standard SQL read-only phrases using ANSI standards. Includes: SELECT, FROM, WHERE, HAVING, ORDER, GROUP BY, IN, DISTINCT, LEFT JOIN, RIGHT JOIN, INNER JOIN + * Support schema-on-read querying and execution + * Build a set of columnar operation primitives including Merge Join, Sort, Streaming Aggregate, Filter, Selection Vector removal. + * Support unlimited level of subqueries and correlated subqueries + * Provided an extensible query-language agnostic JSON-base logical data flow syntax. + * Support complex data type manipulation via logical plan operations + +### Known Issues + +SQL Parsing +Because Apache Drill is built to support late-bound changing schemas while SQL +is statically typed, there are couple of special requirements that are +required writing SQL queries. These are limited to the current release and +will be correct in a future milestone release. + + * All tables are exposed as a single map field that contains + * Drill Alpha doesn't support implicit or explicit casts outside those required above. + * Drill Alpha does not include, there are currently a couple of differences for how to write a query in order to query against UDFs + * Drill currently supports simple and aggregate functions using scalar, repeated and + * Nested data support incomplete. Drill Alpha supports nested data structures as well repeated fields. + + + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/074-m1-alpha-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/074-m1-alpha-rn.md b/_docs/rn/074-m1-alpha-rn.md new file mode 100644 index 0000000..256300a --- /dev/null +++ b/_docs/rn/074-m1-alpha-rn.md @@ -0,0 +1,39 @@ +--- +title: "Apache Drill M1 Release Notes (Apache Drill Alpha)" +parent: "Release Notes" +--- +### Milestone 1 Goals + +The first release of Apache Drill is designed as a technology preview for +people to better understand the architecture and vision. It is a functional +release tying to piece together the key components of a next generation MPP +query engine. It is designed to allow milestone 2 (M2) to focus on +architectural analysis and performance optimization. + + * Provide a new optimistic DAG execution engine for data analysis + * Build a new columnar shredded in-memory format and execution model that minimizes data serialization/deserialization costs and operator complexity + * Provide a model for runtime generated functions and relational operators that minimizes complexity and maximizes performance + * Support queries against columnar on disk format (Parquet) and JSON + * Support the most common set of standard SQL read-only phrases using ANSI standards. Includes: SELECT, FROM, WHERE, HAVING, ORDER, GROUP BY, IN, DISTINCT, LEFT JOIN, RIGHT JOIN, INNER JOIN + * Support schema-on-read querying and execution + * Build a set of columnar operation primitives including Merge Join, Sort, Streaming Aggregate, Filter, Selection Vector removal. + * Support unlimited level of subqueries and correlated subqueries + * Provided an extensible query-language agnostic JSON-base logical data flow syntax. + * Support complex data type manipulation via logical plan operations + +### Known Issues + +SQL Parsing +Because Apache Drill is built to support late-bound changing schemas while SQL +is statically typed, there are couple of special requirements that are +required writing SQL queries. These are limited to the current release and +will be correct in a future milestone release. + + * All tables are exposed as a single map field that contains + * Drill Alpha doesn't support implicit or explicit casts outside those required above. + * Drill Alpha does not include, there are currently a couple of differences for how to write a query in order to query against UDFs + * Drill currently supports simple and aggregate functions using scalar, repeated and + * Nested data support incomplete. Drill Alpha supports nested data structures as well repeated fields. + + + http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/080-m1-alpha-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/080-m1-alpha-rn.md b/_docs/rn/080-m1-alpha-rn.md deleted file mode 100644 index 256300a..0000000 --- a/_docs/rn/080-m1-alpha-rn.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: "Apache Drill M1 Release Notes (Apache Drill Alpha)" -parent: "Release Notes" ---- -### Milestone 1 Goals - -The first release of Apache Drill is designed as a technology preview for -people to better understand the architecture and vision. It is a functional -release tying to piece together the key components of a next generation MPP -query engine. It is designed to allow milestone 2 (M2) to focus on -architectural analysis and performance optimization. - - * Provide a new optimistic DAG execution engine for data analysis - * Build a new columnar shredded in-memory format and execution model that minimizes data serialization/deserialization costs and operator complexity - * Provide a model for runtime generated functions and relational operators that minimizes complexity and maximizes performance - * Support queries against columnar on disk format (Parquet) and JSON - * Support the most common set of standard SQL read-only phrases using ANSI standards. Includes: SELECT, FROM, WHERE, HAVING, ORDER, GROUP BY, IN, DISTINCT, LEFT JOIN, RIGHT JOIN, INNER JOIN - * Support schema-on-read querying and execution - * Build a set of columnar operation primitives including Merge Join, Sort, Streaming Aggregate, Filter, Selection Vector removal. - * Support unlimited level of subqueries and correlated subqueries - * Provided an extensible query-language agnostic JSON-base logical data flow syntax. - * Support complex data type manipulation via logical plan operations - -### Known Issues - -SQL Parsing -Because Apache Drill is built to support late-bound changing schemas while SQL -is statically typed, there are couple of special requirements that are -required writing SQL queries. These are limited to the current release and -will be correct in a future milestone release. - - * All tables are exposed as a single map field that contains - * Drill Alpha doesn't support implicit or explicit casts outside those required above. - * Drill Alpha does not include, there are currently a couple of differences for how to write a query in order to query against UDFs - * Drill currently supports simple and aggregate functions using scalar, repeated and - * Nested data support incomplete. Drill Alpha supports nested data structures as well repeated fields. - - - http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/090-alpha-rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/090-alpha-rn.md b/_docs/rn/090-alpha-rn.md deleted file mode 100644 index 256300a..0000000 --- a/_docs/rn/090-alpha-rn.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: "Apache Drill M1 Release Notes (Apache Drill Alpha)" -parent: "Release Notes" ---- -### Milestone 1 Goals - -The first release of Apache Drill is designed as a technology preview for -people to better understand the architecture and vision. It is a functional -release tying to piece together the key components of a next generation MPP -query engine. It is designed to allow milestone 2 (M2) to focus on -architectural analysis and performance optimization. - - * Provide a new optimistic DAG execution engine for data analysis - * Build a new columnar shredded in-memory format and execution model that minimizes data serialization/deserialization costs and operator complexity - * Provide a model for runtime generated functions and relational operators that minimizes complexity and maximizes performance - * Support queries against columnar on disk format (Parquet) and JSON - * Support the most common set of standard SQL read-only phrases using ANSI standards. Includes: SELECT, FROM, WHERE, HAVING, ORDER, GROUP BY, IN, DISTINCT, LEFT JOIN, RIGHT JOIN, INNER JOIN - * Support schema-on-read querying and execution - * Build a set of columnar operation primitives including Merge Join, Sort, Streaming Aggregate, Filter, Selection Vector removal. - * Support unlimited level of subqueries and correlated subqueries - * Provided an extensible query-language agnostic JSON-base logical data flow syntax. - * Support complex data type manipulation via logical plan operations - -### Known Issues - -SQL Parsing -Because Apache Drill is built to support late-bound changing schemas while SQL -is statically typed, there are couple of special requirements that are -required writing SQL queries. These are limited to the current release and -will be correct in a future milestone release. - - * All tables are exposed as a single map field that contains - * Drill Alpha doesn't support implicit or explicit casts outside those required above. - * Drill Alpha does not include, there are currently a couple of differences for how to write a query in order to query against UDFs - * Drill currently supports simple and aggregate functions using scalar, repeated and - * Nested data support incomplete. Drill Alpha supports nested data structures as well repeated fields. - - - http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/100-0.5.0rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/100-0.5.0rn.md b/_docs/rn/100-0.5.0rn.md deleted file mode 100644 index 2dff978..0000000 --- a/_docs/rn/100-0.5.0rn.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -title: "Apache Drill 0.5.0 Release Notes" -parent: "Release Notes" ---- - -Apache Drill 0.5.0, the first beta release for Drill, is designed to help -enthusiasts start working and experimenting with Drill. It also continues the -Drill monthly release cycle as we drive towards general availability. - -The 0.5.0 release is primarily a bug fix release, with [more than 100 JIRAs](h -ttps://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&versi -on=12324880) closed, but there are some notable features. For information -about the features, see the [Apache Drill Blog for the 0.5.0 -release](https://blogs.apache.org/drill/entry/apache_drill_beta_release_see). - -This release is available as [binary](http://www.apache.org/dyn/closer.cgi/inc -ubator/drill/drill-0.5.0-incubating/apache-drill-0.5.0-incubating.tar.gz) and -[source](http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-0.5.0-incu -bating/apache-drill-0.5.0-incubating-src.tar.gz) tarballs that are compiled -against Apache Hadoop. Drill has been tested against MapR, Cloudera, and -Hortonworks Hadoop distributions. There are associated build profiles and -JIRAs that can help you run Drill against your preferred distribution. - -Apache Drill 0.5.0 Key Notes and Limitations - - * The current release supports in memory and beyond memory execution. However, you must disable memory-intensive hash aggregate and hash join operations to leverage this functionality. - * While the Drill execution engine supports dynamic schema changes during the course of a query, some operators have yet to implement support for this behavior, such as Sort. Others operations, such as streaming aggregate, may have partial support that leads to unexpected results. - * There are known issues with joining text files without using an intervening view. See [DRILL-1401](https://issues.apache.org/jira/browse/DRILL-1401) for more information. - http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/rn/110-0.4.0rn.md ---------------------------------------------------------------------- diff --git a/_docs/rn/110-0.4.0rn.md b/_docs/rn/110-0.4.0rn.md deleted file mode 100644 index d0dac30..0000000 --- a/_docs/rn/110-0.4.0rn.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: "Apache Drill 0.4.0 Release Notes" -parent: "Release Notes" ---- -The 0.4.0 release is a developer preview release, designed to help enthusiasts -start to work with and experiment with Drill. It is the first Drill release -that provides distributed query execution. - -This release is built upon [more than 800 -JIRAs](https://issues.apache.org/jira/browse/DRILL/fixforversion/12324963/). -It is a pre-beta release on the way towards Drill. As a developer snapshot, -the release contains a large number of outstanding bugs that will make some -use cases challenging. Feel free to consult outstanding issues [targeted for -the 0.5.0 -release](https://issues.apache.org/jira/browse/DRILL/fixforversion/12324880/) -to see whether your use case is affected. - -To read more about this release and new features introduced, please view the -[0.4.0 announcement blog -entry](https://blogs.apache.org/drill/entry/announcing_apache_drill_0_4). - -The release is available as both [binary](http://www.apache.org/dyn/closer.cgi -/incubator/drill/drill-0.4.0-incubating/apache-drill-0.4.0-incubating.tar.gz) -and [source](http://www.apache.org/dyn/closer.cgi/incubator/drill/drill-0.4.0- -incubating/apache-drill-0.4.0-incubating-src.tar.gz) tarballs. In both cases, -these are compiled against Apache Hadoop. Drill has also been tested against -MapR, Cloudera and Hortonworks Hadoop distributions and there are associated -build profiles or JIRAs that can help you run against your preferred -distribution. - -Some Key Notes & Limitations - - * The current release supports in memory and beyond memory execution. However, users must disable memory-intensive hash aggregate and hash join operations to leverage this functionality. - * In many cases,merge join operations return incorrect results. - * Use of a local filter in a join âonâ clause when using left, right or full outer joins may result in incorrect results. - * Because of known memory leaks and memory overrun issues you may need more memory and you may need to restart the system in some cases. - * Some types of complex expressions, especially those involving empty arrays may fail or return incorrect results. - * While the Drill execution engine supports dynamic schema changes during the course of a query, some operators have yet to implement support for this behavior (such as Sort). Others operations (such as streaming aggregate) may have partial support that leads to unexpected results. - * Protobuf, UDF, query plan interfaces and all interfaces are subject to change in incompatible ways. - * Multiplication of some types of DECIMAL(28+,*) will return incorrect result. - - http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/_docs/tutorials/020-drill-in-10-minutes.md ---------------------------------------------------------------------- diff --git a/_docs/tutorials/020-drill-in-10-minutes.md b/_docs/tutorials/020-drill-in-10-minutes.md index 258d85f..bf924ef 100644 --- a/_docs/tutorials/020-drill-in-10-minutes.md +++ b/_docs/tutorials/020-drill-in-10-minutes.md @@ -1,6 +1,6 @@ --- title: "Drill in 10 Minutes" -date: 2017-03-15 21:28:37 UTC +date: 2017-07-31 04:43:58 UTC parent: "Tutorials" description: Get started with Drill in 10 minutes or less. --- @@ -45,9 +45,9 @@ The output looks something like this: Complete the following steps to install Drill: 1. In a terminal window, change to the directory where you want to install Drill. -2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) with the command appropriate for your system: - * `wget http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz` - * `curl -o apache-drill-1.10.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz` +2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) with the command appropriate for your system: + * `wget http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz` + * `curl -o apache-drill-1.11.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz` 3. Copy the downloaded file to the directory where you want to install Drill. 4. Extract the contents of the Drill .tar.gz file. Use `sudo` if necessary: `tar -xvzf <.tar.gz file name>` @@ -75,7 +75,7 @@ Start Drill in embedded mode using the `drill-embedded` command: You can install Drill on Windows. First, set the JAVA_HOME environment variable, and then install Drill. Complete the following steps to install Drill: -1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.10.0/apache-drill-1.10.0.tar.gz). +1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.11.0/apache-drill-1.11.0.tar.gz). 2. Move the `apache-drill-<version>.tar.gz` file to a directory where you want to install Drill. 3. Unzip the `TAR.GZ` file using a third-party tool. If the tool you use does not unzip the TAR file as well as the `TAR.GZ` file, unzip the `apache-drill-<version>.tar` to extract the Drill software. The extraction process creates the installation directory named apache-drill-<version> containing the Drill software. http://git-wip-us.apache.org/repos/asf/drill/blob/ec54d586/blog/_posts/2017-07-31-drill-1.11-released.md ---------------------------------------------------------------------- diff --git a/blog/_posts/2017-07-31-drill-1.11-released.md b/blog/_posts/2017-07-31-drill-1.11-released.md new file mode 100644 index 0000000..c379ff4 --- /dev/null +++ b/blog/_posts/2017-07-31-drill-1.11-released.md @@ -0,0 +1,55 @@ +--- +layout: post +title: "Drill 1.11 Released" +code: drill-1.11-released +excerpt: Apache Drill 1.11's highlights are: Cryptography-related functions, spill to disk for the hash aggregate operator, Format plugin support for PCAP files, ability to change the HDFS block Size for Parquet files, ability to store query profiles in memory, configurable CTAS directory and file permissions option, support for network encryption, relative paths stored in the metadata file, and support for ANSI_QUOTES. +authors: ["bbevens"] +--- + +Today, we're happy to announce the availability of Drill 1.11.0. You can download it [here](https://drill.apache.org/download/). + +The release provides the following bug fixes and improvements: + +## Cryptography-Related Functions (DRILL-5634) +Drill provides the following cryptographic-related functions: + +- aes_encrypt() +- aes_decrypt() +- md5() +- sha() +- sha1() +- sha2() + +## Spill to Disk for Hash Aggregate Operator (DRILL-5457) +The Hash aggregate operator can spill data to disk in cases where the operation exceeds the set memory limit. + +## Format Plugin Support for PCAP Files (DRILL-5432) +A âpcapâ format plugin enables Drill to read PCAP files. You must add the âpcapâ format to the dfs storage plugin configuration, as shown: + + "pcap": { + "type": "pcap", + "extensions": [ + "pcap" + ] + } + +## Change the HDFS Block Size for Parquet Files (DRILL-5379) +The `store.parquet.writer.use_single_fs_block` option enables Drill to write a Parquet file as a single file system block without changing the file system default block size. + +## Store Query Profiles in Memory (DRILL-5481) +The `drill.exec.profiles.store.inmemory` option enables Drill to store query profiles in memory instead of writing the query profiles to disk. The `drill.exec.profiles.store.capacity` option sets the maximum number of most recent profiles to retain in memory. + +## Configurable CTAS Directory and File Permissions Option (DRILL-5391) +You can use the `exec.persistent_table.umask` configuration option, at the system or session level, to modify permissions on directories and files that result from running the CTAS command. By default, the option is set to 002, which sets the default directory permissions to 775 and default file permissions to -664. + +## Support for Network Encryption (DRILL-4335) +Drill can use SASL to support network encryption between the Drill client and drillbits, and also between drillbits. + +## Metadata file Stores Relative Paths (DRILL-3867) +Drill now stores the relative path in the metadata file (versus the absolute path), which enables you to move partitioned Parquet directories from one location in DFS to another without having to rebuild the Parquet metadata files; the metadata remains valid in the new location. + +## Support for ANSI_QUOTES (DRILL-3510) +In addition to back ticks, the SQL parser in Drill can use double quotes as identifier quotes. Use the `planner.parser.quoting_identifiers` configuration option, at the system or session level, to set the type of identifier quotes that the SQL parser in Drill uses. + +You can find a complete list of JIRAs resolved in the 1.11.0 release [here](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&version=12339943). +
