GitHub user helifu opened a pull request:
https://github.com/apache/incubator-impala/pull/4
Branch 2.10.0
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-impala branch-2.10.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-impala/pull/4.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4
----
commit ebe8ddd451b3f14d3f778339978a76bcd14b2589
Author: Tim Armstrong <tarmstr...@cloudera.com>
Date: 2017-08-24T20:26:46Z
IMPALA-5830: SET_DENY_RESERVATION_PROBABILITY test
Add a targeted test that confirms that setting the query option will
force spilling.
Testing:
Ran test_spilling locally.
Change-Id: Ida6b55b2dee0779b1739af5d75943518ec40d6ce
Reviewed-on: http://gerrit.cloudera.org:8080/7809
Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com>
Tested-by: Impala Public Jenkins
commit 1faf89f047e7d78c3a1f3b518269a3ae21a4ddea
Author: Lars Volker <l...@cloudera.com>
Date: 2017-08-24T21:59:22Z
IMPALA-5840: Don't write page-level statistics in Parquet files.
Page level statistics in Parquet files are expected to be deprecated in
favor of page indexes (PARQUET-922). This change disables writing
statistics to pages. Impala is currently the only project writing them.
Neither Impala nor other projects make use of these right now and by not
writing them anymore we prevent others from depending on soon-to-be
deprecated fields.
Change-Id: I1b05131320370171d76e93a46b04880a7f9b6d84
Reviewed-on: http://gerrit.cloudera.org:8080/7817
Reviewed-by: Lars Volker <l...@cloudera.com>
Tested-by: Impala Public Jenkins
commit 99fe9b3fd602180d63cbfe73ac2c9171c31ae455
Author: Matthew Jacobs <m...@cloudera.com>
Date: 2017-08-25T20:41:05Z
IMPALA-5838: Improve errors on AC buffer mem rejection
The error message returned when a query is rejected due to
insufficient buffer memory is misleading. It recommended a
mem_limit which would be high enough, but changing the
mem_limit may result in changing the plan, which may result
in further changes to the buffer memory requirement.
In particular, this can happen when the planner compares the
expected hash table size to the mem_limit, and decides to
choose a partitioned join over a broadcast join.
While we might consider other code changes to improve this,
for now lets just be clear in the error message.
Testing:
* Adds tests that verify the expected behavior with the new
error message.
Change-Id: I3dc3517195508d86078a8a4b537ae7d2f52fbcb7
Reviewed-on: http://gerrit.cloudera.org:8080/7834
Reviewed-by: Matthew Jacobs <m...@cloudera.com>
Tested-by: Impala Public Jenkins
commit 73cb9b8b0f6020fb90acf4fa12a00753a3120058
Author: Tim Armstrong <tarmstr...@cloudera.com>
Date: 2017-08-28T22:03:38Z
IMPALA-5852: improve MINIMUM_RESERVATION_UNAVAILABLE error
Augment the error message to mention that oversubscription is likely the
problem and hint at solutions.
Change-Id: I8e367e1b0cb08e11fdd0546880df23b785e3b7c9
Reviewed-on: http://gerrit.cloudera.org:8080/7861
Reviewed-by: Dan Hecht <dhe...@cloudera.com>
Tested-by: Impala Public Jenkins
commit 2912a0f9d9b32caf586b9383c7e027af3fe4c5c4
Author: Tim Armstrong <tarmstr...@cloudera.com>
Date: 2017-08-29T22:29:50Z
IMPALA-5857: avoid invalid free of hedged read metrics
The libHdfs API documents that the output parameter is unchanged on
error, therefore we do not need to attempt to free it on error.
Testing:
The bug only reproduced under stress. I don't know how to trigger this
error path yet.
Change-Id: I93baf3b672429c0283d7f031ff302aca31e05be4
Reviewed-on: http://gerrit.cloudera.org:8080/7885
Reviewed-by: Sailesh Mukil <sail...@cloudera.com>
Reviewed-by: Matthew Jacobs <m...@cloudera.com>
Tested-by: Impala Public Jenkins
commit a58394be7c7998a5dfea53d8a3dbf8beb3370a48
Author: Alex Behm <alex.b...@cloudera.com>
Date: 2017-08-29T02:01:39Z
IMPALA-5850: Cast sender partition exprs under unions.
For a series of partitioned joins within the same fragment we must
cast the sender partition exprs of exchanges to compatible types.
Otherwise, the hashes generated for identical partition values may
differ among senders leading to wrong results.
The bug was that this casting process was only performed for
fragments that are hash-partitioned. However, a union produces a
fragment with RANDOM partition, but the union could still contain
partitioned joins whose senders need to be cast appropriately. The
fix is to add casts regardless of the fragment's data partition.
Testing:
- Core/hdfs run passed
- Added a new regresion test
Change-Id: I0aa801bcad8c2324d848349c7967d949224404e0
Reviewed-on: http://gerrit.cloudera.org:8080/7884
Reviewed-by: Alex Behm <alex.b...@cloudera.com>
Tested-by: Impala Public Jenkins
commit 23d79462da5d0108709e8b1399c97606f4ebdf92
Author: Tim Armstrong <tarmstr...@cloudera.com>
Date: 2017-08-29T15:29:44Z
IMPALA-5855: reserve enough memory for preaggs
The calculation in the planner failed to account for the behaviour of
Suballocator, which needs to obtain at least one buffer to allocate any
memory.
Testing:
Added a regression test that caused a crash before the fix.
Updated planner tests.
Was able to run local stress test binary search to completion (it
previously crashed).
Change-Id: I870fbe2f1da01c6123d3716a1198376f9a454c3b
Reviewed-on: http://gerrit.cloudera.org:8080/7871
Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com>
Tested-by: Impala Public Jenkins
----
---
