Hello Michael Brown, Nithya Janarthanan, David Knupp, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11546
to look at the new patch set (#6).
Change subject: IMPALA-7647: Add HS2/Impyla dimension to TestQueries
......................................................................
IMPALA-7647: Add HS2/Impyla dimension to TestQueries
I used some ideas from Alex Leblang's abandoned patch:
https://gerrit.cloudera.org/#/c/137/ in order to run .test files through
HS2. The advantage of using Impyla is that much of the code will be
reusable for any Python client implementing the standard Python dbapi
and does not require us implementing yet another thrift client.
This gives us better coverage of non-trivial result sets from HS2,
including handling of NULLs, error logs and more interesting result
sets than the basic HS2 tests.
I added HS2 coverage to TestQueries, which has a reasonable variety of
queries and covers the data types in alltypes. I also added
TestDecimalQueries, TestStringQuery and TestCharFormats to get coverage
of DECIMAL, CHAR and VARCHAR that aren't in alltypes. Coverage of
results sets with NULLs was limited so I added a couple of queries.
Places where results differ from Beeswax:
* Impyla is a Python dbapi client so must convert timestamps into python
datetime
objects, which only have microsecond precision. Therefore result
timestamps within nanosecond precision are truncated.
* The HS2 interface reports the NULL type as BOOLEAN as a workaround for
IMPALA-914.
* The Beeswax interface reported VARCHAR as STRING, but HS2 reports
VARCHAR.
I dealt with different results by adding additional result sections so
that the expected differences between the clients/protocols were
explicit.
Limitations:
* Not all of the same methods are implemented as for beeswax, so some
tests that have more complicated interactions with the client will not
work with HS2 yet.
* We don't have a way to get the affected row count for inserts.
I also simplified the ImpalaConnection API by removing some unnecessary
methods and moved some generic methods to the base class.
Testing:
* Confirmed that it detected IMPALA-7588 by re-applying the buggy patch.
* Ran exhaustive and CentOS6 tests.
Change-Id: I9908ccc4d3df50365be8043b883cacafca52661e
---
M bin/start-impala-cluster.py
M testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test
M testdata/workloads/functional-query/queries/QueryTest/chars-formats.test
M testdata/workloads/functional-query/queries/QueryTest/chars.test
M testdata/workloads/functional-query/queries/QueryTest/data-source-tables.test
M testdata/workloads/functional-query/queries/QueryTest/decimal.test
M testdata/workloads/functional-query/queries/QueryTest/inline-view-limit.test
M testdata/workloads/functional-query/queries/QueryTest/inline-view.test
M testdata/workloads/functional-query/queries/QueryTest/limit.test
M testdata/workloads/functional-query/queries/QueryTest/top-n.test
M testdata/workloads/functional-query/queries/QueryTest/union.test
M tests/beeswax/impala_beeswax.py
M tests/common/impala_connection.py
M tests/common/impala_service.py
M tests/common/impala_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_result_verifier.py
M tests/conftest.py
M tests/custom_cluster/test_coordinators.py
M tests/custom_cluster/test_session_expiration.py
M tests/metadata/test_views_compatibility.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_chars.py
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_insert.py
M tests/query_test/test_queries.py
M tests/query_test/test_scanners.py
M tests/util/test_file_parser.py
28 files changed, 763 insertions(+), 149 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/11546/6
--
To view, visit http://gerrit.cloudera.org:8080/11546
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9908ccc4d3df50365be8043b883cacafca52661e
Gerrit-Change-Number: 11546
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: David Knupp <dkn...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Nithya Janarthanan <njanartha...@cloudera.com>
