Michael Brown created IMPALA-5388:
-------------------------------------
Summary: wrong results under stress with secure cluster
Key: IMPALA-5388
URL: https://issues.apache.org/jira/browse/IMPALA-5388
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 2.9.0
Reporter: Michael Brown
Priority: Blocker
The stress test under a secure (Kerberos + SSL) cluster finds that some queries
are producing wrong results. I haven't yet been able to pin down why, but I'm
going ahead and filing this bug to include what I have.
Queries the stress test has reported as producing wrong results:
tpch-q3, tpcds-q34, tpch-q12, tpch-q7
In the case of tpch-q3, I managed to get a complete profile of a correct and
incorrect run of the query. See attached.
I got as far as noticing that in the "wrong results" case, fewer rows are
scanned:
*Results correct*
{noformat}
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak
Mem Est. Peak Mem Detail
----------------------------------------------------------------------------------------------------------------------------------
13:MERGING-EXCHANGE 1 308.431us 308.431us 10 10
0 0 UNPARTITIONED
06:TOP-N 8 15.787ms 18.612ms 80 10 12.00
KB 580.00 B
12:AGGREGATE 8 506.351ms 692.593ms 1.13M 1.73M 14.24
MB 105.17 MB FINALIZE
11:EXCHANGE 8 75.520ms 138.635ms 1.13M 1.73M
0 0 HASH(l_orderkey,o_orderdate...
05:AGGREGATE 8 389.129ms 650.835ms 1.13M 1.73M 13.70
MB 105.17 MB STREAMING
04:HASH JOIN 8 1s901ms 2s576ms 2.99M 1.73M 153.17
MB 12.98 MB INNER JOIN, PARTITIONED
|--10:EXCHANGE 8 235.256ms 552.595ms 3.00M 3.00M
0 0 HASH(c_custkey)
| 00:SCAN HDFS 5 323.828ms 621.551ms 3.00M 3.00M 29.30
MB 176.00 MB tpch_100_parquet.customer
09:EXCHANGE 8 728.297ms 758.348ms 14.57M 6.00M
0 0 HASH(o_custkey)
03:HASH JOIN 8 24s679ms 29s349ms 14.57M 6.00M 777.11
MB 98.35 MB INNER JOIN, PARTITIONED
|--08:EXCHANGE 8 5s521ms 8s310ms 70.97M 15.00M
0 0 HASH(o_orderkey)
| 01:SCAN HDFS 8 3s626ms 7s399ms 70.97M 15.00M 80.88
MB 352.00 MB tpch_100_parquet.orders
07:EXCHANGE 8 14s268ms 15s285ms 323.49M 60.00M
0 0 HASH(l_orderkey)
02:SCAN HDFS 8 11s632ms 17s863ms 323.49M 60.00M 78.65
MB 352.00 MB tpch_100_parquet.lineitem
{noformat}
*Results incorrect:*
{noformat}
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak
Mem Est. Peak Mem Detail
----------------------------------------------------------------------------------------------------------------------------------
13:MERGING-EXCHANGE 1 304.504us 304.504us 10 10
0 0 UNPARTITIONED
06:TOP-N 8 19.261ms 29.196ms 80 10 12.00
KB 580.00 B
12:AGGREGATE 8 305.220ms 449.997ms 1.13M 1.73M 14.24
MB 105.17 MB FINALIZE
11:EXCHANGE 8 66.207ms 96.284ms 1.13M 1.73M
0 0 HASH(l_orderkey,o_orderdate...
05:AGGREGATE 8 516.324ms 653.086ms 1.13M 1.73M 13.58
MB 105.17 MB STREAMING
04:HASH JOIN 8 1s217ms 1s461ms 2.99M 1.73M 153.17
MB 12.98 MB INNER JOIN, PARTITIONED
|--10:EXCHANGE 8 150.899ms 213.929ms 3.00M 3.00M
0 0 HASH(c_custkey)
| 00:SCAN HDFS 5 937.452ms 1s753ms 3.00M 3.00M 29.09
MB 176.00 MB tpch_100_parquet.customer
09:EXCHANGE 8 563.317ms 581.895ms 11.04M 6.00M
0 0 HASH(o_custkey)
03:HASH JOIN 8 24s420ms 28s126ms 11.04M 6.00M 649.11
MB 98.35 MB INNER JOIN, PARTITIONED
|--08:EXCHANGE 8 2s733ms 2s967ms 53.80M 15.00M
0 0 HASH(o_orderkey)
| 01:SCAN HDFS 8 30s937ms 47s728ms 53.80M 15.00M 85.11
MB 352.00 MB tpch_100_parquet.orders
07:EXCHANGE 8 13s816ms 14s173ms 323.49M 60.00M
0 0 HASH(l_orderkey)
02:SCAN HDFS 8 10s053ms 12s288ms 323.48M 60.00M 78.57
MB 352.00 MB tpch_100_parquet.lineitem
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
