[ https://issues.apache.org/jira/browse/IMPALA-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107735#comment-17107735 ]
ASF subversion and git services commented on IMPALA-9712: --------------------------------------------------------- Commit ea4c7e4add8d97822323b4854b5d15f4a6a46878 in impala's branch refs/heads/master from Tim Armstrong [ https://gitbox.apache.org/repos/asf?p=impala.git;h=ea4c7e4 ] IMPALA-9712: fix mem consumption of operators above selective scan This change is motivated by excessive memory consumption of TPC-H Q19 which has a hash join and non-grouping aggregate above a selective scan. This change fixes RowBatch::AtCapacity() to account for the actual memory consumed by the RowBatch. It used total_allocated_bytes(), which does *not* account for unused space in the MemPool chunks. Instead it now uses total_reserved_bytes(), which includes the whole chunks. This reduced memory consumption of the agg from 60+MB to ~16MB. Testing: Ran TPC-H Q19 on parquet with mt_dop=8. Aggregation mem usage was reduced from 60+MB to ~16MB. Added a targeted regression test that ran out of memory before this fix. Ran exhaustive tests. Performance: No significant change on TPC-H single node run with scale factor 30. I also ran TPC-H nested scale factor 1 and there was no measureable change, but generation of the report failed for some reason. +----------+-----------------------+---------+------------+------------+----------------+ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +----------+-----------------------+---------+------------+------------+----------------+ | TPCH(30) | parquet / none / none | 6.18 | -1.31% | 4.54 | -1.03% | +----------+-----------------------+---------+------------+------------+----------------+ +----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------+----------------+---------+-------+ | Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval | +----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------+----------------+---------+-------+ | TPCH(30) | TPCH-Q10 | parquet / none / none | 8.08 | 7.98 | +1.19% | * 14.00% * | * 10.55% * | 20 | +0.63% | 0.60 | 0.30 | | TPCH(30) | TPCH-Q13 | parquet / none / none | 10.03 | 9.98 | +0.44% | 1.22% | 0.92% | 20 | +0.48% | 1.07 | 1.28 | | TPCH(30) | TPCH-Q12 | parquet / none / none | 3.20 | 3.19 | +0.34% | 2.02% | 2.68% | 20 | +0.08% | 0.48 | 0.45 | | TPCH(30) | TPCH-Q21 | parquet / none / none | 22.60 | 22.54 | +0.24% | 2.85% | 2.80% | 20 | +0.17% | 0.22 | 0.27 | | TPCH(30) | TPCH-Q9 | parquet / none / none | 13.80 | 13.77 | +0.17% | 1.99% | 1.70% | 20 | +0.06% | 0.10 | 0.30 | | TPCH(30) | TPCH-Q7 | parquet / none / none | 4.52 | 4.52 | -0.01% | 1.68% | 1.71% | 20 | +0.03% | 0.07 | -0.01 | | TPCH(30) | TPCH-Q8 | parquet / none / none | 5.40 | 5.43 | -0.52% | 1.60% | 1.92% | 20 | -0.23% | -0.98 | -0.94 | | TPCH(30) | TPCH-Q16 | parquet / none / none | 1.82 | 1.83 | -0.63% | 2.98% | 2.62% | 20 | -0.17% | -1.07 | -0.71 | | TPCH(30) | TPCH-Q15 | parquet / none / none | 3.80 | 3.82 | -0.46% | 1.31% | 1.21% | 20 | -0.41% | -1.30 | -1.17 | | TPCH(30) | TPCH-Q4 | parquet / none / none | 2.72 | 2.74 | -0.68% | 3.06% | 2.36% | 20 | -0.23% | -0.92 | -0.79 | | TPCH(30) | TPCH-Q6 | parquet / none / none | 1.63 | 1.64 | -0.98% | 3.28% | 2.66% | 20 | -0.23% | -1.47 | -1.04 | | TPCH(30) | TPCH-Q22 | parquet / none / none | 1.99 | 2.01 | -0.98% | 2.59% | 2.99% | 20 | -0.68% | -1.12 | -1.12 | | TPCH(30) | TPCH-Q1 | parquet / none / none | 5.22 | 5.27 | -0.96% | 1.93% | 2.25% | 20 | -0.93% | -1.39 | -1.45 | | TPCH(30) | TPCH-Q14 | parquet / none / none | 3.61 | 3.64 | -0.73% | 2.51% | 2.40% | 20 | -1.26% | -1.18 | -0.95 | | TPCH(30) | TPCH-Q5 | parquet / none / none | 4.52 | 4.58 | -1.23% | 1.41% | 1.39% | 20 | -1.15% | -2.99 | -2.79 | | TPCH(30) | TPCH-Q19 | parquet / none / none | 4.05 | 4.10 | -1.19% | 2.58% | 2.48% | 20 | -1.27% | -1.77 | -1.49 | | TPCH(30) | TPCH-Q3 | parquet / none / none | 3.96 | 4.01 | -1.32% | 1.82% | 1.82% | 20 | -1.31% | -2.09 | -2.30 | | TPCH(30) | TPCH-Q20 | parquet / none / none | 2.93 | 2.98 | -1.47% | 2.16% | 2.60% | 20 | -1.63% | -1.88 | -1.95 | | TPCH(30) | TPCH-Q2 | parquet / none / none | 2.78 | 2.83 | -1.70% | 2.29% | 2.19% | 20 | -1.77% | -1.97 | -2.41 | | TPCH(30) | TPCH-Q11 | parquet / none / none | 1.09 | 1.12 | -2.64% | 2.99% | 3.81% | 20 | -3.71% | -1.77 | -2.46 | | TPCH(30) | TPCH-Q17 | parquet / none / none | 15.07 | 15.73 | -4.20% | 3.24% | 2.26% | 20 | -5.00% | -3.84 | -4.89 | | TPCH(30) | TPCH-Q18 | parquet / none / none | 13.23 | 14.13 | -6.38% | * 11.87% * | * 26.04% * | 20 | -3.19% | -2.41 | -1.01 | +----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------+----------------+---------+-------+ Change-Id: I6debae562826621411bbcbb757978e227b395441 Reviewed-on: http://gerrit.cloudera.org:8080/15863 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> > Hit OOM on TPC-H Q19 > -------------------- > > Key: IMPALA-9712 > URL: https://issues.apache.org/jira/browse/IMPALA-9712 > Project: IMPALA > Issue Type: Bug > Components: Backend > Reporter: Tim Armstrong > Assignee: Tim Armstrong > Priority: Major > Labels: multithreading, resource-management > > We saw an out-of-memory on TPC-H 1TB with mt_dop=16. > {noformat} > Query Status: Memory limit exceeded: > ParquetColumnChunkReader::ReadDataPage() failed to allocate 65454 bytes for > decompressed data. > HDFS_SCAN_NODE (id=0) could not allocate 63.92 KB without exceeding limit. > Error occurred on backend impala-executor-000-1:22000 by fragment > 4b497d5e4237002a:94c50a5200000041 > Memory left in process limit: 71.74 GB > Memory left in query limit: 52.96 KB > Query(4b497d5e4237002a:94c50a5200000000): Limit=2.35 GB Reservation=1.56 GB > ReservationLimit=1.88 GB OtherMemory=801.69 MB Total=2.35 GB Peak=2.35 GB > Runtime Filter Bank: Reservation=1.00 MB ReservationLimit=1.00 MB > OtherMemory=0 Total=1.00 MB Peak=1.00 MB > Fragment 4b497d5e4237002a:94c50a52000000a7: Reservation=0 OtherMemory=0 > Total=0 Peak=51.91 MB > HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=48.78 MB > KrpcDataStreamSender (dst_id=4): Total=0 Peak=31.17 KB > CodeGen: Total=7.82 KB Peak=2.52 MB > Fragment 4b497d5e4237002a:94c50a52000000a8: Reservation=0 OtherMemory=0 > Total=0 Peak=40.15 MB > HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=36.84 MB > KrpcDataStreamSender (dst_id=4): Total=0 Peak=31.17 KB > Fragment 4b497d5e4237002a:94c50a52000000a9: Reservation=0 OtherMemory=0 > Total=0 Peak=31.79 MB > HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=28.78 MB > KrpcDataStreamSender (dst_id=4): Total=0 Peak=31.17 KB > Fragment 4b497d5e4237002a:94c50a52000000c3: Reservation=544.00 MB > OtherMemory=21.12 KB Total=544.02 MB Peak=544.36 MB > EXCHANGE_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=352.00 KB > KrpcDeferredRpcs: Total=0 Peak=0 > Hash Join Builder (join_node_id=2): Reservation=544.00 MB > OtherMemory=13.12 KB Total=544.01 MB Peak=544.02 MB > Hash Join Builder (join_node_id=2) Exprs: Total=13.12 KB Peak=13.12 KB > Fragment 4b497d5e4237002a:94c50a5200000041: Reservation=66.00 MB > OtherMemory=51.15 MB Total=117.15 MB Peak=117.29 MB > AGGREGATION_NODE (id=3): Total=43.83 MB Peak=49.86 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=6.04 MB Total=6.04 MB > Peak=8.29 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB > Total=67.26 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000042: Reservation=66.00 MB > OtherMemory=52.06 MB Total=118.06 MB Peak=118.06 MB > AGGREGATION_NODE (id=3): Total=50.63 MB Peak=50.63 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB > Peak=7.79 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB > Total=67.35 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > CodeGen: Total=4.06 KB Peak=778.50 KB > Fragment 4b497d5e4237002a:94c50a5200000043: Reservation=66.00 MB > OtherMemory=47.81 MB Total=113.81 MB Peak=116.01 MB > AGGREGATION_NODE (id=3): Total=45.01 MB Peak=48.67 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=1.48 MB Total=1.48 MB > Peak=7.79 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB > Total=67.35 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000044: Reservation=66.00 MB > OtherMemory=56.25 MB Total=122.25 MB Peak=122.25 MB > AGGREGATION_NODE (id=3): Total=54.81 MB Peak=54.81 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB > Peak=8.23 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB > Total=67.35 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000045: Reservation=66.00 MB > OtherMemory=49.68 MB Total=115.68 MB Peak=115.68 MB > AGGREGATION_NODE (id=3): Total=43.95 MB Peak=44.61 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=4.36 MB Total=4.36 MB > Peak=8.29 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB > Total=67.35 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000046: Reservation=66.00 MB > OtherMemory=48.62 MB Total=114.62 MB Peak=115.70 MB > AGGREGATION_NODE (id=3): Total=42.33 MB Peak=48.36 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=4.92 MB Total=4.92 MB > Peak=8.86 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB > Total=67.35 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000047: Reservation=66.00 MB > OtherMemory=49.53 MB Total=115.53 MB Peak=115.53 MB > AGGREGATION_NODE (id=3): Total=44.45 MB Peak=47.74 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=3.79 MB Total=3.79 MB > Peak=8.23 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB > Total=67.26 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000048: Reservation=66.00 MB > OtherMemory=54.38 MB Total=120.38 MB Peak=120.38 MB > AGGREGATION_NODE (id=3): Total=53.13 MB Peak=53.13 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB > Peak=8.29 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB > Total=67.17 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000049: Reservation=66.00 MB > OtherMemory=47.81 MB Total=113.81 MB Peak=115.08 MB > AGGREGATION_NODE (id=3): Total=45.45 MB Peak=47.73 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=1005.84 KB Total=1005.84 > KB Peak=8.23 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB > Total=67.35 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a520000004a: Reservation=66.00 MB > OtherMemory=50.08 MB Total=116.08 MB Peak=117.15 MB > AGGREGATION_NODE (id=3): Total=44.95 MB Peak=47.73 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=5.10 MB Total=5.10 MB > Peak=7.79 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=4.00 KB > Total=66.00 MB Peak=67.42 MB > Exprs: Total=4.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > CodeGen: Total=53.44 KB Peak=13.23 MB > Fragment 4b497d5e4237002a:94c50a520000004b: Reservation=66.00 MB > OtherMemory=47.81 MB Total=113.81 MB Peak=115.08 MB > AGGREGATION_NODE (id=3): Total=45.95 MB Peak=47.74 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=685.92 KB Total=685.92 > KB Peak=8.29 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB > Total=67.17 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a520000004c: Reservation=66.00 MB > OtherMemory=48.34 MB Total=114.34 MB Peak=117.29 MB > AGGREGATION_NODE (id=3): Total=43.83 MB Peak=49.86 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=3.23 MB Total=3.23 MB > Peak=8.29 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB > Total=67.26 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a520000004d: Reservation=66.00 MB > OtherMemory=45.07 MB Total=111.07 MB Peak=115.08 MB > AGGREGATION_NODE (id=3): Total=37.84 MB Peak=47.73 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=6.04 MB Total=6.04 MB > Peak=7.79 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB > Total=67.17 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a520000004e: Reservation=66.00 MB > OtherMemory=45.81 MB Total=111.81 MB Peak=113.58 MB > AGGREGATION_NODE (id=3): Total=43.95 MB Peak=46.24 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=685.92 KB Total=685.92 > KB Peak=7.73 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB > Total=67.17 MB Peak=67.33 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a520000004f: Reservation=66.00 MB > OtherMemory=54.59 MB Total=120.59 MB Peak=120.59 MB > AGGREGATION_NODE (id=3): Total=53.25 MB Peak=53.25 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB > Peak=8.29 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB > Total=67.26 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > Fragment 4b497d5e4237002a:94c50a5200000050: Reservation=66.00 MB > OtherMemory=51.47 MB Total=117.47 MB Peak=117.47 MB > AGGREGATION_NODE (id=3): Total=46.45 MB Peak=48.23 MB > NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB > Exprs: Total=4.00 KB Peak=4.00 KB > HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=3.73 MB Total=3.73 MB > Peak=8.86 MB > Exprs: Total=41.12 KB Peak=41.12 KB > HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB > Total=67.26 MB Peak=67.42 MB > Exprs: Total=8.00 KB Peak=8.00 KB > KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB > {noformat} > The aggregation node is accumulating a lot of memory and it's a non-grouping > agg that shouldn't accumulate much mem: > {noformat} > AGGREGATION_NODE (id=3): Total=43.83 MB Peak=49.86 MB > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org