Amogh Margoor has posted comments on this change. ( http://gerrit.cloudera.org:8080/17592 )
Change subject: IMPALA-7635: Reducing HashTable size by packing it's buckets efficiently. ...................................................................... Patch Set 5: > (1 comment) > > I grabbed this change and was doing some local testing, and I ran > into a scenario that crashes Impala. > > Here's what I did: > 1. I loaded a larger TPC-H dataset in parquet (takes quite a while > and will use up some disk space): > bin/load-data.py -w tpch -s 42 > --table_formats=text/none/none,parquet/none/none > 2. Ran the following multiple times (it is intermittent): > use tpch42_parquet; > SELECT l_orderkey, > count(*) AS cnt > FROM lineitem > GROUP BY l_orderkey > HAVING count(*) > 9999999999999; > > That intermittently crashes with this stack: > C [impalad+0x27e8546] long impala::HashTable::Probe<true, > false>(impala::HashTable::Bucket*, long, impala::HashTableCtx*, > unsigned int, bool*, impala::HashTable::BucketData*)+0x28c > C [impalad+0x27e2141] impala::HashTable::ResizeBuckets(long, > impala::HashTableCtx*, bool*)+0x7d1 > C [impalad+0x27e194d] impala::HashTable::CheckAndResize(unsigned > long, impala::HashTableCtx*, bool*)+0xcb > C [impalad+0x27c8a48] > impala::GroupingAggregator::CheckAndResizeHashPartitions(bool, > int, impala::HashTableCtx*)+0x176 > > This may reproduce on smaller datasets. Thanks for checking this out. I will check it and revert back. -- To view, visit http://gerrit.cloudera.org:8080/17592 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I72912ae9353b0d567a976ca712d2d193e035df9b Gerrit-Change-Number: 17592 Gerrit-PatchSet: 5 Gerrit-Owner: Amogh Margoor <amarg...@gmail.com> Gerrit-Reviewer: Amogh Margoor <amarg...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Thu, 08 Jul 2021 17:06:11 +0000 Gerrit-HasComments: No
