[ https://issues.apache.org/jira/browse/DRILL-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pritesh Maker reassigned DRILL-5891: ------------------------------------ Assignee: Boaz Ben-Zvi (was: Pritesh Maker) > When Drill runs out of memory for a HashAgg, it should tell the user how much > memory to allocate > ------------------------------------------------------------------------------------------------ > > Key: DRILL-5891 > URL: https://issues.apache.org/jira/browse/DRILL-5891 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Relational Operators > Affects Versions: 1.11.0 > Reporter: Robert Hou > Assignee: Boaz Ben-Zvi > > Query is: > select count(\*), max(`filename`) from dfs.`/drill/testdata/hash-agg/data1` > group by no_nulls_col, nulls_col; > Error is: > Error: RESOURCE ERROR: Not enough memory for internal partitioning and > fallback mechanism for HashAgg to use unbounded memory is disabled. Either > enable fallback config drill.exec.hashagg.fallback.enabled using Alter > session/system command or increase memory limit for Drillbit > From drillbit.log: > {noformat} > 2017-10-18 13:30:17,135 [26184629-3f4c-856a-e99e-97cdf0d29321:frag:1:8] TRACE > o.a.d.e.p.i.aggregate.HashAggregator - Incoming sizer: Actual batch schema & > sizes { > no_nulls_col(type: OPTIONAL VARCHAR, count: 1023, std size: 54, actual > size: 130, data size: 132892) > nulls_col(type: OPTIONAL VARCHAR, count: 1023, std size: 54, actual size: > 112, data size: 113673) > EXPR$0(type: REQUIRED BIGINT, count: 1023, std size: 8, actual size: 8, > data size: 8184) > EXPR$1(type: OPTIONAL VARCHAR, count: 1023, std size: 54, actual size: 18, > data size: 18414) > Records: 1023, Total size: 524288, Data size: 273163, Gross row width: 513, > Net row width: 268, Density: 53%} > 2017-10-18 13:30:17,135 [26184629-3f4c-856a-e99e-97cdf0d29321:frag:1:8] TRACE > o.a.d.e.p.i.aggregate.HashAggregator - 2nd phase. Estimated internal row > width: 166 Values row width: 66 batch size: 12779520 memory limit: 63161283 > max column width: 50 > 2017-10-18 13:30:17,139 [26184629-3f4c-856a-e99e-97cdf0d29321:frag:3:2] TRACE > o.a.d.e.p.impl.common.HashTable - HT allocated 4784128 for varchar of max > width 50 > 2017-10-18 13:30:17,139 [26184629-3f4c-856a-e99e-97cdf0d29321:frag:1:15] INFO > o.a.d.e.p.i.aggregate.HashAggregator - User Error Occurred: Not enough > memory for internal partitioning and fallback mechanism for HashAgg to use > unbounded memory is disabled. Either enable fallback config > drill.exec.hashagg.fallback.enabled using Alter session/system command or > increase memory limit for Drillbit > org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Not enough > memory for internal partitioning and fallback mechanism for HashAgg to use > unbounded memory is disabled. Either enable fallback config > drill.exec.hashagg.fallback.enabled using Alter session/system command or > increase memory limit for Drillbit > {noformat} > I would recommend that we add a log message with the "alter" command to > increase the amount of memory allocated, and how much memory to allocate. > Otherwise, the user may not know what to do. > I would also not suggest enabling "drill.exec.hashagg.fallback.enabled" > except as a last resort. -- This message was sent by Atlassian JIRA (v6.4.14#64029)