[jira] [Commented] (IMPALA-7727) compute stats child query errors no longer propagate to parent query

2018-10-18 Thread Michael Brown (JIRA) 


[ 
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655993#comment-16655993
 ] 

Michael Brown commented on IMPALA-7727:
---

We've been hit by bugs like this before, like IMPALA-4981. Suggestion is an 
end-to-end test that does the following:

1. Run compute stats child queries directly (as real queries) with a low memory 
limit and make sure the error string has a reasonable match.
2. Run compute stats with the same low memory limit and assert the same error 
message is propagated.

> compute stats child query errors no longer propagate to parent query
> 
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
>  Labels: regression, stress
>
> [~bharathv] since you have been dealing with stats, please take a look. 
> Otherwise feel free to reassign. This bug prevents the stress test from 
> running with compute stats statements. It triggers in non-stressful 
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but 
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
>   DEBUG MODE WARNING: Query profile created while running a DEBUG build of 
> Impala. Use RELEASE builds to measure query performance.
>   Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory 
> reservation is greater than memory available to the query for buffer 
> reservations. Memory reservation needed given the current plan: 128.00 KB. 
> Adjust either the mem_limit or the pool config (max-query-mem-limit, 
> min-query-mem-limit) for the query to allow the query memory limit to be at 
> least 32.12 MB. Note that changing the mem_limit may also change the plan. 
> See the query profile for more information about the per-node memory 
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build 
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User: 
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner): 
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan: 
> 
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B 
> thread-reservation=1
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> |  output: count:merge(*)
> |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB 
> thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB 
> thread-reservation=4
> 01:AGGREGATE
> |  output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB 
> thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
>  table: rows=15 size=12.34MB
>  columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
> 
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0) 
> mikeb-ub162:22001(128.00 KB)
> Re

[jira] [Commented] (IMPALA-7727) compute stats child query errors no longer propagate to parent query

2018-10-18 Thread bharath v (JIRA) 


[ 
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656134#comment-16656134
 ] 

bharath v commented on IMPALA-7727:
---

Thanks [~mikesbrown]. Will look into this.

> compute stats child query errors no longer propagate to parent query
> 
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
>  Labels: regression, stress
>
> [~bharathv] since you have been dealing with stats, please take a look. 
> Otherwise feel free to reassign. This bug prevents the stress test from 
> running with compute stats statements. It triggers in non-stressful 
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but 
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
>   DEBUG MODE WARNING: Query profile created while running a DEBUG build of 
> Impala. Use RELEASE builds to measure query performance.
>   Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory 
> reservation is greater than memory available to the query for buffer 
> reservations. Memory reservation needed given the current plan: 128.00 KB. 
> Adjust either the mem_limit or the pool config (max-query-mem-limit, 
> min-query-mem-limit) for the query to allow the query memory limit to be at 
> least 32.12 MB. Note that changing the mem_limit may also change the plan. 
> See the query profile for more information about the per-node memory 
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build 
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User: 
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner): 
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan: 
> 
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B 
> thread-reservation=1
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> |  output: count:merge(*)
> |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB 
> thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB 
> thread-reservation=4
> 01:AGGREGATE
> |  output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB 
> thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
>  table: rows=15 size=12.34MB
>  columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
> 
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0) 
> mikeb-ub162:22001(128.00 KB)
> Request Pool: default-pool
> Admission result: Rejected
> Query Compilation: 126.903ms
>- Metadata of all 1 tables cached: 5.484ms (5.484ms)
>- Analysis finished: 16.104ms (10.619ms)
>- Value transfer graph computed: 32.646ms (16.542ms)
>- Single node plan created: 61.289ms (28.642ms)
>

[jira] [Commented] (IMPALA-7727) compute stats child query errors no longer propagate to parent query

2018-10-19 Thread Michael Brown (JIRA) 


[ 
https://issues.apache.org/jira/browse/IMPALA-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657183#comment-16657183
 ] 

Michael Brown commented on IMPALA-7727:
---

I played around with this a little bit more and the *Query Status* is what 
isn't propagating in 3.1 vs. 2.12. I'll attach profiles to this Jira.

> compute stats child query errors no longer propagate to parent query
> 
>
> Key: IMPALA-7727
> URL: https://issues.apache.org/jira/browse/IMPALA-7727
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: bharath v
>Priority: Blocker
>  Labels: regression, stress
>
> [~bharathv] since you have been dealing with stats, please take a look. 
> Otherwise feel free to reassign. This bug prevents the stress test from 
> running with compute stats statements. It triggers in non-stressful 
> conditions, too.
> {noformat}
> $ impala-shell.sh -d tpch_parquet
> [localhost:21000] tpch_parquet> set mem_limit=24m;
> MEM_LIMIT set to 24m
> [localhost:21000] tpch_parquet> compute stats customer;
> Query: compute stats customer
> WARNINGS: Cancelled
> [localhost:21000] tpch_parquet>
> {noformat}
> The problem is that the child query didn't have enough memory to run, but 
> this error didn't propagate up.
> {noformat}
> Query (id=384d37fb2826a962:f4b10357):
>   DEBUG MODE WARNING: Query profile created while running a DEBUG build of 
> Impala. Use RELEASE builds to measure query performance.
>   Summary:
> Session ID: d343e1026d497bb0:7e87b342c73c108d
> Session Type: BEESWAX
> Start Time: 2018-10-18 15:16:34.036363000
> End Time: 2018-10-18 15:16:34.177711000
> Query Type: QUERY
> Query State: EXCEPTION
> Query Status: Rejected query from pool default-pool: minimum memory 
> reservation is greater than memory available to the query for buffer 
> reservations. Memory reservation needed given the current plan: 128.00 KB. 
> Adjust either the mem_limit or the pool config (max-query-mem-limit, 
> min-query-mem-limit) for the query to allow the query memory limit to be at 
> least 32.12 MB. Note that changing the mem_limit may also change the plan. 
> See the query profile for more information about the per-node memory 
> requirements.
> Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build 
> 9f5c5e6df03824cba292fe5a619153462c11669c)
> User: mikeb
> Connected User: mikeb
> Delegated User: 
> Network Address: :::127.0.0.1:46458
> Default Db: tpch_parquet
> Sql Statement: SELECT COUNT(*) FROM customer
> Coordinator: mikeb-ub162:22000
> Query Options (set by configuration): MEM_LIMIT=25165824,MT_DOP=4
> Query Options (set by configuration and planner): 
> MEM_LIMIT=25165824,NUM_SCANNER_THREADS=1,MT_DOP=4
> Plan: 
> 
> Max Per-Host Resource Reservation: Memory=512.00KB Threads=5
> Per-Host Resource Estimates: Memory=146MB
> F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B 
> thread-reservation=1
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 03:AGGREGATE [FINALIZE]
> |  output: count:merge(*)
> |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB 
> thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 03(GETNEXT), 01(OPEN)
> |
> 02:EXCHANGE [UNPARTITIONED]
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 01(GETNEXT)
> |
> F00:PLAN FRAGMENT [RANDOM] hosts=1 instances=4
> Per-Host Resources: mem-estimate=136.00MB mem-reservation=512.00KB 
> thread-reservation=4
> 01:AGGREGATE
> |  output: sum_init_zero(tpch_parquet.customer.parquet-stats: num_rows)
> |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB 
> thread-reservation=0
> |  tuple-ids=1 row-size=8B cardinality=1
> |  in pipelines: 01(GETNEXT), 00(OPEN)
> |
> 00:SCAN HDFS [tpch_parquet.customer, RANDOM]
>partitions=1/1 files=1 size=12.34MB
>stored statistics:
>  table: rows=15 size=12.34MB
>  columns: all
>extrapolated-rows=disabled max-scan-range-rows=15
>mem-estimate=24.00MB mem-reservation=128.00KB thread-reservation=0
>tuple-ids=0 row-size=8B cardinality=15
>in pipelines: 00(GETNEXT)
> 
> Estimated Per-Host Mem: 153092096
> Per Host Min Memory Reservation: mikeb-ub162:22000(0) 
> mikeb-ub162:22001(128.00 KB)
> Request Pool: default-pool
> Admission result: Rejected
> Query Compilation: 126.903ms
>- Metadata of all 1 tables cached: 5.484ms (5.484ms)
>- Analysis finished: 16.104ms (10.619ms)
>- V