[
https://issues.apache.org/jira/browse/IMPALA-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qifan Chen updated IMPALA-10664:
--------------------------------
Description:
In most recent ASAN run (#25), TPCDS q4, q12, q18a, q23_1, q30, q32, q51a, q64
failed to get minimum memory reservation of certain amount.
For q12, the error details are as follows.
{code:java}
Stacktrace
query_test/test_tpcds_queries.py:388: in test_tpcds_q12
self.run_test_case(self.get_workload() + '-decimal_v2-q12', vector)
common/impala_test_suite.py:665: in run_test_case
result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
common/impala_test_suite.py:603: in __exec_in_impala
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:923: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:205: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:365: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:386: in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
E Query aborted:Failed to get minimum memory reservation of 101.81 MB on
daemon impala-ec2-centos74-r5-4xlarge-ondemand-1103.vpc.cloudera.com:27001 for
query b04954c9d5e868f5:6809522800000000 due to following error: Failed to
increase reservation by 101.81 MB because it would exceed the applicable
reservation limit for the "Process" ReservationTracker: reservation_limit=10.20
GB reservation=10.20 GB used_reservation=0 child_reservations=10.20 GB
E The top 5 queries that allocated memory under this tracker are:
E Query(2445142f48b05892:024b8a1200000000): Reservation=9.60 GB
ReservationLimit=9.60 GB OtherMemory=118.01 MB Total=9.71 GB Peak=9.71 GB
E Query(6a4063c66ed82ce4:2e5ed33300000000): Reservation=197.70 MB
ReservationLimit=9.60 GB OtherMemory=1.06 MB Total=198.75 MB Peak=198.75 MB
E Query(b345de7d867ce2b5:8e2b34c500000000): Reservation=104.75 MB
ReservationLimit=9.60 GB OtherMemory=43.55 MB Total=148.30 MB Peak=246.48 MB
E Query(0a4e58b710438a3b:fc00ba6000000000): Reservation=109.94 MB
ReservationLimit=9.60 GB OtherMemory=15.96 MB Total=125.89 MB Peak=274.19 MB
E Query(4f4e8e8119a0415f:98e0d40100000000): Reservation=78.50 MB
ReservationLimit=9.60 GB OtherMemory=4.95 MB Total=83.45 MB Peak=83.45 MB
E
E
E
E
E
E Memory is likely oversubscribed. Reducing query concurrency or configuring
admission control may help avoid this error.
{code}
Similar failures happened to tpch q9, q20 and q13.
was:
In most recent ASAN run (#25), TPCDS q4, q12, q18a, q23_1, q30, q32, q51a, q64
failed to get minimum memory reservation of certain amount.
For q12, the error details are as follows.
Stacktrace
query_test/test_tpcds_queries.py:388: in test_tpcds_q12
self.run_test_case(self.get_workload() + '-decimal_v2-q12', vector)
common/impala_test_suite.py:665: in run_test_case
result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
common/impala_test_suite.py:603: in __exec_in_impala
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:923: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:205: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:365: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:386: in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
E Query aborted:Failed to get minimum memory reservation of 101.81 MB on
daemon impala-ec2-centos74-r5-4xlarge-ondemand-1103.vpc.cloudera.com:27001 for
query b04954c9d5e868f5:6809522800000000 due to following error: Failed to
increase reservation by 101.81 MB because it would exceed the applicable
reservation limit for the "Process" ReservationTracker: reservation_limit=10.20
GB reservation=10.20 GB used_reservation=0 child_reservations=10.20 GB
E The top 5 queries that allocated memory under this tracker are:
E Query(2445142f48b05892:024b8a1200000000): Reservation=9.60 GB
ReservationLimit=9.60 GB OtherMemory=118.01 MB Total=9.71 GB Peak=9.71 GB
E Query(6a4063c66ed82ce4:2e5ed33300000000): Reservation=197.70 MB
ReservationLimit=9.60 GB OtherMemory=1.06 MB Total=198.75 MB Peak=198.75 MB
E Query(b345de7d867ce2b5:8e2b34c500000000): Reservation=104.75 MB
ReservationLimit=9.60 GB OtherMemory=43.55 MB Total=148.30 MB Peak=246.48 MB
E Query(0a4e58b710438a3b:fc00ba6000000000): Reservation=109.94 MB
ReservationLimit=9.60 GB OtherMemory=15.96 MB Total=125.89 MB Peak=274.19 MB
E Query(4f4e8e8119a0415f:98e0d40100000000): Reservation=78.50 MB
ReservationLimit=9.60 GB OtherMemory=4.95 MB Total=83.45 MB Peak=83.45 MB
E
E
E
E
E
E Memory is likely oversubscribed. Reducing query concurrency or configuring
admission control may help avoid this error.
Similar failures happened to tpch q9, q20 and q13.
> TPCDS q4, a12, q18a, q23_1, q30, q32, q51a and q64 failed in a recent ASAN run
> ------------------------------------------------------------------------------
>
> Key: IMPALA-10664
> URL: https://issues.apache.org/jira/browse/IMPALA-10664
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
> Reporter: Qifan Chen
> Priority: Major
>
> In most recent ASAN run (#25), TPCDS q4, q12, q18a, q23_1, q30, q32, q51a,
> q64 failed to get minimum memory reservation of certain amount.
> For q12, the error details are as follows.
> {code:java}
> Stacktrace
> query_test/test_tpcds_queries.py:388: in test_tpcds_q12
> self.run_test_case(self.get_workload() + '-decimal_v2-q12', vector)
> common/impala_test_suite.py:665: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:603: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:923: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:386: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Failed to get minimum memory reservation of 101.81 MB on
> daemon impala-ec2-centos74-r5-4xlarge-ondemand-1103.vpc.cloudera.com:27001
> for query b04954c9d5e868f5:6809522800000000 due to following error: Failed to
> increase reservation by 101.81 MB because it would exceed the applicable
> reservation limit for the "Process" ReservationTracker:
> reservation_limit=10.20 GB reservation=10.20 GB used_reservation=0
> child_reservations=10.20 GB
> E The top 5 queries that allocated memory under this tracker are:
> E Query(2445142f48b05892:024b8a1200000000): Reservation=9.60 GB
> ReservationLimit=9.60 GB OtherMemory=118.01 MB Total=9.71 GB Peak=9.71 GB
> E Query(6a4063c66ed82ce4:2e5ed33300000000): Reservation=197.70 MB
> ReservationLimit=9.60 GB OtherMemory=1.06 MB Total=198.75 MB Peak=198.75 MB
> E Query(b345de7d867ce2b5:8e2b34c500000000): Reservation=104.75 MB
> ReservationLimit=9.60 GB OtherMemory=43.55 MB Total=148.30 MB Peak=246.48 MB
> E Query(0a4e58b710438a3b:fc00ba6000000000): Reservation=109.94 MB
> ReservationLimit=9.60 GB OtherMemory=15.96 MB Total=125.89 MB Peak=274.19 MB
> E Query(4f4e8e8119a0415f:98e0d40100000000): Reservation=78.50 MB
> ReservationLimit=9.60 GB OtherMemory=4.95 MB Total=83.45 MB Peak=83.45 MB
> E
> E
> E
> E
> E
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> Similar failures happened to tpch q9, q20 and q13.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
