[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Riza Suminto (Code Review)
Riza Suminto has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Adds functional query workload tests for both the sys.impala_query_log
and the sys.impala_query_live tables to assert the names and order of
the individual columns within each table.

Renames the python tests for the sys.impala_query_log table removing the
unnecessary "_query_log_table_" string from the name of each test.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Reviewed-on: http://gerrit.cloudera.org:8080/21358
Tested-by: Impala Public Jenkins 
Reviewed-by: Riza Suminto 
---
M be/src/service/workload-management.cc
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-live.test
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-log.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
7 files changed, 492 insertions(+), 515 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Riza Suminto: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 11
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Riza Suminto (Code Review)
Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 10: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 13 May 2024 22:46:35 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 10: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 13 May 2024 21:23:35 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Riza Suminto (Code Review)
Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 10: Code-Review+1

Looks good to me. I will give my +2 if no one else has any new comment and 
DRY_RUN pass.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 13 May 2024 16:19:29 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 10:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10638/ 
DRY_RUN=true


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 13 May 2024 16:19:56 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Jason Fehr (Code Review)
Jason Fehr has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 10:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/9//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/21358/9//COMMIT_MSG@19
PS9, Line 19:
: Refactors workload management code to reduce if-clause nesting.
> nit: Mention new tests added since patch set 6.
Done



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 13 May 2024 16:01:34 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-13 Thread Jason Fehr (Code Review)
Jason Fehr has uploaded a new patch set (#10) to the change originally created 
by Michael Smith. ( http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Adds functional query workload tests for both the sys.impala_query_log
and the sys.impala_query_live tables to assert the names and order of
the individual columns within each table.

Renames the python tests for the sys.impala_query_log table removing the
unnecessary "_query_log_table_" string from the name of each test.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-live.test
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-log.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
7 files changed, 492 insertions(+), 515 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/10
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 10
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 9:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/16169/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 9
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Fri, 10 May 2024 22:04:18 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 8:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/16168/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 8
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Fri, 10 May 2024 21:53:17 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 7:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/16167/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Fri, 10 May 2024 21:51:34 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Riza Suminto (Code Review)
Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 9: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/9//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/21358/9//COMMIT_MSG@19
PS9, Line 19:
: Refactors workload management code to reduce if-clause nesting.
nit: Mention new tests added since patch set 6.



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 9
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Fri, 10 May 2024 21:43:58 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Jason Fehr (Code Review)
Jason Fehr has uploaded a new patch set (#9) to the change originally created 
by Michael Smith. ( http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-live.test
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-log.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
7 files changed, 492 insertions(+), 515 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/9
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 9
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Jason Fehr (Code Review)
Jason Fehr has uploaded a new patch set (#8) to the change originally created 
by Michael Smith. ( http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-live.test
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-log.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
7 files changed, 492 insertions(+), 515 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/8
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 8
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-10 Thread Jason Fehr (Code Review)
Jason Fehr has uploaded a new patch set (#7) to the change originally created 
by Michael Smith. ( http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-live.test
A 
testdata/workloads/functional-query/queries/QueryTest/workload-management-log.test
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
7 files changed, 493 insertions(+), 516 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/7
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-07 Thread Riza Suminto (Code Review)
Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 6: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Tue, 07 May 2024 18:24:07 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-07 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 6: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/6/tests/util/workload_management.py
File tests/util/workload_management.py:

http://gerrit.cloudera.org:8080/#/c/21358/6/tests/util/workload_management.py@26
PS6, Line 26: from SystemTables.ttypes import TQueryTableColumn
> The last addition (line 573) is there to ensure all columns are tested.
ack



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Tue, 07 May 2024 06:52:32 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-06 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/6/tests/util/workload_management.py
File tests/util/workload_management.py:

http://gerrit.cloudera.org:8080/#/c/21358/6/tests/util/workload_management.py@26
PS6, Line 26: from SystemTables.ttypes import TQueryTableColumn
> While these changes simplify the code quite a bit (20% or so), they also re
The last addition (line 573) is there to ensure all columns are tested.

I agree that duplicating the column names helps reinforce they should not be 
changed, but feels kind of excessive. Protocol definitions usually have the 
ability to break things if you change existing fields.



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 06 May 2024 23:06:53 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-06 Thread Jason Fehr (Code Review)
Jason Fehr has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/6/tests/util/workload_management.py
File tests/util/workload_management.py:

http://gerrit.cloudera.org:8080/#/c/21358/6/tests/util/workload_management.py@26
PS6, Line 26: from SystemTables.ttypes import TQueryTableColumn
While these changes simplify the code quite a bit (20% or so), they also remove 
a couple of important assertions for total number of columns and column names.

Asserting the total number of columns against a test constant prompts anyone 
adding a column to also update the python tests.

The column name assertions are important because they ensure columns are not 
inadvertently renamed.  Highly unlikely, but possible where a character could 
be deleted or added accidentally.  Even though the code review should catch 
such a mistake, it is still best for automated checks to ensure those mistakes 
never cause distractions in code reviews.

I like the removal of the index variable.  We should remove that in favor of 
using the indexes from TQueryTableColumn.  Then we could keep the column_val 
function and use it to simplify the code.



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 06 May 2024 23:02:45 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/16103/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 06 May 2024 22:33:06 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-06 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 6:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py@161
PS5, Line 161:   # IMPALA-13051: Add faster default graceful shutdown 
options before processing
> Can you mention the Jira? It may not be evident for the reader why this is
Done


http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py@236
PS5, Line 236:
> I'll probably back this change out, it's weird but doesn't cause issues rig
Done


http://gerrit.cloudera.org:8080/#/c/21358/5/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/custom_cluster/test_query_log.py@411
PS5, Line 411:   """Tests to assert that query_log_table_name works with 
non-default value."""
> nit: I was confused at first that this means that table table can be rename
Done


http://gerrit.cloudera.org:8080/#/c/21358/5/tests/util/workload_management.py
File tests/util/workload_management.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/util/workload_management.py@93
PS5, Line 93:   session_id = re.search(r'\n\s+Session ID:\s+(.*)\n', 
profile_text)
> An idea to make these even more compact:
Haven't had the bandwidth to look at this more. Seems reasonable, but also 
triggers a significant amount of other cleanup in this file that will take me a 
little while.



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Mon, 06 May 2024 22:09:17 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-06 Thread Michael Smith (Code Review)
Hello Quanlong Huang, Andrew Sherman, Riza Suminto, Jason Fehr, Csaba 
Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/21358

to look at the new patch set (#6).

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
5 files changed, 332 insertions(+), 494 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/6
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-05 Thread Csaba Ringhofer (Code Review)
Csaba Ringhofer has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 5: Code-Review+1

(3 comments)

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py@161
PS5, Line 161:   # Add faster default graceful shutdown options before 
processing explicit args.
Can you mention the Jira? It may not be evident for the reader why this is 
important.


http://gerrit.cloudera.org:8080/#/c/21358/5/tests/custom_cluster/test_query_log.py
File tests/custom_cluster/test_query_log.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/custom_cluster/test_query_log.py@411
PS5, Line 411:   """Tests to assert the query log table can be renamed."""
nit: I was confused at first that this means that table table can be renamed 
with ALTER TABLE. Something like  "tests that query_log_table_name works with 
non-default value" seems clearer.


http://gerrit.cloudera.org:8080/#/c/21358/5/tests/util/workload_management.py
File tests/util/workload_management.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/util/workload_management.py@93
PS5, Line 93:   session_id = re.search(r'\n\s+Session ID:\s+(.*)\n', 
profile_text)
An idea to make these even more compact:
the simple cases could be checked with another inline function, e.g.
def check(qtable_col, regex)

check(TQueryTableColumn.SESSION_SESSION_ID, r'\n\s+Session ID:\s+(.*)\n)



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Sun, 05 May 2024 06:36:42 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-03 Thread Michael Smith (Code Review)
Michael Smith has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py@236
PS5, Line 236: super(CustomClusterTestSuite, self).teardown_class()
> Should it be called the very last? I see couple custom cluster tests overri
I'll probably back this change out, it's weird but doesn't cause issues right 
now.



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Fri, 03 May 2024 20:29:03 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-02 Thread Riza Suminto (Code Review)
Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/21358/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/21358/1//COMMIT_MSG@6
PS1, Line 6:
   : IMPALA-13051: Speed up, refa
> Will do, this refactor isn't urgent but I wanted to toss up some ideas.
Done


http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21358/5/tests/common/custom_cluster_test_suite.py@236
PS5, Line 236: super(CustomClusterTestSuite, self).teardown_class()
Should it be called the very last? I see couple custom cluster tests override 
teardown_class()



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Fri, 03 May 2024 01:20:56 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-02 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/16085/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Thu, 02 May 2024 18:25:02 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-02 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 4:

Build Failed

https://jenkins.impala.io/job/gerrit-code-review-checks/16084/ : Initial code 
review checks failed. See linked job for details on the failure.


--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Andrew Sherman 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Jason Fehr 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Thu, 02 May 2024 18:16:57 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-02 Thread Michael Smith (Code Review)
Hello Riza Suminto, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/21358

to look at the new patch set (#5).

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Reorders CustomClusterTestSuite.teardown_method to stop clients, then
Impala, then restart Hive in reverse of the order they're setup in.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
5 files changed, 337 insertions(+), 502 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/5
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Riza Suminto 


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-02 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/21358 )

Change subject: IMPALA-13051: Speed up, refactor query log tests
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21358/4/tests/common/custom_cluster_test_suite.py
File tests/common/custom_cluster_test_suite.py:

http://gerrit.cloudera.org:8080/#/c/21358/4/tests/common/custom_cluster_test_suite.py@162
PS4, Line 162: )
flake8: E501 line too long (91 > 90 characters)



--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Riza Suminto 
Gerrit-Comment-Date: Thu, 02 May 2024 17:58:04 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-13051: Speed up, refactor query log tests

2024-05-02 Thread Michael Smith (Code Review)
Hello Riza Suminto, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/21358

to look at the new patch set (#4).

Change subject: IMPALA-13051: Speed up, refactor query log tests
..

IMPALA-13051: Speed up, refactor query log tests

Sets faster default shutdown_grace_period_s and shutdown_deadline_s when
impalad_graceful_shutdown=True in tests. Impala waits until grace period
has passed and all queries are stopped (or deadline is exceeded) before
flushing the query log, so grace period of 0 is sufficient. Adds them in
setup_method to reduce duplication in test declarations.

Reorders CustomClusterTestSuite.teardown_method to stop clients, then
Impala, then restart Hive in reverse of the order they're setup in.

Re-uses TQueryTableColumn Thrift definitions for testing.

Moves waiting for query log table to exist to setup_method rather than
as a side-effect of get_client.

Refactors workload management code to reduce if-clause nesting.

Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
---
M be/src/service/workload-management.cc
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/util/workload_management.py
5 files changed, 336 insertions(+), 502 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/21358/4
--
To view, visit http://gerrit.cloudera.org:8080/21358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1127ef041a3e024bf2b262767d56ec5f29bf3855
Gerrit-Change-Number: 21358
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Michael Smith 
Gerrit-Reviewer: Riza Suminto