[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17256 ) Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. IMPALA-10596: De-flake teardown in TestAdmissionControllerStress Currently if the threads running queries in TestAdmissionControllerStress hit an error, they close their client which ultimately closes the query that it was running. If teardown() runs after the client is closed, it tries to cancel the query that the thread was running and hits an exception trying to cancel an already closed query. This results in the pytest throws the exception encountered in teardown() instead of the original exception that caused the test to fail in the first place. This patch fixes this by removing the query handle from the thread if the client is closed. Testing: Simulated hitting an error condition in the main thread that initially triggered this condition. Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Reviewed-on: http://gerrit.cloudera.org:8080/17256 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/custom_cluster/test_admission_controller.py 1 file changed, 7 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 3 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17256 ) Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 01 Apr 2021 05:56:24 + Gerrit-HasComments: No
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17246 ) Change subject: Only fetch needed branches in compare_branches.py .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Apr 2021 04:32:29 + Gerrit-HasComments: No
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17246 ) Change subject: Only fetch needed branches in compare_branches.py .. Only fetch needed branches in compare_branches.py Fetching all branches of a remote repo will be time-consuming. This changes compare_branches.py to only fetch the needed branches, i.e. source_branch and target_branch. Tests: - Before this change, it can't finish in 30 mins when comparing two downstream branches. After this change, it finishes in one minute. Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Reviewed-on: http://gerrit.cloudera.org:8080/17246 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/compare_branches.py 1 file changed, 6 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17258 ) Change subject: IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8489/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17258 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifd9820f9944a78811ee7acfa5870a9418902b17b Gerrit-Change-Number: 17258 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 01 Apr 2021 04:25:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17177 ) Change subject: IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations .. IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations Webpage of catalogd operations doesn't sum up requests correctly. Instead, the current meaning is summing by tables. As the column name is "Number of requests", we should sum up by requests. Tests: - Manually run test_concurrent_inserts and verify the number is correct. Change-Id: I1c5361d981832d6f28db5f203a2c2538fe8ebb5e Reviewed-on: http://gerrit.cloudera.org:8080/17177 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/catalog/catalog-server.cc 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1c5361d981832d6f28db5f203a2c2538fe8ebb5e Gerrit-Change-Number: 17177 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17177 ) Change subject: IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c5361d981832d6f28db5f203a2c2538fe8ebb5e Gerrit-Change-Number: 17177 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Thu, 01 Apr 2021 04:15:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state
Wenzhe Zhou has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17258 ) Change subject: IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state .. IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state The previous patch tried to fix the race and make backends avoid to send status report with fragment instance state as "done" and overall_status as OK after fragment instance fails. But it does not work when fragment instance state is updated during generating status report. For failed fragment instance, backend should report the instance as "done" only when overall_statue is reported with error. The final fragment instance state will be reported in final status report. This avoid coordinator to ignore the last status report. Testing: - Manual tests I could only reproduce the situation by adding some artificial delays in the QueryState::ConstructReport() after setting overall_status for the status report when repeatedly running test case test_spilling.py::TestSpillingNoDebugActionDimensions ::test_spilling_no_debug_action. Verified that the issue did not happen after applying this patch. - Passed exhaustive test. Change-Id: Ifd9820f9944a78811ee7acfa5870a9418902b17b --- M be/src/runtime/fragment-instance-state.cc 1 file changed, 4 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/17258/2 -- To view, visit http://gerrit.cloudera.org:8080/17258 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifd9820f9944a78811ee7acfa5870a9418902b17b Gerrit-Change-Number: 17258 Gerrit-PatchSet: 2 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17259 ) Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8488/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Apr 2021 03:14:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10607: Fixed test ctas exprs failure for S3 build
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17234 ) Change subject: IMPALA-10607: Fixed test_ctas_exprs failure for S3 build .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Gerrit-Change-Number: 17234 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 01 Apr 2021 03:07:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10607: Fixed test ctas exprs failure for S3 build
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17234 ) Change subject: IMPALA-10607: Fixed test_ctas_exprs failure for S3 build .. IMPALA-10607: Fixed test_ctas_exprs failure for S3 build New test case TestDecimalOverflowExprs::test_ctas_exprs was added in the patch for IMPALA-10564. But it failed in S3 build with Parquet format and complained the Parquet file had an invalid file length when accessing a table. The table was created by CTAS which finished with error "decimal expression overflowed". Verified this issue does not happen if query option s3_skip_insert_staging is set as false. When s3_skip_insert_staging is set true by default, INSERT writing to S3 goes directly to their final location rather than being copied there by the coordinator. If CTAS finishs with error during INSERT, the parquet partition file is left in un-finalized without file footer. This causes subsequent query failed with error like "have an invalid file length on S3" when the query attemps to access the same table. This patch fixed the issue by deleting the un-finalized file in its final location when AppendRows() return error and staging has been skipped. Testing: - Reproduced the test failure in local box with defaultFS as s3. Verified the fixing by running test_ctas_exprs with defaultFS as s3. - Passed core tests. Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Reviewed-on: http://gerrit.cloudera.org:8080/17234 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/hdfs-table-sink.cc M tests/query_test/test_decimal_queries.py 2 files changed, 47 insertions(+), 14 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Gerrit-Change-Number: 17234 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17259 ) Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8487/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Apr 2021 02:59:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17259 to look at the new patch set (#4). Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. IMPALA-10629: Fix parquet compression codecs for data load scripts Currently, the dataload scripts don't respect non-standard compression codecs when loading Parquet data. It always loads snappy, even when specifying something else like --table_format=parquet/zstd. This fixes the dataload scripts so that they specify the compression_codec query option correctly and thus use the right codec when loading Parquet. For backwards compatibility, this preserves the behavior that parquet/none corresponds to the default compression codec (which is Snappy). This should make it easier to do performance testing on various Parquet codecs (like ZSTD). Testing: - Ran bin/load-data.py -w tpch --table_format=parquet/zstd and checked the codec in the file with the parquet-reader utility Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a --- M testdata/bin/generate-schema-statements.py 1 file changed, 34 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/17259/4 -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17259 ) Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8486/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Apr 2021 02:53:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17259 to look at the new patch set (#3). Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. IMPALA-10629: Fix parquet compression codecs for data load scripts Currently, the dataload scripts don't respect non-standard compression codecs when loading Parquet data. It always loads snappy, even when specifying something else like --table_format=parquet/zstd. This fixes the dataload scripts so that they specify the compression_codec query option correctly and thus use the right codec when loading Parquet. For backwards compatibility, this preserves the behavior that parquet/none corresponds to the default compression codec (which is Snappy). This should make it easier to do performance testing on various Parquet codecs (like ZSTD). Testing: - Ran bin/load-data.py -w tpch --table_format=parquet/zstd and checked the codec in the file with the parquet-reader utility Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a --- M testdata/bin/generate-schema-statements.py 1 file changed, 34 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/17259/3 -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17259 to look at the new patch set (#2). Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. IMPALA-10629: Fix parquet compression codecs for data load scripts Currently, the dataload scripts don't respect non-standard compression codecs when loading Parquet data. It always loads snappy, even when specifying something else like --table_format=parquet/zstd. This fixes the dataload scripts so that they specify the compression_codec query option correctly and thus use the right codec when loading Parquet. For backwards compatibility, this preserves the behavior that parquet/none corresponds to the default compression codec (which is Snappy). This should make it easier to do performance testing on various Parquet codecs (like ZSTD). Testing: - Ran bin/load-data.py -w tpch --table_format=parquet/zstd and checked the codec in the file with the parquet-reader utility Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a --- M testdata/bin/generate-schema-statements.py 1 file changed, 29 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/17259/2 -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17259 ) Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/17259/1/testdata/bin/generate-schema-statements.py File testdata/bin/generate-schema-statements.py: http://gerrit.cloudera.org:8080/#/c/17259/1/testdata/bin/generate-schema-statements.py@163 PS1, Line 163: } flake8: E123 closing bracket does not match indentation of opening bracket's line http://gerrit.cloudera.org:8080/#/c/17259/1/testdata/bin/generate-schema-statements.py@438 PS1, Line 438: def build_impala_parquet_codec_statement(codec): flake8: E302 expected 2 blank lines, found 1 http://gerrit.cloudera.org:8080/#/c/17259/1/testdata/bin/generate-schema-statements.py@493 PS1, Line 493: n flake8: E501 line too long (92 > 90 characters) http://gerrit.cloudera.org:8080/#/c/17259/1/testdata/bin/generate-schema-statements.py@498 PS1, Line 498: " flake8: E501 line too long (94 > 90 characters) http://gerrit.cloudera.org:8080/#/c/17259/1/testdata/bin/generate-schema-statements.py@787 PS1, Line 787: u flake8: E501 line too long (94 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 01 Apr 2021 02:34:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10629: Fix parquet compression codecs for data load scripts
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17259 Change subject: IMPALA-10629: Fix parquet compression codecs for data load scripts .. IMPALA-10629: Fix parquet compression codecs for data load scripts Currently, the dataload scripts don't respect non-standard compression codecs when loading Parquet data. It always loads snappy, even when specifying something else like --table_format=parquet/zstd. This fixes the dataload scripts so that they specify the compression_codec query option correctly and thus use the right codec when loading Parquet. This should make it easier to do performance testing on various Parquet codecs (like ZSTD). Testing: - Ran bin/load-data.py -w tpch --table_format=parquet/zstd and checked the codec in the file with the parquet-reader utility Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a --- M testdata/bin/generate-schema-statements.py 1 file changed, 29 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/17259/1 -- To view, visit http://gerrit.cloudera.org:8080/17259 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1a346de3e5c4e38328e5a8ce8162697b7dd6553a Gerrit-Change-Number: 17259 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 01 Apr 2021 02:07:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17258 ) Change subject: IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8485/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17258 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifd9820f9944a78811ee7acfa5870a9418902b17b Gerrit-Change-Number: 17258 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 01 Apr 2021 01:31:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state
Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17258 Change subject: IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state .. IMPALA-10259 (part 2): Fixed DCHECK error for backend in terminal state The previous patch tried to fix the race and make backends avoid to send status report with fragment instance state as "done" and overall_status as OK after fragment instance fails. But it does not work when fragment instance state is updated during generating status report. For failed fragment instance, backend should report the instance as "done" only when overall_statue is reported with error. The final fragment instance state will be reported in final status report. This avoid coordinator to ignore the last status report. Testing: - Manual tests I could only reproduce the situation by adding some artificial delays in the QueryState::ConstructReport() after setting overall_status for the status report when repeatedly running test case test_spilling.py::TestSpillingNoDebugActionDimensions ::test_spilling_no_debug_action. Verified that the issue did not happen after applying this patch. - Passed exhaustive test. Change-Id: Ifd9820f9944a78811ee7acfa5870a9418902b17b --- M be/src/runtime/fragment-instance-state.cc 1 file changed, 3 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/17258/1 -- To view, visit http://gerrit.cloudera.org:8080/17258 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifd9820f9944a78811ee7acfa5870a9418902b17b Gerrit-Change-Number: 17258 Gerrit-PatchSet: 1 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 01 Apr 2021 00:14:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17256 ) Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7037/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 23:58:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17256 ) Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 2 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 23:58:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Change default values for flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17253 ) Change subject: IMPALA-6671: Change default values for flags .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7030/ -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 31 Mar 2021 23:43:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9910: [DOCS] update retry failed queries query option
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/16819 ) Change subject: IMPALA-9910: [DOCS] update retry failed queries query option .. Patch Set 2: (4 comments) Thanks, this is looking better http://gerrit.cloudera.org:8080/#/c/16819/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16819/2//COMMIT_MSG@9 PS2, Line 9: added why cluster membership changes typically occur This seems to be about Patchset 2. The commit message should describe the whole change. Also please use sentences with capital letters and end with a period. http://gerrit.cloudera.org:8080/#/c/16819/2/docs/topics/impala_retry_failed_queries.xml File docs/topics/impala_retry_failed_queries.xml: http://gerrit.cloudera.org:8080/#/c/16819/2/docs/topics/impala_retry_failed_queries.xml@40 PS2, Line 40: blacklisted by the Impala Coordinator and if a SELECT query fails due to a cluster membership This is too complex. End the sentence with "Coordinator." and then start another with "If ..." http://gerrit.cloudera.org:8080/#/c/16819/2/docs/topics/impala_retry_failed_queries.xml@49 PS2, Line 49: spool_all_results_for_retries. Should this say something like "set the spool_all_results_for_retries query option"? http://gerrit.cloudera.org:8080/#/c/16819/2/docs/topics/impala_retry_failed_queries.xml@53 PS2, Line 53: ENABLED If the type is BOOLEAN won't this be TRUE ? -- To view, visit http://gerrit.cloudera.org:8080/16819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3a65357a6e3d0bffa840b8636171a38bd9b22d17 Gerrit-Change-Number: 16819 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 23:26:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Bump up the GBN number to 11920537
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8484/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 31 Mar 2021 23:06:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9910: [DOCS] update retry failed queries query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16819 ) Change subject: IMPALA-9910: [DOCS] update retry failed queries query option .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/626/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/16819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3a65357a6e3d0bffa840b8636171a38bd9b22d17 Gerrit-Change-Number: 16819 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 22:52:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10624: TestIcebergTable::test alter iceberg tables failed by stale file format
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17248 ) Change subject: IMPALA-10624: TestIcebergTable::test_alter_iceberg_tables failed by stale file format .. Patch Set 1: (1 comment) Patch looks good to me. I left a comment regarding the comment. http://gerrit.cloudera.org:8080/#/c/17248/1/fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/17248/1/fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java@177 PS1, Line 177: only be s Not sure if I understand this comment fully. Is this more readable? Returns true if the table property should only be stored in HMS. If false, the property is stored in HMS as well as iceberg? -- To view, visit http://gerrit.cloudera.org:8080/17248 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff8d5d1d90444aba11d47cfce522aaa45a4a74cc Gerrit-Change-Number: 17248 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 31 Mar 2021 22:38:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Bump up the GBN number to 11920537
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Patch Set 2: > Looked at the GVO failure. > testAcidMinorCompactionLoading failed because our version of Hive > doesn't write '_orc_acid_version' anymore. So I think we should > just change the expected value to 8. > > I'm working on the fix for test_alter_iceberg_tables: > https://gerrit.cloudera.org/#/c/17248/ Thanks for looking into the failures. I updated FileMetadataLoaderTest as suggested. -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 31 Mar 2021 22:34:17 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump up the GBN number to 11920537
Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Bump up the GBN number to 11920537 This change bumps up the GBN to 11920537 which includes several changes to Hive needed to support Catalogd's HMS endpoint for supporting external frontends. Additionally, it excludes some dependencies from the pom.xml which are not uploaded by default to the toolchain. After the GBN bump up Hive doesn't write '_orc_acid_version' files and hence the FileMetadataLoaderTest needed to be modified. Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 --- M bin/impala-config.sh M fe/pom.xml M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java M java/shaded-deps/hive-exec/pom.xml M java/test-hive-udfs/pom.xml 5 files changed, 29 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/17243/2 -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. IMPALA-10608 followup: Detect the virtualenv tarball version When rebasing from an older commit, the version change in virtualenv can cause there to be multiple virtualenv tarballs of different versions in the infra/python/deps directory. bootstrap_virtualenv.py currently doesn't handle this gracefully, because it is looking for all virtualenv*.tar.gz files and fails when it finds more than one. This changes bootstrap_virtualenv.py to get the virtualenv version from the requirements.txt file and only look for the tarball with that version. If it fails to get the version, it falls back to the old method. Testing: - Copied virtualenv-16.7.10.tar.gz to virtualenv-16.7.9.tar.gz and verified that bootstrap_virtualenv.py works Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Reviewed-on: http://gerrit.cloudera.org:8080/17249 Reviewed-by: Joe McDonnell Tested-by: Joe McDonnell --- M infra/python/bootstrap_virtualenv.py 1 file changed, 25 insertions(+), 1 deletion(-) Approvals: Joe McDonnell: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 3: Verified+1 The GVO on patch 2 passed and all the jobs for patch 3 are past the point where this could impact anything. Going ahead with this. -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 31 Mar 2021 22:33:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9910: [DOCS] update retry failed queries query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16819 ) Change subject: IMPALA-9910: [DOCS] update retry failed queries query option .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/626/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/16819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3a65357a6e3d0bffa840b8636171a38bd9b22d17 Gerrit-Change-Number: 16819 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 22:31:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9910: [DOCS] update retry failed queries query option
Hello Andrew Sherman, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16819 to look at the new patch set (#2). Change subject: IMPALA-9910: [DOCS] update retry failed queries query option .. IMPALA-9910: [DOCS] update retry failed queries query option added why cluster membership changes typically occur explained how the coordinator will retry a failed query talked about the new query option spool_all_results_for_retries Change-Id: I3a65357a6e3d0bffa840b8636171a38bd9b22d17 --- M docs/topics/impala_retry_failed_queries.xml 1 file changed, 12 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/16819/2 -- To view, visit http://gerrit.cloudera.org:8080/16819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3a65357a6e3d0bffa840b8636171a38bd9b22d17 Gerrit-Change-Number: 16819 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17246 ) Change subject: Only fetch needed branches in compare_branches.py .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7036/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 22:20:30 + Gerrit-HasComments: No
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17246 ) Change subject: Only fetch needed branches in compare_branches.py .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 22:20:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17177 ) Change subject: IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7035/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c5361d981832d6f28db5f203a2c2538fe8ebb5e Gerrit-Change-Number: 17177 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Wed, 31 Mar 2021 22:20:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17177 ) Change subject: IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c5361d981832d6f28db5f203a2c2538fe8ebb5e Gerrit-Change-Number: 17177 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Wed, 31 Mar 2021 22:20:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10538: [DOCS] Document the newly added NDV parameter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17131 ) Change subject: IMPALA-10538: [DOCS] Document the newly added NDV parameter .. Patch Set 4: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/625/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/17131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec8007b79afac59cdfb3984bb111806213c21c77 Gerrit-Change-Number: 17131 Gerrit-PatchSet: 4 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Wed, 31 Mar 2021 22:18:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10538: [DOCS] Document the newly added NDV parameter
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17131 ) Change subject: IMPALA-10538: [DOCS] Document the newly added NDV parameter .. Patch Set 4: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/625/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/17131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec8007b79afac59cdfb3984bb111806213c21c77 Gerrit-Change-Number: 17131 Gerrit-PatchSet: 4 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Wed, 31 Mar 2021 22:11:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10538: [DOCS] Document the newly added NDV parameter
Hello Andrew Sherman, Tamas Mate, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17131 to look at the new patch set (#4). Change subject: IMPALA-10538: [DOCS] Document the newly added NDV parameter .. IMPALA-10538: [DOCS] Document the newly added NDV parameter Added how this argument maps to a precision used by the HLL algorithm. Incorporated changes from the feedback. Change-Id: Iec8007b79afac59cdfb3984bb111806213c21c77 --- M docs/topics/impala_ndv.xml 1 file changed, 36 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/17131/4 -- To view, visit http://gerrit.cloudera.org:8080/17131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iec8007b79afac59cdfb3984bb111806213c21c77 Gerrit-Change-Number: 17131 Gerrit-PatchSet: 4 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 31 Mar 2021 21:41:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10494: Making use of the min/max column stats to improve min/max filters
Aman Sinha has posted comments on this change. (
http://gerrit.cloudera.org:8080/17075 )
Change subject: IMPALA-10494: Making use of the min/max column stats to improve
min/max filters
..
Patch Set 28: Code-Review+1
(2 comments)
Just a nit and a comment. Should be able to +2 after that.
http://gerrit.cloudera.org:8080/#/c/17075/27/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
File fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java:
http://gerrit.cloudera.org:8080/#/c/17075/27/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java@260
PS27, Line 260: }
> Looks like mixture of files of different format (like Parquet and ORC at the
> same time) is not allowed.
This may not be accurate. In the HdfsScanNode's computeScanRangeLocation(), we
examine the file format for a partition and just record it but just set a flag
if they are not all parquet format. It does not assert or return error.
Here's the snippet in HdfsScanNode.java:
fileFormats_.add(partition.getFileFormat());
if (!isParquetBased(partition.getFileFormat())) {
allParquet = false;
}
However, for statistics, as I mentioned before (and you seem in agreement)
that having at least one parquet partition is sufficient to trigger the min-max
filter checks since it does not affect correctness of results.
http://gerrit.cloudera.org:8080/#/c/17075/28/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/17075/28/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@387
PS28, Line 387: /*
Can you remove this method.
--
To view, visit http://gerrit.cloudera.org:8080/17075
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I08581b44419bb8da5940cbf98502132acd1c86df
Gerrit-Change-Number: 17075
Gerrit-PatchSet: 28
Gerrit-Owner: Qifan Chen
Gerrit-Reviewer: Aman Sinha
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Qifan Chen
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Wed, 31 Mar 2021 21:39:27 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9910: [DOCS] update retry failed queries query option
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/16819 ) Change subject: IMPALA-9910: [DOCS] update retry failed queries query option .. Patch Set 1: (4 comments) I have a few suggestions http://gerrit.cloudera.org:8080/#/c/16819/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16819/1//COMMIT_MSG@9 PS1, Line 9: elaborated the existing content Can you make these into complete sentences please? Take a look at other Impala commit messages to see the standard we are aiming for. http://gerrit.cloudera.org:8080/#/c/16819/1/docs/topics/impala_retry_failed_queries.xml File docs/topics/impala_retry_failed_queries.xml: http://gerrit.cloudera.org:8080/#/c/16819/1/docs/topics/impala_retry_failed_queries.xml@39 PS1, Line 39: If a SELECT query fails due to a cluster membership change, the Coordinator will cancel and I think we should try to include "Cluster membership changes typically occur when an impalad crashes, or if the node is blacklisted by the Impala Coordinator" from the old text. http://gerrit.cloudera.org:8080/#/c/16819/1/docs/topics/impala_retry_failed_queries.xml@40 PS1, Line 40: unregister the running query and then launch a retry of the query. For e.g., if one of the Instead of "For e.g." I think "For example" is clearer http://gerrit.cloudera.org:8080/#/c/16819/1/docs/topics/impala_retry_failed_queries.xml@43 PS1, Line 43: executor nodes. This feature will support retrying the entire query and NOT the individual This feature supports -- To view, visit http://gerrit.cloudera.org:8080/16819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3a65357a6e3d0bffa840b8636171a38bd9b22d17 Gerrit-Change-Number: 16819 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 21:35:57 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10538: [DOCS] Document the newly added NDV parameter
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/17131 ) Change subject: IMPALA-10538: [DOCS] Document the newly added NDV parameter .. Patch Set 3: (5 comments) Thanks for doing this, I have some suggestions/comments for making the docs clearer http://gerrit.cloudera.org:8080/#/c/17131/3/docs/topics/impala_ndv.xml File docs/topics/impala_ndv.xml: http://gerrit.cloudera.org:8080/#/c/17131/3/docs/topics/impala_ndv.xml@49 PS3, Line 49: NDV([DISTINCT | ALL] expression [,scale]) Need to explain somewhere what scale is used if this parameter is not specified. http://gerrit.cloudera.org:8080/#/c/17131/3/docs/topics/impala_ndv.xml@51 PS3, Line 51: The argument scale must be an integer and can be in the range from 1 to 10 This seems a lot of detail for one optional argument. One way to deal with this might be to have some sort of heading like 'notes on the optional scale argument'. http://gerrit.cloudera.org:8080/#/c/17131/3/docs/topics/impala_ndv.xml@52 PS3, Line 52: and maps to a precision used by the HLL algorithm with the following mapping formula: Expand HLL or explain what it means http://gerrit.cloudera.org:8080/#/c/17131/3/docs/topics/impala_ndv.xml@64 PS3, Line 64: need of extra memory. For a given precision p, the amount of memory used by the HLL algorithm Expand or explain HLL http://gerrit.cloudera.org:8080/#/c/17131/3/docs/topics/impala_ndv.xml@69 PS3, Line 69: When provided a scale of 10 against a total of 22 distinct data sets loaded into external I don't understand this sentence, is there a part missing? -- To view, visit http://gerrit.cloudera.org:8080/17131 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec8007b79afac59cdfb3984bb111806213c21c77 Gerrit-Change-Number: 17131 Gerrit-PatchSet: 3 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Wed, 31 Mar 2021 21:21:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10607: Fixed test ctas exprs failure for S3 build
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17234 ) Change subject: IMPALA-10607: Fixed test_ctas_exprs failure for S3 build .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Gerrit-Change-Number: 17234 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 31 Mar 2021 21:18:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10607: Fixed test ctas exprs failure for S3 build
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17234 ) Change subject: IMPALA-10607: Fixed test_ctas_exprs failure for S3 build .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7034/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Gerrit-Change-Number: 17234 Gerrit-PatchSet: 4 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 31 Mar 2021 21:18:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10607: Fixed test ctas exprs failure for S3 build
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17234 ) Change subject: IMPALA-10607: Fixed test_ctas_exprs failure for S3 build .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Gerrit-Change-Number: 17234 Gerrit-PatchSet: 3 Gerrit-Owner: Wenzhe Zhou Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 31 Mar 2021 21:18:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/17177 ) Change subject: IMPALA-10582: Fix wrong summary numbers in the webpage of catalogd operations .. Patch Set 1: Code-Review+2 LGTM -- To view, visit http://gerrit.cloudera.org:8080/17177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c5361d981832d6f28db5f203a2c2538fe8ebb5e Gerrit-Change-Number: 17177 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Wed, 31 Mar 2021 20:59:56 + Gerrit-HasComments: No
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/17246 ) Change subject: Only fetch needed branches in compare_branches.py .. Patch Set 1: Code-Review+2 LGTM -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 20:52:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8483/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:39:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7033/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:26:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 4: gvo aborted because I forgot to commit the fix for Bikram's comment -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:25:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Hello Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17208 to look at the new patch set (#4). Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. IMPALA-10591: Handle failed ReleaseQueryBackends rpcs When the admission control service is in use, coordinators will retry failed ReleaseQueryBackends rpcs 3 times before giving up. This can potentially result in resources not being released when they are no longer in use. This patch fixes the issue by automatically releasing all remaining backends when ReleaseQuery is called in the context of the admission control service (either as the result of a ReleaseQUery rpc or when cleaning up a query's resources with the heartbeat mechanism). Testing: - Added a custom cluster test that simulates failed ReleaseQueryBackends rpcs and ensures that query resources are eventually released. Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 --- M be/src/scheduling/admission-control-service.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M tests/custom_cluster/test_admission_controller.py 4 files changed, 115 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/17208/4 -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7032/ -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:25:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17256 ) Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8482/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 1 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:15:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7032/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:14:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10591: Handle failed ReleaseQueryBackends rpcs
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17208 ) Change subject: IMPALA-10591: Handle failed ReleaseQueryBackends rpcs .. Patch Set 3: Code-Review+2 (1 comment) carrying forward http://gerrit.cloudera.org:8080/#/c/17208/2/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/17208/2/be/src/scheduling/admission-controller.h@855 PS2, Line 855: per_backend_resources > nit: update comment to mention that only the backends currently executing e Done -- To view, visit http://gerrit.cloudera.org:8080/17208 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22842b2fb8ee170b5e91f12cd83f57a5f5502ae9 Gerrit-Change-Number: 17208 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:14:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/17256 ) Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 1 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 31 Mar 2021 20:13:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10596: De-flake teardown in TestAdmissionControllerStress
Bikramjeet Vig has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17256 Change subject: IMPALA-10596: De-flake teardown in TestAdmissionControllerStress .. IMPALA-10596: De-flake teardown in TestAdmissionControllerStress Currently if the threads running queries in TestAdmissionControllerStress hit an error, they close their client which ultimately closes the query that it was running. If teardown() runs after the client is closed, it tries to cancel the query that the thread was running and hits an exception trying to cancel an already closed query. This results in the pytest throws the exception encountered in teardown() instead of the original exception that caused the test to fail in the first place. This patch fixes this by removing the query handle from the thread if the client is closed. Testing: Simulated hitting an error condition in the main thread that initially triggered this condition. Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 --- M tests/custom_cluster/test_admission_controller.py 1 file changed, 7 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/17256/1 -- To view, visit http://gerrit.cloudera.org:8080/17256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8aa8315d9f598ba80d13cd2091e3cc743c64ba77 Gerrit-Change-Number: 17256 Gerrit-PatchSet: 1 Gerrit-Owner: Bikramjeet Vig
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17254 ) Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8481/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 18:56:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17254 Change subject: IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions .. IMPALA-9997/IMPALA-9998: Upgrade compression libraries to latest versions This updates several compression libraries to their latest versions: - Bzip2 1.0.8 - LZ4 1.9.3 - Snappy 1.1.8 - Zlib 1.2.11 - ZStd 1.4.9 Several of these have minor performance improvements. Testing: - Ran release exhaustive job and debug core job TODO: - Run performance job Change-Id: I858f82f773023bd0aea14543f18bd74071758468 --- M be/src/util/compress.cc M be/src/util/decompress.cc M bin/impala-config.sh 3 files changed, 6 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/17254/1 -- To view, visit http://gerrit.cloudera.org:8080/17254 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I858f82f773023bd0aea14543f18bd74071758468 Gerrit-Change-Number: 17254 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7031/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 31 Mar 2021 18:27:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Change default values for flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17253 ) Change subject: IMPALA-6671: Change default values for flags .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8480/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 18:18:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Change default values for flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17253 ) Change subject: IMPALA-6671: Change default values for flags .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8479/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 18:16:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Change default values for flags
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17253 ) Change subject: IMPALA-6671: Change default values for flags .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7030/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 17:58:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6671: Change default values for flags
Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17253 ) Change subject: IMPALA-6671: Change default values for flags .. IMPALA-6671: Change default values for flags This commit changes the default values of following flags which were introduced in the earlier patch for IMPALA-6671. The current default value of topic_update_tbl_max_wait_time_ms of 500ms is too low and may skip the locked tables more aggressively than needed. The new defaults were set based on analysis of a real world deployment. topic_update_tbl_max_wait_time_ms = 12 catalog_max_lock_skipped_topic_updates = 3 (or default) Testing [WIP] Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 --- M be/src/catalog/catalog-server.cc 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/53/17253/2 -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-6671: Change default values for flags
Vihang Karajgaonkar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17253 Change subject: IMPALA-6671: Change default values for flags .. IMPALA-6671: Change default values for flags This commit changes the default values of following flags which were introduced in the earlier patch for IMPALA-6671. The current default value of topic_update_tbl_max_wait_time_ms of 500ms is too low and may skip the locked tables more aggressively than needed. The new defaults were set based on analysis of a real world deployment. topic_update_tbl_max_wait_time_ms = 12 catalog_max_lock_skipped_topic_updates = 3 (or default) Testing [WIP] Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/53/17253/1 -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar
[Impala-ASF-CR] [WIP] IMPALA-10532: TestOverlapMinMaxFilters.test overlap min max filters seems flaky
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17252 ) Change subject: [WIP] IMPALA-10532: TestOverlapMinMaxFilters.test_overlap_min_max_filters seems flaky .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8478/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17252 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I326317833979efcbe02ce6c95ad80133dd5c7964 Gerrit-Change-Number: 17252 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 17:39:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8477/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 31 Mar 2021 17:34:25 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-10532: TestOverlapMinMaxFilters.test overlap min max filters seems flaky
Qifan Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17252 Change subject: [WIP] IMPALA-10532: TestOverlapMinMaxFilters.test_overlap_min_max_filters seems flaky .. [WIP] IMPALA-10532: TestOverlapMinMaxFilters.test_overlap_min_max_filters seems flaky Change-Id: I326317833979efcbe02ce6c95ad80133dd5c7964 --- M be/src/runtime/coordinator-filter-state.h M be/src/runtime/coordinator.cc 2 files changed, 101 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/17252/1 -- To view, visit http://gerrit.cloudera.org:8080/17252 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I326317833979efcbe02ce6c95ad80133dd5c7964 Gerrit-Change-Number: 17252 Gerrit-PatchSet: 1 Gerrit-Owner: Qifan Chen
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 3: Code-Review+2 Carry +2 -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 31 Mar 2021 17:13:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Hello Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17249 to look at the new patch set (#3). Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. IMPALA-10608 followup: Detect the virtualenv tarball version When rebasing from an older commit, the version change in virtualenv can cause there to be multiple virtualenv tarballs of different versions in the infra/python/deps directory. bootstrap_virtualenv.py currently doesn't handle this gracefully, because it is looking for all virtualenv*.tar.gz files and fails when it finds more than one. This changes bootstrap_virtualenv.py to get the virtualenv version from the requirements.txt file and only look for the tarball with that version. If it fails to get the version, it falls back to the old method. Testing: - Copied virtualenv-16.7.10.tar.gz to virtualenv-16.7.9.tar.gz and verified that bootstrap_virtualenv.py works Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 --- M infra/python/bootstrap_virtualenv.py 1 file changed, 25 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/17249/3 -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/17249/2/infra/python/bootstrap_virtualenv.py File infra/python/bootstrap_virtualenv.py: http://gerrit.cloudera.org:8080/#/c/17249/2/infra/python/bootstrap_virtualenv.py@87 PS2, Line 87: line[0] == '#': > the line could be stripped before doing this Good point, done http://gerrit.cloudera.org:8080/#/c/17249/2/infra/python/bootstrap_virtualenv.py@107 PS2, Line 107: virtualenv_tarball = \ > nit: +2 indentation Done -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 31 Mar 2021 17:13:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. Patch Set 3: > Patch Set 1: Code-Review+2 > > Should we have a "the minicluster starts" test in Jenkins that runs on 20.04? Good point. Did you mean it in the context of GVO or postcommit? In general, we should probably revisit the question of our runtime platform for the precommit tests on public Jenkins (there was an earlier discussion about migrating GVO & friends to Ubuntu 18.04) as well as how we handle additional (supported) OS platforms. I'll kick off a discussion on dev@, it is a better medium for a wider discussion. -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Wed, 31 Mar 2021 16:57:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 2: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/17249/2/infra/python/bootstrap_virtualenv.py File infra/python/bootstrap_virtualenv.py: http://gerrit.cloudera.org:8080/#/c/17249/2/infra/python/bootstrap_virtualenv.py@87 PS2, Line 87: line[0] == '#': the line could be stripped before doing this http://gerrit.cloudera.org:8080/#/c/17249/2/infra/python/bootstrap_virtualenv.py@107 PS2, Line 107: virtualenv_tarball = \ nit: +2 indentation -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 16:33:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7029/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 16:07:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8476/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 16:07:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17249 ) Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8475/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 16:03:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10624: TestIcebergTable::test alter iceberg tables failed by stale file format
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17248 ) Change subject: IMPALA-10624: TestIcebergTable::test_alter_iceberg_tables failed by stale file format .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8474/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17248 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iff8d5d1d90444aba11d47cfce522aaa45a4a74cc Gerrit-Change-Number: 17248 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 31 Mar 2021 16:00:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17242 ) Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7028/ -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 31 Mar 2021 15:56:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17249 to look at the new patch set (#2). Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. IMPALA-10608 followup: Detect the virtualenv tarball version When rebasing from an older commit, the version change in virtualenv can cause there to be multiple virtualenv tarballs of different versions in the infra/python/deps directory. bootstrap_virtualenv.py currently doesn't handle this gracefully, because it is looking for all virtualenv*.tar.gz files and fails when it finds more than one. This changes bootstrap_virtualenv.py to get the virtualenv version from the requirements.txt file and only look for the tarball with that version. If it fails to get the version, it falls back to the old method. Testing: - Copied virtualenv-16.7.10.tar.gz to virtualenv-16.7.9.tar.gz and verified that bootstrap_virtualenv.py works Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 --- M infra/python/bootstrap_virtualenv.py 1 file changed, 24 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/17249/2 -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] Bump up the GBN number to 11920537
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17243 ) Change subject: Bump up the GBN number to 11920537 .. Patch Set 1: Looked at the GVO failure. testAcidMinorCompactionLoading failed because our version of Hive doesn't write '_orc_acid_version' anymore. So I think we should just change the expected value to 8. I'm working on the fix for test_alter_iceberg_tables: https://gerrit.cloudera.org/#/c/17248/ -- To view, visit http://gerrit.cloudera.org:8080/17243 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If88ceeaffc94e5bedf2c9953122109e20663f743 Gerrit-Change-Number: 17243 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Wed, 31 Mar 2021 15:45:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10608 followup: Detect the virtualenv tarball version
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17249 Change subject: IMPALA-10608 followup: Detect the virtualenv tarball version .. IMPALA-10608 followup: Detect the virtualenv tarball version When rebasing from an older commit, the version change in virtualenv can cause there to be multiple virtualenv tarballs of different versions in the infra/python/deps directory. bootstrap_virtualenv.py currently doesn't handle this gracefully, because it is looking for all virtualenv*.tar.gz files and fails when it finds more than one. This changes bootstrap_virtualenv.py to get the virtualenv version from the requirements.txt file and only look for the tarball with that version. If it fails to get the version, it falls back to the old method. Testing: - Copied virtualenv-16.7.10.tar.gz to virtualenv-16.7.9.tar.gz and verified that bootstrap_virtualenv.py works Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 --- M infra/python/bootstrap_virtualenv.py 1 file changed, 24 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/17249/1 -- To view, visit http://gerrit.cloudera.org:8080/17249 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40 Gerrit-Change-Number: 17249 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-10624: TestIcebergTable::test alter iceberg tables failed by stale file format
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17248 Change subject: IMPALA-10624: TestIcebergTable::test_alter_iceberg_tables failed by stale file format .. IMPALA-10624: TestIcebergTable::test_alter_iceberg_tables failed by stale file format During table creation we pushed all the table properties to the Iceberg table, e.g. 'iceberg.file_format'. This works well with current Iceberg version, but causes bugs with newer versions. Newer versions of Iceberg refresh the HMS properties from the table properties for every table update. That means if we intially set a table property, we cannot modify it later because Iceberg will rewrite it with the old value on every table update. These table properties are not needed at the Iceberg table level, so we can just skip passing them during table creation. Testing: * ran existing tests * ran tests with newer version Iceberg * checked manually that we don't set these properties at the Iceberg table level Change-Id: Iff8d5d1d90444aba11d47cfce522aaa45a4a74cc --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java 2 files changed, 31 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17248/1 -- To view, visit http://gerrit.cloudera.org:8080/17248 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iff8d5d1d90444aba11d47cfce522aaa45a4a74cc Gerrit-Change-Number: 17248 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 Impala started adding Ubuntu 20.04 support in various places. This patch extends bootstrap_config.sh for Ubuntu 20.04 coverage: 1. The runtime version check error message is updated to claim support for Ubuntu 20.04. 2. Kudu needs libtinfo.5.so on Ubuntu 20.04 for the minicluster binaries. bin/bootstrap_system.sh now installs it when running on Ubuntu 20.04. 3. The OpenJDK default version reset to JDK 8 is extended to Ubuntu 20.04. Tested by running the code using docker/test-with-docker.py using --base-image=ubuntu:20.04 and observing that Kudu was able to start in the minicluster. The test runs completed, but there were test failures, for which separate tickets will be filed. Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Reviewed-on: http://gerrit.cloudera.org:8080/17240 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/bootstrap_system.sh 1 file changed, 6 insertions(+), 4 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Wed, 31 Mar 2021 14:24:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17242 ) Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 31 Mar 2021 10:14:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9331: Add symptom for dataload failing on schema mismatch
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17242 ) Change subject: IMPALA-9331: Add symptom for dataload failing on schema mismatch .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7028/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17242 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fa193740a2764fdda799d6a9cc64f89cab64aba Gerrit-Change-Number: 17242 Gerrit-PatchSet: 3 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 31 Mar 2021 10:14:01 + Gerrit-HasComments: No
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17246 ) Change subject: Only fetch needed branches in compare_branches.py .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/8473/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 31 Mar 2021 09:01:51 + Gerrit-HasComments: No
[Impala-ASF-CR] Only fetch needed branches in compare branches.py
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17246 Change subject: Only fetch needed branches in compare_branches.py .. Only fetch needed branches in compare_branches.py Fetching all branches of a remote repo will be time-consuming. This changes compare_branches.py to only fetch the needed branches, i.e. source_branch and target_branch. Tests: - Before this change, it can't finish in 30 mins when comparing two downstream branches. After this change, it finishes in one minute. Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 --- M bin/compare_branches.py 1 file changed, 6 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/17246/1 -- To view, visit http://gerrit.cloudera.org:8080/17246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia0c70ad4de1fa79498ca32853b6ea99aee2d40a7 Gerrit-Change-Number: 17246 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17171 ) Change subject: IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil .. IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil HDFS FileSystem provides a listStatusIterator() API for listing remote storage using a RemoteIterator. We use it to list files when loading table file metadata. It's not guaranteed that a RemoteIterator can survive when its hasNext() or next() throws IOExceptions. We should stop the loop in this case. Otherwise, we may go into a infinite loop. Without HADOOP-16685, it's also not guaranteed that FileSystem.listStatusIterator() will throw a FileNotFoundException when the path doesn't exist. This patch refactors the file listing iterators so we don't need to depend on these two assumptions. The basic idea is: - On one side, we should not depends on other RemoteIterator's behavior after exception. - On the other side, we try to make our own iterators more robust on transient sub-directories. So table loading won't be failed by them. Tests: - Loop test_insert_stress.py 100 times. Verified the non-existing subdirs are skipped and inserts are stable in a high concurrency. Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Reviewed-on: http://gerrit.cloudera.org:8080/17171 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java 1 file changed, 83 insertions(+), 32 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17171 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Gerrit-Change-Number: 17171 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17171 ) Change subject: IMPALA-10579: Fix usage of RemoteIterator in FileSystemUtil .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17171 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I859bd4f976c51a34eb6a03cefd2ddcdf11656cea Gerrit-Change-Number: 17171 Gerrit-PatchSet: 6 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 31 Mar 2021 08:45:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Wed, 31 Mar 2021 08:42:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10618: Update bootstrap system for Ubuntu 20.04
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17240 ) Change subject: IMPALA-10618: Update bootstrap_system for Ubuntu 20.04 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7027/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I212f6df3657cf9d621a0669573e1e511eae13662 Gerrit-Change-Number: 17240 Gerrit-PatchSet: 2 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Qifan Chen Gerrit-Comment-Date: Wed, 31 Mar 2021 08:42:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10619: Minor refactoring of standardize method for analytic functions
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17237 ) Change subject: IMPALA-10619: Minor refactoring of standardize method for analytic functions .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/17237/1/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java File fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java: http://gerrit.cloudera.org:8080/#/c/17237/1/fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java@833 PS1, Line 833: protected FunctionCallExpr createRewrittenFunction(FunctionName funcName, > This seems a static method for FunctionCallExpr. Nothing special for analyt Right..this was a natural and object-oriented way I could think of where a derived class of AnalyticExpr could override it. The external FE would create something like a new ExternalFEFunctionCallExpr (this is just an example name) which is a derived class of FunctionCallExpr. Making it a static method would prevent the override. I am open to suggestions and will try to think of alternatives. -- To view, visit http://gerrit.cloudera.org:8080/17237 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39e4268c0c5500f09acf98357a80763c28f615c2 Gerrit-Change-Number: 17237 Gerrit-PatchSet: 1 Gerrit-Owner: Aman Sinha Gerrit-Reviewer: Aman Sinha Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 31 Mar 2021 06:44:24 + Gerrit-HasComments: Yes
