[kudu-CR] [compaction] support turn on/off FLAGS enable maintenance manager at runtime
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/19398 ) Change subject: [compaction] support turn on/off FLAGS_enable_maintenance_manager at runtime .. Patch Set 4: (1 comment) > Patch Set 3: > > (1 comment) > > The test failure in TSAN mode seems related. Your advices are very useful, I fix it again. Thanks a lot. http://gerrit.cloudera.org:8080/#/c/19398/3/src/kudu/util/maintenance_manager.cc File src/kudu/util/maintenance_manager.cc: http://gerrit.cloudera.org:8080/#/c/19398/3/src/kudu/util/maintenance_manager.cc@311 PS3, Line 311: MaintenanceOp* op = nullptr; > Seems we need to acquire the 'lock_' before using 'shutdown_' in this place You are right -- To view, visit http://gerrit.cloudera.org:8080/19398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7f7029b22a4c8ce58094501e71a6c22271d4f0b2 Gerrit-Change-Number: 19398 Gerrit-PatchSet: 4 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 06 Feb 2023 06:42:48 + Gerrit-HasComments: Yes
[kudu-CR] [compaction] support turn on/off FLAGS enable maintenance manager at runtime
Hello Tidy Bot, Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19398 to look at the new patch set (#4). Change subject: [compaction] support turn on/off FLAGS_enable_maintenance_manager at runtime .. [compaction] support turn on/off FLAGS_enable_maintenance_manager at runtime Now the gflags FLAGS_enable_maintenance_manager can be on/off, it should set before start. So if we want to change it, we must restart tservers. But restarting a tablet server can sometimes be quite time consuming due to various factors. This patch adds ability to change FLAGS_enable_maintenance_manager at runtime, that can help in avoiding slow restart operations caused due to external factors. Change-Id: I7f7029b22a4c8ce58094501e71a6c22271d4f0b2 --- M src/kudu/util/maintenance_manager.cc 1 file changed, 13 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/19398/4 -- To view, visit http://gerrit.cloudera.org:8080/19398 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7f7029b22a4c8ce58094501e71a6c22271d4f0b2 Gerrit-Change-Number: 19398 Gerrit-PatchSet: 4 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [clock] add sanity check to detect wall clock jumps
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/19473 ) Change subject: [clock] add sanity check to detect wall clock jumps .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/19473/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19473/1//COMMIT_MSG@23 PS1, Line 23: condition nit: condition is http://gerrit.cloudera.org:8080/#/c/19473/1//COMMIT_MSG@40 PS1, Line 40: However, if a problem is observed at some inadequate VMs such : as ones in Azure Cloud, it's now possible to enable the guardrail : to detect such an issue. There are chances that this clock issue may go unnoticed unless there is a specific monitoring for such an error going on. If the clock error goes unnoticed (probably that's what happened in this case as well) and wrong clock timestamp persists and gets propagated to replicas, how would enabling the flag help after that? http://gerrit.cloudera.org:8080/#/c/19473/1//COMMIT_MSG@46 PS1, Line 46: opeations nit: operations http://gerrit.cloudera.org:8080/#/c/19473/1/src/kudu/clock/hybrid_clock.h File src/kudu/clock/hybrid_clock.h: http://gerrit.cloudera.org:8080/#/c/19473/1/src/kudu/clock/hybrid_clock.h@238 PS1, Line 238: captured nit: Did you mean "captured at" ? http://gerrit.cloudera.org:8080/#/c/19473/1/src/kudu/clock/hybrid_clock.cc File src/kudu/clock/hybrid_clock.cc: http://gerrit.cloudera.org:8080/#/c/19473/1/src/kudu/clock/hybrid_clock.cc@165 PS1, Line 165: 15 * 60 Any specific reason to choose 15 minutes of max allowed divergence? -- To view, visit http://gerrit.cloudera.org:8080/19473 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I630783653717d975a9b2ad668e8bd47b7796d275 Gerrit-Change-Number: 19473 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 06 Feb 2023 06:21:41 + Gerrit-HasComments: Yes
[kudu-CR] [web] add maintenance op statistics information on web pages for data retained bytes
Yifan Zhang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19407 ) Change subject: [web] add maintenance op statistics information on web pages for data_retained_bytes .. [web] add maintenance op statistics information on web pages for data_retained_bytes MaintenanceOpStats collects statistics about 'data_retained_bytes_', which means amount of disk capacity in data blocks that would be freed if current maintenence operation ran. But the protobuf file has no such information, so it is not shown on web pages. This patch adds a field in the protobuf file and updates the embedded web server to show the retained bytes for a maintenance operation. Sample screenshot of the maintenance manager page can be found here: https://github.com/shenxingwuying/kudu/blob/dev/flyline/screenshot/data_retained.png Change-Id: Iac8f0307705d57cea48901102b170c88d73d8c2e Reviewed-on: http://gerrit.cloudera.org:8080/19407 Tested-by: Kudu Jenkins Reviewed-by: Yifan Zhang --- M src/kudu/integration-tests/webserver-stress-itest.cc M src/kudu/tserver/tserver_path_handlers.cc M src/kudu/util/maintenance_manager.cc M src/kudu/util/maintenance_manager.proto M www/maintenance-manager.mustache 5 files changed, 28 insertions(+), 1 deletion(-) Approvals: Kudu Jenkins: Verified Yifan Zhang: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/19407 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iac8f0307705d57cea48901102b170c88d73d8c2e Gerrit-Change-Number: 19407 Gerrit-PatchSet: 8 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [web] add maintenance op statistics information on web pages for data retained bytes
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19407 ) Change subject: [web] add maintenance op statistics information on web pages for data_retained_bytes .. Patch Set 7: > Patch Set 7: > > (1 comment) Just ignore my comment. Thanks for your contribution! -- To view, visit http://gerrit.cloudera.org:8080/19407 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac8f0307705d57cea48901102b170c88d73d8c2e Gerrit-Change-Number: 19407 Gerrit-PatchSet: 7 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 06 Feb 2023 02:03:25 + Gerrit-HasComments: No
[kudu-CR] [web] add maintenance op statistics information on web pages for data retained bytes
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19407 ) Change subject: [web] add maintenance op statistics information on web pages for data_retained_bytes .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/19407/7/src/kudu/integration-tests/webserver-stress-itest.cc File src/kudu/integration-tests/webserver-stress-itest.cc: http://gerrit.cloudera.org:8080/#/c/19407/7/src/kudu/integration-tests/webserver-stress-itest.cc@108 PS7, Line 108: data_retained nit: This should be "Data retained". -- To view, visit http://gerrit.cloudera.org:8080/19407 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac8f0307705d57cea48901102b170c88d73d8c2e Gerrit-Change-Number: 19407 Gerrit-PatchSet: 7 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 06 Feb 2023 01:46:25 + Gerrit-HasComments: Yes
[kudu-CR] [web] add maintenance op statistics information on web pages for data retained bytes
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19407 ) Change subject: [web] add maintenance op statistics information on web pages for data_retained_bytes .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19407 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iac8f0307705d57cea48901102b170c88d73d8c2e Gerrit-Change-Number: 19407 Gerrit-PatchSet: 7 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 06 Feb 2023 01:17:35 + Gerrit-HasComments: No
