[kudu-CR] [metrics] Add metrics for tablet copy op time
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tablet/tablet_metrics.cc File src/kudu/tablet/tablet_metrics.cc: http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tablet/tablet_metrics.cc@231 PS5, Line 231: 6LU While the unit is kMicroseconds, so the 'max_val' maybe 60,000,000, or even larger. A tablet disk space is possible to be dozens of GiB, copy a tablet may consume dozens of minitues, so we can define the 'max_val' to several hours. http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc File src/kudu/tserver/tablet_copy_client-test.cc: http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc@336 PS5, Line 336: ASSERT_OK(ResetTabletCopyClient(tablet_replica_->tablet()->metrics())); : auto copy_run_time = : tablet_replica_->tablet()->metrics()->tablet_copy_duration; : const auto before_cnt = copy_run_time->TotalCount(); Can we define these in SetUp() ? http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc@345 PS5, Line 345: client_.reset(); Can we define it in TearDown() ? http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc File src/kudu/tserver/tablet_copy_client.cc: http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc@561 PS5, Line 561: int64_t dur = (GetMonoTimeMicros() - start_time_micros_) / 1000; This metric is in microsecond unit [1], don't divide by 1000. 1. https://gerrit.cloudera.org/c/21356/5/src/kudu/tablet/tablet_metrics.cc#228 http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc@605 PS5, Line 605: int64_t dur = (GetMonoTimeMicros() - start_time_micros_) / 1000; Ditto. -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 29 Apr 2024 03:55:44 + Gerrit-HasComments: Yes
[kudu-CR] Minor refactoring on Op class
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/21342 ) Change subject: Minor refactoring on Op class .. Patch Set 3: Verified+1 Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/21342 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie391d4a55b8da08a62025a05cc466fc2b947099c Gerrit-Change-Number: 21342 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 29 Apr 2024 03:09:15 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) KUDU-3569 fix race in CFileSet::Iterator::OptimizePKPredicates()
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21366 ) Change subject: KUDU-3569 fix race in CFileSet::Iterator::OptimizePKPredicates() .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/21366/1/src/kudu/tablet/cfile_set.cc File src/kudu/tablet/cfile_set.cc: http://gerrit.cloudera.org:8080/#/c/21366/1/src/kudu/tablet/cfile_set.cc@a434 PS1, Line 434: > It seems using 'tablet_schema()' will cause data racing problem. In src/kud Using tablet_schema() that returns a smart pointer doesn't produce a race condition automatically. DumpRowSetInternal() in kudu CLI tools is using a smart pointer while accessing the underlying Schema's data. No race is possible there: Schema key_proj = rs_meta->tablet_schema()->CreateKeyProjection(); -- To view, visit http://gerrit.cloudera.org:8080/21366 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I92daa74cb86a77a4350f42db9ca5dec3a0d4ff75 Gerrit-Change-Number: 21366 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Comment-Date: Sun, 28 Apr 2024 15:34:51 + Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#5). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/5 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#4). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/4 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#3). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/3 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 5: Build Started http://jenkins.kudu.apache.org/job/pre_commit/56/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:54:17 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 4: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/55/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:54:12 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 4: Build Started http://jenkins.kudu.apache.org/job/pre_commit/55/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:51:52 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 3: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/54/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:51:43 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 3: Build Started http://jenkins.kudu.apache.org/job/pre_commit/54/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:50:57 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 2: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/53/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:50:52 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#2). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/2 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 1: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/43/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 1 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 09:13:24 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 2: Build Started http://jenkins.kudu.apache.org/job/pre_commit/53/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 09:13:32 + Gerrit-HasComments: No
