[kudu] branch master updated: [tablet] don't call UpdateMetricsForOp() for rows with errors
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/master by this push:
new a2ba855 [tablet] don't call UpdateMetricsForOp() for rows with errors
a2ba855 is described below
commit a2ba855deaba72846362c95d3614bdb3dd282a87
Author: Alexey Serbin
AuthorDate: Mon Feb 28 10:16:28 2022 -0800
[tablet] don't call UpdateMetricsForOp() for rows with errors
This patch updates the control path in WriteOp::UpdatePerRowErrors()
to avoid calling UpdateMetricsForOp() for a row that contains an error
since that's effectively a no-op.
Also, renamed UpdatePerRowErrors() into UpdatePerRowMetricsAndErrors().
Change-Id: Ic1f57ee7d1b0064569a34ba93d35979426f76812
Reviewed-on: http://gerrit.cloudera.org:8080/18281
Tested-by: Alexey Serbin
Reviewed-by: Andrew Wong
---
src/kudu/tablet/ops/write_op.cc | 21 ++---
src/kudu/tablet/ops/write_op.h | 7 +--
2 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/src/kudu/tablet/ops/write_op.cc b/src/kudu/tablet/ops/write_op.cc
index 83097f9..7f1defb 100644
--- a/src/kudu/tablet/ops/write_op.cc
+++ b/src/kudu/tablet/ops/write_op.cc
@@ -243,19 +243,20 @@ Status WriteOp::Start() {
return Status::OK();
}
-void WriteOp::UpdatePerRowErrors() {
- // Add per-row errors to the result, update metrics.
- for (int i = 0; i < state()->row_ops().size(); ++i) {
-const RowOp* op = state()->row_ops()[i];
+void WriteOp::UpdatePerRowMetricsAndErrors() {
+ // Update metrics or add per-row errors to the result.
+ size_t idx = 0;
+ for (const auto* op : state()->row_ops()) {
if (op->result->has_failed_status()) {
// Replicas disregard the per row errors, for now
// TODO(unknown): check the per-row errors against the leader's, at
least in debug mode
WriteResponsePB::PerRowErrorPB* error =
state()->response()->add_per_row_errors();
- error->set_row_index(i);
+ error->set_row_index(idx);
error->mutable_error()->CopyFrom(op->result->failed_status());
+} else {
+ state()->UpdateMetricsForOp(*op);
}
-
-state()->UpdateMetricsForOp(*op);
+++idx;
}
}
@@ -276,7 +277,7 @@ Status WriteOp::Apply(CommitMsg** commit_msg) {
RETURN_NOT_OK(tablet->ApplyRowOperations(state()));
TRACE("APPLY: Finished.");
- UpdatePerRowErrors();
+ UpdatePerRowMetricsAndErrors();
// Create the Commit message
*commit_msg =
google::protobuf::Arena::CreateMessage(state_->pb_arena());
@@ -477,9 +478,7 @@ void WriteOpState::ReleaseTxResultPB(TxResultPB* result)
const {
}
void WriteOpState::UpdateMetricsForOp(const RowOp& op) {
- if (op.result->has_failed_status()) {
-return;
- }
+ DCHECK(!op.result->has_failed_status());
switch (op.decoded_op.type) {
case RowOperationsPB::INSERT:
DCHECK(!op.error_ignored);
diff --git a/src/kudu/tablet/ops/write_op.h b/src/kudu/tablet/ops/write_op.h
index c8621fa..8c14176 100644
--- a/src/kudu/tablet/ops/write_op.h
+++ b/src/kudu/tablet/ops/write_op.h
@@ -29,7 +29,7 @@
#include
#include "kudu/common/row_operations.h"
-#include "kudu/common/wire_protocol.pb.h"
+#include "kudu/common/row_operations.pb.h"
#include "kudu/consensus/consensus.pb.h"
#include "kudu/gutil/macros.h"
#include "kudu/gutil/ref_counted.h"
@@ -385,7 +385,10 @@ class WriteOp : public Op {
std::string ToString() const override;
private:
- void UpdatePerRowErrors();
+ // For each row of this write operation, update corresponding metrics or set
+ // corresponding error information in the response. The former is for
+ // successfully written rows, the latter is for failed ones.
+ void UpdatePerRowMetricsAndErrors();
// this op's start time
MonoTime start_time_;
[kudu] branch master updated: [docs] add blurb about automatic master addition
This is an automated email from the ASF dual-hosted git repository. awong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/kudu.git The following commit(s) were added to refs/heads/master by this push: new 0335f3e [docs] add blurb about automatic master addition 0335f3e is described below commit 0335f3ea6356011ac88c25a9a353b4fbec952a42 Author: Andrew Wong AuthorDate: Fri Feb 25 13:54:37 2022 -0800 [docs] add blurb about automatic master addition The steps are updated in the latest version of Kudu, and this patch adds a note indicating to readers that they do not need to go through the administrative docs. Change-Id: I15ada3ce0a4319a1e1b5340bdfb7a261ef260f8d Reviewed-on: http://gerrit.cloudera.org:8080/18278 Tested-by: Kudu Jenkins Reviewed-by: Attila Bukor --- docs/administration.adoc | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/administration.adoc b/docs/administration.adoc index 113c1c3..2ba1525 100644 --- a/docs/administration.adoc +++ b/docs/administration.adoc @@ -521,6 +521,13 @@ or because Kudu multi-master support was still experimental at the time. This wo how to migrate to a multi-master configuration. It can also be used to migrate from two masters to three, with straightforward modifications. Note that the number of masters must be odd. +NOTE: From Kudu version 1.16.0 onwards, new Kudu Masters automatically attempt to contact other +masters, determine whether a quorum has already been established, and if so, attempt to join the +quorum. The below steps do not need to be performed in these versions. + +NOTE: From Kudu version 1.15.0 onwards, the `kudu master add` command has been added that simplifies +the orchestration to migrate an existing Kudu cluster to multiple masters. + WARNING: The workflow is unsafe for adding new masters to an existing configuration that already has three or more masters. Do not use it for that purpose. @@ -534,9 +541,6 @@ WARNING: The workflow presupposes at least basic familiarity with Kudu configura using vendor-specific tools the workflow also presupposes familiarity with it and the vendor's instructions should be used instead as details may differ. -NOTE: From Kudu version 1.15.0, a new `kudu master add` command has been added that -simplifies the orchestration to migrate an existing Kudu cluster to multiple masters. - Prepare for the migration . Establish a maintenance window (one hour should be sufficient). During this time the Kudu cluster
[kudu] branch branch-1.16.x updated: [docs] add blurb about automatic master addition
This is an automated email from the ASF dual-hosted git repository. awong pushed a commit to branch branch-1.16.x in repository https://gitbox.apache.org/repos/asf/kudu.git The following commit(s) were added to refs/heads/branch-1.16.x by this push: new baa5c8b [docs] add blurb about automatic master addition baa5c8b is described below commit baa5c8b23c1d49b48022337a082fc1c69c81d3e0 Author: Andrew Wong AuthorDate: Fri Feb 25 13:54:37 2022 -0800 [docs] add blurb about automatic master addition The steps are updated in the latest version of Kudu, and this patch adds a note indicating to readers that they do not need to go through the administrative docs. Change-Id: I15ada3ce0a4319a1e1b5340bdfb7a261ef260f8d Reviewed-on: http://gerrit.cloudera.org:8080/18278 Tested-by: Kudu Jenkins Reviewed-by: Attila Bukor (cherry picked from commit cb016a09994f4b6cf89e62f76b5a5b4bc2a0a3ad) Reviewed-on: http://gerrit.cloudera.org:8080/18279 --- docs/administration.adoc | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/administration.adoc b/docs/administration.adoc index 456665f..995fb12 100644 --- a/docs/administration.adoc +++ b/docs/administration.adoc @@ -515,6 +515,13 @@ or because Kudu multi-master support was still experimental at the time. This wo how to migrate to a multi-master configuration. It can also be used to migrate from two masters to three, with straightforward modifications. Note that the number of masters must be odd. +NOTE: From Kudu version 1.16.0 onwards, new Kudu Masters automatically attempt to contact other +masters, determine whether a quorum has already been established, and if so, attempt to join the +quorum. The below steps do not need to be performed in these versions. + +NOTE: From Kudu version 1.15.0 onwards, the `kudu master add` command has been added that simplifies +the orchestration to migrate an existing Kudu cluster to multiple masters. + WARNING: The workflow is unsafe for adding new masters to an existing configuration that already has three or more masters. Do not use it for that purpose. @@ -528,9 +535,6 @@ WARNING: The workflow presupposes at least basic familiarity with Kudu configura using vendor-specific tools the workflow also presupposes familiarity with it and the vendor's instructions should be used instead as details may differ. -NOTE: From Kudu version 1.15.0, a new `kudu master add` command has been added that -simplifies the orchestration to migrate an existing Kudu cluster to multiple masters. - Prepare for the migration . Establish a maintenance window (one hour should be sufficient). During this time the Kudu cluster
