[kudu-CR] [KUDU-3452] Make validate tablet creating task not affected
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#20). Change subject: [KUDU-3452] Make validate tablet creating task not affected .. [KUDU-3452] Make validate tablet creating task not affected Currently, creating a table with RF=n when the number of healthy tservers is less than n will get stuck. Because catalog manager creates tablets for it will fail and retry continuously. At the same time, creating a table with RF=m also will get stuck even if there are more than m healthy tservers. Because catalog manager will return when finds a tablet-creating task failed and will not try to select replicas for other PREPARING tablets. For example, creating a three replicas table times out when one of three tablet servers becomes unavailable. After that, creating a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table-creating task will be affected by invalidate three-replicas table-creating task. This patch fixes this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/integration-tests/create-table-itest.cc M src/kudu/master/catalog_manager.cc 2 files changed, 99 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/20 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 20 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] KUDU-3452 Allow creating tablets under replicated tables
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19571 to look at the new patch set (#22). Change subject: KUDU-3452 Allow creating tablets under replicated tables .. KUDU-3452 Allow creating tablets under replicated tables Currently, creating a three-replicas table when there are not less than 3 healthy tablet servers will fail and the system catalog will retry continuously and always fail until the unavailable tablet servers becomes healthy again. An under-replicated table is still available for reading and writing, so it's enough to place just a majority of replicas for each tablet at healthy tablet servers to make a newly created table ready to use. This patch adds a new flag: --allow_creating_under_replicated_tables to support this feature. The original logic is kept the same. When this flag is set true, it's possible to create a tablet placing just a majority of replicas at healthy tablet servers. Even if the new tablet is created under-replicated, it's still available for read and write operations. Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 --- M src/kudu/integration-tests/create-table-itest.cc M src/kudu/master/catalog_manager.cc 2 files changed, 198 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/19571/22 -- To view, visit http://gerrit.cloudera.org:8080/19571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19571 Gerrit-PatchSet: 22 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [tests] fix flakiness in TestLogCleanupOnStartup
KeDeng has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19871 Change subject: [tests] fix flakiness in TestLogCleanupOnStartup .. [tests] fix flakiness in TestLogCleanupOnStartup I noticed the TestLogCleanupOnStartup scenario of the LogRollingITest was a bit flaky. This case is used to confirm whether the periodic deletion of the limited number of log files is effective. However, considering that the previous scheduling cycle was 60 seconds, we usually cannot wait until the next scheduling, which can cause the unit test case to fail as follows: ` COULD NOT CREATE A LOGGINGFILE 20230510-110713.1877!I20230510 11:07:14.550666 1226 external_mini_cluster.cc:1362] Started /data/github-kudu/tmp-review/kudu/build/release/bin/kudu as pid 1877 /data/github-kudu/tmp-review/kudu/src/kudu/integration-tests/log-rolling-itest.cc:82: Failure Expected equality of these values: std::min(3, i) Which is: 3 CountInfoLogs(cluster.master()->log_dir()) Which is: 4 /data/github-kudu/tmp-review/kudu/src/kudu/util/test_util.cc:392: Failure Failed Timed out waiting for assertion to pass. ` I have added a new flag '--excess_log_file_deleter_execute_interval_secs' to facilitate the control of the scheduling cycle, so that we can ensure that the deletion operation is scheduled during testing. Change-Id: I6ee02316fb0850a52251f3740309213b69ca79b8 --- M src/kudu/integration-tests/log-rolling-itest.cc M src/kudu/server/server_base.cc 2 files changed, 12 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/19871/1 -- To view, visit http://gerrit.cloudera.org:8080/19871 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6ee02316fb0850a52251f3740309213b69ca79b8 Gerrit-Change-Number: 19871 Gerrit-PatchSet: 1 Gerrit-Owner: KeDeng
[kudu-CR] Change order of operations during InitCatalogManagerTask
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19639 to look at the new patch set (#3). Change subject: Change order of operations during InitCatalogManagerTask .. Change order of operations during InitCatalogManagerTask In MasterStartupTest. StartupWebPage test, WaitForCatalogManagerInit is executed, which waits until catalog_manager_init_status_ is set. It is set by InitCatalogManagerTask and with the previous order of operations it was possible (but very rare) that catalog_manager_init_status_ was set but the timer was not stopped at the time of the startup http request, which resulted an output where initialize_master_catalog_status was 0. With this order initialize_master_catalog_progress_ is always set if WaitForCatalogManagerInit is finished. Change-Id: I64c980a8b357c99d4431a02d58b3c03a60bae1ad --- M src/kudu/master/master.cc 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/39/19639/3 -- To view, visit http://gerrit.cloudera.org:8080/19639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64c980a8b357c99d4431a02d58b3c03a60bae1ad Gerrit-Change-Number: 19639 Gerrit-PatchSet: 3 Gerrit-Owner: Ádám Bakai Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3448 Move password retrieval to master init
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/19873 to review the following change. Change subject: KUDU-3448 Move password retrieval to master init .. KUDU-3448 Move password retrieval to master init Apparently, forks can be expensive, at least on some systems, which makes fetching the password to encrypt/decrypt key materials whenever a TSK or IPKI is generated or loaded into memory not ideal as it may cause the server process to hang. This patch moves the password retrieval to the master initialization phase and the passwords are now stored in memory for later use. Change-Id: I746e657ae8d295f5f34225d63686beea1dff6b7c --- M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.cc M src/kudu/security/token-test.cc M src/kudu/security/token_signer.cc M src/kudu/security/token_signer.h M src/kudu/security/token_signing_key.cc M src/kudu/security/token_signing_key.h 8 files changed, 78 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/19873/1 -- To view, visit http://gerrit.cloudera.org:8080/19873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I746e657ae8d295f5f34225d63686beea1dff6b7c Gerrit-Change-Number: 19873 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Zoltan Chovan
[kudu-CR] KUDU-3448 Add support for encrypting TSKs
Hello Mahesh Reddy, Marton Greber, Zoltan Chovan, Alexey Serbin, Ashwani Raina, Kudu Jenkins, Abhishek Chennaka, Ádám Bakai, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19617 to look at the new patch set (#7). Change subject: KUDU-3448 Add support for encrypting TSKs .. KUDU-3448 Add support for encrypting TSKs In a previous patch, support for encrypting IPKI root CA private keys has been added. This is a follow up patch, to add encryption support for token signing keys as well. It is controlled by a new flag: --tsk_private_key_password_cmd. If this flag is set, the token signing keys will be stored in the syscatalog table in encrypted form (AES-256-CBC with PKCS#8 encoding). Token signing keys rotate automatically in Kudu, but for now, at least, encryption of TSKs can't be turned on or off on an existing master, so if this flag is set on the first startup of a master, it must be set to a command that outputs the same password as on initialization, and vice versa, it must not be provided on later master startups if it wasn't provided on initialization. Change-Id: Id8d770de7ed824cfc725003bbe77f1e42629029b --- M src/kudu/integration-tests/catalog_manager_tsk-itest.cc M src/kudu/master/master-test.cc M src/kudu/security/token_signing_key.cc 3 files changed, 35 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/19617/7 -- To view, visit http://gerrit.cloudera.org:8080/19617 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id8d770de7ed824cfc725003bbe77f1e42629029b Gerrit-Change-Number: 19617 Gerrit-PatchSet: 7 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: Ádám Bakai
[kudu-CR] KUDU-3448 Move password retrieval to master init
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19873 to look at the new patch set (#2). Change subject: KUDU-3448 Move password retrieval to master init .. KUDU-3448 Move password retrieval to master init Apparently, forks can be expensive, at least on some systems, which makes fetching the password to encrypt/decrypt key materials whenever a TSK or IPKI is generated or loaded into memory not ideal as it may cause the server process to hang. This patch moves the password retrieval to the master initialization phase and the passwords are now stored in memory for later use. Change-Id: I746e657ae8d295f5f34225d63686beea1dff6b7c --- M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.cc M src/kudu/security/token-test.cc M src/kudu/security/token_signer.cc M src/kudu/security/token_signer.h M src/kudu/security/token_signing_key.cc M src/kudu/security/token_signing_key.h 8 files changed, 77 insertions(+), 39 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/19873/2 -- To view, visit http://gerrit.cloudera.org:8080/19873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I746e657ae8d295f5f34225d63686beea1dff6b7c Gerrit-Change-Number: 19873 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Zoltan Chovan
[kudu-CR] KUDU-3448 Add support for encrypting TSKs
Attila Bukor has posted comments on this change. (
http://gerrit.cloudera.org:8080/19617 )
Change subject: KUDU-3448 Add support for encrypting TSKs
..
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19617/6/src/kudu/security/token_signing_key.cc
File src/kudu/security/token_signing_key.cc:
http://gerrit.cloudera.org:8080/#/c/19617/6/src/kudu/security/token_signing_key.cc@98
PS6, Line 98: CHECK_OK(key_->ToEncryptedString(&private_key_der_,
DataFormat::DER,
: [&](string* password){
: RETURN_NOT_OK_PREPEND(GetPasswordFromShellCommand(
: FLAGS_tsk_private_key_password_cmd, password),
: "could not get TSK private key password from
configured command");
: return Status::OK();
: }
: ));
> Thanks!
Posted a follow-up patch to address this issue.
--
To view, visit http://gerrit.cloudera.org:8080/19617
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id8d770de7ed824cfc725003bbe77f1e42629029b
Gerrit-Change-Number: 19617
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Bukor
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy
Gerrit-Reviewer: Marton Greber
Gerrit-Reviewer: Zoltan Chovan
Gerrit-Reviewer: Ádám Bakai
Gerrit-Comment-Date: Thu, 11 May 2023 15:38:46 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1945 Update docs with non-unique PK
Abhishek Chennaka has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19809 ) Change subject: KUDU-1945 Update docs with non-unique PK .. KUDU-1945 Update docs with non-unique PK Added small update to cover non-unique primary key. For further info, I added a link to the examples folder. Right now we only have the C++ example in place for non-unique PK, I plan to translate that example to Java and Python as well. Change-Id: I84e1c6b85d4fdb5ac95bad611246c071a63bcd31 Reviewed-on: http://gerrit.cloudera.org:8080/19809 Tested-by: Kudu Jenkins Reviewed-by: Wenzhe Zhou Reviewed-by: Abhishek Chennaka --- M docs/administration.adoc M docs/schema_design.adoc 2 files changed, 40 insertions(+), 1 deletion(-) Approvals: Kudu Jenkins: Verified Wenzhe Zhou: Looks good to me, but someone else must approve Abhishek Chennaka: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/19809 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I84e1c6b85d4fdb5ac95bad611246c071a63bcd31 Gerrit-Change-Number: 19809 Gerrit-PatchSet: 7 Gerrit-Owner: Marton Greber Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR](gh-pages) KUDU-1945 Update faq with non-unique PK
Abhishek Chennaka has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19860 ) Change subject: KUDU-1945 Update faq with non-unique PK .. KUDU-1945 Update faq with non-unique PK Change-Id: I3e588ffb84907deed7cfc8d6be088b09f2b54322 Reviewed-on: http://gerrit.cloudera.org:8080/19860 Reviewed-by: Abhishek Chennaka Reviewed-by: Wenzhe Zhou Tested-by: Abhishek Chennaka --- M faq.md 1 file changed, 6 insertions(+), 4 deletions(-) Approvals: Abhishek Chennaka: Looks good to me, approved; Verified Wenzhe Zhou: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/19860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: merged Gerrit-Change-Id: I3e588ffb84907deed7cfc8d6be088b09f2b54322 Gerrit-Change-Number: 19860 Gerrit-PatchSet: 4 Gerrit-Owner: Marton Greber Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yingchun Lai
[kudu-CR](gh-pages) KUDU-1945 Update faq with non-unique PK
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/19860 ) Change subject: KUDU-1945 Update faq with non-unique PK .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19860 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I3e588ffb84907deed7cfc8d6be088b09f2b54322 Gerrit-Change-Number: 19860 Gerrit-PatchSet: 3 Gerrit-Owner: Marton Greber Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 11 May 2023 15:47:46 + Gerrit-HasComments: No
[kudu-CR] KUDU-1945 Add Python example for non-unique PK
Marton Greber has uploaded this change for review. (
http://gerrit.cloudera.org:8080/19874
Change subject: KUDU-1945 Add Python example for non-unique PK
..
KUDU-1945 Add Python example for non-unique PK
In commit 3019848d00ac721b30d98a1aeb77bc205352b7b5 a C++ client example
has been added to showcase the usage of the non-unique PK. This patch
translates this particular example to Python.
The C++ examples are tested during upstream submission, however the Java
and Python client examples are not tested automatically. Therefore,
I did manual testing and created a ticket to track this improvement:
KUDU-3478.
Manual testing has been done using the following configurations:
Ubuntu 18.04 x86_64: Python2.7
Ubuntu 18.04 x86_64: Python3.7
macOS Monterey M1(arm64): Python3.7
(I couldn't setup the Python2.7 interpreter properly on my mac)
STDOUT of the example:
kudu.Schema {
non_unique_keyint32 NOT NULL
auto_incrementing_id int64 NOT NULL
int_val int32 NOT NULL
PRIMARY KEY (non_unique_key, auto_incrementing_id)
}
Demonstrating scanning ...
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 1)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 2)
Scanned some row(s) WHERE non_unique_key = 1
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 2)
Demonstrating UPDATE ...
Updated row(s) WHERE non_unique_key = 1 AND int_val = 2 to int_val = 98
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 1)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 2)
Updated row(s) WHERE non_unique_key = 2 to int_val = 99
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Updated row(s) WHERE non_unique_key = 2 AND auto_incrementing_id = 5\
to int_val = 100
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 100)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Demonstrating DELETE ...
Deleted row(s) WHERE non_unique_key = 3 AND int_val = 1
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 100)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Deleted row(s) WHERE non_unique_key = 2
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
Deleted row(s) WHERE non_unique_key = 3 AND auto_incrementing_id = 3
(non_unique_key: 3, auto_incrementing_id: 1, int_v
[kudu-CR] KUDU-1945 Add Python example for non-unique PK
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19874
to look at the new patch set (#2).
Change subject: KUDU-1945 Add Python example for non-unique PK
..
KUDU-1945 Add Python example for non-unique PK
In commit 3019848d00ac721b30d98a1aeb77bc205352b7b5 a C++ client example
has been added to showcase the usage of the non-unique PK. This patch
translates this particular example to Python.
The C++ examples are tested during upstream submission, however the Java
and Python client examples are not tested automatically. Therefore,
I did manual testing and created a ticket to track this improvement:
KUDU-3478.
Manual testing has been done using the following configurations:
Ubuntu 18.04 x86_64: Python2.7
Ubuntu 18.04 x86_64: Python3.7
macOS Monterey M1(arm64): Python3.7
(I couldn't setup the Python2.7 interpreter properly on my mac)
STDOUT of the example:
kudu.Schema {
non_unique_keyint32 NOT NULL
auto_incrementing_id int64 NOT NULL
int_val int32 NOT NULL
PRIMARY KEY (non_unique_key, auto_incrementing_id)
}
Demonstrating scanning ...
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 1)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 2)
Scanned some row(s) WHERE non_unique_key = 1
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 2)
Demonstrating UPDATE ...
Updated row(s) WHERE non_unique_key = 1 AND int_val = 2 to int_val = 98
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 1)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 2)
Updated row(s) WHERE non_unique_key = 2 to int_val = 99
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Updated row(s) WHERE non_unique_key = 2 AND auto_incrementing_id = 5\
to int_val = 100
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 100)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Demonstrating DELETE ...
Deleted row(s) WHERE non_unique_key = 3 AND int_val = 1
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 100)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Deleted row(s) WHERE non_unique_key = 2
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
Deleted row(s) WHERE non_unique_key = 3 AND auto_incrementing_i
[kudu-CR] KUDU-3452 Allow creating tablets under replicated tables
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19571 ) Change subject: KUDU-3452 Allow creating tablets under replicated tables .. Patch Set 22: (3 comments) http://gerrit.cloudera.org:8080/#/c/19571/22/src/kudu/integration-tests/create-table-itest.cc File src/kudu/integration-tests/create-table-itest.cc: http://gerrit.cloudera.org:8080/#/c/19571/22/src/kudu/integration-tests/create-table-itest.cc@647 PS22, Line 647: constexpr int kTSUnresponsiveTimeoutMs = 2000; : constexpr int kHeartbeatIntervalMs = 3000; Why is the heartbeat interval even longer than tserver unresponsive interval? In that case, tservers could be considered unavailable between heartbeats. http://gerrit.cloudera.org:8080/#/c/19571/17/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: http://gerrit.cloudera.org:8080/#/c/19571/17/src/kudu/master/catalog_manager.cc@6664 PS17, Line 6664: } : > Done Seems this comment was not resolved in patch set 22. http://gerrit.cloudera.org:8080/#/c/19571/17/src/kudu/master/catalog_manager.cc@6677 PS17, Line 6677: , MasterErrorPB::REPLI > Done ditto. -- To view, visit http://gerrit.cloudera.org:8080/19571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19571 Gerrit-PatchSet: 22 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 11 May 2023 16:39:48 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate tablet creating task not affected
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19594 ) Change subject: [KUDU-3452] Make validate tablet creating task not affected .. Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/19594/20/src/kudu/integration-tests/create-table-itest.cc File src/kudu/integration-tests/create-table-itest.cc: http://gerrit.cloudera.org:8080/#/c/19594/20/src/kudu/integration-tests/create-table-itest.cc@641 PS20, Line 641: // Set tablet server heart beat timeout 1s. nit: This comment about flag value is not correct. -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 20 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 11 May 2023 16:49:25 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3448 Move password retrieval to master init
Hello Marton Greber, Zoltan Chovan, Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19873 to look at the new patch set (#3). Change subject: KUDU-3448 Move password retrieval to master init .. KUDU-3448 Move password retrieval to master init Apparently, forks can be expensive, at least on some systems, which makes fetching the password to encrypt/decrypt key materials whenever a TSK or IPKI is generated or loaded into memory not ideal as it may cause the server process to hang. This patch moves the password retrieval to the master initialization phase and the passwords are now stored in memory for later use. Change-Id: I746e657ae8d295f5f34225d63686beea1dff6b7c --- M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.cc M src/kudu/security/token-test.cc M src/kudu/security/token_signer.cc M src/kudu/security/token_signer.h M src/kudu/security/token_signing_key.cc M src/kudu/security/token_signing_key.h 8 files changed, 79 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/19873/3 -- To view, visit http://gerrit.cloudera.org:8080/19873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I746e657ae8d295f5f34225d63686beea1dff6b7c Gerrit-Change-Number: 19873 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Zoltan Chovan
[kudu-CR] [tests] fix flakiness in TestLogCleanupOnStartup
Mahesh Reddy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19871 ) Change subject: [tests] fix flakiness in TestLogCleanupOnStartup .. Patch Set 1: I cherry-picked this patch and ran a dist-test job as seen here, 255/256 passed: http://dist-test.cloudera.org/job?job_id=mreddy.1683824330.20429 The one failure was unrelated. For context, I also put out a patch to fix the flakiness in the test as seen here: https://gerrit.cloudera.org/c/19842/ I'm not sure which approach is better, maybe others can chime in here. -- To view, visit http://gerrit.cloudera.org:8080/19871 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6ee02316fb0850a52251f3740309213b69ca79b8 Gerrit-Change-Number: 19871 Gerrit-PatchSet: 1 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Comment-Date: Thu, 11 May 2023 17:02:23 + Gerrit-HasComments: No
[kudu-CR] KUDU-3448 Add support for encrypting TSKs
Hello Mahesh Reddy, Marton Greber, Zoltan Chovan, Alexey Serbin, Ashwani Raina, Kudu Jenkins, Abhishek Chennaka, Ádám Bakai, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19617 to look at the new patch set (#8). Change subject: KUDU-3448 Add support for encrypting TSKs .. KUDU-3448 Add support for encrypting TSKs In a previous patch, support for encrypting IPKI root CA private keys has been added. This is a follow up patch, to add encryption support for token signing keys as well. It is controlled by a new flag: --tsk_private_key_password_cmd. If this flag is set, the token signing keys will be stored in the syscatalog table in encrypted form (AES-256-CBC with PKCS#8 encoding). Token signing keys rotate automatically in Kudu, but for now, at least, encryption of TSKs can't be turned on or off on an existing master, so if this flag is set on the first startup of a master, it must be set to a command that outputs the same password as on initialization, and vice versa, it must not be provided on later master startups if it wasn't provided on initialization. Change-Id: Id8d770de7ed824cfc725003bbe77f1e42629029b --- M src/kudu/integration-tests/catalog_manager_tsk-itest.cc M src/kudu/master/master-test.cc M src/kudu/security/token_signing_key.cc 3 files changed, 36 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/19617/8 -- To view, visit http://gerrit.cloudera.org:8080/19617 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id8d770de7ed824cfc725003bbe77f1e42629029b Gerrit-Change-Number: 19617 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: Ádám Bakai
[kudu-CR] KUDU-1945 Add Python example for non-unique PK
Hello Zoltan Chovan, Attila Bukor, Kudu Jenkins, Abhishek Chennaka,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19874
to look at the new patch set (#3).
Change subject: KUDU-1945 Add Python example for non-unique PK
..
KUDU-1945 Add Python example for non-unique PK
In commit 3019848d00ac721b30d98a1aeb77bc205352b7b5 a C++ client example
has been added to showcase the usage of the non-unique PK. This patch
translates this particular example to Python.
The C++ examples are tested during upstream submission, however the Java
and Python client examples are not tested automatically. Therefore,
I did manual testing and created a ticket to track this improvement:
KUDU-3478.
Manual testing has been done using the following configurations:
Ubuntu 18.04 x86_64: Python2.7
Ubuntu 18.04 x86_64: Python3.7
macOS Monterey M1(arm64): Python3.7
(I couldn't setup the Python2.7 interpreter properly on my mac)
STDOUT of the example:
kudu.Schema {
non_unique_keyint32 NOT NULL
auto_incrementing_id int64 NOT NULL
int_val int32 NOT NULL
PRIMARY KEY (non_unique_key, auto_incrementing_id)
}
Demonstrating scanning ...
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 1)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 2)
Scanned some row(s) WHERE non_unique_key = 1
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 2)
Demonstrating UPDATE ...
Updated row(s) WHERE non_unique_key = 1 AND int_val = 2 to int_val = 98
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 1)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 2)
Updated row(s) WHERE non_unique_key = 2 to int_val = 99
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Updated row(s) WHERE non_unique_key = 2 AND auto_incrementing_id = 5\
to int_val = 100
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 100)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Demonstrating DELETE ...
Deleted row(s) WHERE non_unique_key = 3 AND int_val = 1
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
(non_unique_key: 2, auto_incrementing_id: 4, int_val: 99)
(non_unique_key: 2, auto_incrementing_id: 5, int_val: 100)
(non_unique_key: 2, auto_incrementing_id: 6, int_val: 99)
Deleted row(s) WHERE non_unique_key = 2
(non_unique_key: 3, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 3, auto_incrementing_id: 3, int_val: 2)
(non_unique_key: 4, auto_incrementing_id: 4, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 1, int_val: 0)
(non_unique_key: 1, auto_incrementing_id: 2, int_val: 1)
(non_unique_key: 1, auto_incrementing_id: 3, int_val: 98)
Deleted row(s)
[kudu-CR] [KUDU-3452] Make validate tablet creating task not affected
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#21). Change subject: [KUDU-3452] Make validate tablet creating task not affected .. [KUDU-3452] Make validate tablet creating task not affected Currently, creating a table with RF=n when the number of healthy tservers is less than n will get stuck. Because catalog manager creates tablets for it will fail and retry continuously. At the same time, creating a table with RF=m also will get stuck even if there are more than m healthy tservers. Because catalog manager will return when finds a tablet-creating task failed and will not try to select replicas for other PREPARING tablets. For example, creating a three replicas table times out when one of three tablet servers becomes unavailable. After that, creating a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table-creating task will be affected by invalidate three-replicas table-creating task. This patch fixes this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/integration-tests/create-table-itest.cc M src/kudu/master/catalog_manager.cc 2 files changed, 98 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/21 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 21 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] Make validate tablet creating task not affected
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19594 )
Change subject: [KUDU-3452] Make validate tablet creating task not affected
..
Patch Set 21:
(1 comment)
> Patch Set 20:
>
> (1 comment)
http://gerrit.cloudera.org:8080/#/c/19594/20/src/kudu/integration-tests/create-table-itest.cc
File src/kudu/integration-tests/create-table-itest.cc:
http://gerrit.cloudera.org:8080/#/c/19594/20/src/kudu/integration-tests/create-table-itest.cc@641
PS20, Line 641: Substitute("--tserver_unresponsive_timeout_
> nit: This comment about flag value is not correct.
Done
--
To view, visit http://gerrit.cloudera.org:8080/19594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9
Gerrit-Change-Number: 19594
Gerrit-PatchSet: 21
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Fri, 12 May 2023 02:02:37 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3452 Allow creating tablets under replicated tables
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19571 to look at the new patch set (#23). Change subject: KUDU-3452 Allow creating tablets under replicated tables .. KUDU-3452 Allow creating tablets under replicated tables Currently, creating a three-replicas table when there are not less than 3 healthy tablet servers will fail and the system catalog will retry continuously and always fail until the unavailable tablet servers becomes healthy again. An under-replicated table is still available for reading and writing, so it's enough to place just a majority of replicas for each tablet at healthy tablet servers to make a newly created table ready to use. This patch adds a new flag: --allow_creating_under_replicated_tables to support this feature. The original logic is kept the same. When this flag is set true, it's possible to create a tablet placing just a majority of replicas at healthy tablet servers. Even if the new tablet is created under-replicated, it's still available for read and write operations. Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 --- M src/kudu/integration-tests/create-table-itest.cc M src/kudu/master/catalog_manager.cc 2 files changed, 201 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/19571/23 -- To view, visit http://gerrit.cloudera.org:8080/19571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19571 Gerrit-PatchSet: 23 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] KUDU-3452 Allow creating tablets under replicated tables
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19571 ) Change subject: KUDU-3452 Allow creating tablets under replicated tables .. Patch Set 23: (3 comments) http://gerrit.cloudera.org:8080/#/c/19571/22/src/kudu/integration-tests/create-table-itest.cc File src/kudu/integration-tests/create-table-itest.cc: http://gerrit.cloudera.org:8080/#/c/19571/22/src/kudu/integration-tests/create-table-itest.cc@647 PS22, Line 647: constexpr int kTSUnresponsiveTimeoutMs = 4000; : constexpr int kHeartbeatIntervalMs = 3000; > Why is the heartbeat interval even longer than tserver unresponsive interva Done http://gerrit.cloudera.org:8080/#/c/19571/17/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: http://gerrit.cloudera.org:8080/#/c/19571/17/src/kudu/master/catalog_manager.cc@6664 PS17, Line 6664: } : > Seems this comment was not resolved in patch set 22. Done http://gerrit.cloudera.org:8080/#/c/19571/17/src/kudu/master/catalog_manager.cc@6677 PS17, Line 6677: let servers are regist > ditto. Done -- To view, visit http://gerrit.cloudera.org:8080/19571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19571 Gerrit-PatchSet: 23 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 12 May 2023 02:14:52 + Gerrit-HasComments: Yes
[kudu-CR] [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19794 ) Change subject: [cpp-client] KUDU-3455 Reduce space complexity and speed up hash partition pruning for in-list predicate .. Patch Set 7: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie4bea5c10b4ac2c62b85625fe9d2a33ceb4fb2e9 Gerrit-Change-Number: 19794 Gerrit-PatchSet: 7 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 12 May 2023 03:21:04 + Gerrit-HasComments: No
[kudu-CR] KUDU-3371 [fs] make LogBlockContainer a base class
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19610 ) Change subject: KUDU-3371 [fs] make LogBlockContainer a base class .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19610 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieb199de95973aaeba76947ad095907272d84ca67 Gerrit-Change-Number: 19610 Gerrit-PatchSet: 4 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 12 May 2023 03:40:13 + Gerrit-HasComments: No
