[kudu-CR] server: consolidate tablet prepare pools
Todd Lipcon has submitted this change and it was merged. Change subject: server: consolidate tablet prepare pools .. server: consolidate tablet prepare pools Using the new serial token functionality available in ThreadPool, we can now safely consolidate all per-tablet prepare pools into a single server-wide pool. Each replica allocates a token with which to submit to the prepare pool, attaching the existing tablet-specific metrics to it. The prepare pool is configured with no upper bound on the number of running threads because prepare tasks can block on row/schema locks. Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Reviewed-on: http://gerrit.cloudera.org:8080/7150 Tested-by: Kudu Jenkins Reviewed-by: Todd Lipcon--- M src/kudu/integration-tests/delete_tablet-itest.cc M src/kudu/kserver/kserver.cc M src/kudu/kserver/kserver.h M src/kudu/master/sys_catalog.cc M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tablet/transactions/transaction_driver.cc M src/kudu/tablet/transactions/transaction_driver.h M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/ts_tablet_manager.cc 11 files changed, 59 insertions(+), 40 deletions(-) Approvals: Todd Lipcon: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 7 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] server: consolidate tablet prepare pools
Todd Lipcon has posted comments on this change. Change subject: server: consolidate tablet prepare pools .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 6 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] server: consolidate tablet prepare pools
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7150 to look at the new patch set (#5). Change subject: server: consolidate tablet prepare pools .. server: consolidate tablet prepare pools Using the new serial token functionality available in ThreadPool, we can now safely consolidate all per-tablet prepare pools into a single server-wide pool. Each replica allocates a token with which to submit to the prepare pool, attaching the existing tablet-specific metrics to it. The prepare pool is configured with no upper bound on the number of running threads because prepare tasks can block on row/schema locks. Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 --- M src/kudu/integration-tests/delete_tablet-itest.cc M src/kudu/kserver/kserver.cc M src/kudu/kserver/kserver.h M src/kudu/master/sys_catalog.cc M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tablet/transactions/transaction_driver.cc M src/kudu/tablet/transactions/transaction_driver.h M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/ts_tablet_manager.cc 11 files changed, 59 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7150/5 -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 5 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] server: consolidate tablet prepare pools
Adar Dembo has posted comments on this change. Change subject: server: consolidate tablet prepare pools .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/7150/4/src/kudu/kserver/kserver.cc File src/kudu/kserver/kserver.cc: PS4, Line 107: if (tablet_apply_pool_) { : tablet_apply_pool_->Shutdown(); : } : if (tablet_prepare_pool_) { : tablet_prepare_pool_->Shutdown(); : } : if (raft_pool_) { : raft_pool_->Shutdown(); : } > still wondering if ordering is important here. naively I would think that s See what I wrote in the previous patch: the order shouldn't matter since Messenger::Shutdown() guarantees that all RaftConsensus objects are destroyed. I'll add a comment (to the previous patch). -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: Adar Dembo Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] server: consolidate tablet prepare pools
David Ribeiro Alves has posted comments on this change. Change subject: server: consolidate tablet prepare pools .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/7150/4/src/kudu/kserver/kserver.cc File src/kudu/kserver/kserver.cc: PS4, Line 107: if (tablet_apply_pool_) { : tablet_apply_pool_->Shutdown(); : } : if (tablet_prepare_pool_) { : tablet_prepare_pool_->Shutdown(); : } : if (raft_pool_) { : raft_pool_->Shutdown(); : } still wondering if ordering is important here. naively I would think that shutdown should be done in {prepare, raft, apply} or maybe {raft, prepare, apply} order. if you found evidence of the contrary maybe spell it out somewhere? -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: Yes
[kudu-CR] server: consolidate tablet prepare pools
Hello David Ribeiro Alves, Todd Lipcon, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7150 to look at the new patch set (#4). Change subject: server: consolidate tablet prepare pools .. server: consolidate tablet prepare pools Using the new serial token functionality available in ThreadPool, we can now safely consolidate all per-tablet prepare pools into a single server-wide pool. Each replica allocates a token with which to submit to the prepare pool, attaching the existing tablet-specific metrics to it. The prepare pool is configured with no upper bound on the number of running threads because prepare tasks can block on row/schema locks. Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 --- M src/kudu/integration-tests/delete_tablet-itest.cc M src/kudu/kserver/kserver.cc M src/kudu/kserver/kserver.h M src/kudu/master/sys_catalog.cc M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tablet/transactions/transaction_driver.cc M src/kudu/tablet/transactions/transaction_driver.h M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/ts_tablet_manager.cc 11 files changed, 59 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7150/4 -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 4 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] server: consolidate tablet prepare pools
Hello Todd Lipcon, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7150 to look at the new patch set (#3). Change subject: server: consolidate tablet prepare pools .. server: consolidate tablet prepare pools Using the new serial token functionality available in ThreadPool, we can now safely consolidate all per-tablet prepare pools into a single server-wide pool. Each replica allocates a token with which to submit to the prepare pool, attaching the existing tablet-specific metrics to it. The prepare pool is configured with no upper bound on the number of running threads because prepare tasks can block on row/schema locks. Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 --- M src/kudu/integration-tests/delete_tablet-itest.cc M src/kudu/master/sys_catalog.cc M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tablet/transactions/transaction_driver.cc M src/kudu/tablet/transactions/transaction_driver.h M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/ts_tablet_manager.cc 11 files changed, 59 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7150/3 -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 3 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon
[kudu-CR] server: consolidate tablet prepare pools
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7150 to look at the new patch set (#2). Change subject: server: consolidate tablet prepare pools .. server: consolidate tablet prepare pools Using the new serial token functionality available in ThreadPool, we can now safely consolidate all per-tablet prepare pools into a single server-wide pool. Each replica allocates a token with which to submit to the prepare pool, attaching the existing tablet-specific metrics to it. The prepare pool is configured with no upper bound on the number of running threads because prepare tasks can block on row/schema locks. Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 --- M src/kudu/integration-tests/delete_tablet-itest.cc M src/kudu/master/sys_catalog.cc M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tablet/transactions/transaction_driver.cc M src/kudu/tablet/transactions/transaction_driver.h M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/ts_tablet_manager.cc 11 files changed, 59 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/7150/2 -- To view, visit http://gerrit.cloudera.org:8080/7150 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic393f739cc9c1b267142b20e04a1aaa5aed97cb0 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar DemboGerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon