[
https://issues.apache.org/jira/browse/KUDU-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adar Dembo resolved KUDU-3070.
------------------------------
Fix Version/s: 1.12.0
Assignee: wangningito
Resolution: Fixed
> allow skip open block manager in cli cmeta rewrite_raft_config operation
> ------------------------------------------------------------------------
>
> Key: KUDU-3070
> URL: https://issues.apache.org/jira/browse/KUDU-3070
> Project: Kudu
> Issue Type: New Feature
> Components: CLI
> Reporter: wangningito
> Assignee: wangningito
> Priority: Minor
> Fix For: 1.12.0
>
>
> I'm in a bigdata company which served over 1000+ company, we adopted kudu as
> main or auxiliary storage engine, some of our customers are just small
> startups, they had a lot of data but too much nodes are expensive to them.
> So some of cases are based on: few nodes, much data and maybe not compacted
> well data.
> In our scenarios, there exists some migration cases
> # from standalone tserver to another standalone tserver
> # from 3 nodes tserver cluster to another 3 nodes tserver
> In the past, we have to do something like this
> {code:java}
> // First, download tablet data via kudu local_replica copy_from_remote
> // then rewrite all the raft info for each tablet
> echo ${tablet_id_list} | xargs -i kudu local_replica cmeta
> rewrite_raft_config {} PEER_INFO -fs_data_dirs=xxx -fs_wal_dir=yyy{code}
> Download data via copy_from_remote is blazing fast.
> However sometimes it takes us a lot of time to rewrite raft info of all
> tablet, 30s - 60s per tablet as I witnessed. Sometimes it could take more
> time if the data were not fully compacted. So sometimes it take us 2 hours to
> download tablet data, but 6 hours to rewrite meta.
> I noticed some code fragment in RewriteRaftConfig function
> {code:java}
> FsManager fs_manager(env, FsManagerOpts());
> RETURN_NOT_OK(fs_manager.Open());{code}
> This means I have to open the fs_data_dirs and fs_wal_dir 100 times if I want
> to rewrite raft of 100 tablets.
> To saving the overhead of each operation, we can just skip opening block
> manager for rewrite_raft_config, cause all the operations only happened on
> meta files.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
