?????? [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-14 Thread lordcheng10
Reply for Yong:   The migration tool is running in the recovery service. In addition, some changes have been made to the migration tool, for more details, see issue and PR: issue: https://github.com/apache/bookkeeper/issues/3456 PR: https://github.com/apache/bookkeeper/pull/3457

?????? [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-14 Thread lordcheng10
Reply for Yong: The migration tool is running in the recovery service. In addition, some changes have been made to the migration tool, for more details, see issue and PR: The relevant descriptions are as follows: ??. Motivation Currently bookkeeper does not support data migration, only data

?????? [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-14 Thread lordcheng10
Reply for Yong: Q: Does the migration process run in the recovery service? Or run it standalone?A:  The migration tool is running in the recovery service. In addition, some changes have been made to the migration tool, for more details, see issue and PR: issue: https://github.com/apache/bookkeep

?????? [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-14 Thread lordcheng10
Thanks for reviewing this PR! Your comments is very good, I have fixed, PTAL, thanks! After the discussion, I will start a VOTE!

Re: [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-14 Thread Yong Zhang
It's a good idea to have that tool, but I have some questions about it. >The process is as follows: 1. Submit the bookie node to be offline; 5. Traverse each ledgers on the offline bookie, and persist these ledgers and the corresponding offline bookie nodes to the zookeeper directory: ledgers/offl

Re: [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-13 Thread Enrico Olivelli
Thanks for your answers. I support this BP. I have left some comments on the PR, there is some work to be done, but you are on your way When there is consensus about this BP you have to start a VOTE Thanks Enrico Il giorno ven 9 set 2022 alle ore 16:06 lordcheng10 <1572139...@qq.com.invalid> h

?????? [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-09 Thread lordcheng10
Sorry for not describing the function of this BP clearly, this BP is not the same as your proposal. Because the current bookkeeper does not have the ability to migrate data and can only perform data recovery, this BP mainly provides a data migration tool. When bookkeeper has data migration to

??????[Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-09 Thread lordcheng10
How to use this data migration tool?  For example: 1. Migrate all ledger data on bookie1 and bookie2 to other bookie nodes: sh bin/bookkeeper shell replicasMigration -bookieIds bookie1,bookie2 -ledgerIds ALL -readOnly true 2. Migrate ledger1 and ledger3 on bookie1 and bookie2 to other bookie

??????[Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-08 Thread lordcheng10
The main content of this BP is for data migration, and the current title does not match. So I changed the corresponding issue and PR title to BP-56: Support data migration issue: https://github.com/apache/bookkeeper/issues/3456 PR: https://github.com/apache/bookkeeper/pull/3457

??????[Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-08 Thread lordcheng10
The corresponding issue and pr are as follows: issue??https://github.com/apache/bookkeeper/issues/3456 PR??https://github.com/apache/bookkeeper/pull/3457

??????[Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-09-08 Thread lordcheng10
I made some modifications and implemented a replica data migration tool ### Motivation We have a scenario where a large number of bookie nodes are offline, and we find that the bookie's offline solution is very time-consuming. Bookie offline steps are as follows: 1. Set the bookie node to be o

?????? [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-08-30 Thread lordcheng10
1.The first question, I don't understand what you mean, if it is an operation error, which leads to the wrong writing of the offline node, there is no way to avoid it. The current offline command, if the operation is wrong, it may also be offline to the wrong bookie node?? 2.Before going offline

Re: [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-08-28 Thread Yang Yang
Hi Enrico, I'm glad to. Do you have any suggestions for it? Best regards, Yang Yang On Mon, Aug 29, 2022 at 3:45 AM Enrico Olivelli wrote: > Yang, > > Il Sab 27 Ago 2022, 11:05 Yang Yang ha scritto: > > > For the short term, you can see if the `recover` command is able to help > > you: > > >

Re: [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-08-28 Thread Enrico Olivelli
Yang, Il Sab 27 Ago 2022, 11:05 Yang Yang ha scritto: > For the short term, you can see if the `recover` command is able to help > you: > https://bookkeeper.apache.org/docs/reference/cli#bookkeeper-shell-recover > > For the long term, I have proposed a solution to mark the bookie to be > decommi

Re: [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-08-27 Thread Yang Yang
For the short term, you can see if the `recover` command is able to help you: https://bookkeeper.apache.org/docs/reference/cli#bookkeeper-shell-recover For the long term, I have proposed a solution to mark the bookie to be decommissioned in a `draining` state and let the autorecovery mechanism rep

Re: [Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-08-25 Thread steven lu
I think this feature is somewhat custom and not very generic; and there are risks: 1. If you want to go offline on node A (which has already been extracted), but wrong write B, this function will directly go offline on node B, which is likely to cause online failures 2. If the node to be offline

[Discuss] BP-56: Support non-stop bookie data migration and bookie offline

2022-08-25 Thread lordcheng10
Hi Bookkeeper Community,  This is a BP discussion on Support non-stop bookie data migration and bookie offline The issue can be found: https://github.com/apache/bookkeeper/issues/3456 ; I copy the content here for convenience, any suggestions are welcome and appreciated. ### Motivation b