Hello, I upgraded to 2.13.0 and I am able to take sync snapshots now. However, I ran into another problem while restoring from snapshot using manual steps mentioned in documentation.
We run ignite statefulset on kubernetes cluster so when we scale it to N nodes, it brings up one node at a time. Now I am trying to attach init container which will copy /db directory from snapshots to work directory after clearing db directory from work directory and then start main container which runs ignite. It works well on single node, it's able to start cluster with snapshot Data. When I start multiple nodes, init container will run each one of those as first step. Since nodes starts one at a time, it's runs into error saying "too small WAL segments data" I suppose that could be because 2nd node is still in init step while first one is in running mode. There are few which haven't started yet, waiting for 2nd node to be in running state. Any idea how can we make main containers wait until all init containers are completed Asking this here as its related to ignite setup in kubernetes. Any help wil be appreciated. Thanks On Wed, 25 May 2022, 00:04 Surinder Mehra, <redni...@gmail.com> wrote: > Thanks a lot. I will try this. > > On Tue, 24 May 2022, 23:50 Николай Ижиков, <nizhi...@apache.org> wrote: > >> > Does it ensure consistency while copying data which is parallely >> getting updated by application writes >> >> Yes. >> >> From the documentation: >> >> «An Ignite snapshot includes a consistent cluster-wide copy of all data >> records persisted on disk and some other files needed for a restore >> procedure.» >> >> > will this be a stop the world process >> >> No. >> >> >> 24 мая 2022 г., в 21:17, Surinder Mehra <redni...@gmail.com> написал(а): >> >> Hi >> Thanks for reply. >> >> #1: So it's not a stop the world task. Does it ensure consistency while >> copying data which is parallely getting updated by application writes. Or >> does it mark the data to copied and ignore further updates on it. >> >> #2: >> I will try sync snapshot. But just to confirm, will this be a stop the >> world process. Couldn't find anything on Documentation page about it >> >> On Tue, 24 May 2022, 23:12 Николай Ижиков, <nizhi...@apache.org> wrote: >> >>> Hello, Mehra. >>> >>> > 1. Is it stop the world process. >>> >>> No, you can perform any actions. >>> Note, topology changes will cancel snapshot create process. >>> >>> > 2. If so, is it stop the world only during command execution >>> (500millis) or until snapshot Dara is fully copied(takes many minutes) to >>> complete. >>> >>> Please, take a look at `—sync` option of create snapshot command (you >>> can see help in `control.sh` output). >>> `EVT_CLUSTER_SNAPSHOT_FINISHED` raise on snapshot create finish. >>> >>> > 3. Is there a way around to speed up this other than increasing >>> snapshot threads >>> >>> Stop write operations. >>> The less you change the quicker snapshot will be created. >>> >>> 24 мая 2022 г., в 20:12, Surinder Mehra <redni...@gmail.com> написал(а): >>> >>> Hi, >>> I have 3 node ignite cluster each node contains 60G work directory(ebs) >>> and I need to create snapshots. >>> I followed steps to create snapshots and run create snapshot command >>> using control utility. Command completed in 500millis but snapshot >>> directory only had 400Mb data. Later I realised directory size grew up 30G. >>> I suppose it would reach size of work directory. >>> >>> >>> I have few questions. >>> 1. Is it stop the world process. >>> 2. If so, is it stop the world only during command execution (500millis) >>> or until snapshot Dara is fully copied(takes many minutes) to complete. >>> 3. Is there a way around to speed up this other than increasing snapshot >>> threads >>> >>> >>> >>
