Can you, please, send your config and full log file that contains error message.
> 26 мая 2022 г., в 17:50, Surinder Mehra <redni...@gmail.com> написал(а): > > Hello, > I upgraded to 2.13.0 and I am able to take sync snapshots now. However, I ran > into another problem while restoring from snapshot using manual steps > mentioned in documentation. > > We run ignite statefulset on kubernetes cluster so when we scale it to N > nodes, it brings up one node at a time. > > Now I am trying to attach init container which will copy /db directory from > snapshots to work directory after clearing db directory from work directory > and then start main container which runs ignite. > > It works well on single node, it's able to start cluster with snapshot Data. > > When I start multiple nodes, init container will run each one of those as > first step. Since nodes starts one at a time, it's runs into error saying > "too small WAL segments data" > > I suppose that could be because 2nd node is still in init step while first > one is in running mode. There are few which haven't started yet, waiting for > 2nd node to be in running state. > > Any idea how can we make main containers wait until all init containers are > completed > > Asking this here as its related to ignite setup in kubernetes. > > Any help wil be appreciated. Thanks > > On Wed, 25 May 2022, 00:04 Surinder Mehra, <redni...@gmail.com > <mailto:redni...@gmail.com>> wrote: > Thanks a lot. I will try this. > > On Tue, 24 May 2022, 23:50 Николай Ижиков, <nizhi...@apache.org > <mailto:nizhi...@apache.org>> wrote: > > Does it ensure consistency while copying data which is parallely getting > > updated by application writes > > Yes. > > From the documentation: > > «An Ignite snapshot includes a consistent cluster-wide copy of all data > records persisted on disk and some other files needed for a restore > procedure.» > > > will this be a stop the world process > > No. > > >> 24 мая 2022 г., в 21:17, Surinder Mehra <redni...@gmail.com >> <mailto:redni...@gmail.com>> написал(а): >> >> Hi >> Thanks for reply. >> >> #1: So it's not a stop the world task. Does it ensure consistency while >> copying data which is parallely getting updated by application writes. Or >> does it mark the data to copied and ignore further updates on it. >> >> #2: >> I will try sync snapshot. But just to confirm, will this be a stop the world >> process. Couldn't find anything on Documentation page about it >> >> On Tue, 24 May 2022, 23:12 Николай Ижиков, <nizhi...@apache.org >> <mailto:nizhi...@apache.org>> wrote: >> Hello, Mehra. >> >> > 1. Is it stop the world process. >> >> No, you can perform any actions. >> Note, topology changes will cancel snapshot create process. >> >> > 2. If so, is it stop the world only during command execution (500millis) >> > or until snapshot Dara is fully copied(takes many minutes) to complete. >> >> Please, take a look at `—sync` option of create snapshot command (you can >> see help in `control.sh` output). >> `EVT_CLUSTER_SNAPSHOT_FINISHED` raise on snapshot create finish. >> >> > 3. Is there a way around to speed up this other than increasing snapshot >> > threads >> >> Stop write operations. >> The less you change the quicker snapshot will be created. >> >>> 24 мая 2022 г., в 20:12, Surinder Mehra <redni...@gmail.com >>> <mailto:redni...@gmail.com>> написал(а): >>> >>> Hi, >>> I have 3 node ignite cluster each node contains 60G work directory(ebs) and >>> I need to create snapshots. >>> I followed steps to create snapshots and run create snapshot command using >>> control utility. Command completed in 500millis but snapshot directory only >>> had 400Mb data. Later I realised directory size grew up 30G. I suppose it >>> would reach size of work directory. >>> >>> >>> I have few questions. >>> 1. Is it stop the world process. >>> 2. If so, is it stop the world only during command execution (500millis) or >>> until snapshot Dara is fully copied(takes many minutes) to complete. >>> 3. Is there a way around to speed up this other than increasing snapshot >>> threads >> >
