Re: Binary memory restoration

2020-10-09 Thread Ilya Kasnacheev
Hello! You do not need any optimizations here since the time between checkpoints is configurable. More checkpoints - faster recovery but more data needs to be written to disk. Fewer checkpoints - recovery will take more time but overall performance is somewhat higher. Regards, -- Ilya

Re: Binary memory restoration

2020-10-08 Thread Raymond Wilson
Ilya, Does 2.9 have specific optimizations for checkpoints? Thanks, Raymond. On Fri, Oct 9, 2020 at 1:23 AM Ilya Kasnacheev wrote: > Hello! > > I think the real saver is in decreasing amount of time between checkpoints. > > Regards, > -- > Ilya Kasnacheev > > > вт, 6 окт. 2020 г. в 02:10,

Re: Binary memory restoration

2020-10-08 Thread Ilya Kasnacheev
Hello! I think the real saver is in decreasing amount of time between checkpoints. Regards, -- Ilya Kasnacheev вт, 6 окт. 2020 г. в 02:10, Raymond Wilson : > Thanks for the thoughts Ilya and Vladimir. > > We'll do a comparison with 2.9 when it releases to see if that makes any > difference.

Re: Binary memory restoration

2020-10-05 Thread Raymond Wilson
Thanks for the thoughts Ilya and Vladimir. We'll do a comparison with 2.9 when it releases to see if that makes any difference. One of the advantages with persistent storage is that it is effectively 'instant start'. Our WAL size is around 5Gb, perhaps this should be decreased to reduce system

Re: Binary memory restoration

2020-09-29 Thread Vladimir Pligin
It's possible that it happens because of https://issues.apache.org/jira/browse/IGNITE-13068. We need to scan the entire SQL primary index during startup in case you have at least on query entity configured. As far as I can see it's going to be a part of the Ignite 2.9 release. -- Sent from:

Re: Binary memory restoration

2020-09-29 Thread Ilya Kasnacheev
Hello! I guess it is the time to iterate WAL after the last checkpoint to make sure all operations prior to node stop are accounted for. Every DB will do that and the time of 76 seconds seems reasonable. Regards, -- Ilya Kasnacheev вт, 29 сент. 2020 г. в 02:00, Raymond Wilson : > There is a

Re: Binary memory restoration

2020-09-28 Thread Raymond Wilson
There is a summary log line that also references the time taken by the 'restore binary memory' stage. It also references another stage called 'restore logical state'. eg: 2020-09-27 20:50:31,423 [1] INF [] Node started : [stage="Configure system pool" (44 ms),stage="Start managers" (2422

Binary memory restoration

2020-09-28 Thread Raymond Wilson
Just a general query on Ignite server node startup sequence. There are a pair of log lines emitted during node startup that look like this: 2020-09-27 20:48:43,191 [1] INF [] Starting binary memory restore for: [948695530, -1026154091, -2049124072, -552847479, -2024686410, -1820754607,