Re: Using native persistence to "extend" memory

2020-06-16 Thread steve.hostettler
Thank you both for clarifying this. I usually also have large objects 3KB so I thought about increasing the page size to 32KB to reduce the number of pages and thus reduce the speed at which we get to the 2/3 of dirty pages. Good idea? On top of that during the process at some I generate a

Re: Using native persistence to "extend" memory

2020-06-16 Thread Denis Magda
Evgeniy, Thanks for clarifying, I completely forgot about this behavior! - Denis On Tue, Jun 16, 2020 at 5:10 PM Evgenii Zhuravlev wrote: > Steve, > > Actually, disabling WAL is a good option for your use case. Checkpoint > mechanism is the same with disabled WAL, the only difference is that

Re: Using native persistence to "extend" memory

2020-06-16 Thread Evgenii Zhuravlev
Steve, Actually, disabling WAL is a good option for your use case. Checkpoint mechanism is the same with disabled WAL, the only difference is that node is not writing WAL to the disk on each operation. Usually, it might make sense to disable WAL for initial loading - when you can lose the data in

Re: Using native persistence to "extend" memory

2020-06-16 Thread Denis Magda
Steve, Please check these generic recommendations if you haven't done so already: https://apacheignite.readme.io/docs/durable-memory-tuning#native-persistence-related-tuning Otherwise, send us a note if you come across any bottlenecks or issues so that we can give you more specific

Re: Using native persistence to "extend" memory

2020-06-16 Thread steve.hostettler
Thanks a lot for the recommendation. So keeping the WAL, disabling archiving. I understand all records are kept on disk. Thanks again. Anything else? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Using native persistence to "extend" memory

2020-06-16 Thread Denis Magda
Hi Steve, I think that you can get a performance hit for your write operations by disabling the WAL. It serves two purposes in Ignite. The first is the recovery while the second is fast disk writes. The WAL is an append-only structure and Ignite can persist changes to disk really quick. If the

Using native persistence to "extend" memory

2020-06-16 Thread steve.hostettler
Hello, I am trying to use ignite persistence to "extend" memory. That is to replace the swap that is not working very well. Therefore since I do not care about recovery I disabled the WAL. Are there other things you would recommend to configure to use the ignite persistence as a sort of swap.