Denis, 1) Not necessarily. You may want to stream more data than you have memory. In this case, Ignite must spill some data to disk, and in the first phase, this will be done by the same checkpointing mechanics. In the second phase, we can implement a smarter and faster (no-COW) data spill, but still, there might be disk writes. 2) As Vladimir noted, the WAL activation process will internally have a future that will be completed after all nodes flushed their changes. We can either pass this future to the user or make the switch process synchronous.
2017-10-25 18:32 GMT+03:00 Denis Magda <[email protected]>: > Guys, > > Sever things to clarify: > - disabled WAL means disabled checkpointing (no data on disk at all)? > - after turning the WAL back when a preloading is over, how can I know a > moment in time when the disk holds the whole copy of data so that it’s safe > to restart the whole cluster? > > — > Denis > > > On Oct 25, 2017, at 4:35 AM, Vladimir Ozerov <[email protected]> > wrote: > > > > Igniters, > > > > We have a ticket to implement runtime WAL state management [1].It will be > > possible to disable it temporarily. This is very useful for data loading > > case. Our experiments show that we can improve data loading time by a > > factor of 2x-10x depending on configuration, indexes and cluster > topology. > > > > We had a private chat with Anton Vinogradov and Alexey Goncharuk, and > came > > to certain design which you can find inside the ticket. Comments are > > welcomed. > > > > In this email I would like to focus on API of this feature. > Considerations: > > 1) It should be possible to disable WAL globally or on per-cache basis. > > 2) It should be possible to get current state of WAL for the cache. > > > > Quick draft from my side: > > void disableWal(); > > void disableWal(String... cacheNames); > > void enableWal(); > > void enableWal(String... cacheNames); > > boolean isWalEnabled(String... cacheNames); > > > > Please help improving it. > > - Do we really need methods to disable WAL for all caches? This is not > very > > intuitive feature. > > - Where to place these operations? Ignite interface? Ignite + > IgniteCache? > > Separate facade? > > - Do we need async counterparts? > > - Do we need a feature which completes when WAL is enabled back? > > > > Vladimir. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-6411 > >
