Denis,
1) Not necessarily. You may want to stream more data than you have memory.
In this case, Ignite must spill some data to disk, and in the first phase,
this will be done by the same checkpointing mechanics. In the second phase,
we can implement a smarter and faster (no-COW) data spill, but st
Guys,
Sever things to clarify:
- disabled WAL means disabled checkpointing (no data on disk at all)?
- after turning the WAL back when a preloading is over, how can I know a moment
in time when the disk holds the whole copy of data so that it’s safe to restart
the whole cluster?
—
Denis
> On O
Vladimir,
Several questions:
1. On which interface do you plan to add enable/disableWal()?
2. Is it possible that when enabling/disabling WAL for multiple caches,
some fail and some succeed?
3. is enable/disableWal a synchronous operation?
4. What happens if a user starts streaming
Pavel,
No, this is not the case we are trying to cover. WAL disabling should be
separate operation, which is not tied to any API, whether this is streamer,
cache put, or DML.
On Wed, Oct 25, 2017 at 5:20 PM, Vladimir Ozerov
wrote:
> Alex,
>
> My bad, I meant "isWalEnabled(String cacheName)" of
Alex,
My bad, I meant "isWalEnabled(String cacheName)" of course.
On Wed, Oct 25, 2017 at 3:34 PM, Anton Vinogradov
wrote:
> Pavel,
>
> WAL disabling is a very dangerous operation and it seems to be not a good
> idea to allow run regular operation with .disabledWal().
> Let's think twice how to
Pavel,
WAL disabling is a very dangerous operation and it seems to be not a good
idea to allow run regular operation with .disabledWal().
Let's think twice how to make new API safe.
On Wed, Oct 25, 2017 at 3:25 PM, Pavel Tupitsyn
wrote:
> Vladimir,
>
> It would be useful to be able to automatic
Vladimir,
It would be useful to be able to automatically disable WAL when streaming
starts
and re-enable after it ends, don't you think so?
Something like IgniteDataStreamer.disableWal property.
This is in addition to other API calls that you suggested.
On Wed, Oct 25, 2017 at 3:25 PM, Alexey G
I do not like boolean isWalEnabled(String... cacheNames) - it's semantics
is confusing. Should it return true if WAL is enabled for all caches or if
WAL is enabled for at least one cache? IMO, since this is a local-read
operation, single cache per call is enough.
As for the API placement, it looks
Pavel,
This feature will work independently of streamer. If you want to load data
with streamer, then you disable WAL first through some API call or SQL
command, and then start loading.
On Wed, Oct 25, 2017 at 2:41 PM, Pavel Tupitsyn
wrote:
> IMO IgniteCache.disableWal() should be enough.
>
> A
IMO IgniteCache.disableWal() should be enough.
Also what about an option to disable WAL when IgniteDataStreamer is active?
On Wed, Oct 25, 2017 at 2:38 PM, Vladimir Ozerov
wrote:
> One more idea - ability to create a cache with initially disabled WAL.
> Might be useful.
>
> On Wed, Oct 25, 2017
One more idea - ability to create a cache with initially disabled WAL.
Might be useful.
On Wed, Oct 25, 2017 at 2:35 PM, Vladimir Ozerov
wrote:
> Igniters,
>
> We have a ticket to implement runtime WAL state management [1].It will be
> possible to disable it temporarily. This is very useful for
Igniters,
We have a ticket to implement runtime WAL state management [1].It will be
possible to disable it temporarily. This is very useful for data loading
case. Our experiments show that we can improve data loading time by a
factor of 2x-10x depending on configuration, indexes and cluster topolo
12 matches
Mail list logo
