Hi Kirill, Alexey,
On the interface structure.
So, as a user I'll see one interface, WarmUpConfiguration, with no methods.
I choose an implementation and configure it like
cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
Ignite tries to map the LoadEverythingWarmupConfiguration to an implementations
it knows - either to a built-in one or to a plugin.
If it finds the implementation, it passes the configuration to it and it
handles the warmup.
If it doesn't find an existing implementation, it throws an exception.
The implementation will use any internal API of Ignite that it needs to perform
the warmup. It is up to the plugin maintainer to track code changes in Ignite
and adjust the plugin to be compatible from version to version.
Is all of the above correct?
How about
cfg.setWarmUpConfiguration(IgniteWarmupStrategies.LOAD_EVERYTHING);
instead of
cfg.setWarmUpConfiguration(new LoadEverythingWarmupConfiguration());
?
Or do we expect having POJO instead of a string constant to be beneficial?
Agree about preloadPartition(). Fair enough, let's leave it be.
On IgniteConfiguration vs DataRegionConfiguration.
I like DataRegionConfiguration more because it allows to specify different
strategies for different regions naturally.
We already say that all cache groups in the same region share memory management
(e.g. share space and participate in page replacement together).
So it's natural to say that if I want different memory warmup semantics for two
groups then I should be putting them in different regions.
Do you see a good way to have distinct warmup configuration for different
regions while the config is on IgniteConfiguration level?
Thanks,
Stan
> On 6 Aug 2020, at 15:39, ткаленко кирилл <tkalkir...@yandex.ru> wrote:
>
> Hello, Alexey!
>
> Your comments are fair.
>
> 05.08.2020, 15:51, "Alexey Goncharuk" <alexey.goncha...@gmail.com>:
>> Kirill,
>>
>> Thank you for driving this discussion and implementation.
>>
>> A few points from my side:
>> * Agree that it will be best to keep the strategy interface private because
>> it will be very dependent on the persistent storage implementation. We
>> would need to expose page IDs and types to public API, which is very
>> restrictive. The configuration part obviously needs to be public, and
>> ability to pull the strategy implementation from plugin is a good idea.
>> * I was also thinking of adding the warmup configuration straight to the
>> IgniteConfiguration, but I like Stan's idea of adding it to
>> DataRegionConfiguration. No strong preference here.
>> * I do not think we need to deprecate preloadPartition() method. One of the
>> use-cases for this method was to process partitions sequentially while a
>> node is running. This method is able to fetch the partition from disk much
>> (from times to orders of magnitude) faster than sequential scan.
>> * Being able to cancel the warmup during startup is a great feature. We
>> should be able to support it from control.sh because the warmup runs before
>> discovery which starts the last, so the control.sh handler should be
>> already running.
