Maxim,
This way we'll introduce a migration procedure between versions, which
we currently don't have. Different Ignite versions are compatible by
persistence storage. If we add a migration script, we need to decide,
whether we need to run it every time when the version is upgraded, or
only some specific versions are affected.
I suggest having a procedure that will look for metadata in the work
directory, and if it finds it there, then the node will use it.
Otherwise the persistence directory is used.
Denis
On 13.05.2020 21:40, Maxim Muzafarov wrote:
Folks,
I think it's important to discuss the following question regarding this thread:
Should we consider moving the migration procedure from the java
production code to migration scripts?
From my understanding, keeping all such things in java production
source code has some disadvantages:
1. It executes only once at the migration stage.
2. It affects the complexity of the source code and code maintenance.
3. Node crash cases must be covered during the migration procedure.
4. It affects the production usage e.g. the process doesn't have the
right access to the old directory (migration already completed) and
will fail the node start.
The right behavior from my point should be:
1. Change the default path of binary/marshaller directory to the new one.
2. Provide migration scripts for users.
WDYT?
On Wed, 13 May 2020 at 21:10, Denis Mekhanikov <dmekhani...@gmail.com> wrote:
Sounds great!
It happens pretty frequently that users migrate to a new version of
Ignite and preserve persistence files only without caring too much about
the work folder. But it turns out, that the work folder actually has
some important stuff.
This improvement should help with this issue.
What's about in-memory mode? As far as I know, we write binary metadata
to disk even when no persistence is configured. Do you plan to address
it in any way?
Denis
On 12.05.2020 15:56, Sergey Antonov wrote:
Hello Semyon,
This is a good idea!
вт, 12 мая 2020 г. в 15:53, Вячеслав Коптилин <slava.kopti...@gmail.com>:
Hello Semyon,
This is a good and long-awaited improvement! Thank you for your efforts!
Thanks,
S.
вт, 12 мая 2020 г. в 15:11, Данилов Семён <samvi...@yandex.ru>:
Hello!
I would like to propose moving /binary_meta and /marshaller folders to
the
PDS folder.
Motivation: data, directly related to the persistence, is stored outside
the persistence dir, which can lead to various issues and also is not
very
convenient to use. In particular, with k8s, deployment disk that is
attached to a container can not be accessed from other containers or
outside of k8s. In case if support will need to drop persistence except
data, there will be no way to recover due to binary metadata is required
to
process PDS files.
I created an issue (https://issues.apache.org/jira/browse/IGNITE-12994)
and a
pull request(https://github.com/apache/ignite/pull/7792) that fixes the
issue.
In that PR I made the following:
* store binary meta and marshaller data inside db/ folder
* if binary meta of marshaller are found in "legacy" locations --
safely move them to new locations during the node startup
Kind regards,
Semyon Danilov.
