Re: Moving binary metadata to PDS storage folder

2020-06-11 Thread Данилов Семён
Hello everyone, can we move forward with the merge of IGNITE-12994? Semyon 18.05.2020, 19:08, "Denis Mekhanikov" : > Users shouldn't care about binary_meta and marshaller directories. Those > are internal details. Having to go through the documentation on those > things to figure out whether

Re: Moving binary metadata to PDS storage folder

2020-05-18 Thread Denis Mekhanikov
Users shouldn't care about binary_meta and marshaller directories. Those are internal details. Having to go through the documentation on those things to figure out whether you need to run each of the provided scripts or not will lead to terrible user experience. If we add the

Re: Moving binary metadata to PDS storage folder

2020-05-15 Thread Maxim Muzafarov
Folks, I don't agree that it should be a single script for the whole migration process. I don't think we should have some kind of automated `decision-maker` processes for the migration procedure. We should have well-defined documentation pages + the `migration` directory with a single .sh per

Re: Moving binary metadata to PDS storage folder

2020-05-15 Thread Anton Kalashnikov
Hello, I agree with the described disadvantages(which Maxim mentioned) and I think it is a good idea to discuss a new way of pds migration in an extra ticket. But also, I agree with Denis that the script will add ambiguity(it's not clear which script should be used and when) if we add it. In

Re: Moving binary metadata to PDS storage folder

2020-05-15 Thread Данилов Семён
Hello Denis, I found out that without persistence being enabled, no metadata is written on disk (where is an if clause at the start of every method of BinaryMetadataFileStore) and everything is stored in metadataLocalCache. Semyon. 13.05.2020, 21:10, "Denis Mekhanikov" : > Sounds great! > >

Re: Moving binary metadata to PDS storage folder

2020-05-13 Thread Denis Mekhanikov
Maxim, This way we'll introduce a migration procedure between versions, which we currently don't have. Different Ignite versions are compatible by persistence storage. If we add a migration script, we need to decide, whether we need to run it every time when the version is upgraded, or only

Re: Moving binary metadata to PDS storage folder

2020-05-13 Thread Maxim Muzafarov
Folks, I think it's important to discuss the following question regarding this thread: Should we consider moving the migration procedure from the java production code to migration scripts? >From my understanding, keeping all such things in java production source code has some disadvantages: 1.

Re: Moving binary metadata to PDS storage folder

2020-05-13 Thread Denis Mekhanikov
Sounds great! It happens pretty frequently that users migrate to a new version of Ignite and preserve persistence files only without caring too much about the work folder. But it turns out, that the work folder actually has some important stuff. This improvement should help with this issue.

Re: Moving binary metadata to PDS storage folder

2020-05-12 Thread Sergey Antonov
Hello Semyon, This is a good idea! вт, 12 мая 2020 г. в 15:53, Вячеслав Коптилин : > Hello Semyon, > > This is a good and long-awaited improvement! Thank you for your efforts! > > Thanks, > S. > > вт, 12 мая 2020 г. в 15:11, Данилов Семён : > > > Hello! > > > > I would like to propose moving

Re: Moving binary metadata to PDS storage folder

2020-05-12 Thread Вячеслав Коптилин
Hello Semyon, This is a good and long-awaited improvement! Thank you for your efforts! Thanks, S. вт, 12 мая 2020 г. в 15:11, Данилов Семён : > Hello! > > I would like to propose moving /binary_meta and /marshaller folders to the > PDS folder. > > Motivation: data, directly related to the

Moving binary metadata to PDS storage folder

2020-05-12 Thread Данилов Семён
Hello! I would like to propose moving /binary_meta and /marshaller folders to the PDS folder. Motivation: data, directly related to the persistence, is stored outside the persistence dir, which can lead to various issues and also is not very convenient to use. In particular, with k8s,