> > I can’t agree with the «temporary» design. > We have neither design nor IEP or contributor who can fix current behavior. > And, if I understand Alexey Goncharyuk correctly, current behavior was > implemented intentionally.
Alex, what do you think? Are we on the same page that desired behavior for the deactivation is to keep data of all in-memory caches, even though it was intentionally implemented in 2.0 another way? On Tue, Mar 24, 2020 at 12:21 PM Nikolay Izhikov <nizhi...@apache.org> wrote: > Hello, Ivan. > > > I believe we should fix the issue instead of adapting API to temporary > flaws. > > Agree. Let’s fix it. > > > I think that clear description of active(false) impact in the > documentation is more than enough > > I can’t agree with this point. > > We shouldn’t imply the assumption that every user reads the whole > documentation and completely understand the consequences of the > deactivation command. > > This whole thread shows that even active core developers don't understand > it. > > So my proposal is to remove --force flag only after we fix deactivation. > > > To sum it up, the question is whether we should reflect temporary system > design flaws in the API > > I can’t agree with the «temporary» design. > We have neither design nor IEP or contributor who can fix current behavior. > And, if I understand Alexey Goncharyuk correctly, current behavior was > implemented intentionally. > > So, my understanding that current implementation would be here for a while. > And after we fix it I totally support removing —force flag. > > > 24 марта 2020 г., в 12:06, Ivan Rakov <ivan.glu...@gmail.com> > написал(а): > > > >> > >> I think the only question is - Do we need —force flag in Java API or > not. > > > > From my perspective, there's also no agreement that it should be present > > in the thin clients' API. For instance, I think it shouldn't. > > > > As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* > purpose. > >> Can you provide a simple reproducer when in-memory data not cleared on > >> deactivation? > > > > Preserving in-memory data isn't implemented so far, so I can't provide a > > reproducer. My point is that we are halfway through it: we can build a > > solution based on IGNITE_REUSE_MEMORY_ON_DEACTIVATE and additional logic > > with reusing memory pages. > > > > For me, the ultimate value of Ignite into real production environment is > >> user data. > >> If we have some cases when data is lost - we should avoid it as hard as > we > >> can. > >> > >> So, for now, this flag required. > > > > Totally agree that sudden vanishing of user data is unacceptable. But I > > don't see how it implies that we have to solve this issue by tangling > > public API. If we see that system behaves incorrectly, I believe we > should > > fix the issue instead of adapting API to temporary flaws. I think that > > clear description of active(false) impact in the documentation is more > than > > enough: on the one hand, if user didn't read documentation for the method > > he calls, he can't complain about the consequences; on the other hand, if > > user decided to deactivate the cluster for no matter what, -force flag > will > > barely stop him. > > We anyway have enough time before 2.9 to implement a proper solution. > > > > To sum it up, the question is whether we should reflect temporary system > > design flaws in the API. I think, we surely shouldn't: API certainly > lives > > longer and is not intended to collect workarounds for all bugs that are > > already fixed or planned to be fixed. > > We can collect more opinions on this. > > > > On Tue, Mar 24, 2020 at 10:22 AM Nikolay Izhikov <nizhi...@apache.org> > > wrote: > > > >> Alexey. > >> > >> Having the way to silently vanish user data is even worse. > >> So I’m strictly against removing —force flag. > >> > >>> 24 марта 2020 г., в 10:16, Alexei Scherbakov < > >> alexey.scherbak...@gmail.com> написал(а): > >>> > >>> Nikolay, > >>> > >>> I'm on the same page with Ivan. > >>> > >>> Having "force" flag in public API as preposterous as having it in > >>> System.exit. > >>> For me it looks like badly designed API. > >>> If a call to some method is dangerous it should be clearly specified in > >> the > >>> javadoc. > >>> I'm also against some "temporary" API. > >>> > >>> We should: > >>> > >>> 1. Partially remove IGNITE-12701 except javadoc part. Note control.sh > >> for a > >>> long time has support for a confirmation on deactivation (interactive > >> mode). > >>> 2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory > >> content > >>> after deactivation. We should start working on restoring page memory > >> state > >>> after subsequent reactivation. > >>> 3. Describe current behavior for in-memory cache on deactivation in > >> Ignite > >>> documentation. > >>> > >>> > >>> пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <nizhi...@apache.org>: > >>> > >>>> Hello, Ivan. > >>>> > >>>>> Seems like we don't have a final agreement on whether we should add > >> force > >>>> flag to deactivation API. > >>>> > >>>> I think the only question is - Do we need —force flag in Java API or > >> not. > >>>> > >>>> > >>>>> As a final solution, I'd want to see behavior when all in-memory data > >> is > >>>> available after deactivation and further activation. > >>>> > >>>> Agree. > >>>> > >>>>> I believe it’s possible to don't deallocate memory > >>>>> (like mentioned before, we already can achieve that with > >>>> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded > >> data > >>>> pages on next activation and caches start. > >>>> > >>>> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* > >> purpose. > >>>> Can you provide a simple reproducer when in-memory data not cleared on > >>>> deactivation? > >>>> > >>>>> Considering this, do we really need to introduce force flag as a > >>>> temporary precaution? > >>>> > >>>> My answer is yes we need it. > >>>> Right now, we can’t prevent data loss on deactivation for in-memory > >> caches. > >>>> > >>>> For me, the ultimate value of Ignite into real production environment > is > >>>> user data. > >>>> If we have some cases when data is lost - we should avoid it as hard > as > >> we > >>>> can. > >>>> > >>>> So, for now, this flag required. > >>>> > >>>>> I suggest to rollback [2] from AI master, stop working on [1] and > focus > >>>> on how to implement keeping in-memory data after the deactivation. > >>>> > >>>> I think we can remove —force flag only after implementation of keeping > >>>> in-memory data on deactivation would be finished. > >>>> If that happens before 2.9 then we can have clearer API. > >>>> > >>>>> 23 марта 2020 г., в 18:47, Ivan Rakov <ivan.glu...@gmail.com> > >>>> написал(а): > >>>>> > >>>>> Folks, > >>>>> > >>>>> Let's revive this discussion until it's too late and all API changes > >> are > >>>>> merged to master [1]. > >>>>> Seems like we don't have a final agreement on whether we should add > >> force > >>>>> flag to deactivation API. > >>>>> > >>>>> First of all, I think we are all on the same page that in-memory > caches > >>>>> data vanishing on deactivation is counter-intuitive and dangerous. > As a > >>>>> final solution, I'd want to see behavior when all in-memory data is > >>>>> available after deactivation and further activation. I believe it's > >>>>> possible to don't deallocate memory (like mentioned before, we > already > >>>> can > >>>>> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and > carefully > >>>>> reuse all loaded data pages on next activation and caches start. > >>>>> > >>>>> Also, this is a wider question, but: do we understand what cluster > >>>>> deactivation is actually intended for? I can only think of two cases: > >>>>> - graceful cluster shutdown: an ability to cut checkpoints and to end > >>>>> transactional load consistently prior to further stop of all nodes > >>>>> - blocking all API (both reads and writes) due to some maintenance > >>>>> Neither of them requires forcefully clearing all in-memory data on > >>>>> deactivation. If everyone agrees, from now on we should assume data > >>>>> clearing as system design flaw that should be fixed, not as possible > >>>>> scenario which we should support on the API level. > >>>>> > >>>>> Considering this, do we really need to introduce force flag as a > >>>> temporary > >>>>> precaution? I have at least two reasons against it: > >>>>> 1) Once API was changed and released, we have to support it until the > >>>> next > >>>>> major release. If we all understand that data vanishing issue is > >>>> fixable, I > >>>>> believe we shouldn't engrave in the API flags that will become > >> pointless. > >>>>> 2) More personal one, but I'm against any force flags in the API. > This > >>>>> makes API harder to understand; more than that, the presence of such > >>>> flags > >>>>> just highlights that API is poorly designed. > >>>>> > >>>>> I suggest to rollback [2] from AI master, stop working on [1] and > focus > >>>> on > >>>>> how to implement keeping in-memory data after the deactivation. > >>>>> I think we can still require user consent for deactivation via > >> control.sh > >>>>> (it already requires --yes) and JMX. > >>>>> > >>>>> Thoughts? > >>>>> > >>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12614 > >>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12701 > >>>>> > >>>>> -- > >>>>> Ivan > >>>>> > >>>>> > >>>>> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vlads...@gmail.com > > > >>>> wrote: > >>>>> > >>>>>> Nikolay, I think we should reconsider clearing at least system > caches > >>>>>> when deactivating. > >>>>>> > >>>>>> 17.03.2020 14:18, Nikolay Izhikov пишет: > >>>>>>> Hello, Vladimir. > >>>>>>> > >>>>>>> I don’t get it. > >>>>>>> > >>>>>>> What is your proposal? > >>>>>>> What we should do? > >>>>>>> > >>>>>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vlads...@gmail.com> > >>>>>> написал(а): > >>>>>>>> > >>>>>>>> Nikolay, hi. > >>>>>>>> > >>>>>>>>>>> And should be covered with the —force parameter we added. > >>>>>>>> As fix for user cases - yes. My idea is to emphasize overall > ability > >>>> to > >>>>>> lose various objects, not only data. Probably might be reconsidered > in > >>>>>> future. > >>>>>>>> > >>>>>>>> > >>>>>>>> 17.03.2020 13:49, Nikolay Izhikov пишет: > >>>>>>>>> Hello, Vladimir. > >>>>>>>>> > >>>>>>>>> If there is at lease one persistent data region then system data > >>>>>> region also becomes persistent. > >>>>>>>>> Your example applies only to pure in-memory clusters. > >>>>>>>>> > >>>>>>>>> And should be covered with the —force parameter we added. > >>>>>>>>> > >>>>>>>>> What do you think? > >>>>>>>>> > >>>>>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vlads...@gmail.com > > > >>>>>> написал(а): > >>>>>>>>>> > >>>>>>>>>> Hi, all. > >>>>>>>>>> > >>>>>>>>>> Fixes for control.sh and the REST have been merged. Could anyone > >>>> take > >>>>>> a look to the previous email with an issue? Isn't this conductvery > >>>> wierd? > >>>>>>>>>> > >>>>>> > >>>> > >>>> > >>> > >>> -- > >>> > >>> Best regards, > >>> Alexei Scherbakov > >> > >> > >
