Hello Vladimir,
> onto anolugous
> @param forceDeactivation If {@code true}, cluster deactivation will be
forced. //Deactivation clears in-memory caches (without persistence)
including the system caches.
My idea was about providing a link to one place with a complete description
instead of copy&p
Slava, hello.
All right, since we have in public API several
/* Deactivation clears in-memory caches (without persistence) including
the system caches./
We should change in the internals
/ @param forceDeactivation If {@code true}, cluster deactivation will
be forced./
onto anolugous
Hi Vladimir,
> There are about 15 places in inner logic with this description.
> I propose balance between code base size and comment completeness.
I agree with Iva and I also think that this approach is not so good.
Perhaps we can add just a link to the one method which will provide a
comprehensi
Ivan, hello.
Thanks. I vote for keeping the comments as they are now :)
Igniters, it seems we are agreed to merge [1]. And the ticked s to be
reverted in future with new designed solution of keeping in-memory data
after deactivation.
Right?
[1] https://issues.apache.org/jira/browse/IGNITE-
I don't think that making javadocs more descriptive can be considered as
harmful code base enlargement.
I'd recommend to extend the docs, but the last word is yours ;)
On Tue, Mar 31, 2020 at 2:44 PM Vladimir Steshin wrote:
> Ivan, hi.
>
> I absolutely agree this particular description is not en
Ivan, hi.
I absolutely agree this particular description is not enough to see the
deactivation issue. I also vote for brief code.
There are about 15 places in inner logic with this description. I
propose balance between code base size and comment completeness.
Should we enlarge code even if
Vladimir,
@param forceDeactivation If {@code true}, cluster deactivation will be
> forced.
It's true that it's possible to infer semantics of forced deactivation from
other parts of API. I just wanted to highlight that exactly this
description explains something that can be guessed by the paramet
Ivan, hi.
1) >>> Is it correct? If we are on the same page, let's proceed this way
It is correct.
2) - In many places in the code I can see the following javadoc
@param forceDeactivation If {@code true}, cluster deactivation will be forced.
In the internal params/flags. You can also fin
Hello, Ivan.
> Is it correct? If we are on the same page, let's proceed this way.
+1 from me for your plan.
> We have removed GridClientClusterState#state(ClusterState) from Java client
> API. Is it a legitimate thing to do? Don't we have to support API
There is no changes in Public API except
Vladimir, Igniters,
Let's emphasize our final plan.
We are going to add --force flags that will be necessary to pass for a
deactivation if there are in-memory caches to:
1) Rest API (already implemented in [1])
2) Command line utility (already implemented in [1])
3) JMX bean (going to be implemen
Hi, Igniters.
I'd like to remind you that cluster can be deactivated by user with 3
utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
not about control.sh. It suggests same approach regardless of the
utility user executes. The task touches *only* *API of the user calls*,
Hi, Igniters.
Note that DeactivateCommand#confirmationPrompt will be ignored in case
of auto confirmation.
I agree that the force flag should be removed when the user data and
datastructures will not be clearing on deactivation. For now, cmd, jmx
and rest interfaces should be covered.
вт, 24 мар
Hello Nikolay,
I am talking about the interactive mode of the control utility, which
requires explicit confirmation from the user.
Please take a look at DeactivateCommand#prepareConfirmation and its usages.
It seems to me, this mode has the same aim as the forceDeactivation flag.
We can change the
>
> Hello, Alexey.
>
> I just repeat our agreement to be on the same page
>
> > The confirmation should only present in the user-facing interfaces.
>
> 1. We should add —force flag to the command.sh deactivation command.
> 2. We should throw the exception if cluster has in-memory caches and
> —forc
Hello, Slava.
Are you talking about this commit [1] (sorry for commit message it’s due to the
Github issue)?
The message for this command for now
«Deactivation stopped. Deactivation clears in-memory caches (without
persistence) including the system caches.»
Is it clear enough?
[1]
https://
Hi Nikolay,
> 1. We should add —force flag to the command.sh deactivation command.
I just checked and it seems that the deactivation command
(control-utility.sh) already has a confirmation option.
Perhaps, we need to clearly state the consequences of using this command
with in-memory caches.
Than
Hello, Alexey.
I just repeat our agreement to be on the same page
> The confirmation should only present in the user-facing interfaces.
1. We should add —force flag to the command.sh deactivation command.
2. We should throw the exception if cluster has in-memory caches and
—force=false.
3. We s
Igniters, Ivan, Nikolay,
I am strongly against adding confirmation flags to any kind of APIs,
whether we change the deactivation behavior or not (even though I agree
that it makes sense to fix the deactivation to not clean up the in-memory
data). The confirmation should only present in the user-fa
>
> I can’t agree with the «temporary» design.
> We have neither design nor IEP or contributor who can fix current behavior.
> And, if I understand Alexey Goncharyuk correctly, current behavior was
> implemented intentionally.
Alex, what do you think? Are we on the same page that desired behavior
Hello, Ivan.
> I believe we should fix the issue instead of adapting API to temporary flaws.
Agree. Let’s fix it.
> I think that clear description of active(false) impact in the documentation
> is more than enough
I can’t agree with this point.
We shouldn’t imply the assumption that every us
>
> I think the only question is - Do we need —force flag in Java API or not.
From my perspective, there's also no agreement that it should be present
in the thin clients' API. For instance, I think it shouldn't.
As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose.
> Can y
Alexey.
Having the way to silently vanish user data is even worse.
So I’m strictly against removing —force flag.
> 24 марта 2020 г., в 10:16, Alexei Scherbakov
> написал(а):
>
> Nikolay,
>
> I'm on the same page with Ivan.
>
> Having "force" flag in public API as preposterous as having it in
Nikolay,
I'm on the same page with Ivan.
Having "force" flag in public API as preposterous as having it in
System.exit.
For me it looks like badly designed API.
If a call to some method is dangerous it should be clearly specified in the
javadoc.
I'm also against some "temporary" API.
We should:
Hello, Ivan.
> Seems like we don't have a final agreement on whether we should add force
flag to deactivation API.
I think the only question is - Do we need —force flag in Java API or not.
> As a final solution, I'd want to see behavior when all in-memory data is
> available after deactivation
Folks,
Let's revive this discussion until it's too late and all API changes are
merged to master [1].
Seems like we don't have a final agreement on whether we should add force
flag to deactivation API.
First of all, I think we are all on the same page that in-memory caches
data vanishing on deact
Nikolay, I think we should reconsider clearing at least system caches
when deactivating.
17.03.2020 14:18, Nikolay Izhikov пишет:
Hello, Vladimir.
I don’t get it.
What is your proposal?
What we should do?
17 марта 2020 г., в 14:11, Vladimir Steshin написал(а):
Nikolay, hi.
And should be
Hello, Vladimir.
I don’t get it.
What is your proposal?
What we should do?
> 17 марта 2020 г., в 14:11, Vladimir Steshin написал(а):
>
> Nikolay, hi.
>
And should be covered with the —force parameter we added.
>
> As fix for user cases - yes. My idea is to emphasize overall ability to
Nikolay, hi.
And should be covered with the —force parameter we added.
As fix for user cases - yes. My idea is to emphasize overall ability to
lose various objects, not only data. Probably might be reconsidered in
future.
17.03.2020 13:49, Nikolay Izhikov пишет:
Hello, Vladimir.
If the
Hello, Vladimir.
If there is at lease one persistent data region then system data region also
becomes persistent.
Your example applies only to pure in-memory clusters.
And should be covered with the —force parameter we added.
What do you think?
> 17 марта 2020 г., в 13:45, Vladimir Steshin на
Hi, all.
Fixes for control.sh and the REST have been merged. Could anyone take a
look to the previous email with an issue? Isn't this conductvery wierd?
Hi, Igniters.
I've found other deactivation issue. Should we consider this behavior as
a bug?
IgniteEx ignite = startGrids(3);
IgniteAtomicLong atomicLong = ignite.atomicLong("atomic", 10L,
true);
IgniteLock lock = ignite.reentrantLock("lock", true, false, true);
Hello, Alexey.
> We need to agree whether IgniteMxBean#active(boolean) and
> Ignite#active(boolean) behaving differently is ok from the user side.
I think it’s not OK.
We should provide consistent behavior for all management APIs: jmx, control.sh,
REST, Java API.
The primary use-case I keep in
I do not think the change size can even be an argument for not doing a
proper improvement. We need to agree whether IgniteMxBean#active(boolean)
and Ignite#active(boolean) behaving differently is ok from the user side.
Alexey, as I mentioned, at now we have same implementation for IgniteMXBean
and Ignite in IgniteKernal. Already is. I agree, the implementations should
be decoupled. But extracting IgniteMXBean from IgniteKernal seems too bulky
solution to me. Tacking in account lot of variants of deactivation meth
пн, 17 февр. 2020 г. в 14:18, Vladimir Steshin :
> > The only reason is the same implementation IgniteMXBean#active(boolean)
> > and Ignite#active(boolean). Looks like we have to choose whether to allow
> > user erase data unexpectedly or to change behavior of the API call.
>
Vladimir, those two
>
> Hi, Vyacheslav.
>
>
>
> >>> In general, it is a caching layer above an existing 3rd party database.
> That is all.
> >>> I don't think so. It does not really matter which persistence is used
> (Ignite native persistence or RDBMS).
>
>
>
> Ok. I think we might also check if read/write-through i
Hi Nikolay,
> 1. Explicitly warn user about consequences of the deactivation.
I absolutely agree it must be mentioned in the Javadoc, documentation, and
etc.
> Require additional confirmation.
it does make sense to me (control utility and jmx) To be honest, I don't
see a reason to introduce this
Hello, Vyacheslav.
> In my humble opinion, until we have a consensus that the existing behavior is
> a bug (and that is the question for now)
I think we have consensus that current behavior is *NOT A BUG*.
It is a design choice.
But, we have consensus that consequences of this choice is not tr
Hello Vladimir,
> We do not know what exactly does *3-rd party persistence*.
>From my point of view, this is clearly stated here
https://www.jcp.org/en/jsr/detail?id=107 |
https://github.com/jsr107/jsr107spec and here
https://apacheignite.readme.io/docs/3rd-party-store.
In general, it is a caching
Vyacheslav,
>>> I am talking about MIXED cluster with persistent cache and *in-memory*
cache which is backed by *3-rd party persistence*.
We do not know what exactly does *3-rd party persistence*. It may store
only specific filtered data, small part of data. I think it is out of
responsibilit
Hi Vladimir,
> if you have only persistent caches, no warning/confirmation is supposed
at all.
I am talking about MIXED cluster with persistent cache and *in-memory*
cache which is backed by *3-rd party persistence*.
> I’m afraid this won’t stop anyone from using old deprecated
IgniteMXBean#activ
Vyacheslav,
>>> Let's assume that I have a mixed cluster with persistent cache and
in-memory cache which is backed by 3-rd party persistence. I see no reason
to throw an exception in that case at least.
if you have only persistent caches, no warning/confirmation is supposed at
all.
>>> Is i
Soory.
I forgot two more methods
IgniteMXBean {
@Deprecated
public String clusterState();
public void clusterState(String state);
}
> 14 февр. 2020 г., в 17:50, Nikolay Izhikov
> написал(а):
>
> Vyacheslav.
>
>
> Let’s see what we got for now:
>
> ```
> Ignite {
>
>@Depr
Vyacheslav.
Let’s see what we got for now:
```
Ignite {
@Deprecated
public boolean active();
@Deprecated
public void active(boolean active);
}
IgniteMXBean {
@Deprecated
public boolean active();
public void active(boolean active); //Please, note, this method is no
Hello Nikolay,
> Should public java API continue to silently clear in-memory caches?
Let's assume that I have a mixed cluster with persistent cache and
in-memory cache which is backed by 3-rd party persistence. I see no reason
to throw an exception in that case at least.
Anyway, this fact should b
* The pr: https://github.com/apache/ignite/pull/7358
One can see the solution closer.
пт, 14 февр. 2020 г. в 15:57, Вячеслав Коптилин :
> Hello Vladimir,
>
> > adding a new method with force flag means old methods change their
> behavior:
> I don't think that changing the behavior of public API
Вячеслав, Hi.
Even if previous behavior is controversial? It allows to erase data
suddenly. I would suggest to slightly change API of IgniteMXBean. It would
solve the problem of single implementation IgniteKernal#active(boolean).
But changing of API is prohibited. Another solution is to extract
im
Vyacheslav.
What is your suggestion for the API?
Single implementation for both Ignite#active(boolean) and
IgniteMXBean#active(boolean)
Should public java API continue to silently clears in-memory caches?
> 14 февр. 2020 г., в 15:56, Вячеслав Коптилин
> написал(а):
>
> Hello Vladimir,
>
>>
Hello Vladimir,
> adding a new method with force flag means old methods change their
behavior:
I don't think that changing the behavior of public API is the right way.
Moreover, I agree with Alex that there is no need to introduce a
"confirmation" flag to the java API.
Thanks,
S.
пт, 14 февр. 20
Alexey, adding a new method with force flag means old methods change their
behavior: they are considered as executed without ‘force‘ flag and can fail
to prevent data loss. Ignite and IgniteMXBean are different interfaces.
Unfortunately, they have same method
void active(boolean active)
When exe
Alexey.
The issue with the existing method.
Single implementation for both Ignite#active(boolean) and
IgniteMXBean#active(boolean)
Should it continue to silently clears in-memory caches?
> 14 февр. 2020 г., в 15:20, Alexey Goncharuk
> написал(а):
>
> Igniters,
>
> Do we really need the conf
Igniters,
Do we really need the confirmation flag on the public API? I absolutely
agree on the CLI and MXBean, but what is the reason for the flag in the
API? It will be specified at the compile time anyway and does not prevent
any user error.
>From the implementation point of view I see no contra
Hi, guys.
I suggest mark as deprecated the IgniteMXBean#active(boolean) method.
The Ignite#active(boolean), IgniteMXBean#active(),
IgniteCluster#active(boolean) methods already deprecated and told to
use the #state(ClusterState)} instead.
Should the old IgniteMXBean#state(ClusterState) method th
Hello, Vladimir.
I think we should do the following:
* Update Ignite documentation and write down the fact that in-memory cache
cleared on deactivation.
* Disallow, by default, deactivation of the cluster that has in-memory cache
with proper error message
«Your cluster has in-memory cac
Hello, everyone.
I’d like to propose to make behavior of cluster deactivation the same
independently on usage type: in code or manually with control.sh or JMX.
Let’s put it in one place like IgniteCluster#state(ClusterState newState,
boolean force) and require forced call for in-memory data.
Wh
Hi, folks.
Another question has raised. Can I remove / rename / bring substitution for
IgniteMXBean#active(boolean active) ? Or at least to deprecate?
The problem is that it interferes with Ignite#active(boolean active). Same
signature for different interfaces. And only one implementation:
Ignite
> To prevent unexpected data loss on
deactivation CLI and JMX should require 'force' flag
Vladimir, please, go for it.
пт, 7 февр. 2020 г., 11:19 Vladimir Steshin :
> Ok. Then we are at the beginning. To prevent unexpected data loss on
> deactivation CLI and JMX should require 'force' flag. If
Ok. Then we are at the beginning. To prevent unexpected data loss on
deactivation CLI and JMX should require 'force' flag. If there are no extra
proposals I'm going to finish the pull request.
чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov :
> Vladimir,
>
> IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn
Vladimir,
IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
clearing.
It allows allocated memory reuse on re-activation to avoid OS specific
issues if allocated heap is rather large.
You are right to create separate ticket for implementing required behavior.
чт, 6 февр. 2020
Or, is flag [1] is actual only for persistence mode? Related ticket [2] is
completely about disabled persistence.
If previous assumption is right, I think, we can involve flag [1] in ticket
[2].
[1]
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[2] https://issues.apach
Denis, Alexei,
Regarding usage of flag
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[1]
When enabled, I think the following test should work. But it fails.
//
@Test
public void testDataPresent()
I believe there might be a consistency-related reason why this flag was
disabled by default for caches that store data in Ignite native
persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
this.
As for the memory-only caches or caches backed up by a CacheStore such as
an RDBMS,
Denis, but why reuse-data-on-deactivate was disabled by default? There
should be a reason for that. Any data consistency issues when node gets
activated anew? We may use both solutions because the flag can be switched
off again.
ср, 5 февр. 2020 г. в 20:47, Denis Magda :
> Hi Vladimir,
>
> Yes, I
Hi Vladimir,
Yes, I'm suggesting us to enable this flag by
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
default instead of introducing --force flag and showing any warnings.
-
Denis
On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin wrote:
> Hello all.
>
> Denis, wh
Hello all.
Denis, which changes exactly? In current implementation of ticket [2] flag
[1] is checked before requiring --force flag and showing any warnings. Do
we need to set reuse-memory-on-deactivate to true by default?
[1]
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIV
That's the best solution for this scenario. Should we readjust the already
created ticket [1] suggesting to implement the changes of Alex Scherbakov
instead?
[1] https://issues.apache.org/jira/browse/IGNITE-12614
-
Denis
On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
alexey.scherbak...@gma
Alexey Goncharuk,
I see no issues with "lost" partitions. The same is possible right now in
persistence mode. Node can be killed and data erased while grid is
deactivated.
After re-join it should rebalance itself from existing owners (if any
exists).
вт, 4 февр. 2020 г. в 10:54, Alexei Scherbako
Folks,
For a long time we have a flag [1]
It does almost what we want here.
I suggest to make this behavior default and rework it to keep data in
memory as well (we already have special "recovery" mode for caches).
[1] org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
I do not mind making this change if we explicitly and clearly define what
'new inactive state' means. What should happen if a partition is lost in
inactive state? What if such node joins back the cluster after? Etc.
пт, 31 янв. 2020 г. в 20:57, Denis Magda :
> Back up Ivan's opinion here. Initial
Back up Ivan's opinion here. Initially, the activation/deactivation was
created for the baseline topology designed for cases with native
persistence. My thinking was that the mechanism itended to prevent data
inconsistencies while nodes with data on the disk will be in the process of
joining the cl
Go for it, Vladimir!
> 31 янв. 2020 г., в 16:15, Vladimir Steshin написал(а):
>
> Hi folks. I created a ticket for this (IGNITE-12614):
>
>
>
>
>
> Currently, anyone is able to silently deactivate cluster with command line
> utility (control.sh). Probably with JMX too. Same for the API:
Hi folks. I created a ticket for this (IGNITE-12614):
Currently, anyone is able to silently deactivate cluster with command line
utility (control.sh). Probably with JMX too. Same for the API: Java API –
Ignite.cluster.state(). That would lead to data loss when the persistence
is off. In-me
For me it looks like some coincidence effect. I understand that we get
such behavior because deactivation works the same way as for
persistent caches. Was cluster activation/deactivation designed and
described for in-memory caches? Current behavior sounds for me a as
big risk. I expect a lot of ups
Because originally the sole purpose of deactivation was resource
deallocation.
чт, 30 янв. 2020 г. в 22:13, Denis Magda :
> Such a revelation for me that data is purged from RAM if someone
> deactivates the cluster. Alex, do you remember why we decided to implement
> it this way initially?
>
> -
Such a revelation for me that data is purged from RAM if someone
deactivates the cluster. Alex, do you remember why we decided to implement
it this way initially?
-
Denis
On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk
wrote:
> I agree on CLI and JMX because those interfaces can be used by a
I agree on CLI and JMX because those interfaces can be used by a system
administrator and can be invoked by mistake.
As for the Java API, personally, I find it strange to add 'force' or
'confirm' flags to it because it is very unlikely that such an invocation
is done by mistake. Such mistakes are
As far as I know we have several places to manage cluster state:
1. Java API - Ignite#cluster#state
2. CLI utility - control.sh
3. JMX beans(can we manage cluster state from JMX?)
I think we should add new parameter to the all management APIs.
What do you think?
> 30 янв. 2020 г., в 12:51, Alex
Nikolay,
This change will affect only the command-line utility, correct? No changes
for the public API are planned?
чт, 30 янв. 2020 г. в 12:46, Nikolay Izhikov :
> Hello, Alexey.
>
> Thanks for the feedback.
> It seems for me as a very unexpected behavior from the users point of view.
>
> I pro
Hello, Alexey.
Thanks for the feedback.
It seems for me as a very unexpected behavior from the users point of view.
I propose to do the following:
* Update Ignite documentation and write down the fact that in-memory cache
cleared on deactivation.
* Disallow, by default, deactivation of the clus
Nikolay,
Inactive state deallocates all possible resources by design, including the
data regions. If data region is not backed by a persistent storage, the
data is lost, this is an expected behavior.
ср, 29 янв. 2020 г. в 19:18, Nikolay Izhikov :
> Hello, Igniters.
>
> I found really confusing r
Hello, Igniters.
I found really confusing results of the simple test.
Data from the in-memory cache are vanished after change cluster state to
INACTIVE/ACTIVE.
Is this a bug or expected behavior?
```
public class ClusterDeactivateTest extends GridCommonAbstractTest {
String name = "my-cach
81 matches
Mail list logo
