Re: Node is unable to join cluster because it has destroyed caches

[Ilya Kasnacheev]

Hello!

It was disabled not just because of potential data loss but because the
cache was resurrected on such start and could break cluster.

Creating cache per operation and destroying afterwards is an anti-pattern,
it can cause all sorts of issues and is better avoided.

Regards,
-- 
Ilya Kasnacheev


чт, 4 июн. 2020 г. в 01:26, xero :

> Hi,
> I tried your suggestion of using a NodeFilter but, is not solving this
> issue. Using a NodeFilter by consistent-id in order to create the cache in
> only one node is creating persistence information in every node:
>
> In the node for which the filter is true (directory size 75MB):
>
> //work/db/node01-0da087c4-c11a-47ce-ad53-0380f0d2c51a//cache-tempBCK0-cd982aa5-c27f-4582-8a3b-b34c5c60a49c/
>
> In the node for which the filter is false  (directory size 8k):
>
> //work/db/node01-0da087c4-c11a-47ce-ad53-0380f0d2c51a//cache-tempBCK0-cd982aa5-c27f-4582-8a3b-b34c5c60a49c/
>
> If the cache is destroyed while *any* of these nodes is down, it won't join
> the cluster again throwing the exception:
> /Caused by: class org.apache.ignite.spi.IgniteSpiException: Joining node
> has
> caches with data which are not presented on cluster, it could mean that
> they
> were already destroyed, to add the node to cluster - remove directories
> with
> the caches[tempBCK0-cd982aa5-c27f-4582-8a3b-b34c5c60a49c]/
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Node is unable to join cluster because it has destroyed caches

[xero]

Hi,
I tried your suggestion of using a NodeFilter but, is not solving this
issue. Using a NodeFilter by consistent-id in order to create the cache in
only one node is creating persistence information in every node:

In the node for which the filter is true (directory size 75MB):
//work/db/node01-0da087c4-c11a-47ce-ad53-0380f0d2c51a//cache-tempBCK0-cd982aa5-c27f-4582-8a3b-b34c5c60a49c/

In the node for which the filter is false  (directory size 8k):
//work/db/node01-0da087c4-c11a-47ce-ad53-0380f0d2c51a//cache-tempBCK0-cd982aa5-c27f-4582-8a3b-b34c5c60a49c/

If the cache is destroyed while *any* of these nodes is down, it won't join
the cluster again throwing the exception:
/Caused by: class org.apache.ignite.spi.IgniteSpiException: Joining node has
caches with data which are not presented on cluster, it could mean that they
were already destroyed, to add the node to cluster - remove directories with
the caches[tempBCK0-cd982aa5-c27f-4582-8a3b-b34c5c60a49c]/



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Node is unable to join cluster because it has destroyed caches

[xero]

Hi, thanks for the prompt response
We can have several of these caches, one for each query (is an exceptional
case but, with load, there can be several simultaneously) that is being
executed so we would like to preserve the persistence to take advantage of
the swapping in case the amount of memory is not enough. That's the reason
we didn't try to move these caches to a region without persistence.

Do you think that this scenario will only happen with persistence enabled?
is it possible to remove the validation?

Is this an expected behavior when persistence is enabled? I thought the node
would know that those caches were removed in its absence and would delete
the data accordingly





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Node is unable to join cluster because it has destroyed caches

[Aleksandr Shapkin]

Hi,

Have you tried to put the temp cache into the different, non persisted,
memory region?

You can also try to use a node filter to control what nodes should store
the cache.


On Tue, Jun 2, 2020, 19:24 xero  wrote:

> Hi Ignite team, We have a use case where a small portion of the dataset
> must answer successive queries that could be relatively expensive. For
> this, we create a temporary cache with that small subset of the dataset and
> operate on that new cache. At the end of the process, that cache is
> destroyed. The caches are created *partitioned* and *persistence* is
> enabled. The problem we are facing is the following. If during the reboot
> of a node, one of those "temporal" caches is destroyed (which is very
> likely since they have a short life), the node is unable to rejoin the
> cluster because it contains information about a cache that no longer
> exists. This is the exception: 2020-06-02T02: 56: 22.326 + 00: 00
> fa72f64b5d0f ignite: Caused by: class
> org.apache.ignite.spi.IgniteSpiException: Joining node has caches with data
> which are not presented on cluster, it could mean that they were already
> destroyed, to add the node to cluster - remove directories with the caches
> [cache-tempX, cache-tempY] Is this an expected behavior? Is it
> possible to skip this validation? Is there any way to indicate that it is a
> temporary cache? Thanks
> --
> Sent from the Apache Ignite Users mailing list archive
>  at Nabble.com.
>

Node is unable to join cluster because it has destroyed caches

[xero]

Hi Ignite team, We have a use case where a small portion of the dataset must
answer successive queries that could be relatively expensive. For this, we
create a temporary cache with that small subset of the dataset and operate
on that new cache. At the end of the process, that cache is destroyed. The
caches are created *partitioned* and *persistence* is enabled.The problem we
are facing is the following. If during the reboot of a node, one of those
"temporal" caches is destroyed (which is very likely since they have a short
life), the node is unable to rejoin the cluster because it contains
information about a cache that no longer exists. This is the
exception:2020-06-02T02: 56: 22.326 + 00: 00 fa72f64b5d0f ignite: Caused by:
class org.apache.ignite.spi.IgniteSpiException: Joining node has caches with
data which are not presented on cluster, it could mean that they were
already destroyed, to add the node to cluster - remove directories with the
caches [cache-tempX, cache-tempY]Is this an expected behavior? Is it
possible to skip this validation? Is there any way to indicate that it is a
temporary cache?Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/