Re: Regarding Partition Map exchange Triggers

[VeenaMithare]

Hi Ilya,

thank you !

regards,
Veena



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Regarding Partition Map exchange Triggers

[Ilya Kasnacheev]

Hello!

cache.close() will not cause a PME. cache.destroy() (or DROP TABLE) will.

Regards,
-- 
Ilya Kasnacheev


пн, 14 дек. 2020 г. в 12:39, VeenaMithare :

> >>Looking at your stack traces it seems that the cache you accessed in the
> SQL query was stopped right before the new client node is joined.
> >>All topology events are processed one-by-one, so the initial blocking
> time
> was caused by cache stop PME rather than client node join.
>
>
> What is the guidance regarding cleanly shutting down an client or server (
> for deployment etc. )? Is it a good practice to call cache.close when a
> client or a server node is being shutdown ?
>
> Cache closed but not destroyed means it can be still accessed by other
> nodes
> by calling ignite.cache - what is the purpose of calling PME when
> cache.close is invoked ?
>
> Is there any difference in calling cache.close on the client( non baselined
> node ) or server( baselined node ) node ?
>
> regards,
> Veena.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Regarding Partition Map exchange Triggers

[VeenaMithare]

>>Looking at your stack traces it seems that the cache you accessed in the
SQL query was stopped right before the new client node is joined.
>>All topology events are processed one-by-one, so the initial blocking time
was caused by cache stop PME rather than client node join.


What is the guidance regarding cleanly shutting down an client or server (
for deployment etc. )? Is it a good practice to call cache.close when a
client or a server node is being shutdown ? 

Cache closed but not destroyed means it can be still accessed by other nodes
by calling ignite.cache - what is the purpose of calling PME when
cache.close is invoked ?

Is there any difference in calling cache.close on the client( non baselined
node ) or server( baselined node ) node ?

regards,
Veena.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Regarding Partition Map exchange Triggers

[Pavel Kovalenko]

Hi

> According to the exception log of the following topic, a client node
joins the cluster and blocks a SQL query on the transactional cache. Is
this true?
> Now it seems that the relevant explanations are confusing?

Looking at your stack traces it seems that the cache you accessed in the
SQL query was stopped right before the new client node is joined.
All topology events are processed one-by-one, so the initial blocking time
was caused by cache stop PME rather than client node join.
However, SQL and cache endpoints have different mechanisms for dealing with
partitions affinity. My explanations in this topic were about only cache
operations.
For SQL it seems that client node join can still block SQL query for a
time, but client PME is very fast, so its impact should be minimal.



сб, 12 дек. 2020 г. в 04:34, 38797715 <38797...@qq.com>:

> Hi,
>
> According to the exception log of the following topic, a client node joins
> the cluster and blocks a SQL query on the transactional cache. Is this true?
>
>
> http://apache-ignite-users.70518.x6.nabble.com/Failed-to-wait-for-affinity-ready-future-for-topology-version-AffinityTopologyVersion-td34823.html
>
> Now it seems that the relevant explanations are confusing?
> 在 2020/12/11 下午8:21, Pavel Kovalenko 写道:
>
> Hi,
>
> I think it's wrong information on the wiki that PME is not triggered for
> some cases. It should be fixed.
> Actually, PME is triggered in all cases but for some of them it doesn't
> block cache operations or the time of blocking is minimized.
> Most optimizations for minimizing the blocking time of PME have been done
> in Ignite 2.8.
>
> Thick client join/left PME - doesn't block operations at all.
>
> Other events can be ordered by their potential blocking time:
> 1. Non-baseline node left/join - minimal
> 2. Baseline node stop/left
> 3. Baseline node join
> 4. Baseline change - heaviest operation
>
> > *for the end user , is this invoked when we do ignite.getOrCreate( xx )
> and ignite.cache(xx )*
>
> Yes.
>
> пт, 11 дек. 2020 г. в 14:55, VeenaMithare :
>
>> Hi ,
>>
>>
>> I can see the triggers for PME initiation here :
>>
>> https://cwiki.apache.org/confluence/display/IGNITE/%28Partition+Map%29+Exchange+-+under+the+hood
>>
>> Triggers
>> Events which causes exchange
>>
>> Topology events:
>>
>> Node Join (EVT_NODE_JOINED) - new node discovered and joined topology
>> (exchange is done after a node is included into the ring). This event
>> doesn't trigger the PME if a thick client connects the cluster and an
>> Ignite
>> version is 2.8 or later.
>>
>>
>> --> *This means in ignite 2.8 or higher, this is triggered only if nodes
>> that participate in the baseline topology are added ?*
>>
>>
>> Node Left (EVT_NODE_LEFT) - correct shutdown with call ignite.close. This
>> event doesn't trigger the PME in Ignite 2.8 and later versions if a node
>> belonging to an existing baseline topology leaves.
>>
>> --> *This means this is not triggered at all 2.8.1 or higher if shutdown
>> cleanly ? i.e. if this is called : Ignition.stop(false) *
>>
>>
>> Node Failed (EVT_NODE_FAILED) - detected unresponsive node, probably
>> crashed
>> and is considered failed
>>
>> --> *This means this is  triggered at all 2.8.1 or higher for baseline
>> nodes
>> or any thick client node ?*
>>
>> Custom events:
>>
>> Activation / Deactivation / Baseline topology set -
>> ChangeGlobalStateMessage
>> Dynamic cache start / Dynamic cache stop - DynamicCacheChangeBatch
>>
>> --> *for the end user , is this invoked when we do ignite.getOrCreate( xx
>> )
>> and ignite.cache(xx )*
>>
>>
>> Snapshot create / restore - SnapshotDiscoveryMessage
>> Global WAL enable / disable - WalStateAbstractMessage
>> Late affinity assignment - CacheAffinityChangeMessage
>>
>>
>> regards,
>> Veena.
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: Regarding Partition Map exchange Triggers

[38797715]

Hi,

According to the exception log of the following topic, a client node 
joins the cluster and blocks a SQL query on the transactional cache. Is 
this true?


http://apache-ignite-users.70518.x6.nabble.com/Failed-to-wait-for-affinity-ready-future-for-topology-version-AffinityTopologyVersion-td34823.html

Now it seems that the relevant explanations are confusing?

在 2020/12/11 下午8:21, Pavel Kovalenko 写道:

Hi,

I think it's wrong information on the wiki that PME is not triggered 
for some cases. It should be fixed.
Actually, PME is triggered in all cases but for some of them it 
doesn't block cache operations or the time of blocking is minimized.
Most optimizations for minimizing the blocking time of PME have been 
done in Ignite 2.8.


Thick client join/left PME - doesn't block operations at all.

Other events can be ordered by their potential blocking time:
1. Non-baseline node left/join - minimal
2. Baseline node stop/left
3. Baseline node join
4. Baseline change - heaviest operation

> *for the end user , is this invoked when we do ignite.getOrCreate( xx )
and ignite.cache(xx )*

Yes.

пт, 11 дек. 2020 г. в 14:55, VeenaMithare >:


Hi ,


I can see the triggers for PME initiation here :

https://cwiki.apache.org/confluence/display/IGNITE/%28Partition+Map%29+Exchange+-+under+the+hood



Triggers
Events which causes exchange

Topology events:

Node Join (EVT_NODE_JOINED) - new node discovered and joined topology
(exchange is done after a node is included into the ring). This event
doesn't trigger the PME if a thick client connects the cluster and
an Ignite
version is 2.8 or later.


--> *This means in ignite 2.8 or higher, this is triggered only if
nodes
that participate in the baseline topology are added ?*


Node Left (EVT_NODE_LEFT) - correct shutdown with call
ignite.close. This
event doesn't trigger the PME in Ignite 2.8 and later versions if
a node
belonging to an existing baseline topology leaves.

--> *This means this is not triggered at all 2.8.1 or higher if
shutdown
cleanly ? i.e. if this is called : Ignition.stop(false) *


Node Failed (EVT_NODE_FAILED) - detected unresponsive node,
probably crashed
and is considered failed

--> *This means this is  triggered at all 2.8.1 or higher for
baseline nodes
or any thick client node ?*

Custom events:

Activation / Deactivation / Baseline topology set -
ChangeGlobalStateMessage
Dynamic cache start / Dynamic cache stop - DynamicCacheChangeBatch

--> *for the end user , is this invoked when we do
ignite.getOrCreate( xx )
and ignite.cache(xx )*


Snapshot create / restore - SnapshotDiscoveryMessage
Global WAL enable / disable - WalStateAbstractMessage
Late affinity assignment - CacheAffinityChangeMessage


regards,
Veena.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Regarding Partition Map exchange Triggers

[Pavel Kovalenko]

Your thoughts are right.
If cache exists no PME will be started.
If doesn't exist - getOrCreate() method will create it and start PME,
cache() method will throw an exception or return null (doesn't remember
what exactly)

пт, 11 дек. 2020 г. в 17:48, VeenaMithare :

> HI Pavel,
>
> Thank you for the reply.
>
> >>> *for the end user , is this invoked when we do ignite.getOrCreate( xx )
> and ignite.cache(xx )*
>
> >>Yes.
>
>
> getOrCreateCache would create a cache if doesnt exist . I would guess this
> would have the effect of starting the cache if doesnt exist . And I think
> this would start the PME.
>
> And ignite.cache would only get an instance of the existing cache. In this
> case, I would imagine the cache already exists and a reference to the cache
> is returned. I would have thought this WOULD NOT have the effect of
> starting
> the cache and hence should not start the PME.
>
> Please guide. Below is the documentation of getOrCreateCache and
> ignite.cache from the javadocs.
>
> getOrCreateCache(CacheConfiguration cacheCfg)
> Gets existing cache with the given name or creates new one with the given
> configuration.
>
>  IgniteCache cache(String name)
>   throws javax.cache.CacheException
> Gets an instance of IgniteCache API. IgniteCache is a fully-compatible
> implementation of JCache (JSR 107) specification.
>
> regards,
> Veena.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Regarding Partition Map exchange Triggers

[VeenaMithare]

HI Pavel,

Thank you for the reply. 

>>> *for the end user , is this invoked when we do ignite.getOrCreate( xx )
and ignite.cache(xx )*

>>Yes.


getOrCreateCache would create a cache if doesnt exist . I would guess this
would have the effect of starting the cache if doesnt exist . And I think
this would start the PME.

And ignite.cache would only get an instance of the existing cache. In this
case, I would imagine the cache already exists and a reference to the cache
is returned. I would have thought this WOULD NOT have the effect of starting
the cache and hence should not start the PME. 

Please guide. Below is the documentation of getOrCreateCache and
ignite.cache from the javadocs.

getOrCreateCache(CacheConfiguration cacheCfg)
Gets existing cache with the given name or creates new one with the given
configuration.

 IgniteCache cache(String name)
  throws javax.cache.CacheException
Gets an instance of IgniteCache API. IgniteCache is a fully-compatible
implementation of JCache (JSR 107) specification.

regards,
Veena.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Regarding Partition Map exchange Triggers

[Pavel Kovalenko]

Hi,

I think it's wrong information on the wiki that PME is not triggered for
some cases. It should be fixed.
Actually, PME is triggered in all cases but for some of them it doesn't
block cache operations or the time of blocking is minimized.
Most optimizations for minimizing the blocking time of PME have been done
in Ignite 2.8.

Thick client join/left PME - doesn't block operations at all.

Other events can be ordered by their potential blocking time:
1. Non-baseline node left/join - minimal
2. Baseline node stop/left
3. Baseline node join
4. Baseline change - heaviest operation

> *for the end user , is this invoked when we do ignite.getOrCreate( xx )
and ignite.cache(xx )*

Yes.

пт, 11 дек. 2020 г. в 14:55, VeenaMithare :

> Hi ,
>
>
> I can see the triggers for PME initiation here :
>
> https://cwiki.apache.org/confluence/display/IGNITE/%28Partition+Map%29+Exchange+-+under+the+hood
>
> Triggers
> Events which causes exchange
>
> Topology events:
>
> Node Join (EVT_NODE_JOINED) - new node discovered and joined topology
> (exchange is done after a node is included into the ring). This event
> doesn't trigger the PME if a thick client connects the cluster and an
> Ignite
> version is 2.8 or later.
>
>
> --> *This means in ignite 2.8 or higher, this is triggered only if nodes
> that participate in the baseline topology are added ?*
>
>
> Node Left (EVT_NODE_LEFT) - correct shutdown with call ignite.close. This
> event doesn't trigger the PME in Ignite 2.8 and later versions if a node
> belonging to an existing baseline topology leaves.
>
> --> *This means this is not triggered at all 2.8.1 or higher if shutdown
> cleanly ? i.e. if this is called : Ignition.stop(false) *
>
>
> Node Failed (EVT_NODE_FAILED) - detected unresponsive node, probably
> crashed
> and is considered failed
>
> --> *This means this is  triggered at all 2.8.1 or higher for baseline
> nodes
> or any thick client node ?*
>
> Custom events:
>
> Activation / Deactivation / Baseline topology set -
> ChangeGlobalStateMessage
> Dynamic cache start / Dynamic cache stop - DynamicCacheChangeBatch
>
> --> *for the end user , is this invoked when we do ignite.getOrCreate( xx )
> and ignite.cache(xx )*
>
>
> Snapshot create / restore - SnapshotDiscoveryMessage
> Global WAL enable / disable - WalStateAbstractMessage
> Late affinity assignment - CacheAffinityChangeMessage
>
>
> regards,
> Veena.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>