Re: List of running Continuous queries or CacheEntryListener per cache or node

[Dmitry Karachentsev]

Hi Nikolay,

I think it may be useful too. Will try to describe possible API in a ticket.

Thanks!
-Dmitry

21.12.2017 13:18, Nikolay Izhikov пишет:

Hello, Dmitry.

I think it a great idea.

Do we a feature to list all running ComputeTasks?

I, personally, think we have to implement possibility to track all
user-provided tasks - CacheListener, ContinuousQuery, ComputeTasks,
etc.

В Чт, 21/12/2017 в 10:13 +0300, Dmitry Karachentsev пишет:

Crossposting to devlist.

Hi Igniters!

It's might be a nice feature to have - get list of registered
continuous
queries with ability to deregister them.

What do you think?

Thanks!
-Dmitry

20.12.2017 16:59, fefe пишет:

For sanity checks or tests. I want to be sure that I haven't forgot
to
deregister any listener.

Its also very important metric to see how many continuous
queries/listeners
are currently running.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: List of running Continuous queries or CacheEntryListener per cache or node

[Dmitry Karachentsev]

Crossposting to devlist.

Hi Igniters!

It's might be a nice feature to have - get list of registered continuous 
queries with ability to deregister them.


What do you think?

Thanks!
-Dmitry

20.12.2017 16:59, fefe пишет:

For sanity checks or tests. I want to be sure that I haven't forgot to
deregister any listener.

Its also very important metric to see how many continuous queries/listeners
are currently running.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Quick questions on Evictions

[Dmitry Karachentsev]

Hi John,

1. Actually it is. By default data page eviction is disabled 
(DataPageEvictionMode.DISABLED) and when no memory left, it will throw 
IgniteOutOfMemoryException.
2. If you have enabled persistence - no data will be lost, dirty pages 
will be written do disk. In other words Ignite starts swapping pages 
between memory and drive using RANDOM_LRU eviction mode.
3. Ignite 2.0+ works only with pages, so you cannot set eviction per 
entry. If page was modified - it will be marked as dirty and fully 
written to disk.


Thanks!
-Dmitry.

04.09.2017 21:33, John Wilson пишет:

I appreciate the nice explanation. I got a few more questions:

 1. For the case where on-heap caching and persistent are both
disabled, why does Ignite throw out out-dated pages from off-heap?
Why not throw OOM error since the out-dated pages are not backed
by persistent store and throwing away results in data loss?
 2. For off-heap eviction with persistent store enabled, will entries
evicted from data pages be written to disk (in case they are
dirty) or will they be thrown away (which would imply that entries
eligible for eviction must be clean and have already been written
to disk by checkpointing)?
 3.  Checkpointing works by locating dirty pages and writing them out.
If a single entry in a data page is dirty (has been updated since
the last check pointing), will checkpointing write the entire data
page (all entries) to the partition files or just the dirty entry?

Thanks!

On Mon, Sep 4, 2017 at 8:17 AM, dkarachentsev 
mailto:dkarachent...@gridgain.com>> wrote:


Hi,

Assume you have disabled onheapCahe and disabled persistence. In
that case
you may configure only datapage eviction mode, then outdated pages
will be
thrown away, when no free memory will be available for Ignite.
Also you
cannot configure per-entry eviction.

OK, if you enable onheapCache, then Ignite will store on heap
every entry
that was read from off-heap (or disk). Next reads of it will not
require
off-heap readings, and every update will write to off-heap. To
limit size of
onheapCache you may set CacheConfiguration.setEvictionPolicy(),
but it will
not evict off-heap entries.

So, off-heap eviction may be controlled with DataPageEvictionMode
only, and
as you suggested, it clears entries one-by-one from page, checking for
current locks (transaction locks as well). If entry is locked, it
won't be
evicted.

Thanks!
-Dmitry.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Quick questions on Evictions

[Dmitry Karachentsev]

Hi,

Assume you have disabled onheapCahe and disabled persistence. In that 
case you may configure only datapage eviction mode, then outdated pages 
will be thrown away, when no free memory will be available for Ignite. 
Also you cannot configure per-entry eviction.


OK, if you enable onheapCache, then Ignite will store on heap every 
entry that was read from off-heap (or disk). Next reads of it will not 
require off-heap readings, and every update will write to off-heap. To 
limit size of onheapCache you may set 
CacheConfiguration.setEvictionPolicy(), but it will not evict off-heap 
entries.


So, off-heap eviction may be controlled with DataPageEvictionMode only, 
and as you suggested, it clears entries one-by-one from page, checking 
for current locks (transaction locks as well). If entry is locked, it 
won't be evicted.


Thanks!
-Dmitry.

02.09.2017 02:12, John Wilson пишет:

Hi,

I have been reading through Ignite doc and I still have these 
questions. I appreciate your answer.


Assume my Ignite native persistence is *not *enabled:

 1. if on-heap cache is also not enabled, then there are no
entry-based evictions, right?
 2. if on-heap cache is now enabled, does a write to on-heap cache
also results in a write-through or write-behind behavior to the
off-heap entry?
 3. If on-heap cache is not enabled but data page eviction mode is
enabled, then where do evicted pages from off-heap go/written to?

and, need confirmation on how data page eviction is implemented:

4. when a data page eviction is initiated, Ignite works by iterating 
through each entry in the page and evicting entries one by one. It may 
happen that certain entries may be involved in active transactions and 
hence certain entries may not be evicted at all.



Thanks,

Re: In which scenario the ignite nodes will shutdown by itself

[Dmitry Karachentsev]

Hi Aaron,

I was wrong saying that copyOnRead=false consumes heap, it's not.

> if Reside in heap queue and more than batchSize,  will those data be 
lost before DB recover?
Yes, if your store throwing exceptions, and flushQueue is full you may 
start losing updates, but there will be warning in logs.


There should not be so many GridLocalCacheEntry instances. Could you 
find the neares roots for them in dump?


Thanks!
-Dmitry.

25.08.2017 12:21, aa...@tophold.com пишет:

Thanks Dmitry!

At very begin we did not specify the copyOnRead; I remember in another 
thread you suggest us to set to false to save memory?


JavaDoc: "Also if this flag is set copies are created for values 
passed to {@link CacheInterceptor} and to {@link CacheEntryProcessor}."


Also if set this flag to true will the read have to read from 
off-heap? will this meaning be slower?


if Reside in heap queue and more than batchSize,  will those data be 
lost before DB recover?


Thanks for your time!

Reards
Aaron

aa...@tophold.com

*From:* dkarachentsev 
*Date:* 2017-08-25 15:33
*To:* user 
*Subject:* Re: Re: In which scenario the ignite nodes will
shutdown by itself
Hi Aaron,
If data could not be stored to DB, they will reside in heap queue,
but no
more than batchSize.
I see you set copyOnRead to false, this property forces ignite to keep
object on-heap, why do you need it?
Thanks!
-Dmitry.
--
View this message in context:

http://apache-ignite-users.70518.x6.nabble.com/In-which-scenario-the-ignite-nodes-will-shutdown-by-itself-tp16192p16418.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Dmitry Karachentsev]

Hi Nick,

Unfortunately there is no way to use custom class loader without hacks 
with internal API. I've opened a ticket for this feature [1], it could 
be useful in such cases, please check.


[1] https://issues.apache.org/jira/browse/IGNITE-5038

Thanks!
-Dmitry.

19.04.2017 20:09, npordash пишет:

Thanks, Dmitry!

This is using Ignite 1.9. The stack trace is pretty straight forward:



Putting things into caches works just fine (f.e. instances of that Namespace
class), but pulling them out does not since the cache is only taking
Ignite's classloader into account. For the time being I've had to resort to
a hack where read operations use a binary view of the cache and then I have
to cast entries to BinaryObjectImpl or BinaryObjectOffheapImpl in order to
access (or copy) the backing array and then delegate to
Marshaller.unumarshal(byte[], ClassLoader) so the class can be found.

-Nick



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp12055p12087.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Can I let Ignite to do rebalance at a particular time?

[Dmitry Karachentsev]

Hi,

Totally agree wit that. And it should be possible to start re-balancing 
from client node too.


> But I think it is more convenient to start rebalancing manually from 
management console, is it right?


I don't see the difference on how to start re-balancing: from Ignite API 
or console, result will be the same.


-Dmitry.

24.03.2017 02:55, Valentin Kulichenko пишет:

Cross-posting to dev.

Guys,

Is there any reason for such implementation? Does it ever makes sense 
to trigger rebalancing on only one node?


I think rebalance() method should broadcast automatically and do the 
job on all nodes. Just in case, we can also add localRebalance() 
method. This will be more intuitive and also more consistent with 
other APIs, like loadCache/localLoadCache.


-Val

On Wed, Mar 22, 2017 at 11:52 PM, ght230 > wrote:


Thank you for your guidance.

I will try calling IgniteCache.rebalance()  via compute broadcast
task.

But I think it is more convenient to start rebalancing manually from
management console, is it right?



--
View this message in context:

http://apache-ignite-users.70518.x6.nabble.com/Can-I-let-Ignite-to-do-rebalance-at-a-particular-time-tp11301p11381.html


Sent from the Apache Ignite Users mailing list archive at Nabble.com.

DataStreamer pool

[Dmitry Karachentsev]

Hi igniters!

Using data streamer from compute tasks may cause starvation, because 
both operations are executed in the same public pool. Is it possible to 
add configuration for separate thread pool for data streamer?


Thanks!

Dmitry.