Apache Ignite vs alluxio

2016-11-22 Thread Kaiming Wan
Hi, Recently, I have used alluxio for about two month. However, it does't satisfy our requirement. We want to use alluxio as a memory cache layer which can easily interagte with Hadoop or Spark to accelerate our spark or map-reduce job. Unfortunately, it doesn't work. I have done many tests

Re: Blocked get operation problem

2016-11-22 Thread Tolga Kavukcu
Hi Val, When i investigate thread dump of our application, I see a lot of threads waiting on a lock. For example i have 40 threads trying to access same key at same time. So only one of them can read. I realized this situation because, we have switched our cache provider to ignite and see some

Re: Cluster can not let more than one client to do continuous queries

2016-11-22 Thread ght230
When first client start, it is OK. When one more client start, the client will pause at following place: [08:52:08] Ignite documentation: http://ignite.apache.org [08:52:08] [08:52:08] Quiet mode. [08:52:08] ^-- Logging to file '/home/ght/ignite-1.6.8/work/log/ignite-0127ad43.0.log' [08:52:08]

Re: Apache Spark & Ignite Integration

2016-11-22 Thread vkulichenko
This is true, and this is the nature of RDD in my view, which is mostly intended to be read-only. As for embedded mode, it's more for testing in my understanding. In this mode server nodes will be started on executors which I guess will be stopped when the application is stopped. If so, you lose

Re: auto loadcache when server startup

2016-11-22 Thread vkulichenko
Hi Shawn, You can utilize lifecycle bean [1] for this. Configure it on each node and call localLoadCache in the implementation. [1] https://apacheignite.readme.io/docs/ignite-life-cycle -Val -- View this message in context:

Re: OutOfMemoryError with GridLogThrottle

2016-11-22 Thread vkulichenko
GridLogThrottle class indeed doesn't limit the number of saved messages and therefore is not very safe from memory consumption standpoint. However, having more than a million of such unique message in throttled log seems a bit weird to me. Can you investigate the content of the map in more detail?

Re: Blocked get operation problem

2016-11-22 Thread vkulichenko
Hi, Yes, Ignite acquires a short-time Java-level lock on entry each time you access it (either read or write). However, any concurrent updates or reads of different will not interfere with each other. Why do you think this is causing a performance issue? -Val -- View this message in context:

Blocked get operation problem

2016-11-22 Thread Tolga Kavukcu
Hi Everyone, We encountered a performance issue in production. Our code reaches ignite cache so frequently for a get operation. After some investigation we found out that threads are blocked in get opration. Look at stack trace. java.lang.Thread.State: BLOCKED (on object monitor) at

Re: Memory consumption in apache ignite

2016-11-22 Thread rishi007bansod
Hi dkarachentsev, I did loading of data from client node as suggested, but still I think values are getting stored in both serialized and de-serialized format. I have attached my heap dump. Which objects in this dump represents serialized and de-serialized form. Also instead of QueryEntity, I

auto loadcache when server startup

2016-11-22 Thread Shawn Du
Hi, It is possible to load all cache when server startup by configuration? Suppose I have configured store factory in configuration and implement loadcache method for CacheStore Thanks Shawn

Re: Cluster can not let more than one client to do continuous queries

2016-11-22 Thread dkarachentsev
Hi, What do you mean "another client can not join the cluster to do continuous queries"? Is there some specific error? Could you please provide logs from client and server? -- View this message in context:

Re: Restarting Tomcat container and Rebalancing

2016-11-22 Thread dkarachentsev
Hi, You can get to know if rebalancing is started/finished only via events and how you'll use this information is up to you :) Thanks! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Restarting-Tomcat-container-and-Rebalancing-tp9033p9128.html Sent from the

Re: How to apply the distributed lock on all the ignite remote instances

2016-11-22 Thread dkarachentsev
Hi, could you please describe in more details what do you want to achieve? If you want to aquire a distributed lock on some key you may use pessimistic transaction with REPEATEABLE_READ isolation level [1] on transactional cache. [1]

Re: while updating the cache, can I get notification

2016-11-22 Thread dkarachentsev
Don't continuous queries[1] suite your case? [1] https://apacheignite.readme.io/docs/continuous-queries -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/while-updating-the-cache-can-I-get-notification-tp9122p9125.html Sent from the Apache Ignite Users mailing