Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Yakov Zhdanov
+1 for warning about potential deadlock and improving javadocs --Yakov

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Valentin Kulichenko
I like the idea to print out a warning if unsorted map is provided. The fact that there are tons of other ways to get a deadlock doesn't mean that we should ignore this case which is actually very common. -Val On Tue, Oct 31, 2017 at 12:34 PM, Dmitry Pavlov wrote: > Vladimir, thank you. Good po

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Dmitry Pavlov
Vladimir, thank you. Good point for optimistic tx, but still putAll usage require using sorted collections. User, of course, may broke this scenario also by using sorted maps with incorrect custom comparators (one asc, one desc). Producing warning in code for Igntie for potential deadlock is good u

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Denis Magda
Vladimir, That’s an oversight and lack of explanation on our side. The goal is to avoid unexpected deadlocks when a user passed a HashMap in cache.putAll. Before printing out a warning we can filter out OPTIMISTIC/SERIALIZABLE and other suitable scenarios. So you’re free to offer another solut

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Vladimir Ozerov
Guys, Printing a warning in this case is really strange idea. First, how would explain it in case of OPTIMISTIC/SERIALIZABLE transactions where deadlocks are impossible? Second, what would you do in case tow sorted maps are passed one by one in a transaction? User still may have a deadlock. :Last,

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Denis Magda
Here is a ticket for the improvement: https://issues.apache.org/jira/browse/IGNITE-6804 — Denis > On Oct 31, 2017, at 3:55 AM, Dmitry Pavlov wrote: > > I agree with Denis, if we don't have such warning we should continiously warn > users in wiki pages/blogs/presentations. It is simpler to warn

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Dmitry Pavlov
I agree with Denis, if we don't have such warning we should continiously warn users in wiki pages/blogs/presentations. It is simpler to warn from code. What do you think if we will issue warning only if size > 1. HashMap with 1 item will not cause deadlock. Moreover where can be some custom single

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-30 Thread Dmitriy Setrakyan
Denis, We should definitely print out a thorough warning if HashMap is passed into a bulk method (instead of SortedMap). However, we should make sure that we only print that warning once and not ever time the API is called. Can you please file a ticket for 2.4? D. On Thu, Oct 26, 2017 at 11:05

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-26 Thread Denis Magda
+ dev list Igniters, that’s a relevant point below. Newcomers to Ignite tend to stumble on deadlocks simply because the keys are passed in an unordered HashMap. Propose to do the following: - update bulk operations Java doc. - print out a warning if a HashMap is used and its exceeds one element.

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-26 Thread Denis Magda
> Performance is definitely improving. However, the worry is whether or not > OffHeap will be cleaned up properly and whether memory leaks will be caused > by the cache meta information of the Ignite Java Heap. > > Do you have any other concerns? There is nothing I’m concerned about here. — Deni

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread dark
Many people seem to be more likely to send Cache entries in bulk via a HashMap. How do you expose a warning statement by checking if the TreeMap is putAll inside the code? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread dark
Thank you for your kind comments. EagerTttl is usually useful. However, it did not match our situation. We put 40k/s entry to ignite server node. If the backup option is enabled to prevent some loss of data, large cache entries will cause simultaneous expiration. (We expiry the Cache entry after

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread Denis Magda
Hi, Yes, you’ve already grasped how to fix the deadlock - feed the keys sorted in TreeMap to bulk operations such as putAll or removeAll. Keys in HashMap are unordered which leads to a deadlock if there are multiple bulk updates running in parallel. Furthermore, you might consider ‘eagerTtl’ para

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread 김성진
I think I'm talking to myself and giving an answer. lol Maybe this is the issue. I am doing putAllAsync based on HashMap structure. I hope that it will be fine to replace this part with TreeMap. Reference: http://apache-ignite-users.70518.x6.nabble.com/putAll-stoping-at-600-entries-td817.html I

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread 김성진
A similar issue has re-emerged. When I looked at Stackoverflow, there was a user similar to me. https://stackoverflow.com/questions/45028962/possible-starvation-in-striped-pool-with-deadlock-true-apache-ignite To summarize, I am sending a random value of a pattern like Timestamp_a.b.c to the key o

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread 김성진
Additionally, Client use the cache.putAllAsync () call. If you look at the Ignite log, you see a method call like updateAllAsyncInternal0. At the same time, does the client have a lock issue when it asynchronously calls after sending a cache entry? :( 2017-10-21 21:06 GMT+09:00 dark : > Hello, I

Ignite 2.0.0 GridUnsafe unmonitor

2017-10-21 Thread dark
Hello, I'm using Ignite 2.0.0, and I would like to ask if you have any doubts about the deadlock. The first use pattern is to create a new cache time unit, and after a certain period of time, it will perform Destroy. Example) We create a cache that keeps the data of the 3-minute cycle as shown be