Re: Partition Loss Policy options

2018-10-02 Thread Roman Novichenok
Anton, thanks for quick response. Not sure if I'm setting wrong expectations. Just tried sql query and that exhibited the same behavior. Created a pull request with a test: https://github.com/novicr/ignite/pull/3. The test goes through the following steps: 1. creates a 4 node cluster with

Re: Troubles in restarting Ignite with persistence enabled

2018-10-02 Thread akurbanov
Hello Gianluca, There might be a several reasons for this slowdown. Could you please provide a couple of consecutive thread dumps when node startup hangs and a node log? If it is possible, could you do a restart with -DIGNITE_QUIET=false -ea JVM flags? Regards, Anton -- Sent from:

Re: Partition Loss Policy options

2018-10-02 Thread Roman Novichenok
I was looking at scan queries. cache.query(new ScanQuery()) returns partial results. On Tue, Oct 2, 2018 at 1:37 PM akurbanov wrote: > Hello Roman, > > Correct me if I'm mistaken, you are talking about SQL queries. That was > fixed under https://issues.apache.org/jira/browse/IGNITE-8927,

Re: Cluster is not responsive after node segmentation and reconciliation

2018-10-02 Thread Ariel Tubaltsev
You are right, this log is JVM log. Let me post some messages below if it can be of any help. The cluster had less than 1000 records at the time of event. Do you still think eviction can help there? Event: 0.993 Thread 0x7fee4404b800 Exception (0x000587b0aad0) thrown at

Re: Cluster is not responsive after node segmentation and reconciliation

2018-10-02 Thread Ariel Tubaltsev
Yeah, this log is JVM log. Let me past it below if it could be of any help. The cluster had less than 1000 records at the time of event. Do you think eviction still can help? Event: 0.728 Thread 0x7fee4404b800 Implicit null exception at 0x7fee353fef75 to 0x7fee353ff335 Event: 0.728

Re: Cluster is not responsive after node segmentation and reconciliation

2018-10-02 Thread Ariel Tubaltsev
The file I've attached contains JVM logs. Let me post it below. About eviction - the cluster didn't have too many records, less than 1000. Do you still think eviction could be useful here? Event: 0.728 Thread 0x7fee4404b800 Implicit null exception at 0x7fee353fef75 to 0x7fee353ff335

Re: LRU Evicts Most Recent Entries

2018-10-02 Thread akurbanov
Hello, Could you please share full Ignite configuration? Regards, Anton -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Partition Loss Policy options

2018-10-02 Thread akurbanov
Hello Roman, Correct me if I'm mistaken, you are talking about SQL queries. That was fixed under https://issues.apache.org/jira/browse/IGNITE-8927, primary ticket is https://issues.apache.org/jira/browse/IGNITE-8834, will be delivered in 2.7 release. Regards, Anton -- Sent from:

Partition Loss Policy options

2018-10-02 Thread Roman Novichenok
PartitionLossPolicy controls access level to the cache in case any partitions for this cache are not available on the cluster. As far as I can see this policy is only consulted for cache put/get operations. Is there a way to prevent queries from returning results (force an exception) when cache

Re: .NET ContinuousQuery lose cache entries

2018-10-02 Thread Ilya Kasnacheev
Hello! I have spent some time tinkering with your case, and I think that an issue exists. If you have a partitioned cache with non-local continuous query, it works. If you have a replicated cache with non-local continuous query, it works. If you have a replicated cache with local continuous

Re: fieldsQueryCursor.getAll() performance is really bad

2018-10-02 Thread Maxim.Pudov
As long as you use IN statement there is no need in additional id > 0 condition. It might lead to retrieving all data that fit for id > 0 and filtering them on the client side. Use could use "explain" to check whether my guess was right

Re: Cluster is not responsive after node segmentation and reconciliation

2018-10-02 Thread Maxim.Pudov
Please, check the file you attached, it doesn't contain any logs. Persistence will help to save data and prevent running out of memory. Another option is to set an eviction policy: https://apacheignite.readme.io/docs/evictions -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite running on JDK10?

2018-10-02 Thread Maxim.Pudov
https://issues.apache.org/jira/browse/IGNITE-8441 is still open. However, fix version is 2.7, which is coming soon. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Binary marshaller warm up

2018-10-02 Thread Andrey Davydov
Will binary serialization work for all other classes not listed in typeConfiguration? Andrey. От: Evgenii Zhuravlev Отправлено: 2 октября 2018 г. в 16:14 Кому: user@ignite.apache.org Тема: Re: Binary marshaller warm up Hi, You can try to config BinaryConfiguration.setTypeConfigurations, it

Re: Ignite Query Slow

2018-10-02 Thread Skollur
Documentation has example as ccfg.setIndexedTypes(Long.class, Person.class);. I believe this is single field. How to register multiple indexes for CacheConfiguration? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Service Grid classpath changes

2018-10-02 Thread ezhuravlev
When you deploy the service, it serializes the object of this service and puts it to the internal cache. At the moment when it should be redeployed, it will take this object from the cache and deserialize it. So, if after change your object still can be deserialized using the new version of the

Re: Binary marshaller warm up

2018-10-02 Thread Evgenii Zhuravlev
Hi, You can try to config BinaryConfiguration.setTypeConfigurations, it will work like pre-defined types. Evgenii вт, 2 окт. 2018 г. в 14:58, Andrey Davydov : > Hello. > > > > There is difficult transaction in my system, requires to put about 50 > objects to 7 different ignite caches. Objects

Binary marshaller warm up

2018-10-02 Thread Andrey Davydov
Hello. There is difficult transaction in my system, requires to put about 50 objects to 7 different ignite caches. Objects have 7 different classes. Some of them have difficult internal structure with object and collection fileds. The first execution of this transaction after empty cluster

Re: Troubles in restarting Ignite with persistence enabled

2018-10-02 Thread Gianluca Bonetti
Hello Mikael Yes we have more caches, but they are smaller, and the number of incoming data packets are is almost nothing compared to yours (100 per minute compared to 10.000 per second!) The time for restarting excedes 30 minutes, so it is too much and I always kill the process at that point (as

Re: Troubles in restarting Ignite with persistence enabled

2018-10-02 Thread Gianluca Bonetti
Hello Hamed For a dirty and quick test, I have removed the -XX:+AlwaysPreTouch It actually is faster, but I will see how is performing going on with more data. By now, it seems a good hint! Thank you! Cheers Gianluca Il giorno mar 2 ott 2018 alle ore 10:51 Hamed Zahedifar < h.zahedi...@yahoo.com>

Re: BinaryObject vs a HashMap

2018-10-02 Thread Ilya Kasnacheev
Hello! You cannot avoid deserialization since data is stored in off-heap in serialized form and also perhaps sent over network. If you have a lot of different types, using HashMap will be preferred (with primitive keys/values). You could also try storing byte[] values to have precise control

Re: Is there a way to use Ignite optimization and Spark optimization together when using Spark Dataframe API?

2018-10-02 Thread Ilya Kasnacheev
Hello! I suggest that you check those possibilities out: Does performance increase dramatically if you need it on 10% of data, i.e., ~1 million records? Does something change when you have only one client connected? Note that I was running this example on a single node so it should not be hard

Re: Troubles in restarting Ignite with persistence enabled

2018-10-02 Thread Mikael
I use persistent storage but not so many caches and so far no problems, I have 20 caches with around a total of 800.000 entries, each entry is small (<100 bytes), while testing I update around 5000-1 entries per second, restarts do take some time but never more than 2 minutes or so, so far

BinaryObject vs a HashMap

2018-10-02 Thread Mikael
Hi! Is there some meta data storage overhead for binary object "types" ? I need a cache where the value is pretty much a small key value store (5-10 keys and values, all strings, every value different) and with that few values maybe just two arrays with keys and values or a hash map, but as

Re: Troubles in restarting Ignite with persistence enabled

2018-10-02 Thread Hamed Zahedifar
Hi Gianluca Maybe it`s for JVM -XX:+AlwaysPreTouch Switch. try redhat workaround for this problem, i hope it will start faster.Java process takes a long time with -XX:+AlwaysPreTouch - Red Hat Customer Portal | | | | Java process takes a long time with -XX:+AlwaysPreTouch - Red Hat

Troubles in restarting Ignite with persistence enabled

2018-10-02 Thread Gianluca Bonetti
Hello everyone This is my first question to the mailing list, which I follow since some time, to get hints about using Ignite. Until now I used in other softwares development, and Ignite always rocked and made the difference, hence I literally love it :) Now I am facing troubles in restarting an

Re: Ignite Query Slow

2018-10-02 Thread Andrey Mashenkov
Hi, Ignite provides 2 way for configuring indices: queryEntity and annotation based. Seems, you either forget to setIndesTypes or tring mix both of these approaches. Please, find examples in documentation available via link I've provided earlier. пн, 1 окт. 2018 г., 18:05 Skollur : > I tried