Re: Partition map exchange in detail

2018-09-07 Thread eugene miretsky
Thanks! We are using persistence, so I am not sure if shutting down nodes will be the desired outcome for us since we would need to modify the baseline topolgy. A couple more follow up questions 1) Is PME triggered when client nodes join us well? We are using Spark client, so new nodes are

Re: 回复:how does Ignite Client restart ContinuousQuery when Ignite cluster failed and restart

2018-09-07 Thread akurbanov
Try using Near cache , looks like it perfectly fits what are you have described, you may even use your expiry policy form distributed cache to be in sync. For restarting client refer to documentation: Reconnecting a client

Re: Partition map exchange in detail

2018-09-07 Thread Ilya Lantukh
Hi Eugene, 1) PME happens when topology is modified (TopologyVersion is incremented). The most common events that trigger it are: node start/stop/fail, cluster activation/deactivation, dynamic cache start/stop. 2) It is done by a separate ExchangeWorker. Events that trigger PME are transferred

Partition map exchange in detail

2018-09-07 Thread eugene miretsky
Hello, Out cluster occasionally fails with "partition map exchange failure" errors, I have searched around and it seems that a lot of people have had a similar issue in the past. My high-level understanding is that when one of the nodes fails (out of memory, exception, GC etc.) nodes fail to

Re: ignte cluster hang with GridCachePartitionExchangeManager

2018-09-07 Thread eugene miretsky
Hi Wangsan, So what was the original cause of the issue? Was it blocking the listening thread in your test code or something else? We are having similar issues Cheers, Eugene On Mon, Sep 3, 2018 at 1:23 PM Ilya Kasnacheev wrote: > Hello! > > The operation will execute after partition map

Re: Transition from FULL_ASYNC/PRIMARY_SYNC to FULL_SYNC

2018-09-07 Thread slava.koptilin
Hello Dave, > Is there any way to accomplish that, short of adding the ability to change > this cache setting dynamically? Currently, there is no ability to change any cache configuration parameters, except the ability to turn on/of statistics and modifying a schema (DDL). Thanks, S. -- Sent

Re: Question About Persistence

2018-09-07 Thread vkulichenko
Hi Alper, You can create two separate data regions [1], one for the data within the threshold, another for outside. The latter can have very little memory allocated and persistence enabled, which would mean that data in this region is stored only on disc for the most part. -Val -- Sent from:

Query on nested objects

2018-09-07 Thread max904
I have following "Person" data model: public class Address implements Binarylizable { @QuerySqlField private String street; @QuerySqlField private String city; @QuerySqlField private String state; @QuerySqlField private int zip; @QuerySqlField private String country; ...

Re: Off heap constantly grow on use BinaryObject as field of cached data

2018-09-07 Thread Ilya Kasnacheev
Hello! I have ran your test and I don't observe any off-heap growth: [18:17:14,655][INFO ][grid-timeout-worker-#24%ignite.GrowTest0%][GrowTest0] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=23ad15f7, name=ignite.GrowTest0, uptime=00:00:40.015] ^-- H/N/C

Re: Difference between replicated and local cache mode regarding time execution of query

2018-09-07 Thread Vladimir Ozerov
Hi Moti, Could you please attach execution plans for both LOCAL and REPLICATED cases? On Wed, Sep 5, 2018 at 6:13 PM ilya.kasnacheev wrote: > Hello! > > Unfortunately I'm not aware why you see such a big difference in this case. > It should be comparable. Maybe SQL people will chime in? > >

Re: a node fails and restarts in a cluster

2018-09-07 Thread Pavel Kovalenko
Hello Evgeny, Could you please attach full logs from both nodes in your case #2? Make sure, that quiet mode is disabled (-DIGNITE_QUIET=false) to have full info logs. пт, 7 сент. 2018 г. в 17:41, es70 : > I have a cluster of 2 ignite (version 2.6) nodes with enabled persistence > (at > the time

a node fails and restarts in a cluster

2018-09-07 Thread es70
I have a cluster of 2 ignite (version 2.6) nodes with enabled persistence (at the time of writing running on my windows machine for test puposes) and third node (not in the cluster) which I use to run my app in the thick client mode. The app creates a cache (CacheMode.REPLICATED,

Re: $42M ... well done chaps :)

2018-09-07 Thread Denis Magda
Nice summary! Just curious, do you have the same metrics for Hadoop, Spark, Cassandra? -- Denis -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Igniter TCPDiscovery

2018-09-07 Thread Denis Magda
That's because they found each other via the multicast protocol - you're using TcpDiscoveryMulticastIpFinder Replace it with the static IP finder listing the IPs there: https://apacheignite.readme.io/docs/tcpip-discovery#section-static-ip-finder -- Denis On Wed, Sep 5, 2018 at 9:44 AM Skollur

a node fails and restarts in a cluster

2018-09-07 Thread es70
I have a cluster of 2 ignite (version 2.6) nodes with enabled persistence (at the time of writing running on my windows machine for test puposes) and third node (not in the cluster) which I use to run my app in the thick client mode. The app creates a cache (CacheMode.REPLICATED,

Re: How to create tables with JDBC, read with ODBC?

2018-09-07 Thread Ilya Kasnacheev
Maybe we shouldn't bother to quote schemas, assuming that it's the duty of client? Unfortunately after reading ODBC docs I have no idea, but there's no hints that the result will be quoted. Regards, -- Ilya Kasnacheev пт, 7 сент. 2018 г. в 15:38, Igor Sapego : > Well, ODBC applies quotes to

Re: How to create tables with JDBC, read with ODBC?

2018-09-07 Thread Igor Sapego
Well, ODBC applies quotes to all schemas. It makes sense to check and not apply quotes to PUBLIC, but this won't help in all other cases, when cache-name-schema is used. Best Regards, Igor On Fri, Sep 7, 2018 at 2:13 PM Ilya Kasnacheev wrote: > Hello! > > It's actually very strange that we

Re: How to create tables with JDBC, read with ODBC?

2018-09-07 Thread Ilya Kasnacheev
Hello! It's actually very strange that we have quotes around PUBLIC since it's supposed to be used quote-free. I will take a look. Regards, -- Ilya Kasnacheev пт, 7 сент. 2018 г. в 14:07, Igor Sapego : > It happens, because ODBC returns schema name in quotes, > so seems like QLIK adds its

Re: How to create tables with JDBC, read with ODBC?

2018-09-07 Thread Igor Sapego
It happens, because ODBC returns schema name in quotes, so seems like QLIK adds its own quotes around it, as it encounters non standard characters (quotes). I think, it is a QLIK's error, as our ODBC driver explicitly states, that no additional quotes should be used around identifiers. And even

Re: Batch insert into ignite using jdbc thin driver taking long time

2018-09-07 Thread Ilya Kasnacheev
Hello! That's strange. Can you collect a thread dump after it starts hanging, share it with us? Regards, -- Ilya Kasnacheev чт, 6 сент. 2018 г. в 20:28, Sriveena Mattaparthi < sriveena.mattapar...@ekaplus.com>: > Hi Ilya, > > It did compiled...but it is not inserting records to ignite at

Re: Connection reset by peer

2018-09-07 Thread Ilya Kasnacheev
Hello! My recommendation is to increase socketWriteTimeout on TcpCommunicationSpi. It is 2s by default, which is small enough to cause problems under heavy load. Regards, -- Ilya Kasnacheev пт, 7 сент. 2018 г. в 5:48, Justin Ji : > I also encountered this problem before when I am doing

$42M ... well done chaps :)

2018-09-07 Thread Paul Anderson
SLOCDirectory SLOC-by-Language (Sorted) 1072957 modules java=916651,cs=88639,cpp=61325,sh=6277,php=49,jsp=16 12716 examplesjava=12577,php=103,python=36 11738 ipc sh=11343,cpp=395 441 bin sh=441 377 target sh=377 254 scripts

Re: The system cache size was slowly increased

2018-09-07 Thread Justin Ji
I analyzed the log and found that there are many *checkpoint timeout*. The following are the segments of the checkpoint: [db-checkpoint-thread-#46] INFO o.a.i.i.p.c.p.GridCacheDatabaseSharedManager:478 - Checkpoint started [checkpointId=7fdb8ddd-76d7-4ddf-9fd1-cddd8bb093be,

Off heap constantly grow on use BinaryObject as field of cached data

2018-09-07 Thread Serg
Hi All We have got a problem with growing off-heap on each sync data. *Precondition * We have ignite in memory cache. with key Integer and value Contact Each hour we sync full data in cache via api cache.put(key, value); (cache full load) Also we have increment update via the api cache.put(key,

Re: The system cache size was slowly increased

2018-09-07 Thread Justin Ji
Prem - Thank for your reply. You explained why Ignite container grew from 2.5G to more than 4G because there is 2G heap memory and more than 2G off-heap memory, I think what you said is very correct and solve one of my main problems. But another question, why system memory cache also grows slow

Question About Persistence

2018-09-07 Thread Alper Tekinalp
Hi everyone. We have a cache that stores records with timestamp and regularly we run a scan query like "timestamp <= now" and process returned records. As an optimization we implemented such a logic that if timestamp of a record greater than a threshold we don't put it to the cache but store

Re: The system cache size was slowly increased

2018-09-07 Thread Prem Prakash Sharma
Hi Justin, If I am not wrong you have a data region of 3GiB and your ignite node heap size is 2GiB. Since data region was started with 512MB I am guessing your container slowly grew from 2.5Gib to where it is now. The problem in your case is both the off-heap and JVM heaps are different and