Re: “Failed to communicate with Ignite cluster" error when using JDBC Thin driver
We are using mybatis to manage connection, is that matters? BTW, I check the code in JdbcThinTcpIo method sendRequest JdbcResponse sendRequest(JdbcRequest req) throws IOException { int cap = guessCapacity(req); BinaryWriterExImpl writer = new BinaryWriterExImpl(null, new BinaryHeapOutputStream(cap), null, null); req.writeBinary(writer); send(writer.array()); BinaryReaderExImpl reader = new BinaryReaderExImpl(null, new BinaryHeapInputStream(read()), null, null, false); JdbcResponse res = new JdbcResponse(); res.readBinary(reader); return res; } After send() execute, run read() immediately. Is that might introduce synchronized issue? The msg not come back from server when read it from client out? As in our test env, the insert action has big chance to success after several times retry. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Page eviction mode is ignored
Hi Sergey, > But... page eviction doesn't work for "inMemory" memory region... Why? If Ignite persistence is enabled, then the page-based evictions have no effect because the oldest pages will be purged from memory automatically. Updated the text of the warning to makes things clearer. > Does Ignite supports a mix (persistenceEnabled or not) of DataRegion > configurations? Yes, you can have the persistence enabled for region A and disabled for region B. — Denis > On Dec 22, 2017, at 1:13 PM, Sergey Sergeevwrote: > > Hi, > > I'm confused. After the successful launch of the Ignite cluster version > 2.3.0, I saw the WARN message: > WARN GridCacheDatabaseSharedManager - Page eviction mode for [inMemory] > memory region is ignored because Ignite Native Persistence is enabled > > Ignite spring config see below: > > ... > > > > > > class="org.apache.ignite.configuration.DataRegionConfiguration"> > > > > > > > > > class="org.apache.ignite.configuration.DataRegionConfiguration"> > > > > > > > > class="org.apache.ignite.configuration.DataRegionConfiguration"> > > > > > > > > > > > > ... > > > > But... page eviction doesn't work for "inMemory" memory region... Why? > Does Ignite supports a mix (persistenceEnabled or not) of DataRegion > configurations? > > -- > Best regards, > Sergey S. Sergeev > exit(0);
Page eviction mode is ignored
Hi, I'm confused. After the successful launch of the Ignite cluster version 2.3.0, I saw the WARN message: WARN GridCacheDatabaseSharedManager - Page eviction mode for [inMemory] memory region is ignored because Ignite Native Persistence is enabled Ignite spring config see below: ... ... But... page eviction doesn't work for "inMemory" memory region... Why? Does Ignite supports a mix (persistenceEnabled or not) of DataRegion configurations? -- Best regards, Sergey S. Sergeev exit(0);
Re: loadAll and removeAll from cache with custom store
Hi Matt, I've tried the following code based on your configuration and it works. public static void main(String[] args) throws Exception { Ignite ignite = Ignition.start(createIgniteConfiguration()); IgniteCache cache1 = ignite.getOrCreateCache(createULConfiguration("test-cache-1")); // upload 10_000 keys for (int i = 0; i < 10_000; ++i) cache1.put(i, i); // remove all keys cache1.removeAll(); Thread.sleep(10_000); ignite.close(); } private static CacheConfiguration createULConfiguration(String name) { CacheConfigurationcacheConfig = new CacheConfiguration<>(); cacheConfig.setName(name); cacheConfig.setCacheMode(CacheMode.PARTITIONED); cacheConfig.setBackups(2); cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC); cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC); cacheConfig.setWriteBehindEnabled(true); cacheConfig.setWriteBehindBatchSize(512); cacheConfig.setWriteBehindFlushSize(10240); cacheConfig.setWriteBehindFlushFrequency(5_000); cacheConfig.setCacheStoreFactory(FactoryBuilder.factoryOf(CacheStoreExample.class)); cacheConfig.setReadThrough(true); cacheConfig.setWriteThrough(true); return cacheConfig; } public static class CacheStoreExample extends CacheStoreAdapter
How to Load Data from DataBase to Ignite which have native persistence enabled
Hi, I am very new to ignite,I am working around the examples. I have used cache store factory to load the data from sql server database to ignite which is configured with native persistence This is my code: public String getData(String query, String key) throws Exception { String value = ""; Ignite ignite = Ignition.ignite(); String CACHE_NAME = key+"_schemas"; IgniteCachecache= ignite.cache(CACHE_NAME); if(cache==null||cache.size()==0) { IgniteCache schemasCache = createCacheForSchema(key,CacheAtomicityMode.ATOMIC); schemasCache.loadCache(null); } IgniteQueryService service = new IgniteQueryService(); String result = service.query(CACHE_NAME,query); return result; } private IgniteCache createCacheForSchema(String key, CacheAtomicityMode atomic) throws Exception { Ignite ignite = Ignition.ignite(); String CACHE_NAME = key+"_schemas"; CacheConfiguration cacheCfg = new CacheConfiguration(CACHE_NAME); cacheCfg.setReadThrough(true); cacheCfg.setWriteThrough(true); cacheCfg.setCacheMode(CacheMode.PARTITIONED); // Default. cacheCfg.setIndexedTypes(String.class, EntitySchemas.class); cacheCfg.setAtomicityMode(atomic); // get sql connection String connectionString = getSqlConnection(key, atomic); // Configure store cacheCfg.setCacheStoreFactory(new DataCacheStoreFactory(connectionString)); cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC); return ignite.getOrCreateCache(cacheCfg); } Here are my questions: 1. Is this right way to do? Because I am using ignite with native persistence enabled but I am using third party database to load data from database..? 2. What will be the performance issue because it will write in both file system by ignite native persistence and into sql database by third party database.? and so Thanks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: loadAll and removeAll from cache with custom store
Hi, The size in my store is about 9k. I do have loadAll() working now, so once that completes, the cache has 9k items as well. For deleteAll(), I had it work once where it called my adapter multiple times with all of the expected keys. But then restarting my app and trying again (after loading new data) it didn't work at all. Do I need to loadAll() into my Ignite cache before I can delete everything from the backend store? Here's my configuration: String name = "foo"; CacheConfigurationcacheConfig = new CacheConfiguration<>(); cacheConfig.setName(name); cacheConfig.setCacheMode(CacheMode.PARTITIONED); cacheConfig.setBackups(2); cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC); cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC); cacheConfig.setWriteBehindEnabled(true); cacheConfig.setWriteBehindBatchSize(512); cacheConfig.setWriteBehindFlushSize(10240); cacheConfig.setWriteBehindFlushFrequency(5_000); cacheConfig.setCacheStoreFactory(new MyCacheStoreAdapter.MyCacheStoreFactory(name)); cacheConfig.setReadThrough(true); cacheConfig.setWriteThrough(true); -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: When do we get this error - Unknown pair [platformId=0, typeId=1078091073]]
Hi, Looks like not on all nodes exist your classes. Please check if all classes that you're using in cache are available on all nodes. Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: loadAll and removeAll from cache with custom store
Hi Matt, I think you can use IgniteCache.loadCache() [1] in order to load all key-value pairs into the cache before running any job. > For removeAll, I'm seeing that my cache store adapter only ever gets > called once (deleteAll) but the keys only ever have 1000 keys I've tried this use case and it works as expected. Could you please share cache configuration and clarify the size of your cache before IgniteCache.size(..)? [1] https://apacheignite.readme.io/docs/data-loading#ignitecacheloadcache Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to do 'stream processing' and more questions of a Ignite newbie
Hi, Thanks alot! I managed to finally get my interpolation up an running. Most of my questions were caused by confusing due to weird peer class loading errors. The errors I encountered while having peer class loading enabled apparently were caused by classes not getting redeployed on the ignite nodes. Hence, I kept runnining into the very same errors despite editing the code. From what I've read at https://apacheignite.readme.io/docs/deployment-modes it should work with the standard 'shared' mode, so I'm not quite sure why it didn't update the classes. I'll do some further testing with 'continuous mode', as of now I need to restart my nodes to redeploy the class. For now, my only question left is following thing: Here's a method from my class: Ignition.setClientMode(true); Ignite ignite = Ignition.ignite(); IgniteCacheaccCache = Ignition.ignite().cache("AccelerationPoint").withKeepBinary(); (1)double interpolationProgress = ((double) (accPoint.getKey().field("timestamp") - prevGps.field("timestamp"))) / ((double) (curGps.field("timestamp") - prevGps.field("timestamp"))); double interpolatedLat = prevGps.field("lat") + interpolationProgress * (curGps.field("lat") - prevGps.field("lat")); double interpolatedLon = prevGps.field("lon") + interpolationProgress * (curGps.field("lon") - prevGps.field("lon")); (2) System.out.println("Previous lat: " + prevGps.field("lat") + " ; Interpolated lat: " + interpolatedLat + " ; Next lat: " + curGps.field("lat")); (3)accCache. withKeepBinary().invoke( accPoint.getKey(), new CacheEntryProcessor () { public Object process(MutableEntry entry, Object... objects) throws EntryProcessorException { // Create builder from the old value. BinaryObjectBuilder bldr = entry.getValue().toBuilder(); //Update the field in the builder. bldr.setField("lat", interpolatedLat); bldr.setField("lon", interpolatedLon); // Set new value to the entry. entry.setValue(bldr.build()); return null; } }); I'm still trying to grasp how Ignite 'decides' to run the code. For (3) I know that it's run directly on the ignite node. (2) Is printed in my IDE, thus it has to be executed on the client node. Where is (2) being calculated? If its on the client node, whats the best way to transfer that processing to the nodes? If it's on the nodes already, how does it 'know' where to do it? Best regards and thanks alot for the help! Svonn -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Is it possible to import existing mysql database from file in console?
Hi, There are few options: 1) You need to have backups to survive node loss. [1] 2) You may enable persistence to survive grid restart and store more data that available in memory. [2] 3) Checkout nohup command [3] [1] https://apacheignite.readme.io/docs/primary-and-backup-copies [2] https://apacheignite.readme.io/docs/distributed-persistent-store [3] http://linux.101hacks.com/unix/nohup-command/ Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
When do we get this error - Unknown pair [platformId=0, typeId=1078091073]]
Hi Am using 2.3 I have created cache called Account, was trying to access data thru rest API with the below URL http://10.144.114.115:8080/ignite?cmd=get=A10001=Account This is the response I get {"successStatus":1,"sessionToken":null,"error":"Failed to handle request: [req=CACHE_GET, err=Unknown pair [platformId=0, typeId=1078091073]]","response":null} Here is the error log [16:24:26,844][SEVERE][sys-stripe-6-#7%IgnitePOC%][GridRestProcessor] Failed to handle request: CACHE_GET class org.apache.ignite.IgniteCheckedException: Unknown pair [platformId=0, typeId=1078091073] at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7252) at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:171) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140) at org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler$2.applyx(GridCacheCommandHandler.java:329) at org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler$2.applyx(GridCacheCommandHandler.java:325) at org.apache.ignite.internal.util.lang.IgniteClosureX.apply(IgniteClosureX.java:38) at org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:78) at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:70) at org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:30) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383) at org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347) at org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495) at org.apache.ignite.internal.processors.cache.GridCacheFutureAdapter.onDone(GridCacheFutureAdapter.java:55) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.onDone(GridPartitionedSingleGetFuture.java:740) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:462) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.setResult(GridPartitionedSingleGetFuture.java:655) at org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.onResult(GridPartitionedSingleGetFuture.java:506) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processNearSingleGetResponse(GridDhtCacheAdapter.java:348) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1400(GridDhtAtomicCache.java:129) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$15.apply(GridDhtAtomicCache.java:421) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$15.apply(GridDhtAtomicCache.java:416) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=0, typeId=1078091073] at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:376) at
Re: How to connect apache ignite node by static ip address
Mission complete. I created VPN server on my hetzner server and connect my client node to vpn. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/