Re: JDBC store Date deserialization problem

[franck102]

Thanks Val!



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/JDBC-store-Date-deserialization-problem-tp16276p16355.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 2.1 PersistentStoreConfiguration is not configurable by spring injection

[kane]

Hi Slava,

It all works, thanks a lot.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/2-1-PersistentStoreConfiguration-is-not-configurable-by-spring-injection-tp16324p16354.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Re: In which scenario the ignite nodes will shutdown by itself

[aa...@tophold.com]

hi All, 

we notice this cause by the backend DB issue for the JDBC cache store.

So we have a concern, if the backend DB outline for while, will all those  
cached objects be stuck in heap memory. 

Possible those data swap to off-heap? when DB back then write back to DB? 

we already set the onheapCacheEnabled  to false,  but now the reality is every 
time the DB stuck then the on-heap memory increase,  eventually trigger a Full 
GC then the entire node failed. 


Regards
Aaron


aa...@tophold.com
 
From: aa...@tophold.com
Date: 2017-08-18 11:34
To: user
Subject: Re: Re: In which scenario the ignite nodes will shutdown by itself
hi Dmirty, 

Eventually we can reproduce it,  the Node shutdown cause of the long GC pause > 
20s ([Full GC (Allocation Failure)  10182M->7962M(10G), 27.6709967 secs]).

But when we dump the jvm state, Ignite occupied a lot  memory all those data 
holding in the old Generation, cause the full GC failure, an the entire JVM 
stuck there. 

Some interesting beans:  GridLocalCacheEntry & GridCacheObsoleteEntryExtras  
both about 22million;  this quite close to the entire data in our DB. 

Totally Ignite take about of the 4G/8G data in old generation. 

Ignite cached data - transaction usually lifecycle is very short,  there should 
not be so many data onheap, PLUS we configurate off-heap memory for them. 

GC log and our configuration as attached,  thanks again for your time and very 
appreciated 



Regards
Aaron


aa...@tophold.com
 
From: aa...@tophold.com
Date: 2017-08-17 20:25
To: user
Subject: Re: Re: In which scenario the ignite nodes will shutdown by itself
Thanks Dmirty!

we monitor the GC it is health -- the entire java process still running 
smoothly  now.

We adjust a bit of the remote service, not stuck the socket, now seem the node 
not easy to crash, but we notice the instance eat almost all the memory of the 
machine. 

Seems the entire system are at the edge of the crash.  we explicitly set the 
memory policy to 16~27G. 

But I not sure why it still occupy so many memory,  for this process in fact we 
did not query anything from Cache. 

it work just as buffer between the DB, we write to cache and JDBC cache store 
flush to DB,   we notice about 30million data inserted already,  

Will the RANDOM_2_LRU evict the data from page?   What we concern is if crash 
now we will lost some data maybe. 



[root@iZuf62zdiq684kn72aatgjZ 1502945772_19623]# free  -mh
  totalusedfree  shared  buff/cache   available
Mem:62G 17G405M744K 44G 44G
Swap:0B  0B  0B

  PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+ COMMAND 

   
19376 root  20   0 46.715g 0.021t 0.010t S 313.3 34.3   1332:40 java

Thanks for your time!

Regards
Aaron


aa...@tophold.com
 
From: dkarachentsev
Date: 2017-08-16 21:51
To: user
Subject: Re: In which scenario the ignite nodes will shutdown by itself
Hi Aaron,
 
Do you have long GC pauses? As Bob said, large queries or transactions could
move a lot of data to on-heap. F.e. regular SELECT will get all dataset into
heap on server node and will respond to client with pages.
But first of all try to find a reason of node segmentation first, it could
be due to GC or system pauses, lost network, etc.
 
Thanks!
-Dmitry.
 
 
 
--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/In-which-scenario-the-ignite-nodes-will-shutdown-by-itself-tp16192p16227.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster segmentation

[vkulichenko]

Hi Biren,

What is the use case and what are you trying to achieve by all this?

First of all, there is a difference between node_left/failed and
node_segmented events. The former is fired on nodes that are still in
topology to notify that one of the nodes left or failed. But the latter
means that *local* node got segmented, and I don't think it makes sense to
do any checks there.

Segmentation can happen for various reasons, but in vast majority of cases
it's a long GC pause. In this case node does not close connections, but
becomes unresponsive, which causes the cluster to remove it from topology
after failure detection timeout. When GC pause finishes, node tries to
continue to operate, but realizes that it was already kicked out. It then
fires node_segmented event locally and stops immediately. This is correct
behavior.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cluster-segmentation-tp16314p16351.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: JDBC store Date deserialization problem

[vkulichenko]

Franck,

This looks like a bug, I will create tickets.

For now your workaround looks like the best way to solve the issue. Create
your own transformer that will convert java.sql.Date to java.util.Date, this
will fix the issue.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/JDBC-store-Date-deserialization-problem-tp16276p16350.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Streaming test

[vkulichenko]

Jessie,

If it's a preprocessing done on client side, then I think it should happen
on application level. Do required preprocessing, convert to Ignite's data
model and then do cache operations.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Streaming-test-tp14039p16349.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Task Session (compute) API in Ignite .NET

[davida]

Hi All,

I can't find compute grid's checkpoint functionality and
FifoQueueCollisionSpi in .NET's version of Apache Ignite. Are there any
plans to port this to .NET ?

Thx



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Task-Session-compute-API-in-Ignite-NET-tp16348.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: getAsMap ??

[dkarachentsev]

Hi Chris,

1. Indexing may slowdown insertions, but in will not be used if no
CacheConfiguration.setIndexedTypes() or
CacheConfiguration.setQueryEntities() set.

2. It depends on conditions. If you have a lot of data, but you need to
filter out a small set of them, then indexing may greatly help you, because
it will use index but full scan. In other hand, scan query may be greatly
optimized if it's used locally and per partition, but again, it will check
all entries.

3. If you set withKeepBinary(), then you'll be working with BinaryObject and
it's not require any specific config.

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/getAsMap-tp16242p16347.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

ignite.active(true) blocking forever

[userx]

Hi all,

*QUESTION:-Is there a way to timeOut the ignite.active(true) so that the
client does not block forever and moves ahead for some other important
processing ?*

*OBSERVATIONS:-*

I have a client(-Xms512m -Xmx512m) which is represented by the following
piece of code
package org.apache.ignite.examples;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.CacheConfiguration;

public class DataGridClient {

public static void main(String[] args) {
// TODO Auto-generated method stub
Ignition.setClientMode(true);
Ignite ignite = 
Ignition.start("examples/config/example-ignite.xml");
System.out.println("Before Active");
ignite.active(true);
System.out.println("After Active");
CacheConfiguration cfg = new 
CacheConfiguration("1");
IgniteCache cache = 
ignite.getOrCreateCache(cfg);
cache.put(1, 1);
}

}

I have 2 servers (2 different jvms on same machine) *deliberately* started
with -Xms128m -Xmx128m with the following piece of code.
Ignite ignite = Ignition.start("config/example-ignite.xml");

Client and both Servers are started on the same machine.

What I have observed is that the text "After Active" never gets printed and
ignite.active(true) blocks forever. On the other hand, I could see that
there is an OOM error in the server logs
[23:34:47] Ignite node started OK (id=825a2757)
[23:34:47] Topology snapshot [ver=1, servers=1, clients=0, CPUs=4,
heap=0.13GB]
TcpDiscoveryNode [id=825a2757-71b6-4483-90d3-0a0b1d7b1923,
addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.1.3,
fd9c:b2b2:d704:2000:4cbb:b964:b3fb:9269,
fd9c:b2b2:d704:2000:68ac:5531:940e:b851],
sockAddrs=[Garima-PC/192.168.1.3:47501,
/fd9c:b2b2:d704:2000:4cbb:b964:b3fb:9269:47501,
/fd9c:b2b2:d704:2000:68ac:5531:940e:b851:47501, /0:0:0:0:0:0:0:1:47501,
/127.0.0.1:47501], discPort=47501, order=1, intOrder=1,
lastExchangeTime=1503338687228, loc=true, ver=2.1.0#20170720-sha1:a6ca5c8a,
isClient=false]
[23:34:47] Topology snapshot [ver=2, servers=2, clients=0, CPUs=4,
heap=0.25GB]
[23:35:41] Topology snapshot [ver=3, servers=2, clients=1, CPUs=4,
heap=0.75GB]
[23:35:41] Default checkpoint page buffer size is too small, setting to an
adjusted value: 519.5 MiB
[23:35:41,881][ERROR][exchange-worker-#34%null%][GridDhtPartitionsExchangeFuture]
Failed to reinitialize local partitions (preloading will be stopped):
GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=3,
minorTopVer=1], nodeId=825a2757, evt=DISCOVERY_CUSTOM_EVT]
java.lang.OutOfMemoryError: null
at sun.misc.Unsafe.allocateMemory(Native Method) ~[?:1.8.0_111]
at
org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1054)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.mem.unsafe.UnsafeMemoryProvider.nextRegion(UnsafeMemoryProvider.java:80)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.start(PageMemoryImpl.java:276)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.startMemoryPolicies(IgniteCacheDatabaseSharedManager.java:194)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.onActivate(IgniteCacheDatabaseSharedManager.java:949)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.onActivate(GridCacheDatabaseSharedManager.java:459)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.GridCacheSharedContext.activate(GridCacheSharedContext.java:244)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:762)
~[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:574)
[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1901)
[ignite-core-2.1.0.jar:2.1.0]
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
[ignite-core-2.1.0.jar:2.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
[23:35:41,889][ERROR][exchange-worker-#34%null%][GridCachePartitionExchangeManager]
Runtime error caught during grid runnable execution: GridWorker
[name=partition-exchanger, igniteInstanceName=null, finished=false,
hashCode=724661, interrupted=false, 

Re: Performance of persistent store too low when bulb-loading

[dkarachentsev]

Use this command for dstat:
dstat -cmdgs --fs > dstat.txt

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Performance-of-persistent-store-too-low-when-bulb-loading-tp16247p16345.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Amazon AMI not available in a specific region

[Nikolai Tikhonov]

Hello,

Yes, Apache Ignite doesn't provide straight forward way for it, but you can
start Ignite nodes via yarn manually. For it you need to connect to master
node and perform steps from this instruction:
https://apacheignite.readme.io/docs/yarn-deployment

On Tue, Aug 1, 2017 at 9:46 PM, raksja  wrote:

> Thanks for quick turn around that helped.
> Also do you know/any one how to install ignite in EMR worker nodes?
> Looks like there's no straight forward way.
>
> Any help or suggestions?
>
> On Tue, Aug 1, 2017, 5:33 AM Nikolai Tikhonov [via Apache Ignite Users] 
> <[hidden
> email] > wrote:
>
>> Hi,
>>
>> I've copied AMI to Oregon. ami-f07f9b88 is image id.
>>
>> --
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/Amazon-
>> AMI-not-available-in-a-specific-region-tp15838p15854.html
>> To unsubscribe from Amazon AMI not available in a specific region, click
>> here.
>> NAML
>> 
>>
>
> --
> View this message in context: Re: Amazon AMI not available in a specific
> region
> 
>
> Sent from the Apache Ignite Users mailing list archive
>  at Nabble.com.
>

Re: CacheAtomicityMode and CacheMode

[userx]

Thanks Dmitry!



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/CacheAtomicityMode-and-CacheMode-tp16313p16343.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Performance of persistent store too low when bulb-loading

[dkarachentsev]

Hi,

Could you please take dstat, Ignite and GC logs and few thread dumps when
application is experiencing slowdown?

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Performance-of-persistent-store-too-low-when-bulb-loading-tp16247p16342.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Activation: slow and: Ignite node crashed in the middle of checkpoint.

[dkarachentsev]

Hi,

Please share full logs and thread dumps, it's hard to understand the root
cause.

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Activation-slow-and-Ignite-node-crashed-in-the-middle-of-checkpoint-tp16144p16341.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: CacheAtomicityMode and CacheMode

[dkarachentsev]

Hi,

1. In most cases that exception might be thrown if cache store cannot be
updated. Regularly even on unstable topology you won't get it, so the
easiest would be to do putAll() with failed keys, it should be finished
successfully.

2. By default memory shared across all caches, so you may check space
consumed for all of them. To do that: 
consumed memory: MemoryMetrics.getPhysicalMemoryPages() * pageSize
memory and sick: MemoryMetrics.getTotalAllocatedPages() * pageSize
consumed disk: (MemeoryMetrics.getTotalAllocatedPages() -
MemoryMetrics.getPhysicalMemoryPages()) * pageSize

You cannot see metrics for specific cache, but you can apply MemoryPolicy
for specific cache and see metrics for that memory region.

3. Yes, in that case REPLICATED mode is most suitable.

4. By default client waits on put() unless primary nodes updated and backups
are set asynchronously. If you want to make sure that after put() all
changes applied for primary and backups, set
CacheConfiguration.setWriteSynchronizationMode(FULL_SYNC).

5. Try Ignite.cacheNames()

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/CacheAtomicityMode-and-CacheMode-tp16313p16340.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IGFS With HDFS Configuration

[Michael Cherkasov]

Hi Pradeep,

https://issues.apache.org/jira/browse/ZEPPELIN-2674 is resolved,
Zeppelin 0.8 will be issued with ignite 2.0 support.

Thanks,
Mikhail.


2017-08-04 19:09 GMT+03:00 Mikhail Cherkasov :

> Hi Pradeep,
>
> I can't run it with 1.9 too, however, the same config works fine with 2.0.
> You can try to build zeppelin your self, just apply the following patch:
> https://github.com/apache/zeppelin/pull/2445/files
> to some stable version of zeppelin.
> I'm not sure but it should work, anyway, the only way to check this is to
> build with the patch and run.
>
> Thanks,
> Mikhail.
>
> On Thu, Aug 3, 2017 at 9:57 PM, pradeepchanumolu 
> wrote:
>
>> I couldn't get it running with the v1.9.0.
>>
>> I bumped the version to the latest v2.1.0 and could get the ignite
>> services
>> up. (Zeppelin only supports v1.9.0 btw)
>> I am attaching my config file to this message.
>> default-config.xml
>> > default-config.xml>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/IGFS-With-HDFS-Configuration-tp15830p15964.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Thanks,
> Mikhail.
>

答复: 答复: problems in test ignite PersistentStore

[罗 辉]

got it ,thank you Val, I got to have another try.


发件人: vkulichenko 
发送时间: 2017年8月19日 1:54:11
收件人: user@ignite.apache.org
主题: Re: 答复: problems in test ignite PersistentStore

You don't see anything in /ssd folder, because you provided it in
PersistenceStoreConfiguration object which is not even set anywhere. This
code has no effect:

val psc = new PersistentStoreConfiguration
psc.setPersistentStorePath("/ssd")

Actually, Ignite configuration can't be changed in runtime anyway, so it
should be done in XML as part of PersistentStoreConfiguration bean:






-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/problems-in-test-ignite-PersistentStore-tp16244p16297.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

RE: Strange problems with Ignite native Persistence when Data exceeds Memory

[dkarachentsev]

Hi Roger,

The reason of slowdown might be the fact, that Ignite by default consumes
80% of available memory, which forces OS to start a lot of swapping, so all
memory operations limited by disk speed. Try to reduce memory size [1].

[1]
https://apacheignite.readme.io/v2.1/docs/memory-configuration#memory-policies

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Strange-problems-with-Ignite-native-Persistence-when-Data-exceeds-Memory-tp16187p16336.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Performance of persistent store too low when bulb-loading

[dkarachentsev]

Hi,

The reason of slowdown might be the fact, that Ignite by default consumes
80% of available memory, which forces OS to start a lot of swapping, so all
memory operations limited by disk speed. Try to reduce memory size [1].

[1]
https://apacheignite.readme.io/v2.1/docs/memory-configuration#memory-policies

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Performance-of-persistent-store-too-low-when-bulb-loading-tp16247p16335.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 2.1 PersistentStoreConfiguration is not configurable by spring injection

[slava.koptilin]

Hi Kane,

Could you please try the following example in order to set
'persistentStorePath' property?







If you are trying to set an absolute path, you need to provide correct
definition of pathname, which is system dependent.
On UNIX systems, a pathname is absolute if its prefix is "/".  On MS Windows
systems, a pathname is absolute if its prefix is a drive specifier followed
by "\"

For example:
unix: 
windows: 

Apache Ignite log should contain something like the following:
[INFO][main][FilePageStoreManager] Resolved page store work directory:
/projects/persistentstoretest/work/0_0_0_0_0_0_0_1_lo_10_0_2_15_127_0_0_1_47500

Best regards,
Slava.




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/2-1-PersistentStoreConfiguration-is-not-configurable-by-spring-injection-tp16324p16334.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Failure to deserialize simple model object

[franck102]

Awesome, thanks!

Franck



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p16333.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: docker external libs

[Nikolai Tikhonov]

Yes, you can easily to create your own image. As example you can look at
Apache Ignite docker file [1].

1.
https://github.com/apache/ignite/blob/master/modules/docker/2.1.0/Dockerfile

On Mon, Aug 21, 2017 at 1:31 PM, luqmanahmad  wrote:

> Thanks Nikolai for coming back. That's what I am doing and I needed one
> based
> on the centos anyway. So had to create one from scratch.
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/docker-external-libs-tp16254p16330.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: JDBC store Date deserialization problem

[franck102]

BTW, if I remember correctly the server write -2 as a type ID because
java.util.Date is not in this array:

org.apache.ignite.internal.processors.query.QueryUtils#SQL_TYPES

If I use java.sql.Date in the Ignite configuration I get a different
exception (Wrong value has been set).

Franck



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/JDBC-store-Date-deserialization-problem-tp16276p16331.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: docker external libs

[luqmanahmad]

Thanks Nikolai for coming back. That's what I am doing and I needed one based
on the centos anyway. So had to create one from scratch.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/docker-external-libs-tp16254p16330.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: JDBC store Date deserialization problem

[franck102]

I understand for the store implementation, thanks.

I have attached a sample project, simply run this test (Ignite and DB are
embedded):

ignite-binary-sample.zip

  

org.apache.ignite.sample.test.ServerTests#loadItem

... and you get the exception below:

javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
Unexpected field type [pos=24, expected=Date, actual=-2]

at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1312)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.cacheException(IgniteCacheProxy.java:2630)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:1188)
at 
org.apache.ignite.sample.test.ServerTests.loadItem(ServerTests.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at 
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected field
type [pos=24, expected=Date, actual=-2]
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7229)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:258)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:189)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:139)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4499)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4480)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1324)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.get(IgniteCacheProxy.java:1181)
... 25 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Unexpected
field type [pos=24, expected=Date, actual=-2]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.checkFlagNoHandles(BinaryReaderExImpl.java:1676)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readDate(BinaryReaderExImpl.java:1169)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readDate(BinaryReaderExImpl.java:1164)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:757)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:679)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:164)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:843)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
   

Re: sql Injection prevention in ignite

[Alexey Kukushkin]

Hi,

Yes, Ignite prevents SQL injection.

Also, I think PreparedStatement is not about preventing you from SQL
injection. PreparedStatement is about compiling a query so that it can be
efficiently re-used if executed multiple times. What prevents form SQL
injection is binding parameters instead of concatenating them and making
them part of the SQL. If you check
IgniteH2Indexing.preparedStatementWithParams() you would find that this is
exactly what Ignite does - it binds parameters to the statement.

On Mon, Aug 21, 2017 at 10:18 AM, kotamrajuyashasvi <
kotamrajuyasha...@gmail.com> wrote:

> Hi
>
> Using PreparedStatement of JDBC we can avoid/prevent sql Injection. Does
> *SqlFieldsQuery.setArgs(..)* in ignite also have same functionality to
> prevent sql Injection or Are there any other ways to prevent sql Injection
> in ignite.
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/sql-Injection-prevention-in-ignite-tp16321.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Alexey

Re: docker external libs

[Nikolai Tikhonov]

Hello,

I think in this case that the better way it to extend existing a docker
image and add your libs by "ADD" command.

On Thu, Aug 17, 2017 at 4:47 PM, luqmanahmad  wrote:

> Hi there,
>
> In docker deployment can we provide jars from our local system to
> EXTERNAL_LIBS parameter. Let say we have 10 different jars on our local
> system how do we provide them to the EXTERNAL_LIBS parameter.
>
> Thanks,
> Luqman
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/docker-external-libs-tp16254.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: 2.1 PersistentStoreConfiguration is not configurable by spring injection

[kane]

I tried to customize the persistenceStorePath programmatically, but it seems
not work.

I cannot see any file created under the configured folder. could anyone
pointing out if anything is wrong or what should be the right way to
customize the persistence store path.

the sample code is as well

Ignite ignite = Ignition.start(
.class.getClassLoader().getResourceAsStream(
"file name"));

ignite.configuration().getPersistentStoreConfiguration().setPersistentStorePath("/Users/user/Documents/igniteCache");
// an absolute path


- Kane




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/2-1-PersistentStoreConfiguration-is-not-configurable-by-spring-injection-tp16324p16326.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 2.1 new feature - warm up the cache / memory

[Alexey Kukushkin]

Hi,

The data is loaded by pages (somehow similar to file system's virtual
memory).

Persistence is transparent to all kinds of APIs - a page is loaded by
Ignite when you get data from that page.

If you really want to guarantee that the data your app is going to use are
already in memory you might want to "access" that data during your app
initialization, for example, run a SQL query for that data.

There is no housekeeping API in Apache Ignite but Ignite native persistence
exposes extensibility API. Implement
org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteCacheSnapshotManager
and add the implementation as Ignite plugin.

Some vendors might already provide snapshots implementations.



On Mon, Aug 21, 2017 at 11:12 AM, kane  wrote:

> Hi Val,
>
> very appreciate you support, and one more question here.
>
> is there any data housekeeping for the file stored in on the disk ?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/2-1-new-feature-warm-up-the-cache-
> memory-tp16278p16323.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Alexey

2.1 PersistentStoreConfiguration is not configurable by spring injection

[kane]

Most of setter method of the properties in PersistentStoreConfiguration are
returned as PersistentStoreConfiguration, which leads a spring injection
error if we try to simply configure the properties of
PersistentStoreConfiguration by doing the spring bean configure in the
spring bean .xml config file.

which is quite inconvenient, we must have to change it programmatically by
loading the PersistentStoreConfiguration from the spring context, Is it a
oversight of the 2.1 ?

e.g.

/**
 * Returns a path the root directory where the Persistent Store will
persist data and indexes.
 */
public String getPersistentStorePath() {
return persistenceStorePath;
}

/**
 * Sets a path to the root directory where the Persistent Store will
persist data and indexes.
 * By default the Persistent Store's files are located under Ignite work
directory.
 *
 * @param persistenceStorePath Persistence store path.
 */
public PersistentStoreConfiguration setPersistentStorePath(String
persistenceStorePath) {
this.persistenceStorePath = persistenceStorePath;

return this;
}







Got error -

Caused by: org.springframework.beans.NotWritablePropertyException: Invalid
property 'persistenceStorePath' of bean class
[org.apache.ignite.configuration.PersistentStoreConfiguration]: Bean
property 'persistenceStorePath' is not writable or has an invalid setter
method. Did you mean 'persistentStorePath'?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/2-1-PersistentStoreConfiguration-is-not-configurable-by-spring-injection-tp16324.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 2.1 new feature - warm up the cache / memory

[kane]

Hi Val,

very appreciate you support, and one more question here.

is there any data housekeeping for the file stored in on the disk ?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/2-1-new-feature-warm-up-the-cache-memory-tp16278p16323.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

sql Injection prevention in ignite

[kotamrajuyashasvi]

Hi

Using PreparedStatement of JDBC we can avoid/prevent sql Injection. Does
*SqlFieldsQuery.setArgs(..)* in ignite also have same functionality to
prevent sql Injection or Are there any other ways to prevent sql Injection
in ignite.




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/sql-Injection-prevention-in-ignite-tp16321.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Activation: slow and: Ignite node crashed in the middle of checkpoint.

[iostream]

Hi Roger,

I have experienced a similar issue during cluster activation in my setup as
well. I had shared my logs here -
http://apache-ignite-users.70518.x6.nabble.com/Activating-Cluster-taking-too-long-td16093.html

Eagerly seeking a root cause and resolution for this.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Activation-slow-and-Ignite-node-crashed-in-the-middle-of-checkpoint-tp16144p16318.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confused about QueryEntity configuration

[franck102]

empNo is in fact the key. 

I have attached a small sample project that demonstrates the problem; I can
in fact make the query work normally if I add keyFieldName=empNo to the
queryEntity configuration.
I don't see why this is required, for example with another cache that uses a
composite key:
TitlesCache: keyFields: { empNo, date_from }

... I can perfectly execute "select empNo from Titles" and I get non-null
values.

To reproduce the problem:
- mvn clean install
- Run the org.apache.ignite.sample.test.ServerTests#queryKey test

The configuration is in ignite-server/src/main/resources/ignite-server.xml

Franck

ignite-binary-sample.zip

  



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Confused-about-QueryEntity-configuration-tp16281p16317.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.