Java to .NET interop

[Gordon Reid (Nine Mile)]

Hi Igniters,

We are running Ignite 2.2.0, we have a .NET client (runs in ignite client mode) 
and Java servers. We call Ignite Services running on the Java side, from the 
.NET client. We also access our memory grid entities from the .NET client. 
Currently it seems we need to package our java server jar, publish it to nuget, 
and then import this on the .NET side. If we don't have the entity and service 
classes available on the .NET side's Java classpath then we cannot communicate 
from the .NET side to the Java side. We are using the binary marshaller and 
simple name mapper.

Is there any way around this requirement? It is quite a hassle creating this 
tight binary coupling between our .NET client and Java server.

Thanks,
Gordon.


This email and any attachments are proprietary & confidential and are intended 
solely for the use of the individuals to whom it is addressed. Any views or 
opinions expressed are solely for those of the author and do not necessarily 
reflect those of Nine Mile Financial Pty. Limited. If you have received this 
email in error, please let us know immediately by reply email and delete from 
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252

Re: when client node connect to server node, server node throws NotSerializableException

[Jeff Jiao]

ok I tried to comment out this part, the exception I saw on server node
before doesn't appear this time...
so it looks like Cache Factory/Store problem... but why...

















--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: when client node connect to server node, server node throws NotSerializableException

[Jeff Jiao]

yes, PIgniteCacheStoreFactory implements javax.cache.configuration.Factory,
and this factory extends Serializable.
like I said " I already created my own factory to create the cache store to
avoid Serialization problem." without the custom factory, it cannot start
any nodes b/c my own cacheStore doesn't implements Serializable.

The problem I had is when I start a client node to connect to a running
server node, on server node throws exception and client node looks fine,
this is wierd... why the connection trigger the exception..



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Grid freezing

[smurphy]

I added a transaction timeout of 1 millis and a transation size of 100
(transaction is Optimistic and Serializable)...

I see no TransactionTimeoutExceptions, just the following:
CacheException: class org.apache.ignite.IgniteInterruptedException: Got
interrupted while waiting for future to complete..



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How can I get Ignite security plugin to work with JDBC thin client?

[calebs]

Version: Ignite 2.3-SNAPSHOT from ignite-2.3 branch. 

A jar that contains our custom security plugin for the security named
ACSPluginProvider & ACSSecurityProcessor is placed in $IGNITE_HOME/libs
folder.

Run ignite.sh to start the single data node and see
ACSSecurityProcessor.start method is called.

10-23 20:46:16.567 [main ] INFO 
apache.ignite.internal.IgniteKernal%cdev_cluster - Configured caches [in
'sysMemPlc' memoryPolicy: ['ignite-sys-cache']]
10-23 20:46:16.601 [main ] INFO 
apache.ignite.internal.IgniteKernal%cdev_cluster - 3-rd party licenses can
be found at: /opt/ignite/libs/licenses
10-23 20:46:16.663 [main ] INFO 
internal.processors.plugin.IgnitePluginProcessor - Configured plugins:
10-23 20:46:16.664 [main ] INFO 
internal.processors.plugin.IgnitePluginProcessor -   ^-- ACSPluginProvider
1.0.0
10-23 20:46:16.664 [main ] INFO 
internal.processors.plugin.IgnitePluginProcessor -   ^-- MaxPoint
10-23 20:46:16.664 [main ] INFO 
internal.processors.plugin.IgnitePluginProcessor -
10-23 20:46:16.673 [main ] INFO  platform.auth.ignite.ACSSecurityProcessor -
start
10-23 20:46:16.726 [main ] INFO  spi.communication.tcp.TcpCommunicationSpi -
Successfully bound communication NIO server to TCP port [port=47100,
locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, selectorSpins=0, pairedConn=false]

Use Ignite JDBC thin driver to connect to the cluster with user & password
properties. Then I see ACSSecurityProcessor - authenticate as shown below.
But the login is null. Also, I can see ACSSecurityProcessor.authorize is
called for CACHE_PUT when I execute INSERT or DELETE statements, but I do
not see ACSSecurityProcessor.authorize is called for CACHE_READ. 

/opt/ignite/log$ grep platform.auth.ignite ignite.log 
10-25 14:56:35.182 [main ] INFO  platform.auth.ignite.ACSSecurityProcessor -
start
10-25 14:56:35.779 [main ] INFO  platform.auth.ignite.ACSPluginProvider   -
start
10-25 14:56:35.810 [main ] INFO  platform.auth.ignite.ACSSecurityProcessor -
authenticateNode: id=cdb8bd19-d1b0-4d54-a982-01abdc83761a, hosts=[shei1],
address=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.16.128.96]
10-25 14:56:35.858 [main ] INFO  platform.auth.ignite.ACSSecurityProcessor -
onKernalStart(false)
10-25 14:56:35.891 [main ] INFO  platform.auth.ignite.ACSPluginProvider   -
onIgniteStart
10-25 14:57:09.417 [rest-#44%cdev_cluster%] INFO 
platform.auth.ignite.ACSSecurityProcessor - authenticate:
id=b5052d01-5a1c-47ea-9bb1-0ee89519bde7, login=null
10-25 15:01:21.862 [client-connector-#79%cdev_cluster%] WARN 
platform.auth.ignite.ACSSecurityProcessor - authorize:
name=SQL_PUBLIC_TEST1, permission=CACHE_PUT
10-25 15:01:55.818 [client-connector-#80%cdev_cluster%] WARN 
platform.auth.ignite.ACSSecurityProcessor - authorize:
name=SQL_PUBLIC_TEST1, permission=CACHE_PUT

The code for ACSSecurityProcessor.authenticate is

 @Override
public SecurityContext authenticate(AuthenticationContext authCtx)
throws IgniteCheckedException {
ACSSecuritySubject subject =
(ACSSecuritySubject)userMap.get(authCtx.subjectId());
if(subject == null) {
subject = new ACSSecuritySubject(authCtx.subjectId(),
authCtx.credentials(), authCtx.address());
if(logger.isInfoEnabled()) {
logger.info("authenticate: id=" + subject.id() + ", login="
+ subject.login());
}
userMap.put(authCtx.subjectId(), subject);
}
return new ACSSecurityContext(subject);
}

where subject.login() will return null if authCtx.credentials() is null. 

So here, I have two questions:
1. How can I get authCtx.credentials() to return the not null credentials
when I use the thin driver with user/password properties?
2. How can I get ACSSecurityProcessor.authorize invoked for CACHE_READ for
any SELECT query? 




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to query ignite

[iostream]

Reproducer code-

ResultSet rs = null;
try {
Class.forName("org.apache.ignite.IgniteJdbcThinDriver");
Connection conn = DriverManager

.getConnection("jdbc:ignite:thin://127.0.0.1/");

// create table
PreparedStatement stmt = conn.prepareStatement(
"CREATE TABLE test9 (id BIGINT,name 
VARCHAR,PRIMARY KEY (id))WITH
\"backups=1,affinityKey=id\"");
stmt.executeUpdate();

// Try to query
PreparedStatement stmt2 = conn.prepareStatement("select 
* from test8
where id = ?");
stmt2.setObject(1, 1L);
rs = stmt2.executeQuery();
System.out.println(rs);
conn.close();

} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Retrieving keys very slow from a partitioned, no backup, no replicated cache

[Anand Vijai]

Hi Alexey,

Thank you for the response.
I changed my code based on your recommendation and i think with all the
different options I was trying I ended up with a sub-par code!
The below code does the update very fast (few secs)


   Collection res = ignite.compute().broadcast(
new IgniteCallable() {
/** Auto-inject ignite instance. */
@IgniteInstanceResource
private Ignite ignite;

@Override public Integer call() {
IgniteCache cache =
ignite.getOrCreateCache("Fact");
Iterator> iterator =
cache.localEntries().iterator();

FactKey key;
Integer cnt = 0;
while (iterator.hasNext()) {
key = iterator.next().getKey();

cache.invoke(key, (entry, args) -> {
Fact val = entry.getValue();
// do some logic
val.setAmount(val.getAmt1() +
val.getAmt2());
return val;
});
cnt++;
if(cnt%100 == 0) {
System.out.println(">>> Update Count: " +
cnt);
}
}
return cnt;
}
}
);


In regards to your comment on cache will not get updated using invoke. How
do I ensure that the new computed value gets stored in the cache. 
The first goal is to iterate through the cache and update a specific data
field in the cache in the fastest way possible. 
The second goal is to - the cache has writethrough/writebehind enabled and
hence the updates will need to be propagated to the database as well.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: map types in Cassandra

[Igor Rudyak]

Ignite-Cassandra module doesn't support Cassandra complex types like
map. Only BLOB and simple types which could be directly mapped
to appropriate java types are supported.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite/Cassandra failing to use supplied value for where clause

[Kenan Dalley]

Yeah, sorry about the confusion.  I was playing around with the field names
to see if that was a possible issue and had refactored the "Test1Key.id"
field to "Test1Key.myId".  It should have been "myId" originally since the
column name was "my_id" (see the updated Test1Key below).

I'm just not getting this to work.  The "IndexedType" pair example only
deals with a Primitive (Long) key to a POJO (Person) and not the POJO
(Test1Key) key to a POJO (Test1).  Nothing that I am doing is getting this
to use the "test123" actual value being sent in instead of the "null" value
that it's using when attempting to execute the CQL.  I've tried with
String/Test1Key and Test1Key/Test1 and I've tried it with only
String/Test1Key and I've tried it with Test1Key/Test1 and without doing the
IndexedTypes.  Nothing has worked.  I continue to get the error:

Caused by: org.apache.ignite.IgniteException: Failed to execute Cassandra
CQL statement: select "column_1", "column_2", "column_3", "column_4",
"column_5", "column_6", "column_7", "column_8", "column_9" from
"dev_qlty"."test1" where "my_id"=?;Caused by:
com.datastax.driver.core.exceptions.InvalidQueryException: Invalid null
value in condition for column my_id


Application.java updates
Test1Key.java
cassandra-ignite.xml updates
I've tried with just the top QueryEntity xml and with both below with the
Java code using the ".cache" instead of ".getOrCreateCache(config)".



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to access Apache Ignite service grid services from a J2ee application server

[Evgenii Zhuravlev]

Hi,

Have you tried to use TcpDiscoveryVmIpFinder instead of
TcpDiscoveryMulticastIpFinder
?

Evgenii



2017-07-29 12:20 GMT+03:00 tmp :

> Thanks for the node on singleton bean, i did so and it seems to be
> responding
> now.
>
> Please NOTE that I am using " -Djava.net.preferIPv4Stack=true" for all my
> nodes viz: data-node,service-node and the j2ee -app server which is running
> the singelton-bean.
>
> However I  STILL keep seeing the below logs continuously on my
> Ignite-Data-Node.
>
> "[14:46:38,876][ERROR][tcp-disco-ip-finder-cleaner-#211%
> myIgniteDataGrid%][TcpDiscoveryMulticastIpFinder]
> Failed to request nodes addresses.
> java.net.SocketException: Can't assign requested address (Error setting
> socket option)
> at java.net.PlainDatagramSocketImpl.socketSetOption0(Native
> Method)
> at
> java.net.PlainDatagramSocketImpl.socketSetOption(
> PlainDatagramSocketImpl.java:74)
> at
> java.net.AbstractPlainDatagramSocketImpl.setOption(
> AbstractPlainDatagramSocketImpl.java:309)
> at java.net.MulticastSocket.setInterface(MulticastSocket.java:471)
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.
> TcpDiscoveryMulticastIpFinder.requestAddresses(
> TcpDiscoveryMulticastIpFinder.java:565)
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.
> TcpDiscoveryMulticastIpFinder.getRegisteredAddresses(
> TcpDiscoveryMulticastIpFinder.java:454)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(
> TcpDiscoverySpi.java:1711)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$
> IpFinderCleaner.cleanIpFinder(ServerImpl.java:1837)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(
> ServerImpl.java:1812)
> at org.apache.ignite.spi.IgniteSpiThread.run(
> IgniteSpiThread.java:62)
> "
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/How-to-access-Apache-Ignite-service-
> grid-services-from-a-J2ee-application-server-tp14970p15799.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Query performance against table with/out backup

[Andrey Mashenkov]

Hi,

You wrote you use persistence=enabled.
Looks like you try to start cache with changed (backup=2) configuration
without recreating the cache,
grid restore cache from store with its config and doesn't apply a new one.

Looks weird that neither new number of backups wasn't applied nor error
occurs.

Can you confirm if it is so and recreate cache resolve the issue?

On Thu, Oct 19, 2017 at 9:29 PM, blackfield 
wrote:

> Here, I am trying to ascertain that I set backup == 2 properly as I
> mentioned
> above that I do not see query performance difference between backup ==1 and
> backup == 2.
>
> I want to make sure that I configure my cache properly.
>
> When I set the backup==2 (to have three copies), I notice the following via
> visor.
>
> The Affinity Backups is still equal to 1. Is this a different property than
> number of backups? If it is not, how do one see the number of backups a
> cache is configured for?
>
> Invoking "cache -a" to see the detail cache stat, with backup==2, under the
> size column, the sum of entries on all nodes is equal to the number of rows
> in the table * 2.  It appears this is the case for backup >= 1?
>
> As in, only one set of backup will be stored in off heap regardless the
> number of backups are specified?
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Re: insert data took too much time

[Andrey Mashenkov]

Hi Lucky,

Also you can try to remove indices from configuration and create them
dynamically via "CREATE INDEX" sql query after load finished.

On Thu, Oct 12, 2017 at 10:36 AM, Lucky  wrote:

> afedotov
> Thanks a lot !
>
> It took 25 seconds with cache.putAll performance. this is better than
> "insert into...";
> I have send the configuration what you need.
> Please see the attachment.
>
>
> Thanks.
> Lucky
>
>
> At 2017-10-11 18:19:18, "afedotov"  wrote:
> >Hi,
> >
> >Did you have a chance to compare the results with cache.put performance?
> >
> >You mentioned that you set writeBehind to true, so do you have a cache store
> >configured?
> >What is the INSERT rate for the underlying DB in this case?
> >
> >Did you enable persistent store option?
> >What is CPU and memory utilization?
> >Do you observe any GC issues?
> >
> >Please share Ignite and GC logs.
> >
> >Kind regards,
> >Alex
> >
> >
> >
> >--
> >Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
> 【网易自营|30天无忧退货】仅售同款价1/4！MUJI制造商“2017秋冬舒适家居拖鞋系列”限时仅34.9元>>
> 
>
>



-- 
Best regards,
Andrey V. Mashenkov

Re: ComputeGrid API in C++

[asingh]

So, here is what I am trying to do:
1. Launch ignite node on one Linux machine
2. Launch another ignite node on second Linux machine
3. On one of the two machines, run the compute_example containing
PrintWords() class and have its Call() method invoked via compute.RunAsync()
on all three machines in parallel.

All three ignite nodes use the same xml that I had attached earlier.

I was not able to have the example dispatch the calls to the independent
ignite nodes, without explicitly registering the PrintWords class in
ignite's main function.

I am sure there is a better way to do this, but I need some help in figuring
out how!





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite/Cassandra failing to use supplied value for where clause

[Andrey Mashenkov]

Kenan,

I've found there is an issue that can cause unexpected behavior when using
Cassandra keyPersistence configuration with Ignite annotations [1].
But XML way should work.

I'm confused as in first message you wrote Test1Key has 'id' field, in last
message it has 'my_id' field and 'myId' in mapping?
Is it possible to have same field name in Ignite and Cassandra? or
configure proper alias for field in QueryEntity configuration?


[1] https://issues.apache.org/jira/browse/IGNITE-6280

On Wed, Oct 25, 2017 at 4:27 PM, Kenan Dalley  wrote:

> I don't understand what the reasoning behind manually setting the
> IndexedType pair in the configuration. Yes, this is a field that is
> indexed, but the problem is that it's not correctly pulling the data from
> C* and it's worked in the past with other C* tables/caches via @Annotation
> without the need to manually set it. Also, based on the changes that I
> made, it didn't make any difference manually setting the IndexedType pair:
> either by Java or by XML. I may not have set it up correctly, though. Note:
> I did not have both the XML and the Java configuration set at the same
> time. I had one or the other and neither worked. By Java in
> Application.java.test1() method
>
> private void test1() {
> System.out.println(">>> Cassandra cache store Test1 example 
> started.");
> CacheConfiguration config = new 
> CacheConfiguration<>(Application.TEST1_CACHE_NAME);
> config.setIndexedTypes(String.class, Test1Key.class);
> try (IgniteCache cache = this.ignite
> //.cache(Application.TEST1_CACHE_NAME)) {
> .getOrCreateCache(config)) {
>
> By XML in cassandra-ignite.xml
>
>
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
> 
> 
> 
> 
>  class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
> 
>  value="cassandraDataSource" />
> 
>  value="test1Cache_persistence_settings" />
> 
> 
> 
> 
> 
>  value="java.lang.String"/>
>  value="com.gm.test_cassandra.model.Test1Key"/>
> 
> 
>  value="java.lang.String"/>
> 
> 
> 
> 
>  class="org.apache.ignite.cache.QueryIndex">
> 
> 
> 
> 
> 
> 
> 
> 
>
>
> --
> Sent from the Apache Ignite Users mailing list archive
>  at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Failed to query ignite

[Andrey Mashenkov]

Hi,

Looks like a bug.

Can you share a simple repro?
So, it can be easily checked against master or ignite-2.3 branch if it is
already fixed or a ticket should be created.




On Wed, Oct 25, 2017 at 5:28 PM, iostream  wrote:

> I think i have found the problem here. When I query my cache on the PRIMARY
> KEY, the query throws exception. When I removed PRIMARY KEY from "id",
> query
> is working fine. How do I query on the PRIMARY KEY ?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: when client node connect to server node, server node throws NotSerializableException

[Andrey Mashenkov]

Hi,

Looks like node fails while trying to serialize cache configuration.
Would you please check if com.pingan.pilot.cache.impl.ignite.
PIgniteCacheStoreFactory implements Serializable?


On Wed, Oct 25, 2017 at 10:12 AM, Jeff Jiao 
wrote:

> Hi Ignite Community,
>
> I have a Ignite server node running well, it has a cache store connect to
> Oracle, and here I already created my own factory to create the cache store
> to avoid Serialization problem. The server node suppose to store
> BinaryObject as value. I already tested all the get, put, remove, query...
> all these functions on the single node and they all work fine.
>
> Then I start a Ignite client node on the same machine, try to produce one
> of
> the use cases which is two applications on the same machine and they share
> the same data set. There is no error on the client side log, but it shows
> exception on server node(see log below), I don't know why, *what happened
> to
> server node when client node connect to it? It tries to serialize
> something?
> * it seems that they already find each other... but I'm not sure about what
> the exception mean here...
>
> please give some advices, thanks.
>
>
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
>
> >>>__  
> >>>   /  _/ ___/ |/ /  _/_  __/ __/
> >>>  _/ // (7 7// /  / / / _/
> >>> /___/\___/_/|_/___/ /_/ /___/
> >>>
> >>> ver. 2.1.0#20170721-sha1:a6ca5c8a
> >>> 2017 Copyright(C) Apache Software Foundation
> >>>
> >>> Ignite documentation: http://ignite.apache.org
>
> [13:56:34]__  
> [13:56:34]   /  _/ ___/ |/ /  _/_  __/ __/
> [13:56:34]  _/ // (7 7// /  / / / _/
> [13:56:34] /___/\___/_/|_/___/ /_/ /___/
> [13:56:34]
> [13:56:34] ver. 2.1.0#20170721-sha1:a6ca5c8a
> [13:56:34] 2017 Copyright(C) Apache Software Foundation
> [13:56:34]
> [13:56:34] Ignite documentation: http://ignite.apache.org
> [13:56:34]
> [13:56:34] Quiet mode.
> [13:56:34]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
> or "-v" to ignite.{sh|bat}
> [13:56:34]
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> Config URL: n/a
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> Daemon mode: off
> [13:56:34] OS: Windows 7 6.1 amd64
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> OS: Windows 7 6.1 amd64
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> OS
> user: JiaoYiMing601
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> PID: 2236
> [13:56:34] VM information: Java(TM) SE Runtime Environment 1.8.0_40-b25
> Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.40-b25
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> Language runtime: Java Platform API Specification ver. 1.8
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> VM
> information: Java(TM) SE Runtime Environment 1.8.0_40-b25 Oracle
> Corporation
> Java HotSpot(TM) 64-Bit Server VM 25.40-b25
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> VM
> total memory: 3.5GB
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> Remote Management [restart: off, REST: on, JMX (remote: off)]
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> IGNITE_HOME=null
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> VM
> arguments: [-DFAST_DB_URL=jdbc:oracle:thin:@192.168.5.11/ideadev3,
> -DFAST_DB_USERNAME=ideafastdev3,
> -DFAST_DB_PASSWORD=---BPOP---YCUNWakVnX0gBE8RIFP2TxAAbhKitiqUlMj5+6RqsIPc/
> QgQwNGlVKTZQmxfzsBa3CXBuw==---EPOP---,
> -Dfile.encoding=GBK]
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> System cache's MemoryPolicy size is configured to 40 MB. Use
> MemoryConfiguration.systemCacheMemorySize property to change the setting.
> 2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
> Configured caches [in 'sysMemPlc' memoryPolicy: ['ignite-sys-cache'], in
> 'default' memoryPolicy: ['igniteTestCache']]
> 2017-10-25_13:56:34.403 [WARN ] [pub-#14%null%]
> [o.a.ignite.internal.GridDiagnostic] Initial heap size is 256MB (should be
> no less than 512MB, use -Xms512m -Xmx512m).
> [13:56:34] Initial heap size is 256MB (should be no less than 512MB, use
> -Xms512m -Xmx512m).
> [13:56:34] Configured plugins:
> 2017-10-25_13:56:34.443 [INFO ] [main] [o.a.i.i.p.p.IgnitePluginProcessor]
> Configured plugins:
> [13:56:34]   ^-- None
> 2017-10-25_13:56:34.443 [INFO ] [main] [o.a.i.i.p.p.IgnitePluginProcessor]
> ^-- None
> [13:56:34]
> 2017-10-25_13:56:34.453 [INFO ] [main] [o.a.i.i.p.p.IgnitePluginProcessor]
> 2017-10-25_13:56:34.493 [INFO ] [main] [o.a.i.s.c.tcp.TcpCommunicationSpi]
> Successfully bound communication NIO server to TCP port [port=47100,
> locHost=/192.168.6.71, selectorsCnt=4, selectorSpins=0, pairedConn=false]

Re: Failed to query ignite

[iostream]

I think i have found the problem here. When I query my cache on the PRIMARY
KEY, the query throws exception. When I removed PRIMARY KEY from "id", query
is working fine. How do I query on the PRIMARY KEY ?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Retrieving keys very slow from a partitioned, no backup, no replicated cache

[Alexey Popov]

Hi Anand,

1. Actually, you should broadcast your logic to all nodes via
Ignite.compute()
Please see the sample below:

Collection res = ignite.compute().broadcast(
new IgniteCallable() {
/** Auto-inject ignite instance. */
@IgniteInstanceResource
private Ignite ignite;

@Override public Integer call() {
IgniteCache cache =
ignite.getOrCreateCache(CACHE_NAME);
Iterator> iterator =
cache.localEntries().iterator();

Integer key;
Integer cnt = 0;
while (iterator.hasNext()) {
key = iterator.next().getKey();

Integer res = cache.invoke(key, (entry, args) -> {
Integer val = entry.getValue();
// do some logic
val = val + 1;
return val;
});
cnt++;
}
return cnt;
}
}
);

// just ensure that we went through all keys
int entryCount = 0;
for (Integer r : res) {
entryCount += r;
}

2. And please note that cache.invokeAll() from your code does not store a
new cache entry value back to the Cache. So, you will not see any updates in
the Cache after such invokes.

"val.setAmount(val.getAmt1() + val.getAmt2());"

Thank you,
Alexey





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to query ignite

[iostream]

Hi Andrew,

I created a table with BIGINT instead of LONG data type in cache. I am still
getting the same error when I try to query using java.Lang.Long

DDL-

CREATE TABLE test4 
(
  id /BIGINT/,
  name   VARCHAR,
  PRIMARY KEY (id)
  
)
WITH "backups=1,affinityKey=id";




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Job Listeners

[Alexey Kukushkin]

Hi Chandrika,

I cannot reproduce the problem with the code you provided. I created a
public GIT repo with your code from the previous post and suggest this:

   1. You clone the repo: git clone
   https://github.com/kukushal/ignite-userlist-joblisteners.git
   2. Try to reproduce the problem with the code from the repo. If you can
   - let me know. To me that code runs only 2 tasks (12 and 11) and then
   finishes without any deadlocks on any number of nodes (I tried 1, 2, 3, 4
   and 5 nodes)
   3. If you cannot reproduce the problem, please check how your code is
   different. Please give me your github ID and I will add you as a
   collaborator. You modify the code to make sure it reproduce the problem and
   commit it to GIT.
   4. I will get your code and investigate.

Re: Error : Commit produced a runtime exception

[Nikolai Tikhonov]

Hi,

Sorry, for late answer.

It's known usability problem (that Apache Ignite node requests 80% of RAM
from an operating system) which was fixed in 2.2 version. [1][2]  Could you
update to the latest version and check it?

Please, ignore if it is not actual any more.

1
http://apache-ignite-developers.2346864.n4.nabble.com/IGNITE-5717-improvements-of-MemoryPolicy-default-size-td20264.html
2
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Urgent-Ignite-bug-fix-release-td21292.html#a21307



On Mon, Jul 31, 2017 at 4:03 PM, iostream  wrote:

> No I have not set any memory configuration
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Error-Commit-produced-a-runtime-
> exception-tp15768p15824.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Ignite/Cassandra failing to use supplied value for where clause

[Kenan Dalley]

I don't understand what the reasoning behind manually setting the IndexedType
pair in the configuration.  Yes, this is a field that is indexed, but the
problem is that it's not correctly pulling the data from C* and it's worked
in the past with other C* tables/caches via @Annotation without the need to
manually set it.  Also, based on the changes that I made, it didn't make any
difference manually setting the IndexedType pair: either by Java or by XML. 
I may not have set it up correctly, though.Note: I did not have both the XML
and the Java configuration set at the same time.  I had one or the other and
neither worked.
By Java in Application.java.test1() method
By XML in cassandra-ignite.xml




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite-cassandra module issue

[Tobias Eriksson]

Hi Andrey et al
I believe I now know what the problem is, the Cassandra session is refreshed, 
but before it is a prepared statement is created/used and there, and so using a 
new session with an old prepared statement is not working.

The way to reproduce is

1)   Start Ignite Server Node

2)   Start client which inserts a batch of 100 elements

3)   End client

4)   Now Ignite Server Node returns the Cassandra Session to the pool

5)   Wait 5+ minutes

6)   Now Ignite Server Node has does a clean-up of the “unused” Cassandra 
sessions

7)   Start client which inserts a batch of 100 elements

8)   Boom ! The exception starts to happen

Reason is

1)   Execute is called for a BATCH

2)   Prepared-statement is reused since there is a cache of those

3)   It is about to do session().execute( batch )

4)   BUT the call to session() results in refreshing the session, and this 
is where the prepared statements as the old session new them are cleaned up

5)   Now it is looping over 100 times with a NEW session but with an OLD 
prepared statement

This is a bug,

-Tobias


From: Andrey Mashenkov 
Reply-To: "user@ignite.apache.org" 
Date: Wednesday, 25 October 2017 at 14:12
To: "user@ignite.apache.org" 
Subject: Re: Ignite-cassandra module issue

Hi Tobias,

What ignite version do you use? May be this was already fixed in latest one?
I see related fix inclueded in upcoming 2.3 version.

See IGNITE-5897 [1] issue. It is unobvious, but this fix session init\end 
logic, so session should be closed in proper way.

[1] https://issues.apache.org/jira/browse/IGNITE-5897


On Wed, Oct 25, 2017 at 11:13 AM, Tobias Eriksson 
mailto:tobias.eriks...@qvantel.com>> wrote:
Hi
 Sorry did not include the context when I replied
 Has anyone been able to resolve this problem, cause I have it too on and
off
In fact it sometimes happens just like that, e.g. I have been running my
Ignite client and then stop it, and then it takes a while and run it again,
and all by a sudden this error shows up. An that is the first thing that
happens, and there is NOT a massive amount of load on Cassandra at that
time. But I have also seen it when I hammer Ignite/Cassandra with
updates/inserts.

This is a deal-breaker for me, I need to understand how to fix this, cause
having this in production is not an option.

-Tobias


Hi!
I'm using the cassandra as persistence store for my caches and have one
issue by handling a huge data (via IgniteDataStreamer from kafka).
Ignite Configuration:
final IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
igniteConfiguration.setIgniteInstanceName("test");
igniteConfiguration.setClientMode(true);
igniteConfiguration.setGridLogger(new Slf4jLogger());
igniteConfiguration.setMetricsLogFrequency(0);
igniteConfiguration.setDiscoverySpi(configureTcpDiscoverySpi());
final BinaryConfiguration binaryConfiguration = new BinaryConfiguration();
binaryConfiguration.setCompactFooter(false);
igniteConfiguration.setBinaryConfiguration(binaryConfiguration);
igniteConfiguration.setPeerClassLoadingEnabled(true);
final MemoryPolicyConfiguration memoryPolicyConfiguration = new
MemoryPolicyConfiguration();
memoryPolicyConfiguration.setName("3Gb_Region_Eviction");
memoryPolicyConfiguration.setInitialSize(1024L * 1024L * 1024L);
memoryPolicyConfiguration.setMaxSize(3072L * 1024L * 1024L);

memoryPolicyConfiguration.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
final MemoryConfiguration memoryConfiguration = new MemoryConfiguration();
memoryConfiguration.setMemoryPolicies(memoryPolicyConfiguration);
igniteConfiguration.setMemoryConfiguration(memoryConfiguration);

Cache configuration:
final CacheConfiguration cacheConfiguration = new
CacheConfiguration<>();
cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfiguration.setStoreKeepBinary(true);
cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
cacheConfiguration.setBackups(0);
cacheConfiguration.setStatisticsEnabled(false);
cacheConfiguration.setName("TestCache");

cacheConfiguration.setReadThrough(true);
cacheConfiguration.setWriteThrough(true);

cacheConfiguration.setWriteBehindEnabled(true);
cacheConfiguration.setWriteBehindFlushFrequency(1);
cacheConfiguration.setWriteBehindFlushSize(0);
cacheConfiguration.setWriteBehindFlushThreadCount(2);
cacheConfiguration.setWriteBehindBatchSize(1);


final CassandraCacheStoreFactory
cacheStoreFactory = new CassandraCacheStoreFactory<>();
final DataSource dataSource = new DataSource();
dataSource.setContactPoints(contactPoints);
dataSource.setReadConsistency("ONE");
dataSource.setWriteConsistency("ONE");
dataSource.setLoadBalancingPolicy(new TokenAwarePolicy(new
RoundRobinPolicy()));

Re: Re: Where can we get the partition assignment of a Cache after the cluster changed?

[slava.koptilin]

Hi Aaron,

> If I bring up the node X, every events as expected,  but Now If I start
> Node Y,
> I still got a lot CacheA and CacheB 's rebalance events. 
> But Node Y's start or stop should not affect CacheA & CacheB,
> their partition never change or rebalance,  log after Y started:
Yes, I see your point.
When a new node joins the cluster the process of 'partition map exchange' is
triggered
(potentially, some partitions will be moved to a new node) and this the
reason why you get CACHE_REBALACE_STARTED/STOPPED events.
In your particular case, these messages are redundant. Perhaps, it should be
handled on Ignite side...

Thanks!




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Hammering Cassandra with Ignite seems to get ignite into an infinite loop

[Andrey Mashenkov]

Tobias,

Looks it is a duplicate. I've answered in next tread:
http://apache-ignite-users.70518.x6.nabble.com/Ignite-cassandra-module-issue-td13808.html


You can try to build Ignite from master [1] or ignite-2.3 [2] branch and
check against it and let us know if the issue will gone or not.


[1] https://github.com/apache/ignite/tree/master
[2] https://github.com/apache/ignite/tree/ignite-2.3

On Mon, Oct 23, 2017 at 2:19 PM, slava.koptilin 
wrote:

> Hi Tobias,
>
> As far as I know, Cassandra module was tested with Cassandra Database 3.3 &
> Cassandra Driver 3.0.0.
> Perhaps, this issue related to Cassandra v3.11. Could you please check that
> the issue is reproducible with Cassandra Database 3.3 & Cassandra Driver
> 3.0.0?
>
> Thanks!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Ignite-cassandra module issue

[Andrey Mashenkov]

Hi Tobias,

What ignite version do you use? May be this was already fixed in latest one?
I see related fix inclueded in upcoming 2.3 version.

See IGNITE-5897 [1] issue. It is unobvious, but this fix session init\end
logic, so session should be closed in proper way.

[1] https://issues.apache.org/jira/browse/IGNITE-5897


On Wed, Oct 25, 2017 at 11:13 AM, Tobias Eriksson <
tobias.eriks...@qvantel.com> wrote:

> Hi
>  Sorry did not include the context when I replied
>  Has anyone been able to resolve this problem, cause I have it too on and
> off
> In fact it sometimes happens just like that, e.g. I have been running my
> Ignite client and then stop it, and then it takes a while and run it again,
> and all by a sudden this error shows up. An that is the first thing that
> happens, and there is NOT a massive amount of load on Cassandra at that
> time. But I have also seen it when I hammer Ignite/Cassandra with
> updates/inserts.
>
> This is a deal-breaker for me, I need to understand how to fix this, cause
> having this in production is not an option.
>
> -Tobias
>
>
> Hi!
> I'm using the cassandra as persistence store for my caches and have one
> issue by handling a huge data (via IgniteDataStreamer from kafka).
> Ignite Configuration:
> final IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
> igniteConfiguration.setIgniteInstanceName("test");
> igniteConfiguration.setClientMode(true);
> igniteConfiguration.setGridLogger(new Slf4jLogger());
> igniteConfiguration.setMetricsLogFrequency(0);
> igniteConfiguration.setDiscoverySpi(configureTcpDiscoverySpi());
> final BinaryConfiguration binaryConfiguration = new BinaryConfiguration();
> binaryConfiguration.setCompactFooter(false);
> igniteConfiguration.setBinaryConfiguration(binaryConfiguration);
> igniteConfiguration.setPeerClassLoadingEnabled(true);
> final MemoryPolicyConfiguration memoryPolicyConfiguration = new
> MemoryPolicyConfiguration();
> memoryPolicyConfiguration.setName("3Gb_Region_Eviction");
> memoryPolicyConfiguration.setInitialSize(1024L * 1024L * 1024L);
> memoryPolicyConfiguration.setMaxSize(3072L * 1024L * 1024L);
>
> memoryPolicyConfiguration.setPageEvictionMode(
> DataPageEvictionMode.RANDOM_2_LRU);
> final MemoryConfiguration memoryConfiguration = new MemoryConfiguration();
> memoryConfiguration.setMemoryPolicies(memoryPolicyConfiguration);
> igniteConfiguration.setMemoryConfiguration(memoryConfiguration);
>
> Cache configuration:
> final CacheConfiguration cacheConfiguration = new
> CacheConfiguration<>();
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheConfiguration.setStoreKeepBinary(true);
> cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> cacheConfiguration.setBackups(0);
> cacheConfiguration.setStatisticsEnabled(false);
> cacheConfiguration.setName("TestCache");
>
> cacheConfiguration.setReadThrough(true);
> cacheConfiguration.setWriteThrough(true);
>
> cacheConfiguration.setWriteBehindEnabled(true);
> cacheConfiguration.setWriteBehindFlushFrequency(1);
> cacheConfiguration.setWriteBehindFlushSize(0);
> cacheConfiguration.setWriteBehindFlushThreadCount(2);
> cacheConfiguration.setWriteBehindBatchSize(1);
>
>
> final CassandraCacheStoreFactory
> cacheStoreFactory = new CassandraCacheStoreFactory<>();
> final DataSource dataSource = new DataSource();
> dataSource.setContactPoints(contactPoints);
> dataSource.setReadConsistency("ONE");
> dataSource.setWriteConsistency("ONE");
> dataSource.setLoadBalancingPolicy(new TokenAwarePolicy(new
> RoundRobinPolicy()));
> cacheStoreFactory.setDataSource(dataSource);
>
> final String CASSANDRA_PERSISTENCE = " table=\"%s\">" +
> "" +
> "" +
> "";
> final KeyValuePersistenceSettings settings = new
> KeyValuePersistenceSettings(
> String.format(CASSANDRA_PERSISTENCE, "test", "test_table",
> "java.lang.String", "PRIMITIVE",
> "org.apache.ignite.binary.BinaryObject", "BLOB"));
> cacheStoreFactory.setPersistenceSettings(settings);
> cacheConfiguration.setCacheStoreFactory(cacheStoreFactory);
>
> When application works some time (hour or more, may be less - from time to
> time) I see this exceptions on the ignite nodes:
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=4f43d78b, name=null, uptime=00:12:00:072]
> ^-- H/N/C [hosts=3, nodes=3, CPUs=96]
> ^-- CPU [cur=0%, avg=1.86%, GC=0%]
> ^-- PageMemory [pages=118064]
> ^-- Heap [used=4800MB, free=53.12%, comm=10240MB]
> ^-- Non heap [used=78MB, free=-1%, comm=80MB]
> ^-- Public thread pool [active=0, idle=32, qSize=0]
> ^-- System

Re: Ignite/Cassandra failing to use supplied value for where clause

[Andrey Mashenkov]

Kenan,

You can try to set IndexedType pair to cache configuration [1] to force
Ignite process annotations on Key\Value classes.

[1]
https://apacheignite.readme.io/docs/indexes#section-registering-indexed-types

On Tue, Oct 24, 2017 at 11:00 PM, Kenan Dalley  wrote:

> These were included in the project file link in the initial post. Here
> they are in text for reference. The only thing not posted is the pom.xml
> and it's just standard SpringBoot Ignite w/ Cassandra.
> cassandra-ignite.xml
>
> 
>
> 
>
> 
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd";>
>
> 
> 
>
>  class="org.apache.ignite.cache.store.cassandra.persistence.KeyValuePersistenceSettings">
>  value="classpath:test1-cassandra-persistence-settings.xml" />
> 
>
>  class="org.apache.ignite.configuration.IgniteConfiguration">
> 
> 
>
> 
> 
> 
> 
> 
> 
> 
>  class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
> 
> 
> 
> 127.0.0.1:47500..47509
> 
> 
> 
> 
> 
> 
> 
> 
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
> 
> 
> 
> 
>  class="org.apache.ignite.cache.store.cassandra.CassandraCacheStoreFactory">
> 
>  value="cassandraDataSource" />
> 
>  value="test1Cache_persistence_settings" />
> 
> 
> 
> 
> 
> 
> 
>
> test1-cassandra-persistence-settings.xml
>
> 
>  strategy="POJO" />
>  strategy="POJO" />
> 
>
> cassandra-connection-settings.xml
>
> 
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
>
>  class="com.datastax.driver.core.policies.TokenAwarePolicy">
>  type="com.datastax.driver.core.policies.LoadBalancingPolicy">
> 
> 
> 
>
> 
> server-name
> 
>
>  class="org.apache.ignite.cache.store.cassandra.datasource.DataSource">
> 
> 
> 
> 
> 
> 
> 
> 
>
> Test1Key.java
>
> package com.gm.test_cassandra.model;
>
> import org.apache.ignite.cache.query.annotations.QuerySqlField;
>
> public class Test1Key {
>
> //Partition Key
> @QuerySqlField(index = true, groups = {"test1_PK"}, name = "my_id")
> private transient String my_id;
>
> public Test1Key() {
> }
>
> public Test1Key(final String my_id) {
> this.my_id = my_id;
> }
>
> public String getMy_id() {
> return my_id;
> }
>
> public void setMy_id(final String my_id) {
> this.my_id = my_id;
> }
>
> @Override
> public boolean equals(final Object o) {
> if (this == o) {
> return true;
> }
>
> if (!(o instanceof Test1Key)) {
> return false;
> }
>
> final Test1Key other = (Test1Key) o;
>
> if (this.my_id != null ? !this.my_id.equals(other.my_id) : 
> other.my_id != null) {
> return false;
> }
> return true;
> }
>
> @Override
> public int hashCode() {
> return this.my_id != null ? this.my_id.hashCode() : 0;
> }
>
> @Override
> public String toString() {
> final StringBuilder sb = new StringBuilder();
> sb.append("{");
> sb.append(Test1Key.class.getSimpleName() + ": {");
> sb.append("my_id: ");
> sb.append(this.my_id);
> sb.append("}}");
> return sb.toString();
> }
> }
>
> Test1.java
>
> package com.gm.test_cassandra.model;
>
> import java.util.Date;
> import org.apache.ignite.cache.query.annotations.QuerySqlField;
>
> public class Test1 {
>
> /*
> column_1 text,
> column_2 timestamp,
> column_3 text,
> column_4 timestamp,
> column_5 text,
> column_6 text,
> column

Re: Cannot activate Ignite with custom security plugin

[Andrey Mashenkov]

Oh, you are right.
It looks already fixed.

On Tue, Oct 24, 2017 at 9:12 PM, calebs  wrote:

> I just checked out ignite-2.3 branch, and I'm gladly surprised to see the
> following two cases are already included in the GridRestProcessor.authorize
> method.
>
> case CLUSTER_ACTIVE:
> case CLUSTER_INACTIVE:
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: custom restful service using jetty server on top of ignite grid to perform crud operations and configure cache store

[dkarachentsev]

Hi,

In your code Ignite could not inject it's instance, because you have two
instances of your class: one in Ignite as a service, another one is object
that processes requests in Jersey. So when you're doing http query, it goes
to jersey instance.

How do you start Ignite? You may get ignite with Ignition.ignite().

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Retrieving keys very slow from a partitioned, no backup, no replicated cache

[Andrey Mashenkov]

Hi Anand,

Sorry, I've mislooked you use scan query already.
In your case you fetch data from grid and then try to make invoke for every
batch of entries.
I doubt invokeAll will give any speedup in this case comparing to simple
put as you already have full value fetched in your hands.

Why you won't send job to all cache nodes that will run local scan queries
on each node to achieve the same?


To run a job you just need a call:
ignite.compute(igntie.cluster().forCacheNodes("mycache")).execute(...);

Inside job you can make smth like this:

ScanQuery scanqry = new ScanQuery().setLocal(true);

for(Entry<> e:ignite.cache("mycache").query(scanqry))
 //your logic here


Also you can make per partition scan and run them in parallel.

On Tue, Oct 24, 2017 at 8:30 PM, Anand Vijai  wrote:

> Hi Andrew,
>
>
> I did try with the Iterator implementation using iterator.hasNext() (you
> can
> see that its commented in the code i shared) but the performance is only
> slightly higher (hence gave the range 10-100 keys per second)
>
> Do you want me to try to use a cursor explicitly instead of using an
> iterator.
> I need to go through all the keys and perform the computation.
>
>
> Regards
> Anand Vijai
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Cannot find schema for object with compact footer

[slava.koptilin]

Hi,

This feature relates to the underlying representation of binary objects. If
you disable the compact footer then a binary object will require a bit more
memory, but in general, this impact should not be significant, I think.

Could you please clarify where the exception is thrown? On client node?

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite-cassandra module issue

[Tobias Eriksson]

Hi 
 Sorry did not include the context when I replied 
 Has anyone been able to resolve this problem, cause I have it too on and
off 
In fact it sometimes happens just like that, e.g. I have been running my 
Ignite client and then stop it, and then it takes a while and run it again, 
and all by a sudden this error shows up. An that is the first thing that 
happens, and there is NOT a massive amount of load on Cassandra at that 
time. But I have also seen it when I hammer Ignite/Cassandra with 
updates/inserts. 

This is a deal-breaker for me, I need to understand how to fix this, cause 
having this in production is not an option. 

-Tobias 


Hi! 
I'm using the cassandra as persistence store for my caches and have one
issue by handling a huge data (via IgniteDataStreamer from kafka). 
Ignite Configuration: 
final IgniteConfiguration igniteConfiguration = new IgniteConfiguration(); 
igniteConfiguration.setIgniteInstanceName("test"); 
igniteConfiguration.setClientMode(true); 
igniteConfiguration.setGridLogger(new Slf4jLogger()); 
igniteConfiguration.setMetricsLogFrequency(0); 
igniteConfiguration.setDiscoverySpi(configureTcpDiscoverySpi()); 
final BinaryConfiguration binaryConfiguration = new BinaryConfiguration(); 
binaryConfiguration.setCompactFooter(false); 
igniteConfiguration.setBinaryConfiguration(binaryConfiguration); 
igniteConfiguration.setPeerClassLoadingEnabled(true); 
final MemoryPolicyConfiguration memoryPolicyConfiguration = new
MemoryPolicyConfiguration(); 
memoryPolicyConfiguration.setName("3Gb_Region_Eviction"); 
memoryPolicyConfiguration.setInitialSize(1024L * 1024L * 1024L); 
memoryPolicyConfiguration.setMaxSize(3072L * 1024L * 1024L); 
   
memoryPolicyConfiguration.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
 
final MemoryConfiguration memoryConfiguration = new MemoryConfiguration(); 
memoryConfiguration.setMemoryPolicies(memoryPolicyConfiguration); 
igniteConfiguration.setMemoryConfiguration(memoryConfiguration); 

Cache configuration: 
final CacheConfiguration cacheConfiguration = new
CacheConfiguration<>(); 
cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC); 
cacheConfiguration.setStoreKeepBinary(true); 
cacheConfiguration.setCacheMode(CacheMode.PARTITIONED); 
cacheConfiguration.setBackups(0); 
cacheConfiguration.setStatisticsEnabled(false); 
cacheConfiguration.setName("TestCache"); 

cacheConfiguration.setReadThrough(true); 
cacheConfiguration.setWriteThrough(true); 

cacheConfiguration.setWriteBehindEnabled(true); 
cacheConfiguration.setWriteBehindFlushFrequency(1); 
cacheConfiguration.setWriteBehindFlushSize(0); 
cacheConfiguration.setWriteBehindFlushThreadCount(2); 
cacheConfiguration.setWriteBehindBatchSize(1); 


final CassandraCacheStoreFactory
cacheStoreFactory = new CassandraCacheStoreFactory<>(); 
final DataSource dataSource = new DataSource(); 
dataSource.setContactPoints(contactPoints); 
dataSource.setReadConsistency("ONE"); 
dataSource.setWriteConsistency("ONE"); 
dataSource.setLoadBalancingPolicy(new TokenAwarePolicy(new
RoundRobinPolicy())); 
cacheStoreFactory.setDataSource(dataSource); 

final String CASSANDRA_PERSISTENCE = "" + 
"" + 
"" + 
""; 
final KeyValuePersistenceSettings settings = new
KeyValuePersistenceSettings( 
String.format(CASSANDRA_PERSISTENCE, "test", "test_table", 
"java.lang.String", "PRIMITIVE", 
"org.apache.ignite.binary.BinaryObject", "BLOB")); 
cacheStoreFactory.setPersistenceSettings(settings); 
cacheConfiguration.setCacheStoreFactory(cacheStoreFactory); 

When application works some time (hour or more, may be less - from time to
time) I see this exceptions on the ignite nodes: 
Metrics for local node (to disable set 'metricsLogFrequency' to 0) 
^-- Node [id=4f43d78b, name=null, uptime=00:12:00:072] 
^-- H/N/C [hosts=3, nodes=3, CPUs=96] 
^-- CPU [cur=0%, avg=1.86%, GC=0%] 
^-- PageMemory [pages=118064] 
^-- Heap [used=4800MB, free=53.12%, comm=10240MB] 
^-- Non heap [used=78MB, free=-1%, comm=80MB] 
^-- Public thread pool [active=0, idle=32, qSize=0] 
^-- System thread pool [active=0, idle=32, qSize=0] 
^-- Outbound messages queue [size=0] 
[15:28:28,626][INFO][grid-timeout-worker-#39%null%][IgniteKernal] FreeList
[name=null, buckets=256, dataPages=102080, reusePages=0] 
[15:29:02,317][WARNING][sys-#106%null%][CassandraCacheStore] Prepared
statement cluster error detected, refreshing Cassandra session 
com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute
unknown prepared query : 0xcad5832309a512feeb602eec67408130. You may have
used a PreparedStatement

when client node connect to server node, server node throws NotSerializableException

[Jeff Jiao]

Hi Ignite Community,

I have a Ignite server node running well, it has a cache store connect to
Oracle, and here I already created my own factory to create the cache store
to avoid Serialization problem. The server node suppose to store
BinaryObject as value. I already tested all the get, put, remove, query...
all these functions on the single node and they all work fine.

Then I start a Ignite client node on the same machine, try to produce one of
the use cases which is two applications on the same machine and they share
the same data set. There is no error on the client side log, but it shows
exception on server node(see log below), I don't know why, *what happened to
server node when client node connect to it? It tries to serialize something?
* it seems that they already find each other... but I'm not sure about what
the exception mean here... 

please give some advices, thanks.


2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal] 

>>>__    
>>>   /  _/ ___/ |/ /  _/_  __/ __/  
>>>  _/ // (7 7// /  / / / _/
>>> /___/\___/_/|_/___/ /_/ /___/   
>>> 
>>> ver. 2.1.0#20170721-sha1:a6ca5c8a
>>> 2017 Copyright(C) Apache Software Foundation
>>> 
>>> Ignite documentation: http://ignite.apache.org

[13:56:34]__   
[13:56:34]   /  _/ ___/ |/ /  _/_  __/ __/ 
[13:56:34]  _/ // (7 7// /  / / / _/   
[13:56:34] /___/\___/_/|_/___/ /_/ /___/  
[13:56:34] 
[13:56:34] ver. 2.1.0#20170721-sha1:a6ca5c8a
[13:56:34] 2017 Copyright(C) Apache Software Foundation
[13:56:34] 
[13:56:34] Ignite documentation: http://ignite.apache.org
[13:56:34] 
[13:56:34] Quiet mode.
[13:56:34]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
or "-v" to ignite.{sh|bat}
[13:56:34] 
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
Config URL: n/a
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
Daemon mode: off
[13:56:34] OS: Windows 7 6.1 amd64
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
OS: Windows 7 6.1 amd64
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal] OS
user: JiaoYiMing601
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
PID: 2236
[13:56:34] VM information: Java(TM) SE Runtime Environment 1.8.0_40-b25
Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.40-b25
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
Language runtime: Java Platform API Specification ver. 1.8
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal] VM
information: Java(TM) SE Runtime Environment 1.8.0_40-b25 Oracle Corporation
Java HotSpot(TM) 64-Bit Server VM 25.40-b25
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal] VM
total memory: 3.5GB
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
Remote Management [restart: off, REST: on, JMX (remote: off)]
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
IGNITE_HOME=null
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal] VM
arguments: [-DFAST_DB_URL=jdbc:oracle:thin:@192.168.5.11/ideadev3,
-DFAST_DB_USERNAME=ideafastdev3,
-DFAST_DB_PASSWORD=---BPOP---YCUNWakVnX0gBE8RIFP2TxAAbhKitiqUlMj5+6RqsIPc/QgQwNGlVKTZQmxfzsBa3CXBuw==---EPOP---,
-Dfile.encoding=GBK]
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
System cache's MemoryPolicy size is configured to 40 MB. Use
MemoryConfiguration.systemCacheMemorySize property to change the setting.
2017-10-25_13:56:34.403 [INFO ] [main] [o.a.ignite.internal.IgniteKernal]
Configured caches [in 'sysMemPlc' memoryPolicy: ['ignite-sys-cache'], in
'default' memoryPolicy: ['igniteTestCache']]
2017-10-25_13:56:34.403 [WARN ] [pub-#14%null%]
[o.a.ignite.internal.GridDiagnostic] Initial heap size is 256MB (should be
no less than 512MB, use -Xms512m -Xmx512m).
[13:56:34] Initial heap size is 256MB (should be no less than 512MB, use
-Xms512m -Xmx512m).
[13:56:34] Configured plugins:
2017-10-25_13:56:34.443 [INFO ] [main] [o.a.i.i.p.p.IgnitePluginProcessor]
Configured plugins:
[13:56:34]   ^-- None
2017-10-25_13:56:34.443 [INFO ] [main] [o.a.i.i.p.p.IgnitePluginProcessor]  
^-- None
[13:56:34] 
2017-10-25_13:56:34.453 [INFO ] [main] [o.a.i.i.p.p.IgnitePluginProcessor] 
2017-10-25_13:56:34.493 [INFO ] [main] [o.a.i.s.c.tcp.TcpCommunicationSpi]
Successfully bound communication NIO server to TCP port [port=47100,
locHost=/192.168.6.71, selectorsCnt=4, selectorSpins=0, pairedConn=false]
2017-10-25_13:56:34.493 [WARN ] [main] [o.a.i.s.c.tcp.TcpCommunicationSpi]
Message queue limit is set to 0 which may lead to potential OOMEs when
running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message
queues growth on sender and receiver sides.
[13:56:34] Message queue limit is set to 0 which may lead to potential OOMEs
when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due t