Affinity and String key (was: SQLQuery with simple Join return no results)

2017-09-05 Thread Roger Fischer (CW)
e)), I would use them only if it’s impossible to set up the collocation between 2 entities. That’s not your case from what I see. NON collocated joins are slower than collocated ones. — Denis On Sep 1, 2017, at 2:53 PM, Roger Fischer (CW) <rfis...@brocade.com<mailto:rfis...@brocade.com&

RE: SQLQuery with simple Join return no results

2017-09-01 Thread Roger Fischer (CW)
Hi Matt, are the objects to join collocated, ie. do they have the same affinity key? If yes, it should work (it worked for me). If no, you need to enable distributed joins for the query. See the middle line. SqlFieldsQuery qry = new SqlFieldsQuery( stmt);

RE: Ignite as Cache and Query layer for Cassandra.

2017-08-31 Thread Roger Fischer (CW)
Hi Mark, I recently did a Proof-of-Concept with Ignite and Cassandra. There are a number of challenges you need to look out for: 1) In order to do SQL queries, you need to have all the data in Ignite. It is not practical to include other persistence layers in the query process. The exception

Schema vs CacheName

2017-08-29 Thread Roger Fischer (CW)
Hello, I am a bit confused about schemas and cache names. In some cases they seem synonymous, on others the appear different. So far I have defined everything in XML configuration. To execute a query, I get the cache [getOrCreateCache()], and then execute the query statement. When the

RE: Cassandra Persistence Store without POJO on Server

2017-08-24 Thread Roger Fischer (CW)
Thanks, Val. I keep an eye on the ticket. Roger -Original Message- From: vkulichenko [mailto:valentin.kuliche...@gmail.com] Sent: Thursday, August 24, 2017 3:43 PM To: user@ignite.apache.org Subject: Re: Cassandra Persistence Store without POJO on Server Hi Roger, Currently this is

Cassandra Persistence Store without POJO on Server

2017-08-24 Thread Roger Fischer (CW)
Hello, is it possible to use the Cassandra Persistence Store (libs/optional/ignite-cassandra-store) without the need to deploy the key and object POJOs on the server? For basic data grid use, it seems that everything defaults to binary objects. put(), get() SQL queries, native persistence and

How to define the Affinity Key in XML

2017-08-17 Thread Roger Fischer (CW)
Hello, how does one define the affinity key in XML? The documentation describes only the @CacheAffinityKeyMapped annotation. I have found a couple of examples that define the affinity key in the cacheKeyConfiguration. For example: modules/yardstick/config/ignite-bin-multicast-config.xml

Functions returning a result set: supported ?

2017-08-15 Thread Roger Fischer (CW)
Hello, does Ignite SQL support functions that return a results set, and the use of those functions in SQL as a table equivalent? If yes, is there an example? H2 supports it, see http://www.h2database.com/html/features.html#user_defined_functions (need to scroll down a bit). Thanks... Roger

RE: Activation: slow and: Ignite node crashed in the middle of checkpoint.

2017-08-15 Thread Roger Fischer (CW)
Hi Dmitry and Alex, the cache contains 19.2M objects. The work/db directory is 23, 26 and 22 GB respectively. The 3 nodes have 8 GB RAM each. I initiated deactivate at 14:13:39. As of 16:50:00, deactivate has not completed. Only server node 2 continues to log warnings. The client shows the

RE: Strange problems with Ignite native Persistence when Data exceeds Memory

2017-08-15 Thread Roger Fischer (CW)
Hi Dmitry, yes, the data set is bigger than the available memory. I am evaluating native persistence, specifically SQL queries when pages have to be loaded from disk. Thus I need a data set that is significantly bigger than what fits the memory. I made another attempt, loading smaller batches

RE: Activation: slow and: Ignite node crashed in the middle of checkpoint.

2017-08-14 Thread Roger Fischer (CW)
Aug 12, 2017 at 12:39 AM, Roger Fischer (CW) [via Apache Ignite Users] <[hidden email]> wrote: Hello, I am wondering if the following behavior is typical, or if it represents a concern. I have a 3 node cluster with native persistence. Each node as 4 CPU and 16 GB of RAM. Each node has ~45

Strange problems with Ignite native Persistence when Data exceeds Memory

2017-08-14 Thread Roger Fischer (CW)
Hello, I am running into strange problems with Ignite Native Persistence when the data exceeds the size of the cache (memory). I have 3 nodes with 8 GB each, one cache, using 1 backup. I am using SQL queries, and the cache has 3 indexes. I am loading 9.6M objects, using IgniteDataStreamer in

Activation: slow and: Ignite node crashed in the middle of checkpoint.

2017-08-11 Thread Roger Fischer (CW)
Hello, I am wondering if the following behavior is typical, or if it represents a concern. I have a 3 node cluster with native persistence. Each node as 4 CPU and 16 GB of RAM. Each node has ~45 GB used in work/db. Total across the 3 nodes is about 36.5 M objects. I am using SQL queries, and

Visor: cache counts are twice of what is expected

2017-08-10 Thread Roger Fischer (CW)
Hello, the cache counts that are shown in Visor seem to be twice the number that is expected. I am using ver. 2.1.0#20170720-sha1:a6ca5c8a, with native persistence. For a replicated cache, with 363 objects loaded (select count(*) returns 363): Nodes for: FabricCache(@c0)

RE: Cassandra Cache Store: How are loadCache() queries distributed

2017-08-04 Thread Roger Fischer (CW)
=Um-YJWzYXVumPwMqixM6akpUk4J0hAYBgLfaLalcoio=I27CtlmoY0a0bYf8Iw9GSJrC0Yv0mizq3iIx9oo7TcA=> Igor On Thu, Aug 3, 2017 at 2:29 PM, Roger Fischer (CW) <rfis...@brocade.com<mailto:rfis...@brocade.com>> wrote: Hello, could someone please explain to me how loadCache() queries are distributed to the Cassan

Cassandra Cache Store: How are loadCache() queries distributed

2017-08-03 Thread Roger Fischer (CW)
Hello, could someone please explain to me how loadCache() queries are distributed to the Cassandra instances when using the Cassandra Cache Store module. I used Ignite logging and Cassandra server tracing (system_traces.sessions) to try to determine how queries are distributed, but I can't

RE: Missing object when loading from Cassandra with multiple queries

2017-08-02 Thread Roger Fischer (CW)
Hello, it turns out that the problem (missing objects) was caused by cluster configuration issues. Both the Ignite and Cassandra cluster had one node (not the same VM) configured with a different time zone. Since date-time was part of the primary key and query parameters, there were

RE: Missing object when loading from Cassandra with multiple queries

2017-07-28 Thread Roger Fischer (CW)
Hi Val, looking at the debug logs, I see that the query is executed on only one Ignite server. It seems that this happens when the where clause includes a partitioning column. Query " select * from test_11.fc_port_stats; " loads all objects. Query " select * from test_11.fc_port_stats where sid

RE: Missing object when loading from Cassandra with multiple queries

2017-07-28 Thread Roger Fischer (CW)
Thanks, Val. I got confused between the Cassandra and Ignite keys. The configuration (keyPersistence) references the POJO and the Cassandra partition and clustering key. Thus I inadvertently also added the partitioning key elements to the POJO. However, this still does not explain my problem.

RE: Missing object when loading from Cassandra with multiple queries

2017-07-27 Thread Roger Fischer (CW)
Thanks, Val. The two fields are sufficient as the primary key. The "hour" field is the truncated dateTime, and "bucket" is a grouping of the PortIds. Roger -Original Message- From: vkulichenko [mailto:valentin.kuliche...@gmail.com] Sent: Thursday, July 27, 2017 2:56 PM To:

Question on efficient loading from Cassandra

2017-07-26 Thread Roger Fischer (CW)
Hello, what is the best way to efficiently load data from a backing store, like Cassandra. I am looking for a solution that minimizes work in Ignite and Cassandra. As I understand: The simplest way is to call loadCache() with a single select statement. cache.loadCache( null, "select * from

Missing object when loading from Cassandra with multiple queries

2017-07-25 Thread Roger Fischer (CW)
Hello, I am using Ignite with Cassandra, loading data from Cassandra on-demand using multiple query statements. But only a (seemingly random) subset of the rows/objects are loaded into ignite. When I load using a single query, all rows/objects are loaded correctly. In another environment,

RE: Custom SQL Functions - ClassNotFoundException

2017-07-21 Thread Roger Fischer (CW)
Thanks Val. The XML config has to include the package, but _not_ the ".class". Working example: com.abc.poc.icpoc.function.DateTimeFunctions Roger -Original

RE: Custom SQL Functions - ClassNotFoundException

2017-07-20 Thread Roger Fischer (CW)
ass with custom SQL functions via XML just use regular Spring bean notation ... Functions1.class Functions2 ... Kind regards, Alex. On Thu, Jul 20, 2017 at 6:31 AM, Roger Fischer (CW) [via Apache Ignite Users] <[hidden email]&

Custom SQL Functions

2017-07-19 Thread Roger Fischer (CW)
Hello, I have a couple of questions on custom SQL functions: 1) Can functions be overloaded, ie. could there be a to_string( Date) and a to_string( Double) function? 2) If yes, how is the applicable function identified? Based on parameters? Or does the function need to be qualified with the

Problem with Visor and Cassandra Cache Store

2017-07-18 Thread Roger Fischer (CW)
Hello, I am getting the following exceptions from the ignite server(s) when using the visor cache command: [10:03:36,710][SEVERE][mgmt-#415%null%][GridJobWorker] Failed to execute job due to unexpected runtime exception [jobId=c2fd5a65d51-7f82d6ed-0d7f-4cc9-a312-229ba87c8edd,

CodecNotFoundException: [smallint <-> java.lang.Integer] with Cassandra

2017-07-14 Thread Roger Fischer (CW)
Hello, there seems to be a problem with mapping between Cassandra numerical types to Java types. I am using ver. 2.0.0#20170430-sha1:d4eef3c6. In Cassandra, the column is defined as "smallint". In the Java POJO, the field is defined as "Short" In the configuration (QueryEntity), the field is

Provide persistence settings inline for Cassandra KeyValuePersistenceSettings?

2017-07-12 Thread Roger Fischer (CW)
Hello, all the examples for the Cassandra integration load the persistence settings from a resource. Here is one such example: Is it possible to define the persistence settings inline, ie. in the same file as the cache configuration? I tried the following:

Using Ignite as the SQL Engine for Cassandra

2017-07-11 Thread Roger Fischer (CW)
Hello, I have seen the page on using Cassandra as the persistent store for Ignite. Are the same concepts / classes applicable when using Cassandra as the backing database? I have a large data set in Cassandra. At system start I want to load the most recent (ie. most used) data into Ignite.

Large Key, eg UUID or String

2017-07-06 Thread Roger Fischer (CW)
Hello, are there any concerns about larg-ish keys, like a UUID? What if even larger, like a string of variable length? I am using Ignite SQL. Some objects have a single primary key, and in most cases it is a UUID. I am trying to decide if we should a) create an artificial key (long) and have

Unable to connect from Windows Java client to Linux server cluster (ignite 2.0.0)

2017-07-06 Thread Roger Fischer (CW)
Hello, I am unable to make a connection from a client on Windows to an Ignite server cluster running on 3 Linux machines. I have checked firewall settings on both. I am running a 3-node cluster of Ignite (2.0.0) on Centos 7. The nodes are connected to the same Ethernet switch. The cluster is