External data sources

What's the best way to implement SQL that joins an Ignite cache with an external data source, e.g. a Cassandra column family or a MySql table? A very big external table, not one that can be cached in memory prior to running the query. -- View this message in context:

Presenting Apache Ignite in Miami, May 2017

Igniters, I’m presenting two talks related to Apache Ignite in May at ASF Apache Con and Apache BigData conferences. Latest news: https://ignite.apache.org/news.html Direct links: https://apachecon2017.sched.com/event/9zot?iframe=no

Re: Disable WriteBehind

Thank you for the reply. Yes, I disabled both write through and write behind. We're trying evaluate the application's performance on ignite and by taking the persistent store out of equation, we thought the performance shall improve, but on the contrary we saw performance dropped over 30%. What

Re: Stoppin listen for topics in Ignite

Thank you very much.You helped me a lot. p.s. Ignite is very powerfull. :-) -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Stoppin-listen-for-topics-in-Ignite-tp11604p11762.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Client near cache with Apache Flink

I think this kind of what i'm doing, in a abstract way as simple as possible IgniteTest.java Here I get this when executing IgniteTest: gets: 3462000 - seconds past: 15 - rate: 230800 gets/s If i run the same logic

Re: Error in executing hadoop job using ignite

Hi, Please properly subscribe to the mailing list so that the community can receive email notifications for your messages. To subscribe, send empty email to user-subscr...@ignite.apache.org and follow simple instructions in the reply. It looks like you have outdated "objectweb-asm" jar library

Re: Distributed Closures VS Executor Service

Hello Christo, Thank you for your prompt response. I will check your idea and get back to you and the community. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-Closures-VS-Executor-Service-tp11192p11759.html Sent from the Apache Ignite Users

Re: Distributed Closures VS Executor Service

If I understand correctly, the SQL query is executed within every task on each of the nodes and it is not set to be a local query. Correct? If so then what you are really doing is executing the SQL queries from all the nodes to all the nodes. This is bound to be inefficient. In essence what you

Re: Disable WriteBehind

Seems I misunderstood your question. Did you disable both writeThrough and writeBehind properties and got worse perfomance? On Wed, Apr 5, 2017 at 4:39 PM, Nikolai Tikhonov wrote: > Hi, > > Yes, it's expected behavior. If you disable writeBehind then each update > will be

Re: NameNode sync using DUAL_ASYNC mode on IGFS

Hi, You may use it in cases when you use temporary files that should be removed after node stop, or any other data that don't require persistence. -Dmitry. -- View this message in context:

Re: ScanQuery With BinaryObject

Hi David, Thanks for reproducer. I've create a ticket [1] for this bug. [1] https://issues.apache.org/jira/browse/IGNITE-4918 On Wed, Apr 5, 2017 at 5:03 PM, Andrey Mashenkov wrote: > I've creted a ticket [1]. > > [1]

Re: Client near cache with Apache Flink

So, when flink as to process the first record it creates a latch telling the other task managers(jvm) to wait until the cache is loaded. // Ignite ignite = IgniteClient.ignite();

Re: Asp.net client reconnect problem

log.txt Hi Pavel, The log file (after ClientReconnected is fired while server is still down) is attached. thanks -- View this message in context:

Re: Disable WriteBehind

Hi, Yes, it's expected behavior. If you disable writeBehind then each update will be propagated in underlying a store immediately and it's impact to performance. You can find more details in docs [1]. 1. https://apacheignite.readme.io/docs/persistent-store#section-write-behind-caching On Wed,

Re: Asp.net client reconnect problem

And yes, I am sure there's no other server node. (Topology shows 1 server ven I restart the server) -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Asp-net-client-reconnect-problem-tp11735p11751.html Sent from the Apache Ignite Users mailing list archive at

Re: Asp.net client reconnect problem

Hi, Pavel The steps are below: 1- I am starting server node. 2- Then starting asp.net client node. 3- Querying database. Works perfect. 4- Stopping server node. Ignite fires ClientDisconnected as one expected. 5- A few seconds later, while server is still down, Ignite fires ClientReconnected

Re: Apache ignite persistant store implementation

NearCache is designed to store small portion of frequently used data [1] of non-local partitions. [1] https://apacheignite.readme.io/docs/near-caches On Wed, Apr 5, 2017 at 2:19 PM, ALEKSEY KUZNETSOV wrote: > What meaning did u put into "is frontend to" ? > > вт, 4

Re: Non-collocated distributed SQL Joins across caches over separate cluster groups

Thanks Sergi, I understand that non-collocated distributed joins is a last resort very well. I still don't understand why cluster groups would make this any worse since in distributed non-collocated joins the data is NOT expected to be on the same node. Sounds to me that the cross-node calls

Re: Stoppin listen for topics in Ignite

This doesn't make a lot of sense, because "during stop" is not defined. From your code standpoint, stopListen() is an atomic "immediate" operation, and any message can be received before of after this operation. You should call stopListen() only if you don't intend to receive messages anymore.

Re: data consistency in atomic mode

Yes, you're correct, in this case atomic cache could be inconsistent and transactional cache will provide consistency guarantees. -Dmitry. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/data-consistency-in-atomic-mode-tp11719p11746.html Sent from the Apache

Re: Apache ignite persistant store implementation

What meaning did u put into "is frontend to" ? вт, 4 апр. 2017 г. в 13:52, Andrey Mashenkov : > Client can have only LOCAL cache and NearCache that is frontend to > partitioned, see [1]. Moreover, client node requires live server node in > topology. > > However, you

Re: Asp.net client reconnect problem

> Grid is in invalid state to perform this operation When does this happen? Can you provide more details or a project to reproduce? Are you sure you don't call Ignition.Stop somewhere? On Wed, Apr 5, 2017 at 2:08 PM, ozgurnevres wrote: > While we send a query to our

Re: Asp.net client reconnect problem

While we send a query to our cluster, We get the following error message from asp.net client node "Grid is in invalid state to perform this operation. It either not started yet or has already being or have stopped [gridName=myGrid1, state=STOPPED]" Then we restart every thing on client node(app

Re: Distributed Closures VS Executor Service

Hi Christo, I have tried the following: - When app starts, I stream into PARTITIONED CACHE one (1) table needed for the distributed jobs - Now when a Task is submitted, each Job takes over some CATEGORYIDs. Performs an Ignite cache-query (I paste it below), gathers the results, runs calculations

Re: Non-collocated distributed SQL Joins across caches over separate cluster groups

Hi, Moreover distributed joins can be executed only between caches with the same affinity (same partitions on the same nodes). Keep in mind that distributed join is already a "last resort" thing and you have to prefer collocated joins as much as possible, if you want to achieve good performance.

Re: Distributed Closures VS Executor Service

Hi Kyriako, Agree with Nikolai on both points. Regarding point number 2, you want to use affinity data / data colocation and data / compute colocation. You basically need to ensure that when you are inserting data into the cache you use the same affinity for all entries that need to be

Re: Distributed Closures VS Executor Service

Hello Nikolai, all About 1: yes, this was my perception as well, thanks for the confirmation About 2: Even if all the nodes provide result sets of local execution to the query initiator, if we are talking about results being Lists containing a couple of thousands POJOs, then wouldn't be a big

Re: Stoppin listen for topics in Ignite

I mean all concurrently sending messages . During stop I want to know wheather there are messages being done in process so let them be done.And i will execute by hand the ones that just came. -- View this message in context:

Re: Asp.net client reconnect problem

> a few seconds later fires "ClientReconnected" event This is suspicious. Are you sure no other nodes are running on your machine or in the local network? Can you attach the log? I have tested this exact scenario today (ASP.NET runs Ignite in client mode, server is a standalone executable) and it

Re: Runtime error caught during grid runnable execution: GridWorker

Different application instances are usually deployed with different class loaders, so that they are completely independent from each other, including any static content. For example, if there are two instances, they will not know about each other's Ignite instances. Each of them should start

Asp.net client reconnect problem

Hi, I am running Ignite on asp.net as a client node. I have also a server node as console application. M problem is: 1- When I stop the server node when asp.net client node is connected, it fires first "ClientDisconnected" event. Then, a few seconds later fires "ClientReconnected" event. Why? The

Non-collocated distributed SQL Joins across caches over separate cluster groups

Igniters, Is it correct to assume the following: We have an Ignite cluster comprised of 2 cluster groups A & B that have different caches deployed. We use an Ignite client to obtain API access to the whole cluster and execute a join query that joins data across the 2 caches My understanding

Re: Apache Ignite WriteBehind Guarantee during a Graceful Shutdown

Hi Jaipal, All data should be flushed to back store automatically on graceful shutdown. On Wed, Apr 5, 2017 at 10:44 AM, Jaipal Reddy wrote: > Hii, > > We are using Apache Ignite for caching and using its writeBehind feature > for pushing updated data asynchronoulsy

Apache Ignite WriteBehind Guarantee during a Graceful Shutdown

Hii, We are using Apache Ignite for caching and using its writeBehind feature for pushing updated data asynchronoulsy to Oracle DB. The following are relevant properties from our ignite cache configuration. ** With these properties, what I understand is IgniteCacheStore is invoked either