Questions regarding Ignite as hdfs cache

Hello, I am evaluating Ignite to be able to use it as a hdfs cache to speedup my hive queries. I am using hive with tez. Below are my cluster and Ignite configurations, *Cluster: * 4 data nodes with 32gb RAM each, 1 edge node 4 ignite servers, one for each data node. Ignite servers were started w

Re: Node failed to startup due to deadlock

Hi Alexey, Actually i have shared threaddumps from both nodes earlier. I have re-created the scenario, and attached the log and threaddumps from the Node1. Note: Multiple Thread dumps are taken in different times for Node 1 when Node2 was down and started again. Regards, Naresh Node1_AfterNod

Ignite 2.3 still not support group_concat

The new ignite 2.3 release doc said there are aggregation group_concat and related usage https://apacheignite-sql.readme.io/docs/group_concat, but look into the source code, this function still not support /** {@inheritDoc} */ @Override public String getSQL() { String text; swi

Re: map types in Cassandra

Considering that this feature is not supported in Cassandra, I would suggest replacing the latter with Ignite in general. Just enable Ignite persistence [1] and you will get all the features Cassandra offers plus those supported by Ignite only - ACID transactions, SQL with joins, full-fledged

[ANNOUNCE] Apache Ignite 2.3.0 Released

The Apache Ignite Community is pleased to announce the release of Apache Ignite 2.3.0. Apache Ignite [1] is the in-memory computing platform that is durable, strongly consistent and highly available with powerful SQL, key-value and processing APIs. This release adds more SQL capabilities and I

Re: Code deployment throught UriDeploumentSpi

Hi. I have found another work around. I change thread context classloader to parent classloader before proxy method invoke and swich it back after. So the code look like this: public class ServiceHelper { public static T serviceMethodInvoke(Object service, final String methodName, final Ob

Re: Code deployment throught UriDeploumentSpi

Hi. I have found another work around. I change thread context classloader to parent classloader before proxy method invoke and swich it back after. So the code look like this: public class ServiceHelper { public static T serviceMethodInvoke(Object service, final String methodName, final Ob

implement shared RDD in spark streaming in scala

I am very new to Ignite and went through the documentation and samples on the web. Below is the use case I am hoping to resolve using shared spark RDDs. - Spark Streaming App-1 writes incremental records, which does not already exist (by using a hash value generated inside the App) to a shar

Re: Is WAL a memory-mapped file?

Excellent! Thanks! On Wed, Nov 1, 2017 at 12:19 PM, Alexey Kukushkin wrote: > John, > > The default mode is the slowest of the 3 WAL write modes available. The > other two are OS buffered write "LOG_ONLY" and Ignite buffered write > "BACKGROUND". > > You might need some benchmarking in your real

Re: Is WAL a memory-mapped file?

John, The default mode is the slowest of the 3 WAL write modes available. The other two are OS buffered write "LOG_ONLY" and Ignite buffered write "BACKGROUND". You might need some benchmarking in your real environment (hardware and software) with your specific task to understand if it is "slow".

Unable to query Ignite 2.3 table using 2.2 JDBC thin client

Hey, I run *Ignite 2.3* locally for testing...just one node, persistence is enabled. Using DBeaver - using *Ignite 2.2 *JDBC thin client, 1. Create two simple tables, City and Person, from the Getting started example. 2. Insert rows to those tables When I execute 'select * from Person', I got "

Re: Node failed to startup due to deadlock

Hi Alexey, We do have event listeners to handle data loss events. We do 3 things related to ignite (amongst other unrelated things) in the listener (btw, an instance of local listener if running on each node, these listeners have remote filters as well): 1. Read an Ignite cache 2. Publish a message

Re: Is WAL a memory-mapped file?

With the WALMode = DEFAULT, which does a full-sync disk write, individual put operations on a cache are written to disk then. That would make it slow, right? Thanks. On Wed, Nov 1, 2017 at 10:58 AM, Dmitry Pavlov wrote: > Hi John, > > No, WAL consists from several constant sized, append only fi

Re: Is WAL a memory-mapped file?

Hi John, No, WAL consists from several constant sized, append only files (segments). Segments are rotated and deleted after (WAL_History_size) checkpoints. WAL is common for all caches. If you are interested in low-level details of implementation, you can see it here in draft wiki page for Ignite

Is WAL a memory-mapped file?

Hi, Is the WAL a memory mapped file? Is it defined per cache? Thanks.

Where is release notes for 2.3

Hi! URL from downloads page is bad ( https://ignite.apache.org/releases/2.3.0/release_notes.html). Where else can I see it?

Re: Ignite/Cassandra failing to use supplied value for where clause

Got it. Seems, CassandraStore doesn't supports storeKeepBinary(true) flag. On Wed, Nov 1, 2017 at 8:04 PM, Andrey Mashenkov wrote: > Hi Kenan, > > Looks like annotation configuration is not allowed together with xml > comfiguration. > I wonder there was no Exception. I've made an example with a

Re: Ignite/Cassandra failing to use supplied value for where clause

Hi Kenan, Looks like annotation configuration is not allowed together with xml comfiguration. I wonder there was no Exception. I've made an example with all mapping configured in xml and got no errors. Please, check an example that workable for me [1]. [1] https://github.com/AMashenkov/ignite-w

Re: Using two ignite contexts with spark streaming

Hi! I don't really see what you are trying to achieve. In the example, that you provided, the second IgniteContext, called ignitec2, is not used. Do you mean that when you start the second IgniteContext, then both of them stop working? When ignitec.fromCache("Data2") is executed, then cache is ge

Re: Failed to create string representation of binary object.

Ankit, That looks very strange. Your class does not have registrationInfoResponse field which is mentioned in the error. Please confirm that node with id="b2df236f-4fba-4794-b0e4-4e040581ba9d" is a part of your load testing cluster. Do you have peerClassLoadingEnabled=true at your configs? Than

Re: What's the maximum number of nodes for an Apache Ignite cluster?

thanks Alexey On Wed, Nov 1, 2017 at 4:32 AM, Alexey Kukushkin wrote: > Hi, > > As I know there is no limit on the number of nodes. The largest cluster > one of Ignite users is building that I know about will have 2000 nodes - > may be that was what caused confusion. > > Please note that the def

Re: Code deployment throught UriDeploumentSpi

Hello again! I have found one work-around, it is not very sound but seems reliable. Before first invocation of UriDeploymentSpi-deployed Compute Job with, first run a locally-deployed job with same dependency. In this case, ServiceTopologyCallable is called outside of UriDeploymentSpi context, is

Re: Ignite Nodes not connecting to cluster in docker swam mode

Hi Rishi, Check if port 31100 is reachable. Also it is possible you have outdated linux kernel with a bug that return error instead of dropping a packet [1] e.g. network buffer is full. Can you try to start a grid on newer linux\kernel version? [1] https://github.com/playframework/play-plugins/is

Ignite Nodes not connecting to cluster in docker swam mode

Hi, When I connect ignite multiple containers in --net=host mode then they get connected. But when I try to connect Ignite containers in swarm(cluster) mode, they are unable to connect. I am getting following error messege[in verbose(-v) mode]: *[10:46:41,947][SEVERE][grid-time-coordinator-#1

Ignite Nodes not connecting to cluster in docker swam mode

Hi, When I connect ignite multiple containers in --net=host mode then they get connected. But when I try to connect Ignite containers in swarm(cluster) mode, they are unable to connect. I am getting following error messege: *[10:46:41,947][SEVERE][grid-time-coordinator-#102%null%][GridClockSy

Re: What's the maximum number of nodes for an Apache Ignite cluster?

Hi, As I know there is no limit on the number of nodes. The largest cluster one of Ignite users is building that I know about will have 2000 nodes - may be that was what caused confusion. Please note that the default affinity function (RendezvousAffinityFunction) manages 1024 partitions by defaul

Re: Benchmark results questions

Hi Dmitry, It's been a while now, did you check out what happens? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Out of memory streaming to Ignite Apache 2.0

You may be hitting this scenario from my experience: As you have three nodes you begin to get deadlocks between load tasks. These deadlocks cause tasks to be postponed, but real troubles happen when they survive past 30 seconds and are dumped to logs. There's massive amount of data in your tasks

Re: Node failed to startup due to deadlock

Naresh, Rajeev I've looked through the logs. Node2 is stuck because it is waiting for Node1 (id=96eea214-a2e3-4e0a-b8c6-7fdf3f29727c) partition map exchange response. So it looks like the real deadlock is at Node1 (id=96eea214-a2e3-4e0a-b8c6-7fdf3f29727c) Can you also send node1 logs and thread d

Re: when client node connect to server node, server node throws NotSerializableException

Hi Jeff, CacheStore class should be in classpath on all nodes including clients. 1 нояб. 2017 г. 4:52 пользователь "Jeff Jiao" написал: > Hi Andrew, > > Thanks a lot for all the replies. > > yes BoConverter implements Serializable, or Ignite will throw > NotSerializableException. > The "Class"