Re: CacheStore's Performance Drops Dramatically - Why?

I'm writing to confirm I managed to fix my problem by fine tuning the config params for the write behind cache until the performance was fine. I still see single element inserts from time to time, but just a few of them every now and then not like before. You should definitely avoid synchronous sin

Re: Where can I find example-ignite.xml for ignite sink connector?

seen it here also https://apacheignite-mix.readme.io/v1.9/docs/kafka-streamer -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Where-can-I-find-example-ignite-xml-for-ignite-sink-connector-tp12370p12371.html Sent from the Apache Ignite Users mailing list archive a

Where can I find example-ignite.xml for ignite sink connector?

Hi, I am trying to fetch data from my Kafka topic, and insert it to Ignite cache. I followed tutorial here https://github.com/apache/ignite/tree/master/modules/kafka . At the end of instructions, for ignite xml reference, it is written that "For a simple cache configuration file example, see examp

Re: NoSuchMethodError: org.apache.curator.framework.api.CreateBuilder.creatingParentContainersIfNeeded()

Hi, First of all, I would highly recommend to use spark that built with scala 2.11 instead of 2.10, because you use ignite that built with scala 2.11. Please double check that you moved all ZooKeeper module related libs from “{apache_build}/libs/optional/ignite-zookeper” into “{apache_build}/libs

Re: Write-behind Flush Size

It turns out that the flush process is executed when there are "n+1" rows that change. If I set writeBehindFlushSize = 2, then the writing process will be executed on the third change in the table because "with the write-behind approach, only the last update to an entry will be written to the under

Re: Volatile Kubernetes Node Discovery

> Inside my service I'm using a IgniteCache in /Replicated/ mode from Ignite > 1.9. > Some replicas of this service run inside Kubernetes in form of Pods (1 > Container/Pod). > I'm using the > /org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder/ > for the Node D

Re: CacheStore's Performance Drops Dramatically - Why?

Matt, Cross-posting to the dev list. Yes, Ignite switches to the synchronous mode once the buffer is exhausted. However, I do agree that it would be a right solution to flush multiple entries rather than one in the synchronous mode. *Igniters*, I was sure we had a ticket for that optimization

Re: Concurrent job execution and FifoQueueCollisionSpi.parallelJobsNumber=1

Please let me know if I can help in any way. On 4/13/2017 2:18 AM, vkulichenko wrote: Hi Ryan, I will take a look at your sample in the next few days. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Concurrent-job-execution-and-FifoQueueCollisionSpi-par

NoSuchMethodError: org.apache.curator.framework.api.CreateBuilder.creatingParentContainersIfNeeded()

Hello, I got this error but I couldn't solve it. === java.lang.NoSuchMethodError: org.apache.curator.framework.api.CreateBuilder.creatingParentContainersIfNeeded()Lorg/apache/curator/framework/api/ProtectACLCreateModePathAndBytesable; at org.apache.curator.x.discovery.d

Ignite TLS with TrustStore

Hi, I referred to the Ignite docs on TLS: https://apacheignite.readme.io/docs/ssltls. I have the below set in my config XML file for TLS: and the bel

IGFS - Python and R clients

My application needs to read / write data to a distributed memory FS. The application is not Map/Reduce or Spark job. Let's say I need to run a docker orchestration pipeline, where each step is a docker container, and I want to save the intermediate results of each step and pass it to the next ste

Grid is in invalid state to perform this operation

Hi, My application was working and later laptop went into sleep/standBy mode. After awake got below exception. Looks like grid is stopped. Is there a way to handle this? /java.lang.IllegalStateException: Grid is in invalid state to perform this operation. It either not started yet or has already

Serializer

Hi We have multiple off-heap caches defined. We have implemented Binarylizable for each cache object. So far so good. We have a requirement we have to store List or Map of Binarylizable objects in a cache. How do I attempt to write serializer for List and Map? These List and Map are extended o

Re: Volatile Kubernetes Node Discovery

Providing the previously promised log-file. ignite_dmp.txt - Please excuse the DP -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Volatile-Kubernetes-Node-Discovery-tp12357p12358.html S

Volatile Kubernetes Node Discovery

Dear Apache Ignite Users Community, This may be a well-known problem, although the currently available information does not provide enough help for solving this issue. Inside my service I'm using a IgniteCache in /Replicated/ mode from Ignite 1.9. Some replicas of this service run inside Kubernet

Re: Ignite .NET vs Java

You have to implement IComputeFunc. Ignite.NET does not have overloads for anonymous functions & lambdas. We have plans to add them in future versions. On Tue, May 2, 2017 at 9:46 PM, Humphrey wrote: > I'm trying to do the following in .NET with C# using lambda expression and > I > can't figure

Re: HDP, Hive + Ignite

1. Please make sure IGFS is really used: e.g. you may explicitly locate some table data on IGFS, and run the queries upon. IGFS statistics can partially be observed through Visor. Also please note, that upon node(s) start IGFS is empty. In case of dual modes it caches the data upon file reading. I

Ignite .NET vs Java

I'm trying to do the following in .NET with C# using lambda expression and I can't figure out how to do the same thing I can do with Java. In Java I can do the following: List listOfWords = Arrays.asList("How many characters are there?".split(" ")); Collection res = ignite.compute().apply((St

Re: Combined off heap size?

Hi, I have a physical 8x32 server and I am using 4 data nodes in one cluster group(G1) I have the below config: So basically I am assigning 40GB of off-heap memory. I know that this is just assigned, nodes will use that memory as and when data comes in. Based on this scenario - 1.

Re: Failed to send datagram message to remote node

Hi, Seems that you did correct rules. Did you try disable firewall, is it works? Can you check the ports by iperf? On Mon, May 1, 2017 at 1:48 PM, Nikolai Tikhonov wrote: > Hi, > > Are you sure that you allowed to send UDP message through this port? > > > > -- > View this message in context: ht

Re: CacheStore's Performance Drops Dramatically - Why?

No, only with inserts, I haven't tried removing at this rate yet but it may have the same problem. I'm debugging Ignite internal code and I may be onto something. The thing is Ignite has a cacheMaxSize (aka, WriteBehindFlushSize) and cacheCriticalSize (which by default is cacheMaxSize*1.5). When t

Re: CacheStore's Performance Drops Dramatically - Why?

I noticed that behavior when any cache.remove operation is involved. I keep putting stuff in cache seems to be working properly. Do you use remove operation? On Tue, May 2, 2017 at 9:57 AM, Matt wrote: > I'm stuck with that. No matter what config I use (flush size, write > threads, etc) this is

Re: CacheStore's Performance Drops Dramatically - Why?

I'm stuck with that. No matter what config I use (flush size, write threads, etc) this is the behavior I always get. It's as if Ignite internal buffer is full and it's trying to write and get rid of the oldest (one) element only. Any idea people? What is your CacheStore configuration to avoid this

Re: "object not serializable " when initializing igniteContext with Zookeeper Configuration

Hi, Please properly subscribe to the mailing list so that the community can receive email notifications for your messages. To subscribe, send empty email to user-subscr...@ignite.apache.org and follow simple instructions in the reply. Seems, you provide anonymous class to IgniteContext that has i

Re: Distributed Closures VS Executor Service

Dear all, I have created a new Topic: "Ignite SQL Indexing Performance problems" since the conversation has deviate a lot form the original subject. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-Closures-VS-Executor-Service-tp11192p12347.html Sent

Re: CacheStore's Performance Drops Dramatically - Why?

Hello Matt, thank you for posting. I've noticed similar behavior. Would be curious to see the response from the engineering team. Best, Jessie On Tue, May 2, 2017 at 1:03 AM, Matt wrote: > Hi all, > > I have two questions for you! > > *QUESTION 1* > > I'm following the example in [1] (a mix be

Re: tcp-disco-ip-finder-cleaner

Hi, Looks like a bug. Here is a ticket [1]. [1] https://issues.apache.org/jira/browse/IGNITE-5117 On Thu, Apr 27, 2017 at 4:01 PM, gatorjh wrote: > Here is the ignite config > > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/tcp-disco-ip-finder-clea

Re: HDP, Hive + Ignite

Hi Ivan, I have some progress) *1. TEZ on Ignite (with IGFS, without Ignite MR)* I could run Hive queries on TEZ and Ignite with next settings: $IGNITE_HOME/bin/ignite.sh -v -J"-Xms10g -Xmx10g -XX:MaxMetaspaceSize=4g" (every server has RAM 16Gb ) beeline --hiveconf fs.default.name=igfs://dev-dn1

Nodes hanging when accessing queue

Hi, I've been having a couple of issues with a shared IgniteQueue which seem to occur sporadically. I have a single publisher process which pushes data onto a queue, and several worker processes that pull the data from the queue and perform some processing. However these nodes are part of a larger

Ignite SQL Indexing Performance problems

Hello, I have a cache in REPLICATED mode populated with the contents of a single DB table having ~2 million rows. I am running a simple SQL query like: select * from simulation_initial_values where category_id in (1,2,3,4,5,6) and geo_channel_id in (3,4,5,6) and type_id=3

Re: With same load Ignite is not able to respond after enabling SSL

Hi, I see that code from stack trace you provided was refactored at 1.9, possible this will fix your issue. On Fri, Apr 21, 2017 at 3:00 PM, Ankit Singhai wrote: > igniteClient.gz > n12149/igniteClient.gz> > server1.gz >

Continuous Query remote listener misses some events or respond really late

Hi, I am currently facing intermittent issue with continuous query. Cant really reproduce it but if any one faced this issue, please do let me know My application is deployed on 12 nodes with 5-6 services are used to detect respective events using continuous query. Lets say I have a cache of type

CacheStore's Performance Drops Dramatically - Why?

Hi all, I have two questions for you! *QUESTION 1* I'm following the example in [1] (a mix between "jdbc transactional" and "jdbc bulk operations") and I've enabled write behind, however after the first 10k-20k insertions the performance drops *dramatically*. Based on prints I've added to the C

Write-behind Flush Size

Hello, Currently, I'm trying to do a write-behind caching depending on the flush size (I set the writeBehindFlushSize to a number, and set the writeBehindFlushFrequency to 0). I read the documentation [1], and it said that the default writeBehindFlushSize is 10240. What is this number actually? Is

Re: Distributed Closures VS Executor Service

Hello, I don't know about H2, I sent you link from apacheignite.readme.io which describes that IN clause does not support Indexes when run on Ignite. Also at my previous post I have sent a simply query using IN clause that has very poor performance on Ignite even run only on local mode with less t

Re: Distributed Closures VS Executor Service

Actually H2 can use index for IN operator, but using JOIN instead of IN may be still useful. Sergi 2017-05-01 12:40 GMT+03:00 Nikolai Tikhonov : > Hi, > > The page [1] contains also workaround for IN operation. You can use JOIN > instead of IN. > > Sergi, > > Will this tip actual for 2.0 version