What happens when a client gets disconnected

Sorry for the long delay in responding to this issue. I will work on replicating this issue in a more controlled test environment and try to grab thread dumps from there. In a previous post you mentioned that the blocking in this thread dump should only happen when a data node is affected which i

Re: What happens when a client gets disconnected

l delay all further operations until PME is finished. > > Avoid having short-lived clients. > > Regards, > -- > Ilya Kasnacheev > > > вт, 23 апр. 2019 г. в 03:40, Matt Nohelty : > >> I already posted this question to stack overflow here >> https://stackoverflow

What happens when a client gets disconnected

I already posted this question to stack overflow here https://stackoverflow.com/questions/55801760/what-happens-in-apache-ignite-when-a-client-gets-disconnected but this mailing list is probably more appropriate. We use Apache Ignite for caching and are seeing some unexpected behavior across all o

Re: HELLO WORLD GA EXAMPLE

I guess but why isnt it showing up in zeppelin? On Dec 7, 2018 2:52 AM, "Ilya Kasnacheev" wrote: Hello! I don't see any errors here, looks like this example has finished successfully. Regards, -- Ilya Kasnacheev пт, 7 дек. 2018 г. в 07:07, AlphaMufasaOmega : > Dear @Zaleslaw could you ple

Re: Delay queue or similar?

Ok will try that. Cheers! - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Delay queue or similar?

lable keys until the crawl is done. This may introduce a performance hit, if (for example) the only host that is ready for a request is the last one in a very large list of keys. Does that make sense? Apologies if I'm misunderstanding! - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Delay queue or similar?

put into the java work queue if they are not yet ready to be fetched, and I'm slightly worried about the potential number of hosts (in reference to the java Map data-structure). So my question is: is there something that Ignite can provide for making this all work? - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite File System (igfs) spillover to disk

size is much larger than the available RAM? Can igfs be configured to use the (relatively new) native "DataStorage" feature to spillover from RAM to disk, but then also have a specific cache use a custom store? Thanks - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Advice for updating large cache

ution. So I'm wondering if there's something in Ignite that could help? Or if anyone has dealt with this kind of problem before and can offer ideas for a better way? Thanks! - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Leftover Ignite threads after closing cache

Good to know, thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Leftover Ignite threads after closing cache

Hi, I'm using an Ignite ContinuousQuery for processing local cache events: continuousQuery.setLocalListener(new CacheEntryListener<>(myHandler)); I have a "job" that modifies the cache (insert/updates/deletes) and the code in the callback essentially takes items and enqueues them for further pro

Re: Igfs - questions and optimal configuration settings

Thanks Denis! That helps a lot. I'll dig into those settings and see if I get my head around it all. I did notice that the default/max memory settings are based off of system settings/resources, so I'll try the defaults too and see what happens. - Matt -- Sent from: http://apache-ig

Igfs - questions and optimal configuration settings

Hi, I've recently started using the Ignite FileSystem (igfs) in our API, to fully buffer an incoming stream of byte[] values (chunked InputStream). I'm doing this because that stream then needs to be sent along to another remote service, and I'd like the ability to retry without telling the sender

Re: How to stop explicitly write behind

Thanks Alexey, that is exactly what we are doing now actually. We're hoping to find a way to avoid this though, to ensure our nodes aren't doing any unnecessary work, as well as avoiding unexpected results if the backend system comes back up. -- Sent from: http://apache-ignite-users.70518.x6.nab

Re: How to explicitly stop write behind

OK so digging a little further, I found GridCacheWriteBehindStore#stop() - but still no idea how to get access to this. - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to stop explicitly write behind

hammer the non-existent backend and logging goes crazy. So is there a way to tell Ignite to stop calling the CacheStoreAdapter's write behind methods? - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to handle Cache adapter failures?

And I guess this would be for calls to invoke, but also for the asynchronous write-through calls. Thanks, - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: loadAll and removeAll from cache with custom store

Hi, The size in my store is about 9k. I do have loadAll() working now, so once that completes, the cache has 9k items as well. For deleteAll(), I had it work once where it called my adapter multiple times with all of the expected keys. But then restarting my app and trying again (after loading ne

Re: loadAll and removeAll from cache with custom store

For the removeAll() call, I do see that after I do a loadCache((k,v) -> true) I can delete items from the store, but my store's deleteAll() only ever gets 1k items once, deletes them from the store, and then is never called again to delete the rest of the items in the store. -- Sent from: http:/

loadAll and removeAll from cache with custom store

here, but not sure what. What's the best way for me to delete all items in the cache _and_ the backend store in the most efficient way? Thanks, - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to provide a dependency to a custom CacheStoreAdapter

I've created a custom CacheStoreAdapter which requires a dependency. I am now just instantiating directly in this class, but for each cache, this dependency is re-created, which is not what I want. Is there a way I can have this dependency instance, provided to the my adapter? Thanks -

Re: Custom CacheStoreAdapter implementation - fields are null

Yeah, that's definitely happening. 2 instances are being created. Is there a workaround for this that you know of? Thanks, Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Custom CacheStoreAdapter implementation - fields are null

e() method is called, the http client is present and is properly passed to the CacheStoreAdapter instance. But when my loadAll() method is called, it is null. - Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Custom CacheStoreAdapter implementation - fields are null

e is calling on it? Thanks, Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

CacheStoreAdapter#loadAll and ContinuousQuery

s expected, but now with my impl, it's not. The initialQuery cursor actually doesn't ever yield anything. Is there something I'm missing with making these two components work together properly? Anything special I need to do with the impl or config? Thanks, - Matt -- Sent from: h

Re: SQLQuery with simple Join return no results

OK thanks for that. So does that then mean that the key type (K) for my Cache needs to be AffinityKey ? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

SQLQuery with simple Join return no results

I have 2 caches defined, both with String keys, and classes that make use of the Ignite annotations for indexes and affinity. I've got 3 different nodes running, and the code I'm using to populate the cache w/test data works, and I can see each node is updated with its share of the data. My index t

affinityRun then invoke

re processEntry does:cache.invoke(key, (entry, args) -> { if( entry.exists() ){ modify(entry); } else { create(entry); } return true;});Is this generally a valid way to deal with atomically updating entrees within a partitioned cache?Thanks,- Matt -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to configure off heap and "native" distributed persistence

Thanks, got it working. This helped a lot too: https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/persistentstore -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-configure-off-heap-and-native-distributed-persiste

Re: How to configure off heap and "native" distributed persistence

e populated from what was persisted previously. But I'm struggling to figure out how to configure this. The examples in the documentation don't seem to show enough to actually get something working. Thanks, - Matt -- View this message in context: http://apache-ignite-users.70518.x6.n

Re: 1 processing node, n partition nodes

Ah that makes sense! I'll try that out. Thanks, - Matt -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/1-processing-node-n-partition-nodes-tp16485p16498.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How to configure off heap and "native" distributed persistence

Ok thank you, I was looking at CacheConfiguration and not IgniteConfiguration. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-configure-off-heap-and-native-distributed-persistence-tp16488p16497.html Sent from the Apache Ignite Users mailing list archive a

Re: How to configure off heap and "native" distributed persistence

OK thanks Evgenii. For the persistence, I'm trying the Java example here: https://apacheignite.readme.io/v2.1/docs/distributed-persistent-store but the setPersistentStoreConfiguration method does not exist on CacheConfiguration. I'm on Ignite 2.1.0 - what am I doing wrong? Than

How to configure off heap and "native" distributed persistence

additional dependency for that? I don't see it in ignite-core 2.1.0. Thanks, - Matt -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-configure-off-heap-and-native-distributed-persistence-tp16488.html Sent from the Apache Ignite Users mailing list archi

1 processing node, n partition nodes

, regardless of whether it's the node doing the processing or not? Thanks, - Matt -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/1-processing-node-n-partition-nodes-tp16485.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Reusing CacheStores connections?

application spend 10% of the time. Shouldn't Ignite be reusing the connections somehow? Any way to improve this? An example with better performance would be really helpful. Cheers, Matt

Re: Ignite starting by itself when using javax.cache.Caching

Hi, thanks for your help. I understand the problem now... exactly what you said it was. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-starting-by-itself-when-using-javax-cache-Caching-tp15997p16174.html Sent from the Apache Ignite Users mailing list arch

Ignite starting by itself when using javax.cache.Caching

. Why is it starting on its own like this? How can I prevent this from happening? Thanks, - Matt -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-starting-by-itself-when-using-javax-cache-Caching-tp15997.html Sent from the Apache Ignite Users mailing list

Re: Compute Grid Questions

Got it. Thank you. Matt On Tue, Jul 18, 2017 at 8:21 PM, vkulichenko wrote: > Matt, > > undeployTask method is about task deployment [1], it's unrelated to the > discussion. > > [1] https://apacheignite.readme.io/docs/deployment-spi > > -Val > > > >

Re: Compute Grid Questions

ed to work like this? https://gist.github.com/anonymous/3891dfedf26bfcf3314215004e67daa8 Thanks, Matt On Tue, Jul 18, 2017 at 8:50 AM, slava.koptilin wrote: > Hi Matt, > > It seems that your code is not quite correct. > In accordance with the spec Thread.sleep(long millis) clears th

Re: Compute Grid Questions

github.com/anonymous/0a8759e70eddab470f09dcb92644f3c7 Thanks, Matt On Mon, Jul 17, 2017 at 2:26 PM, vkulichenko wrote: > Hi Matt, > > 1. Each task or closure execution creates a session that has an ID. You can > cast returned IgniteFuture to ComputeTaskFuture (unfortunately there is no > other way now) and

Compute Grid Questions

I've seen there's a ignite.compute().withName(...) and ignite.compute().activeTaskFutures() method, but I've no idea how or when to use them, specially since the futures doesn't have a name but a uuid. Also, what exactly is a "task" in Ignite and how does it differs from a closure (ie, IgniteRunnable)? Cheers, Matt

Queue read/write Store

the data in the queue to be persisted to some persistence store, and then read back in later if needed (restart etc.). Thanks, - Matt -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Queue-read-write-Store-tp14682.html Sent from the Apache Ignite Users mailing

Re: ignitevisor starting the node twice

rticular is the case, but I've seen some weird things when visor is running. Matt On Mon, May 29, 2017 at 11:18 PM, Alexey Kuznetsov wrote: > You need to open configuration with proper discovery. > Usually that means - open same config you used to start your "real" nodes

Re: Correct Way to Store Data

Err, I meant "[...] a different memory policy for different classes", not for "different products". On Fri, May 26, 2017 at 6:00 PM, Matt wrote: > I don't think that's correct. > > As far as I know, on Ignite it's fine to put more than one type on

Re: Correct Way to Store Data

t;."Bar" respectively. That's why a model like #2 may make more sense than #1. However, I agree with you that #2 would make it impossible to specify a different memory policy for different products, but that is not required in this case anyway. Matt On Fri, May 26, 2017 at 4:34 P

Re: Correct Way to Store Data

h wrote: > Hi Matt, > > From what I've seen, the most commonly used approach is the one you took: > have caches associated with object classes. This approach is efficient and > completely corresponds to "the Ignite way". > > Having a separate cache for each pro

Correct Way to Store Data

Also, right now I'm storing about 10k products but that number increases as clients arrives, so I expect the cardinality to increase rapidly. Cheers, Matt

Re: CacheStore's Performance Drops Dramatically - Why?

node would flush out in 2 thread, and each > thread flushes out in batch of 512. > > Could someone confirms or clarify the understanding? Thank you! > > On Wed, May 3, 2017 at 12:16 AM, Matt wrote: > >> In fact, I don't see why you would need both batchSize and flushSize. If

Re: CacheStore's Performance Drops Dramatically - Why?

default values should be batchSize > flushSize as I think or not. On Wed, May 3, 2017 at 3:26 AM, Matt wrote: > I'm writing to confirm I managed to fix my problem by fine tuning the > config params for the write behind cache until the performance was fine. I > still see single e

Re: CacheStore's Performance Drops Dramatically - Why?

ng? On Tue, May 2, 2017 at 9:13 PM, Denis Magda wrote: > Matt, > > Cross-posting to the dev list. > > Yes, Ignite switches to the synchronous mode once the buffer is exhausted. > However, I do agree that it would be a right solution to flush multiple > entries rather than one

Re: CacheStore's Performance Drops Dramatically - Why?

> > On Tue, May 2, 2017 at 9:57 AM, Matt wrote: > >> I'm stuck with that. No matter what config I use (flush size, write >> threads, etc) this is the behavior I always get. It's as if Ignite internal >> buffer is full and it's trying to write and get ri

Re: CacheStore's Performance Drops Dramatically - Why?

ration to avoid this? On Tue, May 2, 2017 at 11:50 AM, Jessie Lin wrote: > Hello Matt, thank you for posting. I've noticed similar behavior. > > Would be curious to see the response from the engineering team. > > Best, > Jessie > > On Tue, May 2, 2017 at 1:03 AM, Matt wr

CacheStore's Performance Drops Dramatically - Why?

e read on the docs that using ATOMIC mode (default mode) is better for performance, but I'm not getting why. If I'm not wrong using TRANSACTIONAL mode would cause the CacheStore to reuse connections (not call openConnection(autocommit=true) on each writeAll()). Shouldn't it be bett

Re: SQL "= ANY (?)" supported?

then "*query.setArgs(params)"* (without wrapping it), but I'm looking for a one-param query, which is reusable. Matt On Sun, Apr 30, 2017 at 4:54 AM, Matt wrote: > Hello, > > I'm trying to run a SQL standard query with a "WHERE foo = ANY(?)" but it > se

SQL "= ANY (?)" supported?

t;. I also tried writing the query without the params, hardcoding the values directly on the sql variable, but I'm getting the same result. Is this kind of query supported? Any workaround to load a list of objects dynamically (from an array) without writing multiple "foo = ? OR foo = ? OR ..." on the query? Best, Matt

Ignite Stream Processing?

you would/usually do to process Ignite data as a stream? Any additional insight regarding stream processing on Ignite is very welcome! Best regards, Matt [1] https://ci.apache.org/projects/flink/flink-docs-release- 1.2/dev/windows.html#sliding-windows [2] https://ci.apache.org/projects/flink/fl

Re: getOrCreateCache hang

onListener data source, but I've so far been unable to pin it down beyond that. No doubt it's something simple I'm doing wrong, but whatever it is still eludes me. Matt On Tue, Feb 28, 2017 at 2:14 AM, Nikolai Tikhonov wrote: > I am not able reproduce exactly your test (do not ha

Re: getOrCreateCache hang

Something's not right. I'm seeing this as 100% reproducible on two different OSes. Are you running "mvn package" on both testIgnite1 and testIgnite2 and then starting the shaded JARs simultaneously from two different shell windows using "java-jar ignite?...-standalone

Re: getOrCreateCache hang

Using the test code I sent previously, I changed the ports to be a range but both clients still deadlock on getOrCreateCache. Are you able to reproduce this using the test code I sent? Matt On Mon, Feb 27, 2017 at 3:02 AM, Nikolai Tikhonov wrote: > Hi Matt! > > Try

Re: getOrCreateCache hang

reateCache, the other blocked in Ignition.start. As soon as I kill the process stuck in Iginition.start the other process continues successfully. I've attached the latest test code. Any ideas? Matt On Wed, Feb 22, 2017 at 10:30 AM, Matt Warner [via Apache Ignite Users] < ml-node+s70518

Re: getOrCreateCache hang

One other observation: with the third application acting as just the server, and just one of the clients running, there is no issue. Only when there are multiple clients do I get the deadlock on getOrCreateCache. Matt -- View this message in context: http://apache-ignite-users.70518.x6

Re: getOrCreateCache hang

you're not seeing this with the test code I'm sending when I'm seeing 100% reproducibility. I think I must be missing something very basic...? Matt On Wed, Feb 22, 2017 at 6:21 AM, Matt Warner wrote: > I will try this in about 2 hours and let you know. > > On Feb 22,

Re: getOrCreateCache hang

I will try this in about 2 hours and let you know. > On Feb 22, 2017, at 5:43 AM, Nikolai Tikhonov wrote: > > Could you try start to server from java code and look at results? > > Thanks, > Nikolay > >> On Wed, Feb 22, 2017 at 4:36 PM, Matt Warner >>

Re: getOrCreateCache hang

nto libs dir) contain needed classes? Might be you have a > build issue? > >> On Wed, Feb 22, 2017 at 4:43 PM, Nikolai Tikhonov >> wrote: >> Could you try start to server from java code and look at results? >> >> Thanks, >> Nikolay >> >>&g

Re: getOrCreateCache hang

I am using the default configuration for the server and starting it with ignite.sh. Yes, the Store classes are being deployed into the Ignite libs directory as well as part of the both testIgnite1 and testIgnite2. I feel like I'm missing something? Matt On Wed, Feb 22, 2017 at 4:50 AM, Ni

Re: getOrCreateCache hang

There were some coding errors in the previous attachment. The previous code still illustrates the deadlock, but the attached correctly stores data in tables. I've also included the test input file, for completeness. testCode.gz

Re: getOrCreateCache hang

There is only one Ignite server in my testing and two clients. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/getOrCreateCache-hang-tp10737p10769.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: getOrCreateCache hang

Attached is a file containing the outputs (stack trace, Ignite log) and two Maven test files. test+output.gz <http://apache-ignite-users.70518.x6.nabble.com/file/n10768/test%2Boutput.gz> I'm hoping you can tell me I'm just doing something silly to provoke this... Thanks! Matt

getOrCreateCache hang

I'm experiencing Ignite client hangs when calling getOrCreateCache when both are starting simultaneously. The stack trace shows the clients are hung in the getOrCreateCache method, which is why I'm focusing here. This seems like a deadlock when both clients are trying to simultaneously call getOrC

Re: Data Streamer error

Having similar problems still. I've tried at least 3 different serialization methods for the addData message (latest is a POJO (Serializable) w/3 String fields). Here's the latest error message: http://pastebin.com/b2awykDy -- View this message in context: http://apache-ignite-users.70518.x6.na

Re: Data Streamer error

OK I'll give that a shot. Thanks Val! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Streamer-error-tp7725p7994.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Streamer error

I've been at this for a while but having no luck. Some of this might be the way that we have dependencies setup or something else in our app that's preventing ignite streaming to work fully. But generally, what I'm seeing is that I can send messages into Ignite, and the receiver can take them out.

Re: Data Streamer error

Thanks Alexey, good point. I'm checking out a few things now related to that thought. Will post back any findings here. M -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Streamer-error-tp7725p7739.html Sent from the Apache Ignite Users mailing list archive

Re: Data Streamer error

Looks like it's a problem with the object I'm sending in. I swapped out the object (custom POJO) with an Integer, and all works fine. Is there a way I can find out what is it about this object that's causing the marshaling error? Any sample code around that shows how to run the marshaler stand-alon

Data Streamer error

Hi, I have Ignite (version 1.7.0) setup to do streaming, and a autoFlushFrequency of 15 (for now) - in the logs, I'm getting this during processing - any ideas on what this is coming from? 2016-09-13T17:31:52,580 - ERROR [grid-data-loader-flusher-#55%null%:Slf4jLogger@112] - Runtime error caught

Re: Performance Issue - Threads blocking

(Inline) On Fri, Apr 22, 2016, 4:26 PM vkulichenko wrote: > > Hi Matt, > > I'm confused. The locking does happen on per-entry level, otherwise it's > impossible to guarantee data consistency. Two concurrent updates or reads > for the same key will wait for each

Re: Performance Issue - Threads blocking

It's locking to make sure it's deserialized from swap once and expired once (if necessary; looks like it was in this particular case). matt On Fri, Apr 22, 2016 at 8:07 AM, Vladimir Ozerov wrote: > Hi, > > Could you please explain why do you think that the thread is bl

Re: Ignite "bugs" ?

you. > > > > I have only one Ignite instance started in this case. This is really > strange. > > > > *From:* Matt Hoffman [mailto:m...@mhoffman.org] > *Sent:* mercredi 13 janvier 2016 15:31 > *To:* user@ignite.apache.org > *Subject:* Re: Ignite "bugs" ? &

Re: Ignite "bugs" ?

Using System.properties() to store non-strings is really poor behavior. It violates the contract of the System.setProperty and System.getProperty, as well as the contract of the Properties object itself, from right at the top of the javadoc: "Each key and its corresponding value in the property li

Changing node attributes at runtime

a better way to handle this kind of thing? Thanks, Matt