Re: deep learning over apache ignite

Hello, It's a good idea, machine learning in Apache Ignite is actively developing right now and new features are always appreciated. At the same time, let me point that deep learning is already supported in Apache Ignite via integration with TensorFlow as the most popular deep learning

Ignite client node connect to visor

It has been observed that: 1. No server nodes up, and started client node, it waits for server nodes to start working. 2. Serer nodes are up, and started client node, it connect to Server nodes and start working. 3. No server nodes up, but visor is up, and if we started client node, it connect to

Difference between Ignite Kafka Streamer and Kafka consumer (2.0)

We consume data from Kafka, process (look up on various cache & some business logic) consumed data and add it to the IgniteDataStreamer. Which one should be used to consume data from Kafka in Ignite application? And what is difference between Ignite Kafka Streamer and Kafka consumer (2.0)? --

Did anyone write custom Affinity function?

I want all nodes in a cluster to have equal number of cache partitions. With default Affinity function it is not happening, why default does not use all nodes in a cluster? Will it be okay to write custom Affinity function? And what will we lose doing so? Did anyone write custom Affinity function?

How to benchmark IgniteDataStreamer?

We can get the total number of gets/puts & average time for gets/puts on Caches. But in my case I do not use get/put on cache, I use IgniteDataStreamer to load data in cache. I need to know time taken by IgniteDataStreamer. Thanks -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Partition-exchanger blocked after upgrade to 2.7

Hello, After we upgrade to the 2.7 we meet a wired warn; basically all our ignite cache running in LOCAL model in a internal network. All the configuration are almost default. but we meet a ERROR logger of the tcp-disco-msg-worker* but after that the the cluster still working, no crash

Re: Failed to read data from remote connection

Now the cluster have 100+ nodes, when 'Start check connection process' happens, Some node will throw oom with Direct buffer memory (java nio). When check connections,Many nio socker will be create ,Then oom happens? How to fix the oom except larger xmx? Thanks. -- Sent from:

Ignite Idle CPU usage

Hello, I am currently trying to integrate Ignite 2.7 into an existing Web Application built on top of Apache Tomcat. Therefore I use it in an embedded mode. While everything works fine so far, I came across an issue regarding the idle CPU consumption. On my developing machine (Laptop, Inteil

unsubscribe

unsubscribe

Re: Sql join cause memory exhausted

Hello! Thanks for your reply! Unfortunately,the above query is a sub query of a big query.So I think I can't overcome it.How about integration with spark?Will spark deal with such queries in a smart way? ilya.kasnacheev wrote > Hello! > > The recommendation here is to use Lazy SQL Queries. >

Re: Short lived objects data region recommendation

Hello! 1. I guess you can do both. For long-lived processes the former is preferred to avoid surprises. 2. No. 3. Yes, data is always converted on-heap before given away to any client code. Regards, -- Ilya Kasnacheev пн, 17 дек. 2018 г. в 11:19, Isaeed Mohanna : > Hi > 1. when u say that I

Re: Dealing with changing class definitions in Ignite

Hello! As far as my understanding goes: You can't peer class load your Key/Value types. You also can't redeploy your Key/Value types. They even survive node restart via WORKDIR/marshaller directory, and come back to haunt you. There are plans to maybe ease some of those limitations in 3.0, but

Re: Copying My Service classes to the lib folder of already started nodes (not via visor)

Hello! You could be running Tasks from your service, and you can deploy Tasks via Deployment SPI, e.g. UriDeploymentSpi. https://apacheignite.readme.io/docs/deployment-spi Here it will check if new version is available, redeploy it and start using new JARs. Regards, -- Ilya Kasnacheev пн, 17

Re: How to run Ignite REST API scan query with predicate

Hello! I understand your concern but REST does not support such use case out of box. You could try implementing your own *ConnectorMessageInterceptor* to read additional params from request, put them to thread-local, get them from thread-local in constructor. But I think you will be served

Re: Optimizing Collocated Join

Hello! PUBLIC.EVENTTHETA_PARENTS2CELLID_ASC_S2CELLID_ASC_EVENTDATE_ASC_EVENTHOUR_ASC_IDX Looks like this index is not selective enough for efficient filtering by EVENTDATE. So you have to scan a lot of entries regardless of date range. You can set local=true on SQL query to only return results

Re: Sql join cause memory exhausted

Hello! The recommendation here is to use Lazy SQL Queries. https://apacheignite-sql.readme.io/docs/performance-and-debugging#section-result-set-lazy-loading Then, be careful to process data in small batches and avoid holding on to large subsets of result set. Regards, -- Ilya Kasnacheev пн,

Sql join cause memory exhausted

Hello. I do a simple sql join operation on a ignite node which is version 2.6 and has 22GB heap memory and 22 GB off-heap memory.I also enable persistence.The query exhausted ignite's heap memory quickly.How can I overcome this situation. Here is my query: explain SELECT t2.*,

Re: Dealing with changing class definitions in Ignite

The issue is still present in 2.7. I added a ticket on Jira with sample code that reproduces the issue. https://issues.apache.org/jira/browse/IGNITE-10717 For now I think we can work around the issue by overriding the default BinaryNameMapper, but this feels rather hacky to me. On Mon, Dec 17,

Copying My Service classes to the lib folder of already started nodes (not via visor)

Hi, I have a use case, where a node is already started and I want to copy some jars into that node's libs folder. Visor has the deploy command. But without using visor can we do it somehow? I know peer class loading happens in data and compute. But for services the documentation says no peer

Re: How to Write and Read a Nested JSON Into and From Apache Ignite Table

Hello Dennis, Thank you for the response. Yes, the Objects need to have same schema. With regards to Converting to BinaryObjects, please let me know whether Nested BinaryObject fields more than one level is supported. My understanding is NO. My requirement is, 1. Write Simple and Nested JSON

Ignite native persistence and replicated caches - should it even be possible?

Hi all, Could somebody confirm my conclusion below? It seems it is possible to declare a REPLICATED cache configuration for caches that are mapped to a data region backed by the native persistence layer. Ignite does not complain about this configuration and boots happily. Yet, after cluster

Re: How to Write and Read a Nested JSON Into and From Apache Ignite Table

Hareesh, Are the object supposed to have the same schema? If yes, then JSONs may be converted to BinaryObjects. If no, then you'll have to store them as blobs or text. If you want to access such data from SQL, your tables should have a flat structure. You may have nested binary objects, but in

Re: Python for Ignite for Spark?

I’m not sure there are any — and you’re right, there probably should. Having said that, integration is very straight-forward. You run pyspark (or spark-submit), passing in the Ignite jar files (using the —jars parameter). For example: $SPARK_HOME/bin/spark-submit --jars

Re: error: LNK2001: unresolved external symbol __imp_SHFileOperationW

Hello, Note that in Ignite 2.7 only OpenSSL 1.0 is supported. Now, for your issue. It's weird and I have never faced that one by myself, but according to MSDN [1] you need to link to shell32.dll. [1] - https://docs.microsoft.com/en-us/windows/desktop/api/shellapi/nf-shellapi-shfileoperationa

Re: Dealing with changing class definitions in Ignite

I investigated the issue further and narrowed the issue down to the Binary Marshaller not working as expected given the configured Deployment Mode. When forcing my clients to use unique class names in the metadata of the Binary Marshaller (I forced this by overriding the global BinaryNameMapper

Re: Short lived objects data region recommendation

Hi 1. when u say that I should have a constant amount allocated for heap do u mean initially allocated min:4GB, Max 4GB or having a settings of min:1GB , Max: 4GB is allright as well? 2. If i do not configure any on-heap caches does ignite automatically allocate on-heap caches frequent requests