Re: Using native persistence to "extend" memory

2020-06-16 Thread steve.hostettler
Thank you both for clarifying this. I usually also have large objects 3KB so I thought about increasing the page size to 32KB to reduce the number of pages and thus reduce the speed at which we get to the 2/3 of dirty pages. Good idea? On top of that during the process at some I generate a

Re: Ignite persistence and activation

2020-06-16 Thread Evgenii Zhuravlev
Hi, All caches, including caches for atomic structures and in-memory caches, are not available before activation. I believe it makes sense to move your code for running after the activation event: https://apacheignite.readme.io/docs/baseline-topology#cluster-activationdeactivation-events .

Data streamer hangs

2020-06-16 Thread yangjiajun
Hello.I use SET STREAMING ON and SET STREAMING OFF style to flush data to ignite.Ignite sometimes hangs on SET STREAMING OFF. The thread hangs on client side looks like: daemon prio=5 os_prio=0 tid=0x7fdba0003000 nid=0x3c1 waiting on condition [0x7fdf473f1000]

Re: Using native persistence to "extend" memory

2020-06-16 Thread Denis Magda
Evgeniy, Thanks for clarifying, I completely forgot about this behavior! - Denis On Tue, Jun 16, 2020 at 5:10 PM Evgenii Zhuravlev wrote: > Steve, > > Actually, disabling WAL is a good option for your use case. Checkpoint > mechanism is the same with disabled WAL, the only difference is that

Re: Using native persistence to "extend" memory

2020-06-16 Thread Evgenii Zhuravlev
Steve, Actually, disabling WAL is a good option for your use case. Checkpoint mechanism is the same with disabled WAL, the only difference is that node is not writing WAL to the disk on each operation. Usually, it might make sense to disable WAL for initial loading - when you can lose the data in

Re: Using native persistence to "extend" memory

2020-06-16 Thread Denis Magda
Steve, Please check these generic recommendations if you haven't done so already: https://apacheignite.readme.io/docs/durable-memory-tuning#native-persistence-related-tuning Otherwise, send us a note if you come across any bottlenecks or issues so that we can give you more specific

Re: Using native persistence to "extend" memory

2020-06-16 Thread steve.hostettler
Thanks a lot for the recommendation. So keeping the WAL, disabling archiving. I understand all records are kept on disk. Thanks again. Anything else? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Using native persistence to "extend" memory

2020-06-16 Thread Denis Magda
Hi Steve, I think that you can get a performance hit for your write operations by disabling the WAL. It serves two purposes in Ignite. The first is the recovery while the second is fast disk writes. The WAL is an append-only structure and Ignite can persist changes to disk really quick. If the

Using native persistence to "extend" memory

2020-06-16 Thread steve.hostettler
Hello, I am trying to use ignite persistence to "extend" memory. That is to replace the swap that is not working very well. Therefore since I do not care about recovery I disabled the WAL. Are there other things you would recommend to configure to use the ignite persistence as a sort of swap.

Webinar: Apache Ignite Management & Monitoring

2020-06-16 Thread Greg Stachnick
Hi Everyone, We released a new monitoring and management tool this month for Apache Ignite and GridGain applications; GridGain Control Center. There is a webinar schedule for tomorrow, JUNE 17 2020 10:00AM PDT (-07:00), where I will provide an overview of the features and demo different use

PIVOT with SQL

2020-06-16 Thread narges saleh
Hi All, Is it possible to use pivot function with JDBC SQL used with apache ignite? If not, is there a function available with such a functionality? thanks

Re: Multi-threaded JDBC driver issue/concern

2020-06-16 Thread adipro
This is my SQL table schema ID (Long), URL (Varchar), SCORE (Double), APPNAME_ID (Long) We have composite index on Score, Appname_Id. Based on your answer I've two questions. 1. How can I insert SQL rows using JCache data streamer API (if possible, with example)? Currently I'm using jdbc

Re: Multi-threaded JDBC driver issue/concern

2020-06-16 Thread adipro
Hi stephen. Thanks so much for reply. This is my SQL table schema ID (Long), URL (Varchar), SCORE (Double), APPNAME_ID (Long) We have composite index on Score, Appname_Id. Based on your answer I've two questions. 1. How can I insert SQL rows using JCache data streamer API (if possible, with

Re: Multi-threaded JDBC driver issue/concern

2020-06-16 Thread adipro
Hi stephen. Thanks so much for reply. This is my SQL table schema ID (Long), URL (Varchar), SCORE (Double), APPNAME_ID (Long) We have composite index on Score, Appname_Id. Based on your answer I've two questions. 1. How can I insert SQL rows using JCache data streamer API (if possible, with

Re: IgniteSpringBean IgniteCheckedException

2020-06-16 Thread Denis Magda
It looks like you copied this configuration template from this documentation file: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSpringBean.html It assumes that an Ignite instance will be started separately and then passed into this configuration file via

Re: Apache Ignite with Spring framework

2020-06-16 Thread Denis Magda
It depends on what you assume under a spring bean. If that's an entity of a Spring framework such as Controller, Repository or Entity, then you don't need to register anything with Ignite. Use those Spring annotations throughout your application and have your Controllers & Repositories access an

Apache Ignite with Spring framework

2020-06-16 Thread kay
Hello, Does the ignite operate on a spring framework basis? Can I register a spring controller in classpath at server remote node and use it?(using component , like @Controller) -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Apache Ignite with Spring framework

2020-06-16 Thread kay
Hello, I apologize for sent a lot of e-mails with the same content. I also show your answer on Stack Overflow. I just configure start in xml file like this and There are caches and dataregions. I execute ignite.sh config.xml & What should I configure for registering spring bean??

Re: Multi-threaded JDBC driver issue/concern

2020-06-16 Thread Stephen Darlington
There’s not one, right way of doing it. In Java it’s something like this. Define your classes: public class AppDetailsKey { @QuerySqlField private Long id; public AppDetailsKey(Long id) { this.id = id; } } public class AppDetails { @QuerySqlField private String

How to fix Ignite node segmentation without restart

2020-06-16 Thread Actarus
Hello, I'm running Apache Ignite (2.4.0) embedded into a java application that runs in a master/slave architecture. This means that there are only ever two nodes in a grid, in FULL_SYNC, REPLICATED mode. Only the master application writes to the grid, the slave only reads from it when it gets

Re: ignite work dir

2020-06-16 Thread Ilya Kasnacheev
Hello! Try setting IgniteConfiguration.workDirectory to some writable path. Regards, -- Ilya Kasnacheev чт, 11 июн. 2020 г. в 18:58, narges saleh : > Hi Ilya, > IGNITE_HOME is /opt/ignite and in ignite-log4j, the log file set to > > I don't have any issue on the server side. > > This is

Re: Apache Ignite with Spring framework

2020-06-16 Thread Ilya Kasnacheev
Hello! You have already sent a lot of e-mails with the same content. Please avoid sending more of these. As I have told you on Stack Overflow, Ignite is not a distributed Spring :) it will not register any spring components, but can re-use spring context and make beans from it available to

Apache Ignite with Spring framework

2020-06-16 Thread kay
Hello, Does the Apache Ignite operate on a spring framework basis? Can I register a spring controller in classpath at server remote node and use it? (using component scan, like @Controller) Thank you -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Apache Ignite with Spring framework

2020-06-16 Thread kay
Does the ignite operate on a spring framework basis? Can I register a spring controller in classpath at server node and use it?(using component scan, like @Controller) -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Apache Ignite operation

2020-06-16 Thread kay
Hello, Does the Apache Ignite operate on a spring framework basis? Can I register a spring controller in classpath at server remote node and use it? (using component scan, like @Controller) Thank you -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite with Spring Framework

2020-06-16 Thread kay
Hello, Does the ignite operate on a spring framework basis? Can I register a spring controller in classpath and use it?(using component , like @Controller) -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

IgniteSpringBean start failed

2020-06-16 Thread kay
Hello, I start node using sh ignite.sh /ERP/Domains/CacheDomain/config/config.xml & command It worked well before I added IgniteSpringBean. After I add this code, 2020-06-16T16:20:38,429][INFO ][main][G] Node started : [stage="Configure

Ignite with Spring Framework

2020-06-16 Thread kay
Hello, Does the ignite operate on a spring framework basis? Can I register a spring controller in classpath at server remote node and use it?(using component , like @Controller) Thank you -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Out of memory error in data region with persistence enabled

2020-06-16 Thread Raymond Wilson
Hi Alex, Thanks for providing the additional detail on the checkpointing memory requirements. As you say, this is difficult to calculate statically, but it seems to indicate that small data regions are bad (ie: susceptible to this issue), so should be avoided. Our current system really only has

Re: Out of memory error in data region with persistence enabled

2020-06-16 Thread Alex Plehanov
Raymond, When a checkpoint is triggered you need to have some amount of free page slots in offheap to save metadata (for example free-lists metadata, partition counter gaps, etc). The number of required pages depends on count of caches, count of partitions, workload, and count of CPUs. In worst

IgniteSpringBean IgniteCheckedException

2020-06-16 Thread kay bae
Hello, I start node using command ' sh ignite.sh ./config/config.xml & ' It worked well before I added IgniteSpringBean. After I add this code, 2020-06-16T16:20:38,429][INFO ][main][G] Node started : [stage="Configure system pool" (53

IgniteCheckedException

2020-06-16 Thread kay bae
Hello, I start node using command ' sh ignite.sh ./config/config.xml & ' It worked well before I added IgniteSpringBean. After I add this code, 2020-06-16T16:20:38,429][INFO ][main][G] Node started : [stage="Configure system pool" (53