Re: Node requires maintenance, non-empty set of maintainance tasks is found - node is not coming up

[Gianluca Bonetti]

Hello Naveen

Apache Ignite 2.13 is more than 2 years old, 25 months old in actual fact.
Three bugfix releases had been rolled out over time up to 2.16 release.

It seems you are restarting your cluster on a regular basis, so you'd
better upgrade to 2.16 as soon as possible.
Otherwise it will also be very difficult for people on a community based
mailing list, on volunteer time, to work out a solution with a 2 years old
version running.

Besides that, you are not providing very much information about your
cluster setup.
How many nodes, what infrastructure, how many caches, overall data size.
One could only guess you have more than 1 node running, with at least 1
cache, and non-empty dataset. :)

This document from GridGain may be helpful but I don't see the same for
Ignite, it may still be worth checking it out.
https://www.gridgain.com/docs/latest/perf-troubleshooting-guide/maintenance-mode

On the other hand you should also check your failing node.
If it is always the same node failing, then there should be some root cause
apart from Ignite.
Indeed if the nodes configuration is the same across all nodes, and just
this one fails, you should also consider some network issues (check
connectivity and network latency between nodes) and hardware related issues
(faulty disks, faulty memory)
In the end, one option might be to replace the faulty machine with a brand
new one.
In cloud environments this is actually quite cheap and easy to do.

Cheers
Gianluca

On Wed, 29 May 2024 at 08:43, Naveen Kumar  wrote:

> Hello All
>
> We are using Ignite 2.13.0
>
> After a cluster restart, one of the node is not coming up and in node logs
> are seeing this error - Node requires maintenance, non-empty set of
> maintainance  tasks is found - node is not coming up
>
> we are getting errors like time out is reached before computation is
> completed error in other nodes as well.
>
> I could see that, we have control.sh script to backup and clean up the
> corrupted files, but when I run the command, it fails.
>
> I have removed the node from baseline and tried to run as well, still its
> failing
>
> what could be the solution for this, cluster is functioning, however there
> are requests failing
>
> Is there anyway we can start ignite node in  maintenance mode and try
> running clean corrupted commands
>
> Thanks
> Naveen
>
>
>

Re: Ignite process memory usage continues to grow in docker

[Gianluca Bonetti]

Hello

You should give some information otherwise the problem statement is too
generic.
Information such as Ignite version, JVM version, JVM memory parameters, and
Docker deployment resources limits are required.

Cheers
Gianluca

On Sat, 20 Apr 2024 at 10:22, 38797715 <38797...@qq.com> wrote:

> Hi team,
>
> Start a simple Ignite process using Docker, which is the default
> configuration and does nothing. After a period of time, you will find that
> the memory occupied by this Ignite process increases until the OOM Killer
> is triggered.
>
> If Docker is not used and the Ignite process is started on the host, this
> problem does not exist.
>
> After testing, this issue is not related to the Docker version, JDK
> version, or Ignite version.
>
> Through monitoring, it can be seen that the JVM heap space, non-heap
> space, and offHeap space are all normal.
>
> Has anyone in the community encountered the same problem?
>

Re: [ANNOUNCE] Apache Ignite 2.16.0 Released

[Gianluca Bonetti]

Hello

I think rolling upgrades are available only for GridGain.
https://www.gridgain.com/docs/latest/installation-guide/rolling-upgrades

Cheers
Gianluca Bonetti

On Fri, 5 Jan 2024 at 16:53, John Smith  wrote:

> Can we Upgrade from 2.13.0 to 2.16.0 Opensource or we need to kill the
> whole cluster and all clients?
>
> On Tue, Dec 26, 2023 at 2:33 PM Nikita Amelchev 
> wrote:
>
>> The Apache Ignite Community is pleased to announce the release of
>> Apache Ignite 2.16.0.
>>
>> Apache Ignite® is an in-memory computing platform for transactional,
>> analytical, and streaming workloads delivering in-memory speeds at a
>> petabyte scale.
>> https://ignite.apache.org
>>
>> The Apache Ignite community has made a lot of changes in the 2.16.0
>> release. This blog post will help you to know about some valuable
>> improvements:
>> https://ignite.apache.org/blog/apache-ignite-2-16-0.html
>>
>> For the full list of changes, you can refer to the RELEASE_NOTES list
>> which is trying to catalogue the most significant improvements for
>> this version of the platform.
>> https://ignite.apache.org/releases/2.16.0/release_notes.html
>>
>> Download the latest Ignite version from here:
>> https://ignite.apache.org/download.cgi
>>
>> Please let us know if you encounter any problems:
>> https://ignite.apache.org/our-community.html#faq
>>
>> Regards,
>> Nikita Amelchev on behalf of the Apache Ignite community.
>>
>

Re: Info about time series support

[Gianluca Bonetti]

Hello Jagat

There are Ignite thin clients for a number of languages, including Python.
For a full list of functionalities and comparison, please always refer to
the official documentation.
https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients

All thin clients should perform around the same in tasks such as storing
and retrieving data as they use the Apache Ignite binary protocol.
As you know performance also varies case by case, because of different
setups, configurations, and versions of software/frameworks/libraries
being used, and of course the performance of the code that you will write
yourself.

For my specific use cases, Apache Ignite always performed extremely well.
As I don't know anything about your project, there are far too many
possible variables to be able to reduce to a yes/no answer.
The advice is to run your own benchmarks on your infrastructure to get some
meaningful figures for your specific project and infrastructure.

Cheers
Gianluca Bonetti

On Fri, 5 Jan 2024 at 12:40, Jagat Singh  wrote:

> Hi Gianluna,
>
> Does the Python client miss any functionality or performance compared to
> Java?
>
> Thanks
>
> On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti 
> wrote:
>
>> Hello Angelo
>>
>> It seems to be an interesting use case for Ignite.
>>
>> However, you should consider what Ignite is, and what is not.
>> Essentially, Ignite is a distributed in-memory database/cache/grid/etc...
>> It also has some distributed computing API capabilities.
>>
>> You can store data easily in Ignite, and consume data by your code
>> written in Java.
>> You can also use Python since there is a Python Ignite Client available
>> if it makes your time series analysis easier.
>> You can also use the Ignite Computing API to execute code on your cluster
>> https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing
>> but in this case I think Python is not supported.
>>
>> Cheers
>> Gianluca Bonetti
>>
>> On Fri, 5 Jan 2024 at 08:52, Angelo Immediata 
>> wrote:
>>
>>> I'm pretty new to Apache Ignite
>>>
>>>
>>> I asked this also on stackoverflow (
>>> https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting)
>>> but I received no answer
>>>
>>> I need to make some forecasting analysis
>>>
>>> Basically I can collect data in Ignite in real time. Ignite will store
>>> data in its own caches
>>>
>>> Now I need to make some forecasting showing me the distribution of data
>>> in the next X months/years by starting from observed and collected data.
>>>
>>> As far as I know, this kind of forecasting can be realized by time
>>> series forecasting. In Ignite I see no time series based algorithm. Am I
>>> right?
>>>
>>> If I'm correct I may use or tensor flow or Deep Java Library. But in
>>> this case what I don't understand is: where should I use these libraries?
>>> Inside my thick client microservice or should I write an Ignite plugin in
>>> order to use the scalability feature provided by Ignite?
>>>
>>> Thank you
>>>
>>> Angelo
>>>
>>

Re: Info about time series support

[Gianluca Bonetti]

Hello Angelo

It seems to be an interesting use case for Ignite.

However, you should consider what Ignite is, and what is not.
Essentially, Ignite is a distributed in-memory database/cache/grid/etc...
It also has some distributed computing API capabilities.

You can store data easily in Ignite, and consume data by your code written
in Java.
You can also use Python since there is a Python Ignite Client available if
it makes your time series analysis easier.
You can also use the Ignite Computing API to execute code on your cluster
https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing
but in this case I think Python is not supported.

Cheers
Gianluca Bonetti

On Fri, 5 Jan 2024 at 08:52, Angelo Immediata  wrote:

> I'm pretty new to Apache Ignite
>
>
> I asked this also on stackoverflow (
> https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting)
> but I received no answer
>
> I need to make some forecasting analysis
>
> Basically I can collect data in Ignite in real time. Ignite will store
> data in its own caches
>
> Now I need to make some forecasting showing me the distribution of data in
> the next X months/years by starting from observed and collected data.
>
> As far as I know, this kind of forecasting can be realized by time series
> forecasting. In Ignite I see no time series based algorithm. Am I right?
>
> If I'm correct I may use or tensor flow or Deep Java Library. But in this
> case what I don't understand is: where should I use these libraries? Inside
> my thick client microservice or should I write an Ignite plugin in order to
> use the scalability feature provided by Ignite?
>
> Thank you
>
> Angelo
>

Re: Request handler for HTTP call for getting the topology

[Gianluca Bonetti]

Helo

Please refer to the latest version, 2.9.0 is pretty old.

Have a look to the section about security for the REST APIs, on the same
documentation page.
https://ignite.apache.org/docs/2.15.0/restapi#security

You would have problems when looking for support from the community with a
version so old.
In case you need to use an older release, you'd better plan for commercial
support from GridGain (I am not affiliated to the company)

Cheers
Gianluca

On Tue, 25 Jul 2023 at 18:38, Amitkumar Maheshwari <
amitkumar.maheshw...@automationanywhere.com> wrote:

> I have been looking into the document
> https://ignite.apache.org/docs/2.9.0/restapi#topology
>
>
>
> From this, I am making sense that there exists an endpoint to get the
> topology regarding information out of ignite.
>
>
>
> Now that I am looking for the relevant code, I could find one test case as
> well in “JettyRestProcessorSignedSelfTest”
>
> But this test only checks w.r.t. “Authorized” and “Unauthorized” request.
>
>
>
> I need to understand what is expected from this end point, and also where
> is the associated request handler implementation.
>
>
>
> Thanks and Regards,
>
> *Amitkumar Maheshwari ** | * Sr, Software Engineer
>
> www.automationanywhere.com/in
>
> Automation Anywhere
>
> Ground Floor (North West Part) Alembic Business Park Premises Alembic Road
> Gorwa.
>
>
>
>
>

Re: Starting Ignite Nodes

[Gianluca Bonetti]

Hello Arunima

I suppose you run Spark in containers so you can create a custom Docker
image to start Apache Ignite.

You can also start the server nodes programmatically by Java code.

Cheers
Gianluca

On Mon, 3 Jul 2023 at 16:03, Arunima Barik  wrote:

> I have around 20 spark worker nodes running on different hosts
>
> I need to start an Ignite node on every such spark worker
>
> I know that we can ssh into that host and run ignite.sh script to start
> the node
>
> Is there a simpler way to do so ..
> Without having to ssh into 20 hosts manually??
>
> Regards
> Arunima
>

Re: Re: Cache size getting zero after first restart of server node in 2.13 version

[Gianluca Bonetti]

Hello

You start a client node, do you start a server node too?

Cheers
Gianluca

On Mon, 15 May 2023 at 13:26, Krishna.Mulumudi.ril.com via user <
user@ignite.apache.org> wrote:

> It’s persisting data after reboot and able to retrieve inserted records
> from cache. Only problem is, cache.size getting reset to zero after first
> restart of the cluster.
>
>
>
> Please find output of requested commands and code snippet for getting
> cache.size .
>
> 
>
>
>
> [tibusr@**~]$ df
>
> Filesystem1K-blocks Used Available Use% Mounted on
>
> devtmpfs78122680   7812268   0% /dev
>
> tmpfs   78233924   7823388   1% /dev/shm
>
> tmpfs   7823392   795764   7027628  11% /run
>
> tmpfs   78233920   7823392   0% /sys/fs/cgroup
>
> /dev/mapper/vg_os-lv_root 103080888 47604952  50216672  49% /
>
> /dev/sda2999320   106456824052  12% /boot
>
> /dev/mapper/vg_os-lv_app  103080888 22497672  75323952  23% /app
>
> tmpfs   15646800   1564680   0% /run/user/1002
>
> tmpfs   15646800   1564680   0% /run/user/7071
>
> [tibusr@** ~]$ lsattr /app
>
> -e-- /app/snapshots1
>
> -e-- /app/datastore1
>
> -e-- /app/logs
>
> [tibusr@**~]$ lsattr /app/datastore1
>
> -e-- /app/datastore1/archive
>
> -e-- /app/datastore1/data
>
> -e-- /app/datastore1/wal
>
> -e-- /app/datastore1/metadataWorkDir
>
> [tibusr@**~]$
>
>
>
> ## JAVA CODE Snippet to get cache size
> 
>
>
>
> /**
>
> *
>
>  */
>
> package com.jio.digitalapi.edif.util.operations;
>
>
>
> import java.util.Arrays;
>
> import java.util.Properties;
>
> import java.util.UUID;
>
>
>
> import org.apache.ignite.Ignite;
>
> import org.apache.ignite.IgniteCache;
>
> import org.apache.ignite.Ignition;
>
> import org.apache.ignite.cache.CachePeekMode;
>
> import org.apache.ignite.configuration.BinaryConfiguration;
>
> import org.apache.ignite.configuration.IgniteConfiguration;
>
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
>
> import
> org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
>
>
>
>
>
> import com.jio.digitalapi.jtl.edif.model.JtlCustomer;
>
>
>
> import javax.swing.*;
>
>
>
> /**
>
> * @author Gangaiah Gundeboina
>
> *
>
> */
>
> public class CacheSizes {
>
>
>
>   public static  void main(String[] args) {
>
>
>
>  String igniteNodeIpAddress = "IPADDRESS";
>
>
>
>  String igniteInstanceName = null;
>
>  try {
>
>String[] ipAddrArray =
> igniteNodeIpAddress.split(",", -1);
>
>
> System.setProperty("java.net.preferIPv4Stack", "true");
>
>IgniteConfiguration cfg = new
> IgniteConfiguration();
>
>
> cfg.setPeerClassLoadingEnabled(true);
>
>cfg.setClientMode(true);
>
>String random =
> (UUID.randomUUID().getMostSignificantBits() + "").toString().replace("-",
> "");
>
>String igniteFullInstaneName =
> igniteInstanceName + random;
>
>
> cfg.setIgniteInstanceName(igniteFullInstaneName);
>
>cfg.setDiscoverySpi((new
> TcpDiscoverySpi().setJoinTimeout(240)
>
>
> .setIpFinder((new
> TcpDiscoveryVmIpFinder()).setAddresses(Arrays.asList(ipAddrArray);
>
>Ignite ignite =
> Ignition.start(cfg);
>
>IgniteCache JtlCustomer> cache =
> ignite.cache("JtlCustomerCache");
>
>
> System.out.println("PRIMARY:"+Integer.valueOf(cache.size(new
> CachePeekMode[]{CachePeekMode.PRIMARY})).toString());
>
>
>  } catch (Exception e) {
>
>e.printStackTrace();
>
>System.exit(0);
>
>  }
>
>  System.exit(0);
>
>
>
>   }
>
>
>
> }
>
> ##
>
>
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> 

Re: Re: Cache size getting zero after first restart of server node in 2.13 version

[Gianluca Bonetti]

Hello

I can make the same setup working on my computer.
What I mean is that when you write to the cache, you should see some files
being created in storage and these should persist across software reboots.
It is still possible there are some configuration settings in your Java
code, and also how the cache is being fed is undisclosed.
Also, are you running inside Kubernetes?

Can we have the output of:
- mount
- df
- lsattr /app
- lsattr /app/datastore1
- lsattr /app/datastore1

Cheers
Gianluca

On Mon, 15 May 2023 at 11:32, Krishna.Mulumudi.ril.com via user <
user@ignite.apache.org> wrote:

> After first restart of the cluster, Data is there and able retrieve from
> get calls but getting cache size zero, it's not retaining value.
>
>
>
> Able to replicate the issue with single server node also.
>
>
>
> 1. start server node
>
> 2. activate cluster.
>
> 3. write few records to cache.
>
> 4. restart the cluster
>
>
>
> Below are inline comments for requested info,
>
> Do these paths exist?
>
> -- Yes
>
> What are the permission flags for these three paths? (owner, group,
>
> permissions)
>
> -- drwxrwxr-x.
>
> Are you running the process by a user who has owner or group permission?
>
> -- runing with a user (tibusr)
>
> Any content in these directories from previous executions?
>
> -- No, we have setup new cluster and then inserted few records and then
> restarted the cluster.
>
> Any particular warning or failure description in the log files?
>
> -- could not see any warnings/errors
>
>
>
> Please find directories info for above requested info,
>
> #
>
> [tibusr@*** ~]$ cd /app/Ignite/scripts
>
> [tibusr@*** scripts]$ cat config.properties
>
> nodeIpAddresses=***:47500
>
> maxSize=#{10L * 1024 * 1024 * 1024}
>
> checkpointPageBufferSize=#{24L * 1024 * 1024 }
>
> pageSize=#{4 * 1024}
>
> #pageSize=#{16 * 1024}
>
> storagePath=/app/datastore1/data
>
> walMode=BACKGROUND
>
> walFlushFrequency=3
>
> rebalanceThreadPoolSize=4
>
> rebalanceThrottle=100
>
> rebalanceBatchSize=#{16 * 1024 * 1024}
>
> walPath=/app/datastore1/wal
>
> metadataWorkDir=/app/datastore1/metadataWorkDir
>
> walArchivePath=/app/datastore1/archive
>
> #IGNITE_LOG4J2=/app/Ignite/scripts/ignite-log4j.xml
>
> IGNITE_LOG4J2=/app/Ignite/scripts/ignite-log4j2.xml
>
> IGNITE_INSTANCE_NAME=EDIF_IOT_ST
>
> snapshotsDirPath=/app/snapshots1/
>
> [tibusr@*** scripts]$ cd /app/
>
> [tibusr@*** app]$ ls -ltr
>
> total 27552
>
> drwx--.  2 tibusr tibusr16384 Apr  6  2018 lost+found
>
> drwxrwxr-x.  2 tibusr tibusr 4096 May 11 17:11 snapshots1
>
> drwxrwxr-x.  3 tibusr tibusr 4096 May 13 15:34 logs
>
> drwxrwxr-x.  6 tibusr tibusr 4096 May 13 17:02 datastore1
>
> [tibusr@*** app]$ cd datastore1
>
> [tibusr@*** datastore1]$ ls -ltr
>
> total 16
>
> drwxrwxr-x. 3 tibusr tibusr 4096 May 13 17:02 data
>
> drwxrwxr-x. 4 tibusr tibusr 4096 May 13 17:02 metadataWorkDir
>
> drwxrwxr-x. 3 tibusr tibusr 4096 May 13 17:02 wal
>
> drwxrwxr-x. 3 tibusr tibusr 4096 May 13 17:02 archive
>
> [tibusr@*** datastore1]$
>
> ###
>
>
>
> And also please find attached server log.
>
>
>
>
>
> *From:* Krishna Mulumudi
> *Sent:* Friday, May 12, 2023 11:41 PM
> *To:* user@ignite.apache.org
> *Subject:* RE: Re: Cache size getting zero after first restart of server
> node in 2.13 version
>
>
>
> Hi Gianluca Bonetti,
>
> Please find below details
>
>
>
> CACHE CONFIGURATION :
>
>
>
>
>
>   
>
>  
>
>  class="org.apache.ignite.configuration.CacheConfiguration"
> parent="cache-template">
>
>
>
>
>
>
>
>
>
>
>
>
>
>   
>
>  
>
>  value="com.jio.digitalapi.jtl.edif.model.JtlMapCustKey" />
>
> 
>
>
> SERVER CONFIGURATION :
>
>
>
> 
>
>
>
> 
> xmlns="http://www.springframework.org/schema/beans;
>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>
> xmlns:util="http://www.springframework.org/schema/util;
>
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://w

Re: Re: Cache size getting zero after first restart of server node in 2.13 version

[Gianluca Bonetti]

Hello

You are using params inside the configuration.
What are the injected values for ${storagePath}, ${walPath}, and
${walArchivePath} ?
Do these paths exist?
What are the permission flags for these three paths? (owner, group,
permissions)
Are you running the process by a user who has owner or group permission?
Any content in these directories from previous executions?
Any particular warning or failure description in the log files?

Cheers
Gianluca

On Fri, 12 May 2023 at 19:13, Krishna.Mulumudi.ril.com via user <
user@ignite.apache.org> wrote:

> Hi Gianluca Bonetti,
>
> Please find below details
>
>
>
> CACHE CONFIGURATION :
>
>
>
>
>
>   
>
>  
>
>  class="org.apache.ignite.configuration.CacheConfiguration"
> parent="cache-template">
>
>
>
>
>
>
>
>
>
>
>
>
>
>   
>
>  
>
>  value="com.jio.digitalapi.jtl.edif.model.JtlMapCustKey" />
>
> 
>
>
> SERVER CONFIGURATION :
>
>
>
> 
>
>
>
> 
> xmlns="http://www.springframework.org/schema/beans;
>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>
> xmlns:util="http://www.springframework.org/schema/util;
>
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd
> <http://www.springframework.org/schema/beans%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20http:/www.springframework.org/schema/beans/spring-beans.xsd%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20http:/www.springframework.org/schema/util%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20http:/www.springframework.org/schema/util/spring-util.xsd>
> ">
>
> 
>
> 
>
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
>
> 
>
> 
>
> 
>
>  value="${IGNITE_INSTANCE_NAME}" />
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
>
>
>
>
>
>
>  value="${rebalanceThreadPoolSize}"/>
>
> 
>
> 
>
> 
>
>  value="${IGNITE_SCRIPT}/ignite-log4j2.xml" />
>
> 
>
> 
>
> 
>
> 
>
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
> 
>
> 
>
> class="org.apache.ignite.configuration.DataRegionConfiguration">
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> value="${checkpointPageBufferSize}" />
>
> 
>
> 
>
> 
>
> 
>
>  />
>
> 
>
>  value="${walMode}" />
>
>  value="${walFlushFrequency}"/>
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> class="org.apache.ignite.configuration.LoadAllWarmUpConfiguration"/>
>
> 
>
> 
>
> 
>
> 
>
> 
>
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>
> 
>
> 
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
> 
>
>  class="org.springframework.util.StringUtils"
>
>
> factory-method="commaDelimitedListToSet">
>
> 
>
> value="${nodeIpAddre

Re: Cache size getting zero after first restart of server node in 2.13 version

[Gianluca Bonetti]

Hello

Not enough information, you didn't provide the current full configuration
nor a reproducible code test case.

Cheers
Gianluca

On Fri, 12 May 2023 at 09:07, Ganguly Gundeboina 
wrote:

> Hi,
>
> We recently setup the 2.13 ignite cluster having three server nodes and
> enabled write services to write the records to the cluster.
>
> After few days, want to restart the cluster as part of maintenance
> activity. After restart all cache sizes showing zero even though data is
> there and getting retrieved.
>
> Below one used to get the cache size,
> cache.size(new CachePeekMode[]{Cache.PeekMode.PRIMARY})
>
> cache size is important to validate the data during maintenance.
>
> Replication steps:
> 1. setup cluster 2.13
> 2. insert few records in any cache.
> 3. restart the cluster.
>
> Same issue not observed in 2.11 and 2.7.6 versions.
>
> Regards,
> Ganguly
>

Re: Storage Exception using Ignite

[Gianluca Bonetti]

Hello

It might be a number of things starting from no space available on your
disk, partition, and/or permissions and/or other constraints on your
workstation imposed by your organization.
I suggest you check this with your internal support team to exclude any
permission issue on your development machine.

I am not affiliated to Ignite/GridGain but I strongly encourage you to
subscribe for a support plan if you need support so frequently and so
quickly, I am confident your organization has the budget for it.

Cheers
Gianluca


On Thu, 2 Mar 2023 at 08:47, satyajit.mandal.barclays.com via user <
user@ignite.apache.org> wrote:

> Hi  team,
>
>
>
> Any  update  on  this  what might  be  causing  this  issue  while
> storing  data in  Ignite  with  persistence  enabled?
>
>
>
> class o.a.i.i.processors.cache.persistence.StorageException: Failed to
> initialize partition file
>
>
>
> Thanks
>
> Satyajit
>
>
>
>
>
>
>
> *From:* Mandal, Satyajit: IT (PUN)
> *Sent:* Thursday, March 2, 2023 7:00 AM
> *To:* 'user@ignite.apache.org' 
> *Subject:* Storage Exception using Ignite
>
>
>
> Hi  Team,
>
>
>
> Am  getting below  storage  exception while  loading  a  heavy  table
> into  cache. Am running  Ignite  on  windows .
>
>
>
> 17:29:29,731][SEVERE][exchange-worker-#62][] JVM will be halted
> immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.StorageException: Failed to initialize
> partition file:
> D:\Data\apache-ignite\work\db\node00-382c3dd6-8cca-4469-b041-bd2f24c31ab3\cache-unittest.RuleGroup\index.bin]]
>
>
>
> Could  you  please  suggest  if  any  settings  is  missing ?
>
>
>
> Thanks
>
> Satyajit
>
> Barclays Execution Services Limited registered in England. Registered No.
> 1767980. Registered office: 1 Churchill Place, London, E14 5HP
>
> Barclays Execution Services Limited provides support and administrative
> services across Barclays group. Barclays Execution Services Limited is an
> appointed representative of Barclays Bank UK plc, Barclays Bank plc and
> Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays
> Bank plc are authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Clydesdale Financial Services Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> This email and any attachments are confidential and intended solely for
> the addressee and may also be privileged or exempt from disclosure under
> applicable law. If you are not the addressee, or have received this email
> in error, please notify the sender and immediately delete it and any
> attachments from your system. Do not copy, use, disclose or otherwise act
> on any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or virus-free. The
> Barclays group does not accept responsibility for any loss arising from
> unauthorised access to, or interference with, any internet communications
> by any third party, or from the transmission of any viruses. Replies to
> this email may be monitored by the Barclays group for operational or
> business reasons.
>
> Any opinion or other information in this email or its attachments that
> does not relate to the business of the Barclays group is personal to the
> sender and is not given or endorsed by the Barclays group.
>
> Unless specifically indicated, this e-mail is not an offer to buy or sell
> or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays.
>

Re: Unable to start Ignite server node through java API

[Gianluca Bonetti]

Hello

I only use:

  
org.apache.ignite
ignite-core
2.14.0
  

Which brings in what is required and actually does NOT bring in H2, because
it's not required any more.

I guess you should fix your pom.xml, which I have no information about, as
you didn't share.

Cheers
Gianluca

On Wed, 1 Mar 2023 at 13:22, Abhishek Ubhe 
wrote:

> Okay, Can you suggest any changes in maven dependencies?
>
> On Wed, Mar 1, 2023 at 2:07 PM Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>> Hello
>>
>> It's very strange that this method does not exist in the H2 library.
>> Can you please check you're not including the H2 library twice?
>> For what I know, Ignite works with h2-1.4.197.jar which is included in
>> the last bundle.
>> I think you are including a newer version instead, or aside.
>>
>> Cheers
>> Gianluca
>>
>> On Wed, 1 Mar 2023 at 06:43, Abhishek Ubhe 
>> wrote:
>>
>>> Hello,
>>>
>>> I am unable to start ignite server node with default configurations.
>>> Getting below error
>>>
>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.registerH2(Lorg/apache/ignite/internal/processors/cache/persistence/tree/io/IOVersions;Lorg/apache/ignite/internal/processors/cache/persistence/tree/io/IOVersions;)V
>>>
>>> *stacktrace :* at
>>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.(IgniteH2Indexing.java:197)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:264)
>>> at
>>> org.apache.ignite.internal.util.IgniteUtils.inClassPath(IgniteUtils.java:1753)
>>> at
>>> org.apache.ignite.internal.IgniteComponentType.inClassPath(IgniteComponentType.java:172)
>>> at
>>> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1999)
>>> at
>>> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1758)
>>> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1143)
>>> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:663)
>>> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589)
>>> at org.apache.ignite.Ignition.start(Ignition.java:328)
>>>
>>>
>>> Please help with this and let me know why this error is getting and how
>>> I can avoid it in future.
>>> My API :
>>> IgniteConfiguration cfg = new IgniteConfiguration();
>>> cfg.setClientMode(false);
>>>
>>> cfg.setWorkDirectory("/home/developers/apache_ignite_home/apache-ignite-2.14.0-bin/work");
>>>  Ignite ignite = Ignition.start(cfg);
>>>
>>> version using : 2.14.0
>>> --
>>> *Regards,*
>>> *Abhishek Ubhe*
>>>
>>>
>
> --
> *Regards,*
> *Abhishek Ubhe*
>
>

Re: Unable to start Ignite server node through java API

[Gianluca Bonetti]

Hello

It's very strange that this method does not exist in the H2 library.
Can you please check you're not including the H2 library twice?
For what I know, Ignite works with h2-1.4.197.jar which is included in the
last bundle.
I think you are including a newer version instead, or aside.

Cheers
Gianluca

On Wed, 1 Mar 2023 at 06:43, Abhishek Ubhe 
wrote:

> Hello,
>
> I am unable to start ignite server node with default configurations.
> Getting below error
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.registerH2(Lorg/apache/ignite/internal/processors/cache/persistence/tree/io/IOVersions;Lorg/apache/ignite/internal/processors/cache/persistence/tree/io/IOVersions;)V
>
> *stacktrace :* at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.(IgniteH2Indexing.java:197)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.ignite.internal.util.IgniteUtils.inClassPath(IgniteUtils.java:1753)
> at
> org.apache.ignite.internal.IgniteComponentType.inClassPath(IgniteComponentType.java:172)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1999)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1758)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1143)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:663)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589)
> at org.apache.ignite.Ignition.start(Ignition.java:328)
>
>
> Please help with this and let me know why this error is getting and how I
> can avoid it in future.
> My API :
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setClientMode(false);
>
> cfg.setWorkDirectory("/home/developers/apache_ignite_home/apache-ignite-2.14.0-bin/work");
>  Ignite ignite = Ignition.start(cfg);
>
> version using : 2.14.0
> --
> *Regards,*
> *Abhishek Ubhe*
>
>

Re: H2database dependency in Apache Ignite

[Gianluca Bonetti]

Hello

I am also using Apache Ignite for some projects, but I don't see any
dependency on h2 in my projects.
I think h2 dependency is coming from somewhere else.
Can you run a "mvn dependency:tree" and share the results?

Cheers
Gianluca

On Fri, 20 Jan 2023 at 09:56, David Cussen  wrote:

> Hi,
>
> I am an employee in Workday and our team uses Apache Ignite for one of our
> products. There is a dependency on com.h2database:h2:jar:1.4.197 :
> https://github.com/apache/ignite/blob/master/parent/pom.xml#L92
>
>
>
> We are wondering if there is a plan to upgrade this dependency to remediate
>  CVE-2021-42392
>  and if
> so, do you have an ETA on when this would be available?
>
>
>
> Thank you.
>
>
>
> Kind regards,
>
> David Cussen
>
> Workday
>
>
>

Re: Cluster shutdown by "to many files open"

[Gianluca Bonetti]

Hello

I had the same problem, with far more caches in use, as total number (but
each cache was very small in size).

32768 files is definitely too low.
In my ase, I had to raise it to 262144 hard limit and 131072 soft limit.
Please update your /etc/security/limits.conf records for the user you run
your app with.

I also raised fs.file-max to 2097152 which may be excessive, but I don't
see a problem with setting it that high.

Cheers
Gianluca

On Fri, 16 Dec 2022 at 01:39, John Smith  wrote:

> Hi it seems the JVM was forcefully shutdown when I tried to create a new
> partitioned cache.
>
> The error seems to indicate that it was "too many files" can someone from
> Ignite confirm this?
>
> I have checked with lsof and Ignite only has about 3600 files open. It's
> the only service running on that server. So I don't see how this could
> happen? I have a total of 10 caches mixed between replicated and
> partitioned (1 backup) over 3 nodes.
>
> I have
>
> fs.file-max:30
> and
> - softnofile  32768
> - hardnofile  32768
> respectively on each node.
>
> What I did was delete the db/folder for that specific cache on that node
> and when I restarted it. It worked and recreated the folder for that cache.
>
> https://www.dropbox.com/s/zwf28akser9p4dt/ignite-XX.0.log?dl=0
>

Re: Ignite-schedule

[Gianluca Bonetti]

Hello

I don't think that piece of software is being used anywhere.
You can see it's an old jar from year 2015, so definitely ancient and not
up to date.
I never had to use that ignite-schedule anywhere in my projects.
Any particular reason to resurrect this old component?

Cheers
Gianluca

On Fri, 26 Aug 2022 at 18:08, Surinder Mehra  wrote:

> Hi,
> Could you please suggest if it is safe to use below version of
> ignite-schedule
> It has warnings about CVE
>
>
> https://mvnrepository.com/artifact/org.apache.ignite/ignite-schedule/1.2.0-incubating
>
> Vulnerabilities from dependencies:
> CVE-2020-1963
> 
> CVE-2018-8018
> 
> CVE-2018-1295
> 
> CVE-2017-7686
> 
>

Re: Cache directory full path on disk

[Gianluca Bonetti]

Hello

Linux rmdir command deletes only non-empty directories.

Another option for you is to commit the cleaning source code to Apache
Ignite as open source contributor, either as fix on the destroy cache API,
or other new APIs.

To my humble point of view, removing the empty directory is not mission
critical fix for the core team to work on, as I suppose they are mainly
working on Apache Ignite 3.0

Thanks
Gianluca

On Mon, 11 Jul 2022 at 09:16, Sumit Deshinge 
wrote:

> Hi,
> Thanks for your suggestion.
>
> Before I go with custom delete logic with checking each directory empty or
> not and based on that deleting it, I wanted to see if ignite gives any such
> functionality out of the box through its APIs.
>
> On Fri, Jul 8, 2022 at 9:19 PM Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>> Hello Sumit
>>
>> You can simply clean up with a scheduled cronjob does executes rmdir, if
>> the directory is not empty rmdir will not proceed.
>>
>> Cheers
>> Gianluca
>>
>> On Fri, 8 Jul 2022 at 15:38, Sumit Deshinge 
>> wrote:
>>
>>> Any pointers here?
>>>
>>> On Wed, Jun 29, 2022 at 12:55 PM Sumit Deshinge <
>>> sumit.deshi...@gmail.com> wrote:
>>>
>>>> After cache.destroy operation is performed, it clears all the cache
>>>> data, but directory for that cache exists with empty data.
>>>> I am creating many caches for specific workflow and destroy them, same
>>>> keeps repeating for each workflow.
>>>> This results into cache directories lying on the machine and that count
>>>> goes on increasing.
>>>> So I want to cleanup them.
>>>>
>>>> On Tue, Jun 28, 2022 at 6:38 PM Stephen Darlington <
>>>> stephen.darling...@gridgain.com> wrote:
>>>>
>>>>> What are you trying to achieve?
>>>>>
>>>>> On 28 Jun 2022, at 13:14, Sumit Deshinge 
>>>>> wrote:
>>>>>
>>>>> How do I get a full cache directory path?
>>>>> E.g. ignite_storage_path/host/cache_name
>>>>>
>>>>> I want this full directory path from the machine where cache data is
>>>>> physically being stored. (not only storage directory path, but actual 
>>>>> cache
>>>>> data directory)
>>>>> Is there any ignite API for the same?
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Sumit Deshinge
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Sumit Deshinge
>>>>
>>>>
>>>
>>> --
>>> Regards,
>>> Sumit Deshinge
>>>
>>>
>
> --
> Regards,
> Sumit Deshinge
>
>

Re: Cache directory full path on disk

[Gianluca Bonetti]

Hello Sumit

You can simply clean up with a scheduled cronjob does executes rmdir, if
the directory is not empty rmdir will not proceed.

Cheers
Gianluca

On Fri, 8 Jul 2022 at 15:38, Sumit Deshinge 
wrote:

> Any pointers here?
>
> On Wed, Jun 29, 2022 at 12:55 PM Sumit Deshinge 
> wrote:
>
>> After cache.destroy operation is performed, it clears all the cache data,
>> but directory for that cache exists with empty data.
>> I am creating many caches for specific workflow and destroy them, same
>> keeps repeating for each workflow.
>> This results into cache directories lying on the machine and that count
>> goes on increasing.
>> So I want to cleanup them.
>>
>> On Tue, Jun 28, 2022 at 6:38 PM Stephen Darlington <
>> stephen.darling...@gridgain.com> wrote:
>>
>>> What are you trying to achieve?
>>>
>>> On 28 Jun 2022, at 13:14, Sumit Deshinge 
>>> wrote:
>>>
>>> How do I get a full cache directory path?
>>> E.g. ignite_storage_path/host/cache_name
>>>
>>> I want this full directory path from the machine where cache data is
>>> physically being stored. (not only storage directory path, but actual cache
>>> data directory)
>>> Is there any ignite API for the same?
>>>
>>> --
>>> Regards,
>>> Sumit Deshinge
>>>
>>>
>>>
>>
>> --
>> Regards,
>> Sumit Deshinge
>>
>>
>
> --
> Regards,
> Sumit Deshinge
>
>

Re: Concurrently update some keys

[Gianluca Bonetti]

Hello

Unfortunately reiterating the question won't help.
If you have no positive answer in days, it means there's no such features
in the making.
Also this is a community-based mailing list, it's not a commercial support
service and there's no ETA for tickets, just best efforts from the Apache
Ignite team who is actually doing an amazing job with development and
support.

SQL transactions are still marked as beta, so you really shouldn't rely on
beta software for your work which seems to be urgent, and not even plan on
top of beta features.
I think the focus is on the 3.0 release and features with lower user
demand will be postponed.
Please remember free software is provided "as is" and with no guarantee of
"fitness for a particular purpose".

Your alternatives are...
- switch to key/value approach (widely used, super fast, and rock solid)
- if you have extra developer resources, contribute to development
- get a support subscription from GridGain (I don't work for that company)
and push for developing that feature as a paying customer of the
commercially supported version
- if you have a wealthy end customer you're developing for, ask them to
fund development by GridGain developers, extra developers of your own, or a
bountysource

I know I am not being particularly helpful on the specific topic, but I
just wanted to help in sorting out what options you have out there.

Thank you
Cheers
Gianluca

On Fri, 24 Jun 2022 at 08:14, Ajay Babu  wrote:

> Hello Team,
>
> Please update.
>
>1. Any roadmap will be  solved in upcoming releases.
>2. In case of exception, if we want to re-try any specific exception
>type will throw, so that will catch and do the re-try specific case.
>
> Br,
> Ajay Babu Maguluri.
>
> On Thu, Jun 23, 2022 at 11:56 AM Ajay Babu  wrote:
>
>> Hello Team,
>>
>> Please update.
>>
>>1. Any roadmap will be  solved in upcoming releases.
>>2. In case of exception, if we want to re-try any specific exception
>>type will throw, so that will catch and do the re-try.
>>
>> Br,
>> Ajay Babu Maguluri.
>>
>>
>> On Wed, Jun 22, 2022 at 11:53 PM Ajay Babu 
>> wrote:
>>
>>> Hi,
>>>
>>> Any roadmap will be  solved in upcoming releases.
>>>
>>> Br
>>> Ajay Babu Maguluri.
>>>
>>>
>>> On Wed, Jun 22, 2022 at 11:34 PM Николай Ижиков 
>>> wrote:
>>>
 Hello.

 Ignite SQL is not transactional [1] [2]
 It seems this exception are expected.

 For transactional updates [3] you should use key-value API.

 [1] https://ignite.apache.org/docs/latest/sql-reference/transactions
 [2] https://ignite.apache.org/docs/latest/transactions/mvcc
 [3] https://ignite.apache.org/docs/latest/key-value-api/transactions

 22 июня 2022 г., в 20:59, Ajay Babu 
 написал(а):

 Hello Team,

 Please update, Will Ignite support this without any exception.

 Br,
 Ajay Babu Maguluri,

 On Wed, Jun 22, 2022 at 6:29 PM Ajay Babu 
 wrote:

> Hi Stephen,
>
> Thanks for your reply.
>
> Yes, I am doing SQL update via IgniteJdbcThinDriver and  I am using
> Ignite 2.13.x for one of my requirement. I need to maintain cache in 
> Ignite
> cluster and some of the keys values will be update concurrently and
> selection also happens parallel from multiple jdbc clients. So here I am
> facing an issue.
>
> Br,
> Ajay Babu Maguluri.
>
>
> On Wed, Jun 22, 2022 at 6:04 PM Stephen Darlington <
> stephen.darling...@gridgain.com> wrote:
>
>> You don’t give much information about what you’re doing. I’m guessing
>> that you’re doing a SQL update from a JDBC connection? Can you add some
>> more colour?
>>
>> On 22 Jun 2022, at 13:29, Ajay Babu  wrote:
>>
>> Hi Team,
>>
>> Please update.
>>
>> Br,
>> Ajay Babu Maguluri.
>>
>>
>> On Tue, Jun 21, 2022 at 1:44 PM Ajay Babu 
>> wrote:
>>
>>>
>>> Hello Team,
>>>
>>> I have a requirement to keep cache in ignite and that will update
>>> frequently and selection also happens parallel,  while i am executing
>>> queries iam getting,
>>>
>>> ava.sql.SQLException: Failed to update some keys because they had
>>> been modified concurrently [keys=[ROWKEY [idHash=628930863,
>>> hash=1637498765, rowkey=6, id=765436]]]
>>> at
>>> org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:1009)
>>> at
>>> org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:234)
>>> at
>>> org.apache.ignite.internal.jdbc.thin.JdbcThinPreparedStatement.executeWithArguments(JdbcThinPreparedStatement.java:252)
>>> at
>>> org.apache.ignite.internal.jdbc.thin.JdbcThinPreparedStatement.executeUpdate(JdbcThinPreparedStatement.java:96)
>>> at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown
>>> Source)
>>> 

Re: gridgain ultimate edition snapshot error

[Gianluca Bonetti]

Hello there

Yes it's not a bug.
The open files limit is the default and needs to be raised.
Maybe apply a ulimit action in your start script.

Very likely it worked for you on Ignite vs GridGain because you're using
Ignite on another machine (a testing vm?) where you have less caches hence
less files.

Cheers
Gianluca

On Tue, 7 Jun 2022 at 10:35, Surinder Mehra  wrote:

> Hi,
> Thanks for your reply. Current limits are highlighted below. As suggested
> in prev reply, I will change limits and try again.
>
> Limit Soft Limit   Hard Limit   Units
>
> Max cpu time  unlimitedunlimited
>  seconds
> Max file size unlimitedunlimitedbytes
>
> Max data size unlimitedunlimitedbytes
>
> Max stack size8388608  unlimitedbytes
>
> Max core file sizeunlimitedunlimitedbytes
>
> Max resident set  unlimitedunlimitedbytes
>
> Max processes 6330663306
>  processes
> *Max open files1024 4096 files
>   *
> Max locked memory 6553665536bytes
>
> Max address space unlimitedunlimitedbytes
>
> Max file locksunlimitedunlimitedlocks
>
> Max pending signals   6330663306
>  signals
> Max msgqueue size 819200   819200   bytes
>
> Max nice priority 00
> Max realtime priority 00
> Max realtime timeout  unlimitedunlimitedus
>
> On Tue, Jun 7, 2022 at 1:38 PM Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>> Hello
>>
>> What is returned by this command?
>>
>> # cat /proc/PID/limits
>>
>> Cheers
>> Gianluca
>> Gianluca
>>
>> On Tue, 7 Jun 2022 at 07:35, Surinder Mehra  wrote:
>>
>>> Hi,
>>> I was going through this post on stackoverflow which is about the same
>>> issue. The fact that snapshot works for apache ignite bit not in ultimate
>>> edition indicates there is some bug in later. Could you please confirm. We
>>> have around 15 caches with 2 backups. I changed backups to zero but still
>>> see this issue. Could you please advise further.
>>>
>>>
>>> https://stackoverflow.com/questions/72041292/is-there-a-fix-for-too-many-open-files-error-in-gridgain-cluster
>>>
>>> On Mon, Jun 6, 2022 at 9:13 PM Surinder Mehra 
>>> wrote:
>>>
>>>> Hi,
>>>> I was experimenting with the GG ultimate edition to take snapshots and
>>>> encountered the below error and cluster stops. Please note that this works
>>>> in the ignite free version and we don't see too many files open error. Is
>>>> this a bug or we are missing some configuration?
>>>>
>>>> version:  gridgain-8.8.19
>>>>
>>>> /bin./snapshot-utility.sh snapshot -type=full
>>>>
>>>> [21:03:51,693][SEVERE][db-snapshot-executor-stripe-0-#35][] Critical
>>>> system error detected. Will be handled accordingly to configured handler
>>>> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
>>>> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
>>>> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
>>>> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
>>>> o.a.i.i.processors.cache.persistence.StorageException: Failed to initialize
>>>> partition file:
>>>> /home/usr/tools/gridgain-ultimate-8.8.19/work/db/node00-c221fe71-5d29-4cd7-ab0f-9fa8240711b2/cache-name/part-88.bin]
>>>> class
>>>> org.apache.ignite.internal.processors.cache.persistence.StorageException:
>>>> Failed to initialize partition file:
>>>> /home/usr/tools/gridgain-ultimate-8.8.19/work/db/node00-c221fe71-5d29-4cd7-ab0f-9fa8240711b2/cache-name/part-88.bin
>>>> at
>>>> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.init(FilePageStore.java:519)
>>>> at
>>>> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:405)
>>>> at
>>>> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageReadWriteManagerImpl.read(PageReadWriteManagerImpl.java:68)
>>>&

Re: gridgain ultimate edition snapshot error

[Gianluca Bonetti]

Hello

What is returned by this command?

# cat /proc/PID/limits

Cheers
Gianluca
Gianluca

On Tue, 7 Jun 2022 at 07:35, Surinder Mehra  wrote:

> Hi,
> I was going through this post on stackoverflow which is about the same
> issue. The fact that snapshot works for apache ignite bit not in ultimate
> edition indicates there is some bug in later. Could you please confirm. We
> have around 15 caches with 2 backups. I changed backups to zero but still
> see this issue. Could you please advise further.
>
>
> https://stackoverflow.com/questions/72041292/is-there-a-fix-for-too-many-open-files-error-in-gridgain-cluster
>
> On Mon, Jun 6, 2022 at 9:13 PM Surinder Mehra  wrote:
>
>> Hi,
>> I was experimenting with the GG ultimate edition to take snapshots and
>> encountered the below error and cluster stops. Please note that this works
>> in the ignite free version and we don't see too many files open error. Is
>> this a bug or we are missing some configuration?
>>
>> version:  gridgain-8.8.19
>>
>> /bin./snapshot-utility.sh snapshot -type=full
>>
>> [21:03:51,693][SEVERE][db-snapshot-executor-stripe-0-#35][] Critical
>> system error detected. Will be handled accordingly to configured handler
>> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
>> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
>> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
>> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
>> o.a.i.i.processors.cache.persistence.StorageException: Failed to initialize
>> partition file:
>> /home/usr/tools/gridgain-ultimate-8.8.19/work/db/node00-c221fe71-5d29-4cd7-ab0f-9fa8240711b2/cache-name/part-88.bin]
>> class
>> org.apache.ignite.internal.processors.cache.persistence.StorageException:
>> Failed to initialize partition file:
>> /home/usr/tools/gridgain-ultimate-8.8.19/work/db/node00-c221fe71-5d29-4cd7-ab0f-9fa8240711b2/cache-name/part-88.bin
>> at
>> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.init(FilePageStore.java:519)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:405)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageReadWriteManagerImpl.read(PageReadWriteManagerImpl.java:68)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:577)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:911)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:730)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:711)
>> at
>> org.gridgain.grid.internal.processors.cache.database.snapshot.SnapshotCreateFuture.completeSavingAllocatedIndex(SnapshotCreateFuture.java:1304)
>> at
>> org.gridgain.grid.internal.processors.cache.database.snapshot.SnapshotCreateFuture.completeSnapshotCreation(SnapshotCreateFuture.java:1486)
>> at
>> org.gridgain.grid.internal.processors.cache.database.snapshot.SnapshotCreateFuture.doFinalStage(SnapshotCreateFuture.java:1171)
>> at
>> org.gridgain.grid.internal.processors.cache.database.snapshot.SnapshotOperationFuture.completeStagesLocally(SnapshotOperationFuture.java:2352)
>> at
>> org.gridgain.grid.internal.processors.cache.database.snapshot.SnapshotOperationFuture$10.run(SnapshotOperationFuture.java:2286)
>> at
>> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:567)
>> at
>> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
>> at java.base/java.lang.Thread.run(Thread.java:829)
>> Caused by: java.nio.file.FileSystemException:
>> /home/usr/tools/gridgain-ultimate-8.8.19/work/db/node00-c221fe71-5d29-4cd7-ab0f-9fa8240711b2/cache-name/part-88.bin:
>> Too many open files
>> at
>> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
>> at
>> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
>> at
>> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
>> at
>> java.base/sun.nio.fs.UnixFileSystemProvider.newAsynchronousFileChannel(UnixFileSystemProvider.java:201)
>> at
>> java.base/java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:253)
>> at
>> java.base/java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:311)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.(AsyncFileIO.java:65)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory.create(AsyncFileIOFactory.java:43)
>> at
>> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.init(FilePageStore.java:491)
>> ... 14 more
>> [21:03:51,695][SEVERE][db-snapshot-executor-stripe-0-#35][FailureProcessor]
>> No deadlocked threads 

Re: Unknown connection detected & Failed to process selector key

[Gianluca Bonetti]

Hello Siva

If running in a shared cloud environment, you should put best security
practices in place to secure all of your servers from connections from
unknown hosts (iptables rules) especially on sensitive ports as databases
or Apache Ignite.

It does seem that this address in the range 10.0.0.0/8 is somehow belonging
to your organization.
It's very unlikely any cloud provider would share 10.0.0.0/8 among
different tenants.

Cheers
Gianluca


Il giorno mar 30 nov 2021 alle ore 04:15  ha
scritto:

> Hi,
>
>
>
> I am seeing below exception in our ignite thick client logs. I am seeing
> remote address 10.136.96.174 which is not in our ignite cluster list and
> what I infer from the below logs is that someone is trying to connect to
> out ignite instances. Please let me know if my assumption is wrong. Also is
> there a better way to handle these scenarios ? Appreciate any help/pointers
> on this.
>
> 2021-11-27T13:35:59,656
> -0500|grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%|||WARN
> |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.warning.295| -
> Unknown connection detected (is some other software connecting to this
> Ignite port?) [rmtAddr=/10.136.96.174:52490, locAddr=/10.148.207.245:47101
> ]
>
> 2021-11-27T13:35:59,658
> -0500|grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%|||INFO
> |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| -
> Accepted incoming communication connection [locAddr=/10.148.207.245:47101,
> rmtAddr=/10.136.96.174:52493]
>
> 2021-11-27T13:35:59,676
> -0500|grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%|||WARN
> |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.warning.295| -
> Unknown connection detected (is some other software connecting to this
> Ignite port?) [rmtAddr=/10.136.96.174:52493, locAddr=/10.148.207.245:47101
> ]
>
> 2021-11-27T13:35:59,678
> -0500|grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%|||INFO
> |org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.info.285| -
> Accepted incoming communication connection [locAddr=/10.148.207.245:47101,
> rmtAddr=/10.136.96.174:52496]
>
> *2021-11-27T13:36:01,712
> -0500|grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%|||ERROR|org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.error.310|
> - Failed to process selector key [ses=GridSelectorNioSessionImpl
> [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=0,
> bytesRcvd=533640, bytesSent=2405277, bytesRcvd0=0, bytesSent0=0,
> select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-0,
> igniteInstanceName=TcpCommunicationSpi, finished=false,
> heartbeatTs=1638038160689, hashCode=161343674, interrupted=false,
> runner=grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%]]],
> writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
> readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
> inRecovery=null, outRecovery=null, closeSocket=true,
> outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
> super=GridNioSessionImpl [locAddr=/10.148.207.245:47101
> , rmtAddr=/10.136.96.174:52496
> , createTime=1638038159678, closeTime=0,
> bytesSent=18, bytesRcvd=0, bytesSent0=0, bytesRcvd0=0,
> sndSchedTime=1638038159678, lastSndTime=1638038159678,
> lastRcvTime=1638038159678, readsPaused=false,
> filterChain=FilterChain[filters=[GridNioCodecFilter
> [parser=o.a.i.i.util.nio.GridDirectParser@6613bbbf, directMode=true],
> GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
> java.io.IOException: Connection reset by peer at
> sun.nio.ch.FileDispatcherImpl.read0(Native Method) at
> sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at
> sun.nio.ch.IOUtil.read(IOUtil.java:192) at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) at
> org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1337)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2494)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2259)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1896)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)*
>
> 2021-11-27T13:36:04,703 -0500|grid-timeout-worker-#22|||WARN
> |org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.warning.295|
> - Unable to perform handshake within timeout [timeout=1, remoteAddr=/
> 10.136.96.174:51991]
>
> 2021-11-27T13:36:17,069
> -0500|grid-nio-worker-client-listener-2-#34|||ERROR|org.apache.ignite.internal.processors.odbc.ClientListenerProcessor.error.310|
> - Failed to process selector key 

Re: security issues question

[Gianluca Bonetti]

Hello Timothy

I usually add iptables rules on top of every deployment, to block access
from unknown locations to Apache Ignite and other services (Tomcat to name
one, and others)

My typical iptables rules, embedded into /etc/rc.local looks like this:

iptables -A INPUT -p tcp --match multiport --dport
10800,10801,11211,47100:47109,47400:47409,47500:47509 -s 127.0.0.1 -j ACCEPT
iptables -A INPUT -p tcp --match multiport --dport
10800,10801,11211,47100:47109,47400:47409,47500:47509 -s 10.192.192.192/26
-j ACCEPT
iptables -A INPUT -p tcp --match multiport --dport
10800,10801,11211,47100:47109,47400:47409,47500:47509 -j REJECT

So connection to all Ignite ports (known to me) is permitted from
localhost, from private network space in the cloud, then forbidden from
anywhere else.
You may also want to limit other ports exposed to the wild you may notice
by netstat -nat
This is a simple solution, other experts may have better solutions, and I'm
also interested in them :)

On the other hand, I noticed your running Ignite on IPv6, but I think
running on IPv4 is still preferred.

Cheers
Gianluca

Il giorno gio 18 nov 2021 alle ore 02:08 Timothy Peng 
ha scritto:

> Hello,
>
> I saw Ignite is listening on all interfaces by default:
>
> tcp6   0  0 :::10800:::*LISTEN
> 3211/java
>
>
> Does this have security issues since everyone can access the port and do
> cache reading/writing?
>
>
> Thanks
>

Re: Ignite on a server

[Gianluca Bonetti]

Hello Timothy

I usually deploy on more servers, but just 2 to 4, I don't have larger
deployments.
I also have single server deployments.
In my opinion and personal experience, native Java integration by Ignite
with persistent storage is definitely better than a "hybrid" solution (some
other software plus a Java client) like going to external caches like Redis.
I've seen benchmarks in the past where Ignite also performed better.
Don't know the current status as of today, but not my concern at all, as
using Apache Ignite is definitely too easy and too fast to even consider
alternatives.
Some other developers may have had different experiences, just wanted to
share my point of view.

Cheers
Gianluca


Il giorno mar 16 nov 2021 alle ore 04:33 Timothy Peng 
ha scritto:

> @Surinder, @Ilya,
>
> Can you tell what's the advantages for memory storage compared to other
> products such as Redis?
>
> Thanks.
>
>
> On Tue, Nov 16, 2021 at 11:27 AM Surinder Mehra 
> wrote:
>
>> So it is a very broad question. Sure you can use it for caching and
>> compute as well as it has enough cores. Depending upon your requirements,
>> you can read about data grid part on ignite documentation to get details
>> about various ignite cache configurations
>> - in memory
>> - in memory with cache stores (IMDG)
>> - caches with  native persistence(IMDB)
>> - on heap or off heap caches
>>
>> Just some pointers to read
>>
>> On Tue, Nov 16, 2021, 08:34 Timothy Peng  wrote:
>>
>>> Hello
>>>
>>> I just have one dedicated server who has 128GB memory and 16 cores.
>>> Can I deploy Ignite on this server and just use the memory cache feature?
>>>
>>> Thanks
>>>
>>

Re: [External]Re: Client taking long time to connect to cluster

[Gianluca Bonetti]

Hello there

I would give it a try with -Djava.security.egd=file:/dev/urandom and
-XX:+UseG1GC on both sides and see results.

Another question, are both client and server running in the cloud, within
the same datacentre?
Or client and server run in different locations?

Cheers
Gianluca

Il giorno gio 2 set 2021 alle ore 18:45 Ganguly Gundeboina <
gangaiah.ya...@gmail.com> ha scritto:

>
> Hi Gianluca,
>
> Thanks for your response.
>
> Below are jvm details for both server and client,
>
> *SERVER:*
>
> [tibusr@SERVERIP bin]$ ./java -version
>
> java version "1.8.0_251"
>
> Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
>
> Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)
>
>
>
> export JVM_OPTS="$JVM_OPTS_SERVER -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC
> -XX:+UnlockCommercialFeatures -XX:+PrintGCDetails
> -XX:+PrintAdaptiveSizePolicy -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps
> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=65 -XX:GCLogFileSize=10M
> -Xloggc:/backup/logs/gclog/gc.log -Djava.net.preferIPv4Stack=true
> -DIGNITE_LONG_OPERATIONS_DUMP_TIMEOUT=60
> -DIGNITE_THREAD_DUMP_ON_EXCHANGE_TIMEOUT=true"
>
>
>
> *CLIENT*:
>
> [sitibco@CLIENTIP bin]$ ./java -version
>
> java version "1.8.0_271"
>
> Java(TM) SE Runtime Environment (build 1.8.0_271-b09)
>
> Java HotSpot(TM) 64-Bit Server VM (build 25.271-b09, mixed mode)
>
>
>
>
>
> java.heap.size.max=2048M
>
> java.heap.size.initial=512M
>
> java.extended.properties=-XX\:+UseConcMarkSweepGC
> -XX\:+CMSPermGenSweepingEnabled -XX\:+CMSClassUnloadingEnabled
> -XX\:PermSize\=512m -XX\:MaxPermSize\=512m -Djava.net.preferIPv4Stack=true
>
>
> Regards,
>
> Gangaiah
>
>
> On Thu, Sep 2, 2021 at 5:57 PM Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>> Hello
>>
>> I don't know if I can help you, but I'll give it a try, as I had problems
>> for slow startup time in the past, and I solved them.
>>
>> Can you share a couple of information about JVM?
>> Which JVM version are you using on the two different setups?
>> Which JVM params are you using for them?
>>
>> Thank you
>> Cheers
>> Gianluca
>>
>> Il giorno gio 2 set 2021 alle ore 12:01 Kamlesh Joshi <
>> kamlesh.jo...@ril.com> ha scritto:
>>
>>> Yohan, we have tried with JVM param as well, still its taking long time
>>> to connect.
>>>
>>>
>>>
>>> *Thanks and Regards,*
>>>
>>> *Kamlesh Joshi*
>>>
>>>
>>>
>>> *From:* Yohan Fernando 
>>> *Sent:* 02 September 2021 15:00
>>> *To:* user@ignite.apache.org
>>> *Subject:* Re: [External]Re: Client taking long time to connect to
>>> cluster
>>>
>>>
>>>
>>> The e-mail below is from an external source. Please do not open
>>> attachments or click links from an unknown or suspicious origin.
>>>
>>> Seems like that VM arg is only read at startup time of the JVM according
>>> to this post, so setting it in code may not work
>>>
>>>
>>>
>>>
>>> https://stackoverflow.com/questions/9882357/how-to-set-java-net-preferipv4stack-true-at-runtime
>>>
>>>
>>>
>>> You might want to try it as a VM argument
>>>
>>>
>>>
>>> On Thu, Sep 2, 2021 at 10:22 AM Kamlesh Joshi 
>>> wrote:
>>>
>>> Hi Yohan,
>>>
>>>
>>>
>>> We have already added this but not as a JVM param but via a code, like
>>> setting in system properties. Still the issue persists.
>>>
>>>
>>>
>>> Any other work around?
>>>
>>>
>>>
>>> *Thanks and Regards,*
>>>
>>> *Kamlesh Joshi*
>>>
>>>
>>>
>>> *From:* Yohan Fernando 
>>> *Sent:* 02 September 2021 14:44
>>> *To:* user@ignite.apache.org
>>> *Subject:* [External]Re: Client taking long time to connect to cluster
>>>
>>>
>>>
>>> The e-mail below is from an external source. Please do not open
>>> attachments or click links from an unknown or suspicious origin.
>>>
>>> We had a similar problem. Try adding -Djava.net.preferIPv4Stack=true to
>>> the VM parameters.
>>>
>>>
>>>
>>> On Wed, Sep 1, 2021 at 3:22 PM Kamlesh Joshi 
>>> wrote:
>>>
>>> Hi Igniters,
>>>
>>>
>>>
>>&g

Re: [External]Re: Client taking long time to connect to cluster

[Gianluca Bonetti]

Hello

I don't know if I can help you, but I'll give it a try, as I had problems
for slow startup time in the past, and I solved them.

Can you share a couple of information about JVM?
Which JVM version are you using on the two different setups?
Which JVM params are you using for them?

Thank you
Cheers
Gianluca

Il giorno gio 2 set 2021 alle ore 12:01 Kamlesh Joshi 
ha scritto:

> Yohan, we have tried with JVM param as well, still its taking long time to
> connect.
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Yohan Fernando 
> *Sent:* 02 September 2021 15:00
> *To:* user@ignite.apache.org
> *Subject:* Re: [External]Re: Client taking long time to connect to cluster
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Seems like that VM arg is only read at startup time of the JVM according
> to this post, so setting it in code may not work
>
>
>
>
> https://stackoverflow.com/questions/9882357/how-to-set-java-net-preferipv4stack-true-at-runtime
>
>
>
> You might want to try it as a VM argument
>
>
>
> On Thu, Sep 2, 2021 at 10:22 AM Kamlesh Joshi 
> wrote:
>
> Hi Yohan,
>
>
>
> We have already added this but not as a JVM param but via a code, like
> setting in system properties. Still the issue persists.
>
>
>
> Any other work around?
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Yohan Fernando 
> *Sent:* 02 September 2021 14:44
> *To:* user@ignite.apache.org
> *Subject:* [External]Re: Client taking long time to connect to cluster
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> We had a similar problem. Try adding -Djava.net.preferIPv4Stack=true to
> the VM parameters.
>
>
>
> On Wed, Sep 1, 2021 at 3:22 PM Kamlesh Joshi 
> wrote:
>
> Hi Igniters,
>
>
>
> We have upgraded OS version from Red Hat 6.10(Santiago) to 7.8(Maipo) in
> Client server. Ignite cluster nodes’ OS are already on 7.8, we just
> upgraded client server OS.
>
>
>
> After upgrading client server, we are facing slow connections with cluster
> almost 5 mints. Sometimes it's not even able to connect to cluster from the
> client server, so had to retry multiple times.
>
> We have another client server with OS version 6.10 from which we are not
> facing such issue.
>
>
>
> *Is there any issue compatibility issue with Red Hat 7.8 version. We are
> using ignite 2.7.6 version.*
>
>
>
> *Please find netstat output below of upgraded client server:*
>
>
>
> Proto Recv-Q Send-Q Local Address   Foreign Address
> State   PID/Program name
>
> (Not all processes could be identified, non-owned process info
>
> will not be shown, you would have to be root to see it all.)
>
> tcp0  0 0.0.0.0:47101
> 
> 0.0.0.0:*   LISTEN  38647/bwengine
>
> tcp0  0 0.0.0.0:10801
> 
> 0.0.0.0:*   LISTEN  38647/bwengine
>
> *tcp   6674800  0 ClientIP:49917  ServerIP:47500
> ESTABLISHED 38647/bwengine*
>
> udp0  0 127.0.0.1:49634
> 
> 127.0.0.1:53
> 
> ESTABLISHED 38647/bwengine
>
> unix  2  [ ] STREAM CONNECTED 142186514 38647/bwengine
>
> unix  3  [ ] STREAM CONNECTED 142148803 38647/bwengine
>
> unix  2  [ ] STREAM CONNECTED 142053176 38647/bwengine
>
> unix  3  [ ] STREAM CONNECTED 142112733 38647/bwengine
>
> unix  3  [ ] STREAM CONNECTED 142148802 38647/bwengine
>
> unix  3  [ ] STREAM CONNECTED 142112732 38647/bwengine
>
>
>
> Any pointers on this would be helpful.
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the 

Re: Slow startup of server node and query

[Gianluca Bonetti]

Hi there

Il giorno dom 4 apr 2021 alle ore 21:39 DonTequila  ha
scritto:

> Hi,
>
> thanks for your suggestions. Unfortunately this didn't help improving the
> startup time.
>

I only had such slow startup times because of the two problems I described
before.

I had a look at your logs and there's no glaring problem, at least to my
eyes.
Only a couple of things I spotted...

You're using JVM 11 but you didn't add the required VM arguments.
https://ignite.apache.org/docs/latest/quick-start/java#running-ignite-with-java-11-or-later

You're also running an old Ignite version, please consider updating to the
latest.

Just a side note, how much data is stored in your caches?
Especially MediaFan_Jobs and MediaFan_MediaSetStatusHistory

Cheers
Gianluca

Re: Slow startup of server node and query

[Gianluca Bonetti]

Hello

It happened to me on a few occasions.
I have a couple of suggestions which may, or may not, solve your problem.
Please try :)

First, please ensure you use /dev/urandom as an entropy source.
/dev/random does a better job, but may cause long pauses (10 minutes or
longer may happen easier than expected) and /dev/urandom is good enough for
most cases.

Second, please ensure not to use -XX:+AlwaysPreTouch in your JVM flags as
zeroing large heaps on initialization may take a very long time.

Cheers
Gianluca

Il giorno mer 31 mar 2021 alle ore 21:12 DonTequila  ha
scritto:

> Has anyone been able to look at this and give me some advise? Thanks!
>
>
> DonTequila wrote
> > Hi,
> >
> > I'm seeing a pretty slow Ignite server node startup time, of approx
> > 20mins, with persistence enabled for most caches. Of course I can see
> > startup-time highly depends on disk speed, but I wonder if there are
> > settings that could improve this?
> >
> > I've attached a server log of the time when it starts. Also the Ignite
> > configs attached.
> >
> > Also, when first accessed from a client with SQL there's a long running
> > query which shouldn't be the case in my opinion for binary storage and
> > indexes?
> >
> > Can you spot anything that is wrong with my system startup time?
> >
> > Thanks,
> > Thomas.
> >
> >
> >
> >
> > server.log (102K)
> > 
> http://apache-ignite-users.70518.x6.nabble.com/attachment/35676/0/server.log
> ;
> > ignite-caches-config.xml (39K)
> > 
> http://apache-ignite-users.70518.x6.nabble.com/attachment/35676/1/ignite-caches-config.xml
> ;
> > ignite-caches-dataregions.xml (2K)
> > 
> http://apache-ignite-users.70518.x6.nabble.com/attachment/35676/2/ignite-caches-dataregions.xml
> ;
> > ignite-mediafan-db.xml (4K)
> > 
> http://apache-ignite-users.70518.x6.nabble.com/attachment/35676/3/ignite-mediafan-db.xml
> ;
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite Restart takes long time

[Gianluca Bonetti]

Hello Krumar

Full explanation about the slowness when using AlwaysPreTouch comes from
this article https://access.redhat.com/solutions/2685771

Is it still taking 5 minutes to start, after removing the AlwaysPreTouch
parameter?

Cheers
Gianluca

Il giorno mar 7 apr 2020 alle ore 18:20 krkumar24061...@gmail.com <
krkumar24061...@gmail.com> ha scritto:

> Hi Gianluca Bonetti - Thanks for the help. I actually have that JVM
> parameter
> and now I have removed that. Looks like its working and I will do few more
> rounds of testing and then  update you.
>
> Also any idea why ignite.active(true) taking almost 5 minutes whenever i
> restart the node?
>
> Again a big thanks,
>
> KR Kumar
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite Restart takes long time

[Gianluca Bonetti]

Hello Kumar

It seems a problem similar to the one I had in the past.

Please check that you are running the JVM *without* the
"-XX:+AlwaysPreTouch" parameter.

Also, in my experience, using /dev/random as entropy source could block
startup for long time, hence I prefer using /dev/urandom
(set securerandom.source=file:/dev/random in
/etc/java-8-openjdk/security/java.security for Debian-like systems)

Cheers
Gianluca

Il giorno mar 7 apr 2020 alle ore 14:33 krkumar24061...@gmail.com <
krkumar24061...@gmail.com> ha scritto:

> Hi Guys - I have this problem that's very recent and nothing significant
> changed in the system but all of a sudden still takes lot of time to
> restart. I do a graceful shutdown of one of the nodes and restart the node
> after some updates. Now the ignite takes almost 30 minutes to initiate and
> that's killing us.
>
> Here is the snippet of my ignite configuration:
>
>  depends-on="dataSource">
> 
>  class="org.apache.ignite.configuration.IgniteConfiguration">
> 
> 
>  class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
>  name="connectTimeout" value="1">
>  name="socketWriteTimeout" value="3">
> 
> 
>  value="12">
>  value="8">
>  value="64" />
>  value="32" />
> 
>  class="org.apache.ignite.configuration.DataStorageConfiguration">
>  name="writeThrottlingEnabled" value="true">
>  name="defaultDataRegionConfiguration">
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
>  name="initialSize" value="#{512L *  1024 *
> 1024}">
>  name="maxSize" value="#{20L * 1024 * 1024 * 1024}" />
>  name="persistenceEnabled" value="true" />
> 
> 
>  name="dataRegionConfigurations">
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
>
> 
>
> 
>
> 
>
> 
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
>
> 
>
> 
>
> 
>
> 
> 
> 
> 
>  value="#{4 * 1024}" />
>  name="storagePath" value="${grid.data}" />
>  value="${grid.wal}" />
>  name="walArchivePath" value="${grid.wal}" />
>  value="BACKGROUND" />
>  name="walFlushFrequency" value="1">
> 
> 
>
> Here is the code snippet of how we create the cache
>
> CacheConfiguration cacheConfig = new
> CacheConfiguration<>();
> cacheConfig.setCacheMode(CacheMode.PARTITIONED);
> cacheConfig.setRebalanceMode(CacheRebalanceMode.ASYNC);
> cacheConfig.setRebalanceDelay(6);
> cacheConfig.setName("eventCache-" + xxx + "-" + xxx);
> cacheConfig.setBackups(1);
> cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);
>
>
> cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
>
> We start the ignite like the following
>
> context = new ClassPathXmlApplicationContext("xxx.xml");
>
> logger.info("Cluster kernel is initializing ...Done");
> engine = (Ignite) context.getBean("igniteContainer");
> engine.cluster().active(true);
>
>
> Thanx and Regards,
> KR Kumar
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Cannot connect to Ignite

[Gianluca Bonetti]

Hello Stéphane

GCP should allow internal traffic using the internal IP address (static or
not), usually a 10.xxx.xxx.xxx, but should deny connections on special
ports using the external IP address (static or not).
If you need to use an external IP address you have to add a firewall rule,
but I think it would be fixed by using the internal IP addresses.

Best regards
Gianluca Bonetti

Il giorno dom 1 mar 2020 alle ore 08:39 Stéphane Thibaud <
snthib...@gmail.com> ha scritto:

> Hello Ignite users,,
>
> I suddenly am unable to connect to the Ignite instance. I am running it on
> GCP with Container OS/Docker. I checked by SSH-ing into the instance and
> running netstat that port 10800 is listening:
>
> Proto Recv-Q Send-Q Local Address   Foreign Address State
>   PID/Program name
> ...
> tcp0  0 0.0.0.0:10800   0.0.0.0:*
>   LISTEN  -
> I have allocated a static IP to this instance and connecting to it
> previously worked (firewall rule should allow internal GCP connections).
> Now I get 'timeout' error even though I can ping the machine without
> problem.
>
> Any ideas for additional checks?
>
>
> Kind regards,
>
> Stéphane Thibaud
>

Re: Version Mismatch Check

[Gianluca Bonetti]

Hello

For what I know, Apache Ignite does not allow different versions to run on
the same cluster.
I think that GridGain commercial version has support for running nodes with
different versions on same cluster.

Cheers
Gianluca

Il giorno ven 5 lug 2019 alle ore 08:51 Ankit Batra 
ha scritto:

> Hi,
>
> I am facing an issue with Ignite's version mismatch check. I have a
> deployment where 3 micro-service instances also run as Ignite server nodes.
> The process to update these microservices is an automated one, where 1
> microservice is rebooted with a newer build at one time and until that
> microservice comes up, another microservice does not reboot with a newer
> build for the upgrade to complete across all the 3 instances.
>
> This automated process of deployment exposes an issue when I bump up the
> Ignite version for the micro-services and publish a newer build. When the
> first microservice reboots with a newer version of Apache-Ignite (while the
> other microservices are still running an older version) it throws a version
> mismatch error and never comes up.
>
> I am not sure if Ignite allows skipping this version mismatch check.
> Looking for some guidance on how this can be solved?
>
> Regards
> Ankit Batra
>

Re: starting ignite in docker

[Gianluca Bonetti]

Hello
I was able to run the software, with the exact docker command line that you
wrote.
I just used a stock configuration file (config/default-config.xml file from
the Ignite 2.7.0 tarball), copied in $PWD as ignite-config.xml (as called
in your example)
Please double check to have the proper configuration file in $PWD, and the
name of the file itself.
Best regards
Gianluca Bonetti

Il giorno mer 3 apr 2019 alle ore 14:31 Coleman, JohnSteven (Agoda) <
johnsteven.cole...@agoda.com> ha scritto:

> I’d like to pass a local configuration file to docker as below, but ignite
> doesn’t find it, any ideas? The file is in the PWD directory. Using windows
> powershell.
>
>
>
> *docker *run *-it *--rm --net=host *-v *${PWD}*:/apache-ignite/config **-e
> **"CONFIG_URI=file:///apache-ignite/config/ignite-config.xml" *--name
> ignite apacheignite/ignite
>
>
>
>
>
> /opt/ignite/apache-ignite/bin/ignite.sh, WARN: Failed to resolve JMX host
> (JMX will be disabled): linuxkit-00155d0f670e
>
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context
> [springUrl=file:/apache-ignite/config/ignite-config.xml, err=IOException
> parsing XML document from URL
> [file:/apache-ignite/config/ignite-config.xml]; nested exception is
> java.io.FileNotFoundException: /apache-ignite/config/ignite-config.xml (No
> such file or directory)]
>
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1025)
>
> at org.apache.ignite.Ignition.start(Ignition.java:351)
>
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301)
>
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> instantiate Spring XML application context
> [springUrl=file:/apache-ignite/config/ignite-config.xml, err=IOException
> parsing XML document from URL
> [file:/apache-ignite/config/ignite-config.xml]; nested exception is
> java.io.FileNotFoundException: /apache-ignite/config/ignite-config.xml (No
> such file or directory)]
>
> --
> This message is confidential and is for the sole use of the intended
> recipient(s). It may also be privileged or otherwise protected by copyright
> or other legal rules. If you have received it by mistake please let us know
> by reply email and delete it from your system. It is prohibited to copy
> this message or disclose its content to anyone. Any confidentiality or
> privilege is not waived or lost by any mistaken delivery or unauthorized
> disclosure of the message. All messages sent to and from Agoda may be
> monitored to ensure compliance with company policies, to protect the
> company's interests and to remove potential malware. Electronic messages
> may be intercepted, amended, lost or deleted, or contain viruses.
>

Re: When will Apache Ignite support Java 11?

[Gianluca Bonetti]

Hello Loredana

You can switch to OpenJDK 8, which would be supported also for security
updates by all the major Linux distribution.
OpenJDK is production ready since years, actually Oracle's JDK is now a
build of OpenJDK, with some additions.

Cheers
Gianluca

Il giorno lun 3 dic 2018 alle ore 21:17 Stanislav Lukyanov <
stanlukya...@gmail.com> ha scritto:

> The support for Java 9 **should** mean support for Java 11, the
> compatibility gap between the two is not big.
>
>
>
> Moreover, I would (and going to) push for almost completely skipping the
> testing on Java 9 – it is
>
> in end-of-life already, so providing support for it is kind of pointless.
> Java 11 is what should be supported by Ignite 2.8.
>
>
>
> That said, I honestly don’t see everyone jumping from the Java 8 train any
> time soon.
>
> Gap between 8 and 9+ (although not to big in reality) still makes people
> stay on 8,
>
> and Oracle’s competitors are ready to offer alternative support.
>
> So, I’d say that Java 8 is still going to be the main target for at least
> Ignite 2.8.
>
>
>
> But this is all just a speculation for now as no plans were set in stone
> yet.
>
> Stay tuned at d...@ignite.apache.org.
>
>
>
> Stan
>
>
>
> *From: *Loredana Radulescu Ivanoff 
> *Sent: *26 ноября 2018 г. 21:23
> *To: *user@ignite.apache.org
> *Subject: *Re: When will Apache Ignite support Java 11?
>
>
>
> Hello,
>
>
>
> The current plan is that Oracle will stop updates for Java 8 commercial
> users after January 2019, and Java 11 is the next LTS release, so is there
> a plan to have Ignite working with Java 11 by then?
>
>
>
> Thank you.
>
>
>
> On Thu, Nov 22, 2018 at 10:49 PM Petr Ivanov  wrote:
>
> Hi!
>
>
> Full Java 9+ support is planned for 2.8 at least.
> Currently it will work more or less on Java9. Java10/11 work is not
> guaranteed.
>
> > On 22 Nov 2018, at 21:22, monstereo  wrote:
> >
> > Is there any plan to support Java 11 for Apache Ignite?
> >
> > If the next version of the Apache Ignite (2.7) will support Java 11,
> when it
> > will be released?
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>

Re: Long activation times with Ignite persistence enabled

[Gianluca Bonetti]

Hello

In my case of slow startup, as suggested from a member of this mailing
list, I deleted the -XX:+AlwaysPreTouch command line option from JVM
launch, and the cluster got back to very fast startup.
Don't know if you are using this option, hope it helps.

Cheers
Gianluca

Il giorno lun 5 nov 2018 alle ore 14:10 Naveen 
ha scritto:

> Hi Denis
>
> We have only 40 caches in our cluster.
> If we introduce grouping of caches, guess we need to reload the data from
> scratch, right ??
>
> We do have very powerful machines as part of cluster, they are 128 CPU very
> high end boxes and huge resources available, by increasing any of the below
> thread pools, can we reduce the cluster activation time.
>
> System Pool
> Public Pool
> Striped Pool
> Custom Thread Pools
>
> Thanks
> Naveen
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Cluster freeze with SSL enabled and JDK 11

[Gianluca Bonetti]

Hello

I think that the only supported version is Java 8 right now, and Java 11
support should come with 2.7 or 2.8 release.
https://issues.apache.org/jira/browse/IGNITE-8441

Cheers
Gianluca

Il giorno gio 18 ott 2018 alle ore 23:56 Loredana Radulescu Ivanoff <
lradu...@tibco.com> ha scritto:

> Hello,
>
> I can consistently reproduce this issue with Ignite 2.6.0, JDK 11 and SSL
> enabled:
>
>
>- the second node that I bring up joins, and then shortly after
>freezes and prints this message every minute:
>
> "WARN ...[*Initialization*]
> processors.cache.GridCachePartitionExchangeManager: Still waiting for
> initial partition map exchange"
>
>
>- once the second node joins, the first node starts experiencing very
>frequent 100% CPU spikes; these are the messages I see:
>
> WARN 2018-10-18T13:50:52,728-0700 []
> communication.tcp.TcpCommunicationSpi: Communication SPI session write
> timed out (consider increasing 'socketWriteTimeout' configuration property)
> [remoteAddr=/10.100.36.82:51620, writeTimeout=15000]
> WARN 2018-10-18T13:50:52,737-0700 []
> communication.tcp.TcpCommunicationSpi: Failed to shutdown SSL session
> gracefully (will force close) [ex=javax.net.ssl.SSLException: Incorrect SSL
> engine status after closeOutbound call [status=OK,
> handshakeStatus=NEED_WRAP,
> WARN 2018-10-18T13:51:01,441-0700 []
> dht.preloader.GridDhtPartitionsExchangeFuture: Unable to await partitions
> release latch within timeout: ServerLatch [permits=1,
> pendingAcks=[aeba8bb7-c9b8-4d46-be8a-df361eaa8fc5], super=CompletableLatch
> [id=exchange, topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0]]]
>
> Other observations:
>
> I can reproduce this every time I start the nodes, and it doesn't matter
> which node comes up first.
>
>
> The issue goes away if I disable SSL.
>
>
> Increasing the socketWriteTimeout, networkTimeout or the
> failureDetectionTimeout does not help.
>
> It seems to be happening only with JDK 11, and not with JDK 8.
>
>
> Do you have any suggestions/known issues about this?
>
> Thank you,
>
> Loredana
>
>
>
>
>

Re: Troubles in restarting Ignite with persistence enabled

[Gianluca Bonetti]

Hello everyone

I have made some tests following Hamed Zahedifar suggestion about the
-XX:+AlwaysPreTouch and pointing to the RedHat thread.
By now, I simply run the tests cutting off -XX:+AlwaysPreTouch from JVM
startup command.
It starts back exaggeratedly faster than before.
With the same amount of data, around 4 GB, it starts in less than 15
seconds, while with -XX:+AlwaysPreTouch flag, it won't finish startup even
in 20 minutes.
By now I feel completely satisfied with the performance.
Thanks everyone for the support, and expecially Hamed for pointing out
about the flag behaviour.

Cheers
Gianluca

Il giorno mar 2 ott 2018 alle ore 10:31 Gianluca Bonetti <
gianluca.bone...@gmail.com> ha scritto:

> Hello everyone
>
> This is my first question to the mailing list, which I follow since some
> time, to get hints about using Ignite.
> Until now I used in other softwares development, and Ignite always rocked
> and made the difference, hence I literally love it :)
>
> Now I am facing troubles in restarting an Apache Ignite instance on a new
> product we are developing and testing.
> Previously, I have been developing using Apache Ignite with custom loader
> from database, but this time we wanted to go with a "cache centric"
> approach and use only Ignite Persistence, as there is no need of
> integrating with databases or JDBC tools.
> So Ignite Instance is the main and only storage.
>
> The software is a monitoring platform, which receives small chunks of data
> (more or less 500 bytes) and stores in different caches, depending on the
> source address.
> The number of incoming data packets is really low as we are only in
> testing, let's say around 100 packes per minute.
> The software is running in testing enviroment, so only one server is
> deployed at the moment.
>
> The software can run for weeks with no problem, the caches get bigger and
> bigger and everything runs fine and fast.
> Then if we restart the software, it takes ages to restart, and actually
> most of the times it does not ever complete the initial restart of Ignite.
> So we have to delete the persistence storage files, to be able to start
> again.
> As we are only in testing, we can still withstand it.
>
> We get just a message in the logs: "Ignite node stopped in the middle of
> checkpoint. Will restore memory state and finish checkpoint on node start."
> The client instances connecting to Ignite gets the log: "
> org.apache.ignite.logger.java.JavaLogger.info Join cluster while cluster
> state transition is in progress, waiting when transition finish."
> But it never finishes.
>
> Speaking of sizes, when running tests with no interruption, the cache grew
> up to 50 GBs, with no degradation in performance or data loss.
> The issues with restarting start just when the cache grows up to ~4 GBs.
> The other softwares I developed using Ignite, with custom database loader,
> never had problems with large caches in memory.
>
> The testing server is a dedicated Linux machine with 8 cores Xeon
> processor, 64 GB RAM, and SATA disks on software mdraid.
> The JVM is OpenJDK 8, started with "-server -Xms24g -Xmx24g
> -XX:MaxMetaspaceSize=1g -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AggressiveOpts"
>
> For starting Ignite instance, I am one (the last?) which prefers Java code
> instead of XML files.
> I recently switched off PeerClassLoading and added the
> BinaryTypeConfiguration, which previosly I hadn't specified, but didn't
> help.
>
> public static final Ignite newInstance(List remotes) {
> DataStorageConfiguration storage = new DataStorageConfiguration();
> DataRegionConfiguration region =
> storage.getDefaultDataRegionConfiguration();
> BinaryConfiguration binary = new BinaryConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder();
> TcpDiscoverySpi discovery = new TcpDiscoverySpi();
> IgniteConfiguration config = new IgniteConfiguration();
> storage.setStoragePath("/home/ignite/data");
> storage.setWalPath("/home/ignite/wal");
> storage.setWalArchivePath("/home/ignite/archive");
> region.setPersistenceEnabled(true);
> region.setInitialSize(16L * 1024 * 1024 * 1024);
> region.setMaxSize(16L * 1024 * 1024 * 1024);
> binary.setCompactFooter(false);
> binary.setTypeConfigurations(Arrays.asList(new
> BinaryTypeConfiguration(Datum.class.getCanonicalName(;
> finder.setAddresses(remotes);
> discovery.setIpFinder(finder);
> config.setDataStorageConfiguration(storage);
> config.setBinaryConfiguration(binary);
> config.setPeerClassLoadingEnabled(false);
> config.setDiscoverySpi(discovery);
> config.setClientMode(false);
> Ignite ignite = Ignition.start(config);

Re: Troubles in restarting Ignite with persistence enabled

[Gianluca Bonetti]

Hello Mikael
Yes we have more caches, but they are smaller, and the number of incoming
data packets are is almost nothing compared to yours (100 per minute
compared to 10.000 per second!)
The time for restarting excedes 30 minutes, so it is too much and I always
kill the process at that point (as it is only test)
So when the cache grows up, I have never been able to complete a full
restart.
2 minutes would be very positive result for me, looking forward to achieve
that result!
Thank you for the information and the feedback on config through Java code!
Have nothing against XML configs, I use in many enviroments, but
specifically for Ignite, it seems more readable to my eyes.
Thank you
Cheers
Gianluca

Il giorno mar 2 ott 2018 alle ore 11:02 Mikael 
ha scritto:

> I use persistent storage but not so many caches and so far no problems, I
> have 20 caches with around a total of 800.000 entries, each entry is small
> (<100 bytes), while testing I update around 5000-1 entries per second,
> restarts do take some time but never more than 2 minutes or so, so far only
> tested with one node, I have not run for that long time yet though, around
> 5 days at the time so far.
>
> Oh, you are not the last one by the way, I do all configuration of Ignite
> in Java ;)
>
> Mikael
>
> Den 2018-10-02 kl. 10:50, skrev Hamed Zahedifar:
>
> Hi Gianluca
>
> Maybe it`s for JVM -XX:+AlwaysPreTouch Switch. try redhat workaround for
> this problem, i hope it will start faster.
> Java process takes a long time with -XX:+AlwaysPreTouch - Red Hat Customer
> Portal <https://access.redhat.com/solutions/2685771>
>
>
> Java process takes a long time with -XX:+AlwaysPreTouch - Red Hat Custom...
> <https://access.redhat.com/solutions/2685771>
>
>
> ----
> Hamed
>
>
>
>
>
>
> On Tuesday, October 2, 2018, 12:01:38 PM GMT+3:30, Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>
>
>
>
> Hello everyone
>
> This is my first question to the mailing list, which I follow since some
> time, to get hints about using Ignite.
> Until now I used in other softwares development, and Ignite always rocked
> and made the difference, hence I literally love it :)
>
> Now I am facing troubles in restarting an Apache Ignite instance on a new
> product we are developing and testing.
> Previously, I have been developing using Apache Ignite with custom loader
> from database, but this time we wanted to go with a "cache centric"
> approach and use only Ignite Persistence, as there is no need of
> integrating with databases or JDBC tools.
> So Ignite Instance is the main and only storage.
>
> The software is a monitoring platform, which receives small chunks of data
> (more or less 500 bytes) and stores in different caches, depending on the
> source address.
> The number of incoming data packets is really low as we are only in
> testing, let's say around 100 packes per minute.
> The software is running in testing enviroment, so only one server is
> deployed at the moment.
>
> The software can run for weeks with no problem, the caches get bigger and
> bigger and everything runs fine and fast.
> Then if we restart the software, it takes ages to restart, and actually
> most of the times it does not ever complete the initial restart of Ignite.
> So we have to delete the persistence storage files, to be able to start
> again.
> As we are only in testing, we can still withstand it.
>
> We get just a message in the logs: "Ignite node stopped in the middle of
> checkpoint. Will restore memory state and finish checkpoint on node start."
> The client instances connecting to Ignite gets the log: "
> org.apache.ignite.logger.java.JavaLogger.info Join cluster while cluster
> state transition is in progress, waiting when transition finish."
> But it never finishes.
>
> Speaking of sizes, when running tests with no interruption, the cache grew
> up to 50 GBs, with no degradation in performance or data loss.
> The issues with restarting start just when the cache grows up to ~4 GBs.
> The other softwares I developed using Ignite, with custom database loader,
> never had problems with large caches in memory.
>
> The testing server is a dedicated Linux machine with 8 cores Xeon
> processor, 64 GB RAM, and SATA disks on software mdraid.
> The JVM is OpenJDK 8, started with "-server -Xms24g -Xmx24g
> -XX:MaxMetaspaceSize=1g -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AggressiveOpts"
>
> For starting Ignite instance, I am one (the last?) which prefers Java code
> instead of XML files.
> I recently switched off PeerClassLoading and added the
> BinaryTypeConfiguration, which previosl

Re: Troubles in restarting Ignite with persistence enabled

[Gianluca Bonetti]

Hello Hamed
For a dirty and quick test, I have removed the -XX:+AlwaysPreTouch
It actually is faster, but I will see how is performing going on with more
data.
By now, it seems a good hint!
Thank you!
Cheers
Gianluca

Il giorno mar 2 ott 2018 alle ore 10:51 Hamed Zahedifar <
h.zahedi...@yahoo.com> ha scritto:

> Hi Gianluca
>
> Maybe it`s for JVM -XX:+AlwaysPreTouch Switch. try redhat workaround for
> this problem, i hope it will start faster.
> Java process takes a long time with -XX:+AlwaysPreTouch - Red Hat Customer
> Portal <https://access.redhat.com/solutions/2685771>
>
> Java process takes a long time with -XX:+AlwaysPreTouch - Red Hat Custom...
>
> <https://access.redhat.com/solutions/2685771>
>
>
> 
> Hamed
>
>
>
>
>
>
> On Tuesday, October 2, 2018, 12:01:38 PM GMT+3:30, Gianluca Bonetti <
> gianluca.bone...@gmail.com> wrote:
>
>
>
>
>
> Hello everyone
>
> This is my first question to the mailing list, which I follow since some
> time, to get hints about using Ignite.
> Until now I used in other softwares development, and Ignite always rocked
> and made the difference, hence I literally love it :)
>
> Now I am facing troubles in restarting an Apache Ignite instance on a new
> product we are developing and testing.
> Previously, I have been developing using Apache Ignite with custom loader
> from database, but this time we wanted to go with a "cache centric"
> approach and use only Ignite Persistence, as there is no need of
> integrating with databases or JDBC tools.
> So Ignite Instance is the main and only storage.
>
> The software is a monitoring platform, which receives small chunks of data
> (more or less 500 bytes) and stores in different caches, depending on the
> source address.
> The number of incoming data packets is really low as we are only in
> testing, let's say around 100 packes per minute.
> The software is running in testing enviroment, so only one server is
> deployed at the moment.
>
> The software can run for weeks with no problem, the caches get bigger and
> bigger and everything runs fine and fast.
> Then if we restart the software, it takes ages to restart, and actually
> most of the times it does not ever complete the initial restart of Ignite.
> So we have to delete the persistence storage files, to be able to start
> again.
> As we are only in testing, we can still withstand it.
>
> We get just a message in the logs: "Ignite node stopped in the middle of
> checkpoint. Will restore memory state and finish checkpoint on node start."
> The client instances connecting to Ignite gets the log: "
> org.apache.ignite.logger.java.JavaLogger.info Join cluster while cluster
> state transition is in progress, waiting when transition finish."
> But it never finishes.
>
> Speaking of sizes, when running tests with no interruption, the cache grew
> up to 50 GBs, with no degradation in performance or data loss.
> The issues with restarting start just when the cache grows up to ~4 GBs.
> The other softwares I developed using Ignite, with custom database loader,
> never had problems with large caches in memory.
>
> The testing server is a dedicated Linux machine with 8 cores Xeon
> processor, 64 GB RAM, and SATA disks on software mdraid.
> The JVM is OpenJDK 8, started with "-server -Xms24g -Xmx24g
> -XX:MaxMetaspaceSize=1g -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AggressiveOpts"
>
> For starting Ignite instance, I am one (the last?) which prefers Java code
> instead of XML files.
> I recently switched off PeerClassLoading and added the
> BinaryTypeConfiguration, which previosly I hadn't specified, but didn't
> help.
>
> public static final Ignite newInstance(List remotes) {
> DataStorageConfiguration storage = new DataStorageConfiguration();
> DataRegionConfiguration region =
> storage.getDefaultDataRegionConfiguration();
> BinaryConfiguration binary = new BinaryConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder();
> TcpDiscoverySpi discovery = new TcpDiscoverySpi();
> IgniteConfiguration config = new IgniteConfiguration();
>
> storage.setStoragePath("/home/ignite/data");
> storage.setWalPath("/home/ignite/wal");
> storage.setWalArchivePath("/home/ignite/archive");
>
> region.setPersistenceEnabled(true);
> region.setInitialSize(16L * 1024 * 1024 * 1024);
> region.setMaxSize(16L * 1024 * 1024 * 1024);
>
> binary.setCompactFooter(false);
> binary.setTypeConfigurations(Arrays.asList(new
> BinaryTypeConfiguration(Datum.class.getCanonicalName(;
>
> finder.setAddresses(remotes);
>
> discovery.s

Troubles in restarting Ignite with persistence enabled

[Gianluca Bonetti]

Hello everyone

This is my first question to the mailing list, which I follow since some
time, to get hints about using Ignite.
Until now I used in other softwares development, and Ignite always rocked
and made the difference, hence I literally love it :)

Now I am facing troubles in restarting an Apache Ignite instance on a new
product we are developing and testing.
Previously, I have been developing using Apache Ignite with custom loader
from database, but this time we wanted to go with a "cache centric"
approach and use only Ignite Persistence, as there is no need of
integrating with databases or JDBC tools.
So Ignite Instance is the main and only storage.

The software is a monitoring platform, which receives small chunks of data
(more or less 500 bytes) and stores in different caches, depending on the
source address.
The number of incoming data packets is really low as we are only in
testing, let's say around 100 packes per minute.
The software is running in testing enviroment, so only one server is
deployed at the moment.

The software can run for weeks with no problem, the caches get bigger and
bigger and everything runs fine and fast.
Then if we restart the software, it takes ages to restart, and actually
most of the times it does not ever complete the initial restart of Ignite.
So we have to delete the persistence storage files, to be able to start
again.
As we are only in testing, we can still withstand it.

We get just a message in the logs: "Ignite node stopped in the middle of
checkpoint. Will restore memory state and finish checkpoint on node start."
The client instances connecting to Ignite gets the log: "
org.apache.ignite.logger.java.JavaLogger.info Join cluster while cluster
state transition is in progress, waiting when transition finish."
But it never finishes.

Speaking of sizes, when running tests with no interruption, the cache grew
up to 50 GBs, with no degradation in performance or data loss.
The issues with restarting start just when the cache grows up to ~4 GBs.
The other softwares I developed using Ignite, with custom database loader,
never had problems with large caches in memory.

The testing server is a dedicated Linux machine with 8 cores Xeon
processor, 64 GB RAM, and SATA disks on software mdraid.
The JVM is OpenJDK 8, started with "-server -Xms24g -Xmx24g
-XX:MaxMetaspaceSize=1g -XX:+AlwaysPreTouch -XX:+UseG1GC
-XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AggressiveOpts"

For starting Ignite instance, I am one (the last?) which prefers Java code
instead of XML files.
I recently switched off PeerClassLoading and added the
BinaryTypeConfiguration, which previosly I hadn't specified, but didn't
help.

public static final Ignite newInstance(List remotes) {
DataStorageConfiguration storage = new DataStorageConfiguration();
DataRegionConfiguration region =
storage.getDefaultDataRegionConfiguration();
BinaryConfiguration binary = new BinaryConfiguration();
TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder();
TcpDiscoverySpi discovery = new TcpDiscoverySpi();
IgniteConfiguration config = new IgniteConfiguration();
storage.setStoragePath("/home/ignite/data");
storage.setWalPath("/home/ignite/wal");
storage.setWalArchivePath("/home/ignite/archive");
region.setPersistenceEnabled(true);
region.setInitialSize(16L * 1024 * 1024 * 1024);
region.setMaxSize(16L * 1024 * 1024 * 1024);
binary.setCompactFooter(false);
binary.setTypeConfigurations(Arrays.asList(new
BinaryTypeConfiguration(Datum.class.getCanonicalName(;
finder.setAddresses(remotes);
discovery.setIpFinder(finder);
config.setDataStorageConfiguration(storage);
config.setBinaryConfiguration(binary);
config.setPeerClassLoadingEnabled(false);
config.setDiscoverySpi(discovery);
config.setClientMode(false);
Ignite ignite = Ignition.start(config);
ignite.cluster().active(true);
return ignite;
}

Datum is a small POJO class, with nearly 100 fields and should be less than
500 bytes of data.
Then there are nearly 200 caches in use, all containing Datum objects (at
least for now).

I am quite sure I am missing something when starting the instance, but
cannot understand what.

Is there a way to inspect the progress of the checkpoint at startup?
I cannot do anything by Ignite Visor as it would not connect until the
cluster activation finishes.

If you have any suggestions, let me know.

Thank you very much!
Best regards
Gianluca