Hi Alex, there were no other relevant logs than what I already listed in the first email.
<beans xmlns="http://www.springframework.org/schema/beans"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd";> <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"> <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> <property name="addresses"> <list> <value>10.24.51.190</value> <value>10.24.51.187</value> <value>10.24.51.150</value> </list> </property> </bean> </property> </bean> </property> <property name="persistentStoreConfiguration"> <bean class="org.apache.ignite.configuration.PersistentStoreConfiguration"/> </property> <property name="cacheConfiguration"> <list> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <property name="name" value="FcPortStatsCache"/> <property name="cacheMode" value="PARTITIONED"/> <property name="backups" value="1"/> <property name="queryEntities"> <list> <bean class="org.apache.ignite.cache.QueryEntity"> <property name="keyType" value="com.abc.poc.inppoc.model.FcPortStatsKey"/> <property name="valueType" value="com.abc.poc.inppoc.model.FcPortStats"/> <property name="fields"> <map> <entry key="dateTime" value="java.util.Date"/> <entry key="portId" value="java.util.UUID"/> <entry key="portType" value="java.lang.Integer"/> <entry key="switchId" value="java.util.UUID"/> <entry key="fabricId" value="java.util.UUID"/> <entry key="rxUtil" value="java.lang.Float"/> <entry key="txUtil" value="java.lang.Float"/> <entry key="higherUtil" value="java.lang.Float"/> <entry key="lowerUtil" value="java.lang.Float"/> <entry key="rxRate" value="java.lang.Float"/> <entry key="txRate" value="java.lang.Float"/> <entry key="higherRate" value="java.lang.Float"/> <entry key="lowerRate" value="java.lang.Float"/> <entry key="c3Discards" value="java.lang.Float"/> <entry key="crcErrors" value="java.lang.Float"/> </map> </property> <property name="indexes"> <list> <bean class="org.apache.ignite.cache.QueryIndex"> <constructor-arg> <list> <value>dateTime</value> <value>portId</value> </list> </constructor-arg> <constructor-arg value="SORTED"/> </bean> <bean class="org.apache.ignite.cache.QueryIndex"> <constructor-arg> <list> <value>portId</value> <value>dateTime</value> </list> </constructor-arg> <constructor-arg value="SORTED"/> </bean> <bean class="org.apache.ignite.cache.QueryIndex"> <constructor-arg> <list> <value>switchId</value> <value>dateTime</value> </list> </constructor-arg> <constructor-arg value="SORTED"/> </bean> </list> </property> </bean> </list> </property> </bean> </list> </property> </bean> </beans> All 3 servers (and the client) are on VMs on the same host. No network hop latency. But all 3 VMs use the same physical disk (on the host). Servers have 16 GB of RAM. Data on disk (work/db) was about 35 GB per mode. About 36M objects. Please also note http://apache-ignite-users.70518.x6.nabble.com/Strange-problems-with-Ignite-native-Persistence-when-Data-exceeds-Memory-td16187.html. There were some odd problems at time that may have affected the activation. Roger From: afedotov [mailto:alexander.fedot...@gmail.com] Sent: Monday, August 14, 2017 11:05 AM To: user@ignite.apache.org Subject: Re: Activation: slow and: Ignite node crashed in the middle of checkpoint. Hi, Could you please share the logs and configuration? Actually, the activation time depends on the amount of data, network connectivity and other variables. Kind regards, Alex. On Sat, Aug 12, 2017 at 12:39 AM, Roger Fischer (CW) [via Apache Ignite Users] <[hidden email]</user/SendEmail.jtp?type=node&node=16176&i=0>> wrote: Hello, I am wondering if the following behavior is typical, or if it represents a concern. I have a 3 node cluster with native persistence. Each node as 4 CPU and 16 GB of RAM. Each node has ~45 GB used in work/db. Total across the 3 nodes is about 36.5 M objects. I am using SQL queries, and there are 3 indexes. The servers start up normally and join the cluster, as expected. When I start the client, which calls active(), all 3 servers report the following: [12:41:28] Topology snapshot [ver=5, servers=3, clients=1, CPUs=16, heap=4.8GB] [12:41:29] Default checkpoint page buffer size is too small, setting to an adjusted value: 2.0 GiB [12:41:29] Ignite node crashed in the middle of checkpoint. Will restore memory state and enforce checkpoint on node start. 1) Should I worry about the “crashed” log? The activation takes more than 30 minutes (until active() returns). 2) Is that normal for activate to take that long? ver. 2.1.0#20170720-sha1:a6ca5c8a OS: Linux 3.10.0-514.el7.x86_64 amd64 Thanks… Roger ________________________________ If you reply to this email, your message will be added to the discussion below: http://apache-ignite-users.70518.x6.nabble.com/Activation-slow-and-Ignite-node-crashed-in-the-middle-of-checkpoint-tp16144.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_Activation-2Dslow-2Dand-2DIgnite-2Dnode-2Dcrashed-2Din-2Dthe-2Dmiddle-2Dof-2Dcheckpoint-2Dtp16144.html&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=qsKTpUnWgB48OZA1huxDdJW-LyQpHaNdTlxMrCCW-Xk&s=DO1_9o_iSHBJutWMps2XTrpB04xC7ifpFgWgH-GCQKI&e=> To start a new topic under Apache Ignite Users, email [hidden email]</user/SendEmail.jtp?type=node&node=16176&i=1> To unsubscribe from Apache Ignite Users, click here. NAML<https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_template_NamlServlet.jtp-3Fmacro-3Dmacro-5Fviewer-26id-3Dinstant-5Fhtml-2521nabble-253Aemail.naml-26base-3Dnabble.naml.namespaces.BasicNamespace-2Dnabble.view.web.template.NabbleNamespace-2Dnabble.view.web.template.NodeNamespace-26breadcrumbs-3Dnotify-5Fsubscribers-2521nabble-253Aemail.naml-2Dinstant-5Femails-2521nabble-253Aemail.naml-2Dsend-5Finstant-5Femail-2521nabble-253Aemail.naml&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=qsKTpUnWgB48OZA1huxDdJW-LyQpHaNdTlxMrCCW-Xk&s=UnxulBBZr8p2NytrAfQlV_zN5CuQMKdMmEzmwxTWre4&e=> ________________________________ View this message in context: Re: Activation: slow and: Ignite node crashed in the middle of checkpoint.<https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_Activation-2Dslow-2Dand-2DIgnite-2Dnode-2Dcrashed-2Din-2Dthe-2Dmiddle-2Dof-2Dcheckpoint-2Dtp16144p16176.html&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=qsKTpUnWgB48OZA1huxDdJW-LyQpHaNdTlxMrCCW-Xk&s=EniF7rtveceC33Z8NsqSuKuTjP6oxO1SLh9wC4S4S1o&e=> Sent from the Apache Ignite Users mailing list archive<https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=qsKTpUnWgB48OZA1huxDdJW-LyQpHaNdTlxMrCCW-Xk&s=YZ5IcVzR9jK6NBv3sOl_ifmo1gC14Pa01R0HCHWBLPY&e=> at Nabble.com.
