Hello! For compute jobs, I believe it's public pool. For cache operations, I imagine it's striped pool.
Regards, -- Ilya Kasnacheev ср, 20 февр. 2019 г. в 13:34, <nadav.h...@docauthority.com>: > Hey, > > > > Which thread pools are responsible for the compute jobs and cache > operations ? > > > > Thanks, > > > > Nadav > > > > *From:* Ilya Kasnacheev <ilya.kasnach...@gmail.com> > *Sent:* Wednesday, February 20, 2019 12:26 PM > *To:* user@ignite.apache.org > *Subject:* Re: Apache Ignite starts fast and then become really slow with > out of memory > > > > Hello! > > > > Maybe you could do that by decreasing thread pool sizes as well as > limiting message queue lengths. However, it might not be worth the hassle. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > вт, 19 февр. 2019 г. в 21:26, <nadav.h...@docauthority.com>: > > Hey Ilya, > > > > I managed to stable the memory and avoid extreme slowness by send compute > jobs one by one and not in bulk. > > The compute jobs takes objects from cache by bulks, so I guess that firing > 50 compute jobs on a node where each job takes bulk of 100 is too heavy for > it to handle… > > My node has 32GB (Can go even higher) and 16 Cores. > > > > The one by one bits the purpose of scaling (i.e. multiple nodes serving > the same request), Is there a way to send a bulk of compute jobs and > manage it won't "explode" > > > > Thank you very much!! > > > > Nadav > > > > *From:* Ilya Kasnacheev <ilya.kasnach...@gmail.com> > *Sent:* Monday, February 18, 2019 11:39 AM > *To:* user@ignite.apache.org > *Subject:* Re: Apache Ignite starts fast and then become really slow with > out of memory > > > > Hello! > > > > I recommend starting from simplest configuration to more complex one. > Remove onheap cache, see if problem goes away. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > вс, 17 февр. 2019 г. в 12:09, <nadav.h...@docauthority.com>: > > Hey Ilya, > > > > Yes you are right, the LinkedHashMaps entries are the dominate part. > > I was concentrating on the reader because I thought there may be an > implicit connection.. > > Those LinkedHashMaps entries are part of the object structure I'm loading > to cache, if needed I can send the model code (Java). > > > > As you can see from my config, I use onheapCacheEnabled=true, my objects > can be pretty heavy (2 MG each), but I've place a rigid eviction policy of > only 1000 entries. > > > > <property name="evictionPolicy"> > > <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy"> > > <property name="maxSize" value="1000"/> > > </bean> > > </property> > > > > My heap is 16GB (can be more..) > > > > if I set onheapCacheEnabled=false does it mean that the object loaded will > not reside in the heap..? > > > > My general assumption is that I have problem with my GC tuning and it does > not handle the cache load well and struggles to clear/allocate memory. > > > > What will help you to identify my problem more precisely ? (GC Logs, > behavioral code, model code) ? > > > > Thank you very much. > > > > Nadav > > > > *From:* Ilya Kasnacheev <ilya.kasnach...@gmail.com> > *Sent:* Friday, February 15, 2019 4:33 PM > *To:* user@ignite.apache.org > *Subject:* Re: Apache Ignite starts fast and then become really slow with > out of memory > > > > Hello! > > > > As far as I can see, the highlighted class does not dominate heap in any > meaningful way. > > > > You seem to have huge LinkedHashMaps, any idea where they are used/held? > > > > Regards, > > > > -- > > Ilya Kasnacheev > > > > > > чт, 14 февр. 2019 г. в 22:11, <nadav.h...@docauthority.com>: > > Hey, > > > > Guys is this normal…? > > > > [image: cid:image001.png@01D4C556.6CEBD080] > > > > So many binary readers are opened without reuse? I saw an improvement > issue on this https://jira.apache.org/jira/browse/IGNITE-5721 > > > > I'm running a simple compute tasks that split around 4K ids to compute job > that each one takes from cache bulk of 500 ids. > > Each Object size return is ~1.8MB. > > > > When I run 3-4 iterations it is super fasts completing the tasks under 100 > ms. > > But then memory rises and it becomes really slow, fully occupied by GC and > after a while it caves and get OOME. > > > > I'm evaluating ignite for a POC and this is a real showstopper. > > I'm sure that I'm missing something because I do nothing out of the > ordinary. > > > > I tried different GC tuning G1 and CMS, different evection policies, > different region sizes. Nothing helped. > > > > I'm using the latest ignite 2.7, my machine is windows server 12 R2 with 8 > cores and 32GB RAM, this is my node config: > > > > <beans xmlns="http://www.springframework.org/schema/beans"; > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > > xmlns:context="http://www.springframework.org/schema/context"; > > xsi:schemaLocation=" > > http://www.springframework.org/schema/beans > > http://www.springframework.org/schema/beans/spring-beans.xsd > > http://www.springframework.org/schema/context > > > http://www.springframework.org/schema/context/spring-context.xsd > > "> > > > > <context:annotation-config/> > > > > <context:property-placeholder > location="classpath:application.properties"/> > > > > <bean id="similarityConfiguration" > class="com.cla.filecluster.service.files.pv.SimilarityConfiguration"/> > > > > <bean id="similarityCalculator" > class="com.cla.filecluster.service.files.pv.SimilarityCalculator" > > factory-method="create"> > > <constructor-arg ref="similarityConfiguration" /> > > </bean> > > > > > > > > <bean id="ignite.cfg" > class="org.apache.ignite.configuration.IgniteConfiguration"> > > <property name="userAttributes"> > > <map key-type="java.lang.String" > value-type="java.lang.Boolean"> > > <entry key="data.node" value="true"/> > > </map> > > </property> > > > > <property name="binaryConfiguration"> > > <bean > class="org.apache.ignite.configuration.BinaryConfiguration"> > > <property > name="compactFooter" value="false"/> > > </bean> > > </property> > > > > <!-- property name="asyncCallbackPoolSize" > value="2"/ --> > > > > > > > > <property > name="dataStorageConfiguration"> > > <bean > class="org.apache.ignite.configuration.DataStorageConfiguration"> > > <property > name="pageSize" value="#{4 * 1024}"/> > > > > <property > name="defaultDataRegionConfiguration"> > > > <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> > > > <property name="name" value="analysis_region"/> > > <property name="persistenceEnabled" value="true"/> > > > <property name="initialSize" value="#{100 * 1024 * 1024}"/> > > > <property name="maxSize" value="8589934592"/> > > </bean> > > </property> > > <property name="storagePath" value="E:\asm-repo\db"/> > > <property name="walPath" value="E:\asm-repo\wal"/> > > <property > name="walArchivePath" value="E:\asm-repo\wal\archive"/> > > </bean> > > </property> > > > > <property name="cacheConfiguration"> > > <list> > > <!-- Partitioned cache example configuration (Atomic > mode). --> > > <bean > class="org.apache.ignite.configuration.CacheConfiguration"> > > > <property name="storeKeepBinary" value="true"/> > > > <property name="eagerTtl" value="true"/> > > <property name="name" value="PvAnalysisData"/> > > > <property name="dataRegionName" value="analysis_region"/> > > <property name="atomicityMode" value="ATOMIC"/> > > <property name="backups" value="0"/> > > > <property name="onheapCacheEnabled" value="true"/> > > > <property name="expiryPolicyFactory"> > > > <bean class="javax.cache.expiry.CreatedExpiryPolicy" > factory-method="factoryOf"> > > > <constructor-arg> > > > <bean class="javax.cache.expiry.Duration"> > > > <constructor-arg value="SECONDS"/> > > > > <constructor-arg value="5"/> > > > </bean> > > > </constructor-arg> > > > </bean> > > > </property> > > > <property name="evictionPolicy"> > > > > > > <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy"> > > > <!-- property name="maxMemorySize" value="104857600"/ --> > > > <property name="maxSize" value="1000"/> > > > </bean> > > > </property> > > > <property name="nodeFilter"> > > > <bean class="com.cla.filecluster.common.DataNodeFilter"/> > > > </property> > > </bean> > > </list> > > </property> > > > > <!-- Explicitly configure TCP discovery SPI to provide list of > initial nodes. --> > > <property name="discoverySpi"> > > <bean > class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > > <property name="ipFinder"> > > <!-- > > Ignite provides several options for automatic > discovery that can be used > > instead os static IP based discovery. For > information on all options refer > > to our documentation: > http://apacheignite.readme.io/docs/cluster-config > > --> > > <!-- Uncomment static IP finder to enable static-based > discovery of initial nodes. --> > > <!--<bean > class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">--> > > <bean > class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder"> > > <property name="addresses"> > > <list> > > <!-- In distributed environment, replace > with actual host IP address. --> > > <value>127.0.0.1:47500..47509</value> > > </list> > > </property> > > </bean> > > </property> > > </bean> > > </property> > > </bean> > > </beans> > > > > > > > > Thanks, > > Nadav > > System Architect > > [image: cid:image002.jpg@01D4C556.6CEBD080] > > +972-544821606 > DocAuthority.com <http://www.docauthority.com/> > > > > > > > >
