Hi Stanislav, Thanks for info and the note on the terminology. So in my setup im partitioning by chunks of time. If I turn eagerttl to false how would the cleanup look? Would I periodically scan the partitions that i want to be expired? Are there any best practices to that end?
Additionally are there any truncate-like commands I can use to achieve a lightweight cleanup of my data? Thanks, Scott > On Mar 9, 2018, at 1:41 AM, Stanislav Lukyanov <[email protected]> wrote: > > Hi, > > A terminology nitpicking: it seems that you’re talking about expiry; eviction > is a similar mechanism but is based on the data size, not time. > > Have you tried setting CacheConfiguration.eagerTtl = false > (https://apacheignite.readme.io/v2.3/docs/expiry-policies#section-eager-ttl)? > With that setting the expired entries will be removed on the next access > instead of doing that in the background. > If you add entries in large batches but access them more uniformly over time, > it could redistribute the cost of expiring the data more evenly. > > Thanks, > Stan > > From: scottmf > Sent: 9 марта 2018 г. 6:51 > To: [email protected] > Subject: eviction performance > > Hi, > I am prototyping using ignite to ingest lots of short lived events in a > system. In order to ingest the data i'm using the kafka streamer mechanism. > I'm pushing 20k events / second into the system into a partitioned off heap > cache. I've run into a few performance issues but I've been able to wiggle > my way out of it using information from the forums and ignite docs. But the > latest issue I can't seem to find any information about. > > My cache is setup to evict the data after one hour. The insertion > performance degrades substantially when the eviction mechanism kicks in > where the latency of the insertion continues to degrade over time. > > WRT to the OS, memory and disk none of them are saturated. The load avg is > low, the gc is not under pressure and the disk is not fully utilized. > > I've experimented around with different eviction times to make sure that it > is indeed the eviction that is causing this. After some idle time I startup > our event simulator and wait for the evictions to start and sure enough the > latency of insertions increases immediately when evictions start. > > Each node is 32 GB / 8 cpus. > > Here are my jvm opts: "-Xmx12g -Xms12g -XX:MaxDirectMemorySize=10g > -XX:PermSize=192m -XX:MaxPermSize=192m -XX:+UseG1GC -XX:ConcGCThreads=12 > -XX:ParallelGCThreads=22 -XX:MaxGCPauseMillis=1000 -XX:G1HeapWastePercent=2 > -XX:G1ReservePercent=15 -XX:+UnlockExperimentalVMOptions > -XX:G1OldCSetRegionThresholdPercent=15 -XX:G1MixedGCLiveThresholdPercent=90 > -XX:G1MaxNewSizePercent=25 -Dcom.sun.management.jmxremote > -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError" > > I was wondering if there is no tuning I can do to avoid this overhead, I am > partitioning my data by time - is there a performant way to clear a > partition in a cache one at a time? (i was thinking of something similar to > a truncate command in a rdbms) > > Here is my schema: > CachePojo { > @QuerySqlField(index=false) > private String buffer0; > @QuerySqlField(index=true) > private String buffer1; > @QuerySqlField > private String buffer2; > @QuerySqlField(index=true, inlineSize=8, descending=true) > private Timestamp timestamp; > @QuerySqlField(index=false) > private Map<String, String> fields = new HashMap<>(); > } > > Ignite config: > IgniteConfiguration igniteConfiguration = new IgniteConfiguration(); > igniteConfiguration.setIgniteHome(igniteHome); > // this is probably not needed > igniteConfiguration.setTransactionConfiguration(new > TransactionConfiguration()); > igniteConfiguration.setPeerClassLoadingEnabled(true); > igniteConfiguration.setIncludeEventTypes(new int[0]); > igniteConfiguration.setDiscoverySpi(tcpDiscoverySpi()); > igniteConfiguration.setDataStreamerThreadPoolSize(8); > igniteConfiguration.setPublicThreadPoolSize(16); > igniteConfiguration.setSystemThreadPoolSize(16); > DataStorageConfiguration dsc = dataStorageConfiguration(); > igniteConfiguration.setDataStorageConfiguration(dsc); > > dataStorageConfig: > DataStorageConfiguration dataStorageConfiguration = new > DataStorageConfiguration(); > DataRegionConfiguration dataRegionConfiguration = > dataRegionConfiguration(); > > dataStorageConfiguration.setDefaultDataRegionConfiguration(dataRegionConfiguration); > > dataStorageConfiguration.setDataRegionConfigurations(largeDataRegionConfiguration()); > dataStorageConfiguration.setPageSize(8192); > dataStorageConfiguration.setMetricsEnabled(true); > dataStorageConfiguration.setWriteThrottlingEnabled(true); > > dataStorageConfiguration.setStoragePath("/var/lib/ignite/persistence"); > dataStorageConfiguration.setWalMode(WALMode.NONE); > dataStorageConfiguration.setWalPath("/var/lib/ignite/wal"); > > dataStorageConfiguration.setWalArchivePath("/var/lib/ignite/wal/archive"); > > cacheDataRegionConfig: > DataRegionConfiguration dataRegionConfiguration = new > DataRegionConfiguration(); > dataRegionConfiguration.setPersistenceEnabled(true); > dataRegionConfiguration.setName("dataRegion"); > > dataRegionConfiguration.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU); > dataRegionConfiguration.setInitialSize(500l * 1024 * 1024); > dataRegionConfiguration.setMaxSize(8l * 1024 * 1024 * 1024); > dataRegionConfiguration.setMetricsEnabled(true); > dataRegionConfiguration.setEmptyPagesPoolSize(1024); > > cacheConfig: > CacheConfiguration<Key, CachePojo> cacheConfig = new > CacheConfiguration<>(); > cacheConfig.setSqlFunctionClasses(SearchFunctions.class); > cacheConfig.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new > Duration(TimeUnit.MINUTES, 60))); > cacheConfig.setName("mycache"); > cacheConfig.setTypes(Key.class, CachePojo.class); > cacheConfig.setIndexedTypes(Key.class, CachePojo.class); > cacheConfig.setCacheMode(CacheMode.PARTITIONED); > cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC); > cacheConfig.setQueryParallelism(64); > cacheConfig.setStatisticsEnabled(true); > cacheConfig.setGroupName("cache"); > cacheConfig.setDataRegionName("dataRegion"); > // partitions in one minute chunks evenly across all nodes > cacheConfig.setAffinity(new CacheAffinityFunction()); > cacheConfig.setRebalanceBatchSize(1024 * 1024 * 4); > cacheConfig.setRebalanceThreadPoolSize(4); > cacheConfig.setRebalanceTimeout(TimeUnit.MINUTES.toMillis(10)); > cacheConfig.setRebalanceDelay(TimeUnit.MINUTES.toMillis(2)); > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
