Re: [infinispan-dev] Translating our events to JSR 107 events
On Mon, Jan 28, 2013 at 5:41 PM, Manik Surtani msurt...@redhat.com wrote: On 28 Jan 2013, at 15:22, Vladimir Blagojevic vblag...@redhat.com wrote: On 13-01-28 7:31 AM, Manik Surtani wrote: If you're ok with changing the core, you could add a getValue() method to CacheEntryCreatedEvent, and an isCreated() method to CacheEntryModifiedEvent (as I suppose you don't want to call the updates listener when an entry is created). Both changes should be backwards-compatible. That could work. The second one Manik? As I was researching how to do this I found you were against the first option https://issues.jboss.org/browse/ISPN-881 Yes, isCreated(). The second I have no idea how to implement as we do not have CacheEntryExpired event. True, spec is not rigorous that such an event has to be fired immediately after an entry has expired but eventually (which might be on access). Either way, I am all ears on suggestions how to implement this one. I guess @CacheEntryEvicted/@CacheEntriesEvicted would be the closest thing we have in Infinispan. But you can't check in the listener if the entry was evicted because it expired or because there wasn't enough space in the data container (yet). There could definitely be something clever we could do here. Adding the (expired or evicted) entry to a queue for later notification. But that would definitely need to be something we explicitly enable rather than have running all the time, since it kinda defeats the purpose of evicting something to save memory only to have it put in a different queue elsewhere until an event is fired. Exactly! One thing we could do is what RI does. Check for expired on entry access from JSR module and during normal expired cleanup cycle. Be mindful of performance considerations here - this could get very expensive. Exactly. I don't think you can use a queue for later notification, because you also have to cancel the notification if the user updates the same key again. I mean if an entry was supposed to expire at 12:00 and the user did a put at 12:59 with expiration time 12:30, then he shouldn't get an expired notification at 12:00 - even if the entry was evicted in the meantime. I think you'd need something like a cache store that only keeps the keys and their expiration time, and with passivation enabled. I don't think you can reuse the cache store code, though. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Lucene 4 / Infinispan performance
Wow. That's impressive. I presume the numbers change quite drastically the moment we have a cache store configured though? On 28 Jan 2013, at 23:53, Sanne Grinovero sa...@infinispan.org wrote: These are preliminary results of our stressor; looks quite promising as I haven't yet looked into profiling / tuning: Stock Lucene RAMDirectory Searches: 14.799.852 Writes: 195.935 Stock Lucene FSDirectory (Memory mapping on SSD) Searches: 9.628.593 Writes: 105.930 Our custom Infinispan Directory (LOCAL) Searches: 17.815.874 Writes: 184.140 Figures represent operations performed in 15 minutes on a relatively small index. Cheers, Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Translating our events to JSR 107 events
The second I have no idea how to implement as we do not have CacheEntryExpired event. True, spec is not rigorous that such an event has to be fired immediately after an entry has expired but eventually (which might be on access). Either way, I am all ears on suggestions how to implement this one. I guess @CacheEntryEvicted/@CacheEntriesEvicted would be the closest thing we have in Infinispan. But you can't check in the listener if the entry was evicted because it expired or because there wasn't enough space in the data container (yet). There could definitely be something clever we could do here. Adding the (expired or evicted) entry to a queue for later notification. But that would definitely need to be something we explicitly enable rather than have running all the time, since it kinda defeats the purpose of evicting something to save memory only to have it put in a different queue elsewhere until an event is fired. Exactly! One thing we could do is what RI does. Check for expired on entry access from JSR module and during normal expired cleanup cycle. Be mindful of performance considerations here - this could get very expensive. Exactly. I don't think you can use a queue for later notification, because you also have to cancel the notification if the user updates the same key again. I mean if an entry was supposed to expire at 12:00 and the user did a put at 12:59 with expiration time 12:30, then he shouldn't get an expired notification at 12:00 - even if the entry was evicted in the meantime. Yes; definitely not a simple thing to do. -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Lucene 4 / Infinispan performance
No I'm not comparing with Lucene 3.6 now with this configuration. It is well known that Lucene 4 is significantly faster than Lucene 3, so that would be unfair. What is interesting is that when comparing our implementations vs. the Apache stock ones while using Lucene 3 we where very close, often a bit faster but not too exciting. Now comparing with the stock ones using Lucene 4 it seems we're getting into a better position... and I didn't even profile it, this is the first run after finishing coding the functional requirements. Still these figures are produced by a stress test whose primary purpose is to verify consistency and no-corruptions under stress.. I happened to add some metrics for fun, but to provide realistic figures one should run Lucene's own bench suite.. I'll do that next week. @Manik yes I'm not using a CacheStore, I would presume the same. This is why I've created the Lucene-specific CacheLoader, maybe I should complete the job and make it a CacheStore. Sanne On 29 January 2013 13:14, Tristan Tarrant ttarr...@redhat.com wrote: Have you got numbers comparing against Lucene 3.6 ? Tristan On 01/29/2013 12:53 AM, Sanne Grinovero wrote: These are preliminary results of our stressor; looks quite promising as I haven't yet looked into profiling / tuning: Stock Lucene RAMDirectory Searches: 14.799.852 Writes: 195.935 Stock Lucene FSDirectory (Memory mapping on SSD) Searches: 9.628.593 Writes: 105.930 Our custom Infinispan Directory (LOCAL) Searches: 17.815.874 Writes: 184.140 Figures represent operations performed in 15 minutes on a relatively small index. Cheers, Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] ClockService
Thinking about Sanne's idea re: a clock service and not continuously relying on System.cTM, I stumbled upon this: https://blogs.oracle.com/ksrini/entry/we_take_java_performance_very The last section re: caching is interesting. Having all code use a Clock.currentTimeMillis() and having an implementation that kicks off a scheduled task to update a cached value every $frequency millis (by using the System call) is probably the correct way to implement something like this, but this adds complexities/concerns: * CPU cache line blowing away (see article) * Context switching * Managing an extra service thread * Probably an additional configuration option to configure that thread, its priority and frequency. If frequency is dropped (e.g., every second) I suppose the perf gains would be huge. JGroups - does JGroups make many and frequent calls to System.cTM as well? Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Lucene 4 / Infinispan performance
On 01/29/2013 02:30 PM, Sanne Grinovero wrote: @Manik yes I'm not using a CacheStore, I would presume the same. This is why I've created the Lucene-specific CacheLoader, maybe I should complete the job and make it a CacheStore. Oh yeah, a Lucene CacheStore would be nice. Tristan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ClockService
Glad you started work on that :) Any currentTimeMillis() even today will blow away your cache line and probably trigger a context switch. Having it as a service we will be able to experiment: the first thing I'll do is replace it with a NOOPService and see how much it improves. Ideally I'd like it to have multiple methods so that different needs can hint about a different level of precision, or maybe expect an enum as parameter which represents the hint. Cheers, Sanne On 29 January 2013 16:02, Manik Surtani msurt...@redhat.com wrote: Thinking about Sanne's idea re: a clock service and not continuously relying on System.cTM, I stumbled upon this: https://blogs.oracle.com/ksrini/entry/we_take_java_performance_very The last section re: caching is interesting. Having all code use a Clock.currentTimeMillis() and having an implementation that kicks off a scheduled task to update a cached value every $frequency millis (by using the System call) is probably the correct way to implement something like this, but this adds complexities/concerns: * CPU cache line blowing away (see article) * Context switching * Managing an extra service thread * Probably an additional configuration option to configure that thread, its priority and frequency. If frequency is dropped (e.g., every second) I suppose the perf gains would be huge. JGroups - does JGroups make many and frequent calls to System.cTM as well? Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ClockService
Interesting; looking at the code a bit more, I don't think this offers any improvements except in the case where expiry is set. Granted, this is a useful thing to have at that point, but I'm not entirely sure whether it will affect the majority of users. Anyway, to measure any benefit, I'm also going to have to enhance RadarGun to (optionally) set expiry information on data stored. - M On 29 Jan 2013, at 16:25, Sanne Grinovero sa...@infinispan.org wrote: Glad you started work on that :) Any currentTimeMillis() even today will blow away your cache line and probably trigger a context switch. Having it as a service we will be able to experiment: the first thing I'll do is replace it with a NOOPService and see how much it improves. Ideally I'd like it to have multiple methods so that different needs can hint about a different level of precision, or maybe expect an enum as parameter which represents the hint. Cheers, Sanne On 29 January 2013 16:02, Manik Surtani msurt...@redhat.com wrote: Thinking about Sanne's idea re: a clock service and not continuously relying on System.cTM, I stumbled upon this: https://blogs.oracle.com/ksrini/entry/we_take_java_performance_very The last section re: caching is interesting. Having all code use a Clock.currentTimeMillis() and having an implementation that kicks off a scheduled task to update a cached value every $frequency millis (by using the System call) is probably the correct way to implement something like this, but this adds complexities/concerns: * CPU cache line blowing away (see article) * Context switching * Managing an extra service thread * Probably an additional configuration option to configure that thread, its priority and frequency. If frequency is dropped (e.g., every second) I suppose the perf gains would be huge. JGroups - does JGroups make many and frequent calls to System.cTM as well? Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ClockService
On 1/29/13 5:02 PM, Manik Surtani wrote: Thinking about Sanne's idea re: a clock service and not continuously relying on System.cTM, I stumbled upon this: https://blogs.oracle.com/ksrini/entry/we_take_java_performance_very The last section re: caching is interesting. Having all code use a Clock.currentTimeMillis() and having an implementation that kicks off a scheduled task to update a cached value every $frequency millis (by using the System call) is probably the correct way to implement something like this, but this adds complexities/concerns: * CPU cache line blowing away (see article) * Context switching * Managing an extra service thread * Probably an additional configuration option to configure that thread, its priority and frequency. If frequency is dropped (e.g., every second) I suppose the perf gains would be huge. JGroups - does JGroups make many and frequent calls to System.cTM as well? No; currentTimeMillis() is used in tasks that run every few seconds, otherwise we use nanoTime(). -- Bela Ban, JGroups lead (http://www.jgroups.org) ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ClockService
On 29 Jan 2013, at 17:17, Bela Ban b...@redhat.com wrote: On 1/29/13 5:25 PM, Sanne Grinovero wrote: Glad you started work on that :) Any currentTimeMillis() even today will blow away your cache line and probably trigger a context switch. I understand the context switch (in general, it's not recommended anyway to invoke a system call in synchronized code), but I fail to see why this would blow the cache line. Are you referring to the cached Date value here ? No, if you have a separate maint thread that updates a reusable currentTimeMillis value. Do you use nanoTime() a lot then? Because that too is inefficient (as per the Oracle blog) ... -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Platform Architect, JBoss Data Grid http://red.ht/data-grid ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] 5.2.0.Final moves to 31 Jan
In order to accommodate some critical bug fixes, $Subject. Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org) ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Lucene 4 / Infinispan performance
On 29 Jan 2013, at 14:08, Dan Berindei wrote: Very nice, looking forward to the Lucene bench results. I hope you'll run it with a distributed cache as well! +1 :-) Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org) ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
