Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Sanne Grinovero
2011/6/10 Manik Surtani : > > On 10 Jun 2011, at 15:35, Sanne Grinovero wrote: > >> 2011/6/10 Manik Surtani : >>> Anyway, to summarise, does this cover everything? >>> >>> 1)  Passivate before evict, as described below >>> 2)  AsyncStore checks async queue before delegating to actual cache store >

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
Ok, pls have a look and comment, guys: https://github.com/infinispan/infinispan/pull/380 On 10 Jun 2011, at 15:45, Manik Surtani wrote: > > On 10 Jun 2011, at 15:35, Sanne Grinovero wrote: > >> 2011/6/10 Manik Surtani : >>> Anyway, to summarise, does this cover everything? >>> >>> 1) Passiv

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
On 10 Jun 2011, at 15:35, Sanne Grinovero wrote: > 2011/6/10 Manik Surtani : >> Anyway, to summarise, does this cover everything? >> >> 1) Passivate before evict, as described below >> 2) AsyncStore checks async queue before delegating to actual cache store on >> load >> >> Cheers >> Manik >

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Sanne Grinovero
2011/6/10 Manik Surtani : > Anyway, to summarise, does this cover everything? > > 1)  Passivate before evict, as described below > 2)  AsyncStore checks async queue before delegating to actual cache store on > load > > Cheers > Manik Yes, looks good! 2) won't be easy, AFAIR the AsyncStore has sev

Re: [infinispan-dev] frequent stack in testsuite

2011-06-10 Thread Galder Zamarreño
On Jun 10, 2011, at 3:51 PM, Sanne Grinovero wrote: > 2011/6/10 Galder Zamarreño : >> >> On Jun 10, 2011, at 11:42 AM, Sanne Grinovero wrote: >> >>> 2011/6/10 Galder Zamarreño : On Jun 9, 2011, at 5:47 PM, Manik Surtani wrote: > +1 to writing the error marker to the stream.

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
Anyway, to summarise, does this cover everything? 1) Passivate before evict, as described below 2) AsyncStore checks async queue before delegating to actual cache store on load Cheers Manik On 10 Jun 2011, at 14:18, Sanne Grinovero wrote: > It looks as a good improvement, especially as we re

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
On 10 Jun 2011, at 14:33, Manik Surtani wrote: >> >> 2) with async stores, the issue is back. The value is not going to be >> found while it's "in flight" in the async storage queue; so unless you >> want to avoid supporting it for now, the actual removal from the >> container should be performe

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
On 10 Jun 2011, at 14:44, Sanne Grinovero wrote: >> With passivation enabled, the entry is only written to the store when >> removed from memory. And when activating (store -> memory) it is removed >> from the store. So you wouldn't really write twice to the store. > > Thanks, with that you

Re: [infinispan-dev] frequent stack in testsuite

2011-06-10 Thread Sanne Grinovero
2011/6/10 Galder Zamarreño : > > On Jun 10, 2011, at 11:42 AM, Sanne Grinovero wrote: > >> 2011/6/10 Galder Zamarreño : >>> >>> On Jun 9, 2011, at 5:47 PM, Manik Surtani wrote: >>> +1 to writing the error marker to the stream. At least prevent false alarms. Re: unit testing our

Re: [infinispan-dev] frequent stack in testsuite

2011-06-10 Thread Galder Zamarreño
On Jun 10, 2011, at 11:42 AM, Sanne Grinovero wrote: > 2011/6/10 Galder Zamarreño : >> >> On Jun 9, 2011, at 5:47 PM, Manik Surtani wrote: >> >>> +1 to writing the error marker to the stream. At least prevent false alarms. >>> >>> Re: unit testing our externalizers, Galder, any thoughts there?

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Sanne Grinovero
2011/6/10 Manik Surtani : > > On 10 Jun 2011, at 14:18, Sanne Grinovero wrote: > >> It looks as a good improvement, especially as we remove the locking, >> but I still have some doubts: >> >> 1) as the passivation happens in the eviction thread, is that going to >> be fast enough or hindering the e

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Galder Zamarreño
On Jun 10, 2011, at 3:18 PM, Sanne Grinovero wrote: > > > 3) so every time we evict an entry, we write it to the store.. I guess > that leads to re-write quite often values which are already in the > store, and for which there's no need to store again. I think this > argument came up before, bu

Re: [infinispan-dev] Timing related tests

2011-06-10 Thread Manik Surtani
This certainly helps, I'll give it a go, On 10 Jun 2011, at 13:15, Sanne Grinovero wrote: > 2011/6/10 Manik Surtani : >> Some tests - like ExpiryTest - rely on certain timings for the test to run, >> and due to thread scheduling on our parallel test suite, tend to >> occasionally fail on certain

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
On 10 Jun 2011, at 14:18, Sanne Grinovero wrote: > It looks as a good improvement, especially as we remove the locking, > but I still have some doubts: > > 1) as the passivation happens in the eviction thread, is that going to > be fast enough or hindering the eviction performance? especially wi

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
This has an added implication on Galder's recent fix re: @CacheEntryPassivated -> @CacheEntriesPassivated notifications. With this fix, these notifications *cannot* be batched as they will happen once per entry, as and when the entry is removed from memory. So we now have @CacheEntryPassivated

Re: [infinispan-dev] Timing related tests

2011-06-10 Thread Pete Muir
IDK if it's possible (or if it changes the purpose of the test too much), but can we add a new "expiry policy" which is actually deterministic, so we mimic expirations? This won't test the default expiry policy, but would test the logic managing expiration. On 10 Jun 2011, at 14:20, Mircea Mar

Re: [infinispan-dev] Timing related tests

2011-06-10 Thread Mircea Markus
This is a very tricky one indeed. These might fail as well e.g. if we increase the number of concurrent threads running the test suite. I remember when switching from 1 thread to 10 there were many of these failing. I think your suggestion makes sense. On 10 Jun 2011, at 13:02, Manik Surtani wro

Re: [infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Sanne Grinovero
It looks as a good improvement, especially as we remove the locking, but I still have some doubts: 1) as the passivation happens in the eviction thread, is that going to be fast enough or hindering the eviction performance? especially with LIRS, being this the user thread it means that put operati

[infinispan-dev] Passivation and BCHM WAS: fixing eviction with transactions (critical for Hibernate Search)

2011-06-10 Thread Manik Surtani
So I've taken on this sub-task from Sanne (ISPN-1169), and here is the deal: BCHM analyses whether or not to evict stuff every time an operation (put, get, replace) is run, based on the configured eviction policy (LIRS, LRU or NONE). It does so by calling evictionStrategy.execute(), which retur

Re: [infinispan-dev] Timing related tests

2011-06-10 Thread Sanne Grinovero
2011/6/10 Manik Surtani : > Some tests - like ExpiryTest - rely on certain timings for the test to run, > and due to thread scheduling on our parallel test suite, tend to > occasionally fail on certain environments such as CloudBees: > https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-

[infinispan-dev] Timing related tests

2011-06-10 Thread Manik Surtani
Some tests - like ExpiryTest - rely on certain timings for the test to run, and due to thread scheduling on our parallel test suite, tend to occasionally fail on certain environments such as CloudBees: https://infinispan.ci.cloudbees.com/job/Infinispan-master-JDK6-tcp/org.infinispan$infinispan-c

Re: [infinispan-dev] frequent stack in testsuite

2011-06-10 Thread Dan Berindei
We may have a deadlock because we hold the processing lock (for reading) while invoking commands remotely (in JGroupsTransport). The remote commands might block waiting for the remote cache to start, and the remote cache won't start because it is waiting for this cache to acquire the processing log

Re: [infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

2011-06-10 Thread Sanne Grinovero
2011/6/10 Manik Surtani : > > On 10 Jun 2011, at 04:48, Tristan Tarrant wrote: > >> I don't know if I'm actually contributing something here or just creating >> noise. >> >> Are these buffers reused over time ? If not, from a GC point of view >> it would be better then not to reduce the size of th

Re: [infinispan-dev] (topic changed) For interposed sync support, building a patch CR4 or new CR5...

2011-06-10 Thread Scott Marlow
Manik, I'd like to try doing that and revert back to using a patched CR4 if we hit any issues/regressions (would be nice to know what they are for Monday's build :-) If we get stuck this weekend (or late today), we can always go the patch route. I'll talk to Paul about this as well, as he wil

Re: [infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

2011-06-10 Thread Tristan Tarrant
> Somewhere in this thread there was discussion of creating a buffer per thread > (thread-local again) but was determined to be too much of a mem leak (and I > agree with this). We should avoid thread locals :) > Maybe it makes sense to create a pool of buffers, to be shared?  It would > certa

Re: [infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

2011-06-10 Thread Manik Surtani
On 10 Jun 2011, at 04:48, Tristan Tarrant wrote: > I don't know if I'm actually contributing something here or just creating > noise. > > Are these buffers reused over time ? If not, from a GC point of view > it would be better then not to reduce the size of the buffer just to > save a few byte

Re: [infinispan-dev] Adaptive marshaller buffer sizes - ISPN-1102

2011-06-10 Thread Manik Surtani
I was referring to generating byte arrays (sending state), not generating objects (receiving state). A buffer is maintained in the AbstractMarshaller and used. I did see a comment from Bela on this thread about seeing this on the receiver too though - Bela, care to clarify? I presume on the r

Re: [infinispan-dev] locking optimisations reloaded

2011-06-10 Thread Manik Surtani
Thanks for the paper, Paolo - I'll have a read of it over the weekend. On 9 Jun 2011, at 21:47, Paolo Romano wrote: > There has been a lot of discussion on this topic also in the Transactional > Memory area, triggered to the best of my knowledge by this paper [1]. > > Mixing transactional and

Re: [infinispan-dev] frequent stack in testsuite

2011-06-10 Thread Sanne Grinovero
2011/6/10 Galder Zamarreño : > > On Jun 9, 2011, at 5:47 PM, Manik Surtani wrote: > >> +1 to writing the error marker to the stream. At least prevent false alarms. >> >> Re: unit testing our externalizers, Galder, any thoughts there? > > The debugging done by Sanne/Dan seems to be correct. > > EOFE

Re: [infinispan-dev] discussion about impact of using TransactionSynchronizationRegistry in AS7...

2011-06-10 Thread Manik Surtani
Scott, if you want I can cut CR5 this morning based on what we have and then a CR6 for next week, with the bits we originally had in mind for CR5. I'm trying to compete with Bob McW for the largest number of release candidates. ;) On 9 Jun 2011, at 21:42, Scott Marlow wrote: > As a short t

Re: [infinispan-dev] frequent stack in testsuite

2011-06-10 Thread Galder Zamarreño
On Jun 9, 2011, at 5:47 PM, Manik Surtani wrote: > +1 to writing the error marker to the stream. At least prevent false alarms. > > Re: unit testing our externalizers, Galder, any thoughts there? The debugging done by Sanne/Dan seems to be correct. EOFException is simply saying: "hey, i'm exp