[infinispan-dev] Warnings vs. Fail Fast
Currently if a cache is configured with indexing enabled, but the Query module isn't on classpath, you get a simple warning. I think this should fail with a configuration validation error; it's not just safer but also consistent with many other validations. I've created ISPN-3583 and patch is ready.. any good reason to not apply it? Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Infinispan Query API changes
Ok, noted. -Ales On Oct 3, 2013, at 17:44, Sanne Grinovero wrote: > Hi, > just a heads up that I've been moving some APIs around from > org.infinispan.query: > > https://github.com/infinispan/infinispan/pull/2131 > > In particular I think that CapeDwarf is using some methods from > > org.infinispan.query.SearchManager > > which are no longer exposed; the fix is simple, cast any SearchManager > you might have into a > org.infinispan.query.spi.SearchManagerImplementor > > SearchManagerImplementor is meant to be an SPI, so a relatively stable > interface for projects like CD but not expected to be used by every > day users. > > Sanne > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Configuration compatibility expectations ?
Ok, as long as it's expected I guess it's fine. Thanks! Sanne On 3 October 2013 18:32, Tristan Tarrant wrote: > You need the 52x compatibility jar. > > Tristan > > On 10/03/2013 06:53 PM, Sanne Grinovero wrote: >> Hello, >> I just tried to undust an old test I had shelved. When I run it I got >> an exception about the configuration having an unexpected header: >> >> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xsi:schemaLocation="urn:infinispan:config:5.2 >> http://www.infinispan.org/schemas/infinispan-config-5.2.xsd"; >> xmlns="urn:infinispan:config:5.2"> >> >> Everything works fine after I change it to: >> >> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xsi:schemaLocation="urn:infinispan:config:6.0 >> http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"; >> xmlns="urn:infinispan:config:6.0"> >> >> Is that expected? >> >> Sanne >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Configuration compatibility expectations ?
You need the 52x compatibility jar. Tristan On 10/03/2013 06:53 PM, Sanne Grinovero wrote: > Hello, > I just tried to undust an old test I had shelved. When I run it I got > an exception about the configuration having an unexpected header: > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="urn:infinispan:config:5.2 > http://www.infinispan.org/schemas/infinispan-config-5.2.xsd"; > xmlns="urn:infinispan:config:5.2"> > > Everything works fine after I change it to: > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="urn:infinispan:config:6.0 > http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"; > xmlns="urn:infinispan:config:6.0"> > > Is that expected? > > Sanne > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Configuration compatibility expectations ?
Hello, I just tried to undust an old test I had shelved. When I run it I got an exception about the configuration having an unexpected header: http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="urn:infinispan:config:5.2 http://www.infinispan.org/schemas/infinispan-config-5.2.xsd"; xmlns="urn:infinispan:config:5.2"> Everything works fine after I change it to: http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="urn:infinispan:config:6.0 http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"; xmlns="urn:infinispan:config:6.0"> Is that expected? Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Infinispan Query API changes
Hi, just a heads up that I've been moving some APIs around from org.infinispan.query: https://github.com/infinispan/infinispan/pull/2131 In particular I think that CapeDwarf is using some methods from org.infinispan.query.SearchManager which are no longer exposed; the fix is simple, cast any SearchManager you might have into a org.infinispan.query.spi.SearchManagerImplementor SearchManagerImplementor is meant to be an SPI, so a relatively stable interface for projects like CD but not expected to be used by every day users. Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Which QueryBuilder ?
On 3 October 2013 14:10, Adrian Nistor wrote: > I know, was just joking. Anyway, I don't see any confusion having two > classes with the same name. It's going to be hard enough to explain to people why we are providing two different approaches, if we can't even think of a different name to properly highlight the different usage then we have a problem. Try figuring the forum / support question "I'm using QueryBuilder on Infinispan 7.3 and this happens... " The javadoc index will have it listed twice -> annoying. Google for "QueryBuilder Infinispan" -> annoying Or try figuring out the documentation: # Chapter 5: Queries. There are two approaches to run Queries in Infinispan. Either you use the QueryBuilder, which provides simple domain oriented properties and can work both in embedded and remote mode, or you use the more powerfull QueryBuilder. # 5.1 QueryBuilder blah blah # 5.2 QueryBuilder blah blah If they are different, the should really have different names, even just to avoid confusion among ourselves when talking about hem. If you feel they're the same, the interesting alternative is to literally merge them in one single interface, potentially exposing multiple methods. Sanne > > On 10/03/2013 02:29 PM, Emmanuel Bernard wrote: >> It's already productized code. >> >> On Thu 2013-10-03 14:16, Adrian Nistor wrote: >>> I would suggest renaming the old one :)) >>> >>> On 10/02/2013 11:13 PM, Sanne Grinovero wrote: It seems we have now 2 different interfaces both names "QueryBuilder" when using Infinispan Query. One is coming from Hibernate Search, and represents the "classic" way to build queries for Infinispan Query in embedded mode. The other one is new, and represents the simplified approach, also implemented for remote queries. Could we find an alternative name for the new API? It's certainly going to be confusing, even more when we'll have to document the differences, and which one is more suited for one use cases vs. another. Cheers, Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> ___ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Which QueryBuilder ?
I know, was just joking. Anyway, I don't see any confusion having two classes with the same name. On 10/03/2013 02:29 PM, Emmanuel Bernard wrote: > It's already productized code. > > On Thu 2013-10-03 14:16, Adrian Nistor wrote: >> I would suggest renaming the old one :)) >> >> On 10/02/2013 11:13 PM, Sanne Grinovero wrote: >>> It seems we have now 2 different interfaces both names "QueryBuilder" >>> when using Infinispan Query. >>> One is coming from Hibernate Search, and represents the "classic" way >>> to build queries for Infinispan Query in embedded mode. >>> >>> The other one is new, and represents the simplified approach, also >>> implemented for remote queries. >>> >>> Could we find an alternative name for the new API? >>> >>> It's certainly going to be confusing, even more when we'll have to >>> document the differences, and which one is more suited for one use >>> cases vs. another. >>> >>> Cheers, >>> Sanne >>> ___ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Which QueryBuilder ?
It's already productized code. On Thu 2013-10-03 14:16, Adrian Nistor wrote: > I would suggest renaming the old one :)) > > On 10/02/2013 11:13 PM, Sanne Grinovero wrote: > > It seems we have now 2 different interfaces both names "QueryBuilder" > > when using Infinispan Query. > > One is coming from Hibernate Search, and represents the "classic" way > > to build queries for Infinispan Query in embedded mode. > > > > The other one is new, and represents the simplified approach, also > > implemented for remote queries. > > > > Could we find an alternative name for the new API? > > > > It's certainly going to be confusing, even more when we'll have to > > document the differences, and which one is more suited for one use > > cases vs. another. > > > > Cheers, > > Sanne > > ___ > > infinispan-dev mailing list > > infinispan-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Which QueryBuilder ?
I would suggest renaming the old one :)) On 10/02/2013 11:13 PM, Sanne Grinovero wrote: > It seems we have now 2 different interfaces both names "QueryBuilder" > when using Infinispan Query. > One is coming from Hibernate Search, and represents the "classic" way > to build queries for Infinispan Query in embedded mode. > > The other one is new, and represents the simplified approach, also > implemented for remote queries. > > Could we find an alternative name for the new API? > > It's certainly going to be confusing, even more when we'll have to > document the differences, and which one is more suited for one use > cases vs. another. > > Cheers, > Sanne > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] ISPN-3557: interactions between a clear() operation and a Transaction
On 3 October 2013 10:29, Dan Berindei wrote: > > On Wed, Oct 2, 2013 at 12:48 PM, Pedro Ruivo wrote: >> >> >> >> On 10/02/2013 12:03 AM, Sanne Grinovero wrote: >> > I'd love to brainstorm about the clear() operation and what it means >> > on Infinispan. >> > >> > I'm not sure to what extent, but it seems that clear() is designed to >> > work in a TX, or even create an implicit transaction if needed, but >> > I'm not understanding how that can work. > > > I think it's pretty straight-forward to define the semantics of operations > after clear(): they should assume that every key in the cache was removed. > > True, like Pedro mentioned, the current implementation allows subsequent > operations in the same tx to see keys inserted by other txs. But I don't > think fixing that would be harder than ensuring that if a tx inserts key k > owned by nodes A and B, clear() from outside the tx either deletes k on A > and B or doesn't delete k on either node. > > >> >> > >> > Obviously a clear() operation isn't listing all keys explicitly. >> >> (offtopic) like entrySet(), keySet(), values() and iterator() when >> executed in a transactional context. however, I think that this >> operations are more difficult to handle, because they have to return >> consistent data... > > > I agree, defining how entrySet() and the others should work in a tx seems > harder than defining clear(). I agree, those other nasty beasts are next on my list: see below.. But we're not arguing about which method deserves the gold medal for complex design ;-) > Will, I'm guessing your fix for ISPN-3560 still allows entrySet() after > clear() to see entries inserted by another tx in the meantime? > >> >> > Which >> > implies that it's undefined on which keys it's going to operate when >> > it's fired.. that seems like terribly wrong in a distributed key/value >> > store as we can't possibly freeze the global state and somehow define >> > a set of keys which are going to be affected, while an explicit >> > enumeration is needed to acquire the needed locks. >> >> you may not need to explicit acquire the lock for the affected key. you >> can use a global lock that is share acquired for write transactions >> without clear operation and exclusively acquire for transactions with >> clear operation (exactly what I use for Total Order). also, the clear >> transactions (i.e. the transactions with clear operation) can also only >> acquired the global lock. > > > +1 for the global lock. It may slow things down a bit in the general case, > because every tx will have to acquire it in shared mode, but it will > guarantee consistency for clear(). I had thought of that too, but it will slow down all operations, at which benefit? To have a nice transactional clear() operation which guarantees consistency? The goal of this thread is to challenge if that makes sense for real use cases, it seems in fact we would highly prefer a non transactional version, and considering that we can't run a single Cache in both modes anymore that's a problem. Side note on the implementation complexity: even the AsyncCacheStore decorator code is made extremely more tricky just to handle clear() appropriately, to keep it in the correct order of execution compared to the other operations, while still allowing "coaleshing" of other operations, which essentially implies we allow reordering of operations as long as they affect independent keys.. where clear() is of course a jolly which requires it's own execution path, its own global lock and generally makes it far more complex. > > It still won't help entrySet() and its brethren... > >> >> > >> > It might give a nice safe feeling that, when invoking a clear() >> > operation in a transaction, I can still abort the transaction to make >> > it cancel the operation; that's the only good part I can think of: we >> > can cancel it. >> >> yep > > > Wouldn't it be harder to integrate it with Total Order if it didn't have a > tx? > > But most of all, I think we have to keep it because it's a part of the Map > interface and a part of JCache's Cache interface. > >> >> >> > >> > I don't think it has anything to do with consistency though? To make >> > sure you're effectively involving all replicas of all entries in a >> > consistent way, a lock would need to be acquired on each affected key, >> > which again implies a need to enumerate all keys, including the >> > unknown keys which might be hiding in a CacheStore: it's not enough to >> > broadcast the clear() operation to all nodes and have them simply wipe >> > their local state as that's never going to deal correctly >> > (consistently) with in-flight transactions working on different nodes >> > at different times (I guess enabling Total Order could help but you'd >> > need to make it mandatory). >> > >> >> see the comment above about the locks... may not be the most efficient, >> but handles the problem. > > > Note that we need to iterate the keys anyway, in order to fire the > CacheEntryRe
Re: [infinispan-dev] ISPN-3557: interactions between a clear() operation and a Transaction
On Wed, Oct 2, 2013 at 12:48 PM, Pedro Ruivo wrote: > > > On 10/02/2013 12:03 AM, Sanne Grinovero wrote: > > I'd love to brainstorm about the clear() operation and what it means > > on Infinispan. > > > > I'm not sure to what extent, but it seems that clear() is designed to > > work in a TX, or even create an implicit transaction if needed, but > > I'm not understanding how that can work. > I think it's pretty straight-forward to define the semantics of operations after clear(): they should assume that every key in the cache was removed. True, like Pedro mentioned, the current implementation allows subsequent operations in the same tx to see keys inserted by other txs. But I don't think fixing that would be harder than ensuring that if a tx inserts key k owned by nodes A and B, clear() from outside the tx either deletes k on A and B or doesn't delete k on either node. > > > > Obviously a clear() operation isn't listing all keys explicitly. > > (offtopic) like entrySet(), keySet(), values() and iterator() when > executed in a transactional context. however, I think that this > operations are more difficult to handle, because they have to return > consistent data... > I agree, defining how entrySet() and the others should work in a tx seems harder than defining clear(). Will, I'm guessing your fix for ISPN-3560 still allows entrySet() after clear() to see entries inserted by another tx in the meantime? > > Which > > implies that it's undefined on which keys it's going to operate when > > it's fired.. that seems like terribly wrong in a distributed key/value > > store as we can't possibly freeze the global state and somehow define > > a set of keys which are going to be affected, while an explicit > > enumeration is needed to acquire the needed locks. > > you may not need to explicit acquire the lock for the affected key. you > can use a global lock that is share acquired for write transactions > without clear operation and exclusively acquire for transactions with > clear operation (exactly what I use for Total Order). also, the clear > transactions (i.e. the transactions with clear operation) can also only > acquired the global lock. > +1 for the global lock. It may slow things down a bit in the general case, because every tx will have to acquire it in shared mode, but it will guarantee consistency for clear(). It still won't help entrySet() and its brethren... > > > > It might give a nice safe feeling that, when invoking a clear() > > operation in a transaction, I can still abort the transaction to make > > it cancel the operation; that's the only good part I can think of: we > > can cancel it. > > yep > Wouldn't it be harder to integrate it with Total Order if it didn't have a tx? But most of all, I think we have to keep it because it's a part of the Map interface and a part of JCache's Cache interface. > > > > > I don't think it has anything to do with consistency though? To make > > sure you're effectively involving all replicas of all entries in a > > consistent way, a lock would need to be acquired on each affected key, > > which again implies a need to enumerate all keys, including the > > unknown keys which might be hiding in a CacheStore: it's not enough to > > broadcast the clear() operation to all nodes and have them simply wipe > > their local state as that's never going to deal correctly > > (consistently) with in-flight transactions working on different nodes > > at different times (I guess enabling Total Order could help but you'd > > need to make it mandatory). > > > > see the comment above about the locks... may not be the most efficient, > but handles the problem. > Note that we need to iterate the keys anyway, in order to fire the CacheEntryRemoved listeners. Even if we optimized it skip the iteration when there are no registered listeners, it would still have to handle the general case. > > So let's step back a second and consider what is the use case for > > clear() ? I suspect it's primarily a method needed during testing, or > > maybe cleanup before a backup is restored (operations), maybe a > > manually activated JMX operation to clear the cache in exceptional > > cases. > > I'm not aware of the uses cases of a clear() operation, but use it as > JMX operation can be a problem in transactional caches (depending the > semantic you have in mind). > > If it is going to clear all the caches (i.e. from all replicas/nodes), > then we have to find a way to set an order with the current transactions > or we will have something some nodes delivering a transaction (say tx1) > before the clear() and other nodes delivering tx1 after the clear(), > leading to an inconsistent state. > > (offtopic) I'm wondering if we have that problem in the current > implementation if tx1 is only insert new keys... in that case, we have > no lock to serialize an order between the clear() and tx1... > I'm pretty sure we do have this problem. We definitely need to add some tests for clear concu
