[hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
Incase this email didn't go to the full dev-list. I got it as a separate thread so forwarding on. Begin forwarded message: From: johng@gmail.com Date: 25 September 2009 16:00:53 BST To: Navin Surtani Subject: Re: Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new All, I think Hardy's original push back came from the first pass' use of the decorator pattern to try to come up with a DSL. That really isn't much better than knowing the API. The alternate is to come up with a more natural language implementation but that leads to parsers, lexers, etc... I'm not saying it's not worth while but it may be a lot of work. John Griffin On Sep 25, 2009 8:12am, Navin Surtani wrote: > Just wanted to get this topic re-started again. > > > > > > Essentially what I think this project/DSL/module/thingy-bob is thought > > to become: - > > > > A simple package where a user can build Lucene queries without having > > to know too much about Lucene itself. If I'm headed down the wrong > > thought path then just thwack me. > > > > > > > > On 26 Aug 2009, at 21:08, Hardy Ferentschik wrote: > > > > > On Wed, 2009-08-26 at 13:39 +0200, Emmanuel Bernard wrote: > > >> I've been thinking about a DSL to build Lucene queries in the last > > >> day. > > >> What do you think of this proposal? > > > > > > What do you really gain compared to native Lucene queries? > > > > What's gained I believe is the fact that people can build complex > > lucene queries easier. Currently, it's a bit clunky imo so if we > > provide a cleaner way to build them it can prove beneficial to any > > lucene user (myself included for querying on Infinispan). > > > > Any other thoughts? > > > > > > > If your API achieves exactly the same as what's possible with Lucene > > > it is just a 'useless' wrapper. > > > > > > A wrapper around native Lucene queries would make sense if it could > > > somehow use some of the Hibernate Search specific meta data. As an > > > extreme example one could generate some meta classes a la JPA2. This > > > way > > > one could ensure that you can get help with which field names are > > > available. > > > > > > --Hardy > > > > > > ___ > > > infinispan-dev mailing list > > > infinispan-...@lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > Navin Surtani > > > > Intern Infinispan > > Intern JBoss Cache Searchable > > > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > Navin Surtani Intern Infinispan Intern JBoss Cache Searchable ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
Lexers are parsers are not difficult (a simple ANTLR grammar takes very little effort, for example), but that is orthoginal to this discussion. I believe the very term DSL is misleading (since DSL implies a separate grammar), but this is more for an intuitive builder API for query creation. And yes, while this does imply knowledge of the API, it is certainly far less verbose and more expressive than the existing query construction mechanism. Perhaps as a next step, a grammar can be built on top of this. - Manik On 25 Sep 2009, at 18:41, Navin Surtani wrote: Incase this email didn't go to the full dev-list. I got it as a separate thread so forwarding on. Begin forwarded message: From: johng@gmail.com Date: 25 September 2009 16:00:53 BST To: Navin Surtani Subject: Re: Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new All, I think Hardy's original push back came from the first pass' use of the decorator pattern to try to come up with a DSL. That really isn't much better than knowing the API. The alternate is to come up with a more natural language implementation but that leads to parsers, lexers, etc... I'm not saying it's not worth while but it may be a lot of work. John Griffin On Sep 25, 2009 8:12am, Navin Surtani wrote: > Just wanted to get this topic re-started again. > > > > > > Essentially what I think this project/DSL/module/thingy-bob is thought > > to become: - > > > > A simple package where a user can build Lucene queries without having > > to know too much about Lucene itself. If I'm headed down the wrong > > thought path then just thwack me. > > > > > > > > On 26 Aug 2009, at 21:08, Hardy Ferentschik wrote: > > > > > On Wed, 2009-08-26 at 13:39 +0200, Emmanuel Bernard wrote: > > >> I've been thinking about a DSL to build Lucene queries in the last > > >> day. > > >> What do you think of this proposal? > > > > > > What do you really gain compared to native Lucene queries? > > > > What's gained I believe is the fact that people can build complex > > lucene queries easier. Currently, it's a bit clunky imo so if we > > provide a cleaner way to build them it can prove beneficial to any > > lucene user (myself included for querying on Infinispan). > > > > Any other thoughts? > > > > > > > If your API achieves exactly the same as what's possible with Lucene > > > it is just a 'useless' wrapper. > > > > > > A wrapper around native Lucene queries would make sense if it could > > > somehow use some of the Hibernate Search specific meta data. As an > > > extreme example one could generate some meta classes a la JPA2. This > > > way > > > one could ensure that you can get help with which field names are > > > available. > > > > > > --Hardy > > > > > > ___ > > > infinispan-dev mailing list > > > infinispan-...@lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > Navin Surtani > > > > Intern Infinispan > > Intern JBoss Cache Searchable > > > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > Navin Surtani Intern Infinispan Intern JBoss Cache Searchable ___ infinispan-dev mailing list infinispan-...@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
On 27 sept. 09, at 12:08, Manik Surtani wrote: > Lexers are parsers are not difficult (a simple ANTLR grammar takes > very little effort, for example), but that is orthoginal to this > discussion. I believe the very term DSL is misleading (since DSL > implies a separate grammar), but this is more for an intuitive > builder API for query creation. And yes, while this does imply > knowledge of the API, it is certainly far less verbose and more > expressive than the existing query construction mechanism. That's untrue. External DSLs imply a separate grammar. Internal DSLs imply using the hosting language to create a fluent API. ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
On 27 Sep 2009, at 11:26, Sanne Grinovero wrote: > Next version of Lucene will provide helpers and tools to make it easy > to create your own QueryParser, so that everyone can make his one > based on business-specific needs > (Everybody can already, but is not as easy). > So I would avoid reinventing that, and focus on a good API. > > To make this API very cool IMHO it should integrate with Hibernate > Search to exploit all knowledge about object mapping, declarative > Analyzer and Filter definitions, and so on... > We concluded in another thread that to make best use of this > information we need to give the type of what you're searching for as a > parameter, so that from the specific > mapping the analyzers, fields and fieldbridges can be matched. > Should be fine, it means we can provide typesafe results? Are you suggesting that this is made to be specific to Hibernate Search, rather than more generic, for Lucene? As far as Infinispan is concerned I couldn't care either way since we just depend on HS. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
On 28 sept. 09, at 10:48, Manik Surtani wrote: > > On 27 Sep 2009, at 11:26, Sanne Grinovero wrote: > >> Next version of Lucene will provide helpers and tools to make it easy >> to create your own QueryParser, so that everyone can make his one >> based on business-specific needs >> (Everybody can already, but is not as easy). >> So I would avoid reinventing that, and focus on a good API. >> >> To make this API very cool IMHO it should integrate with Hibernate >> Search to exploit all knowledge about object mapping, declarative >> Analyzer and Filter definitions, and so on... >> We concluded in another thread that to make best use of this >> information we need to give the type of what you're searching for >> as a >> parameter, so that from the specific >> mapping the analyzers, fields and fieldbridges can be matched. >> Should be fine, it means we can provide typesafe results? > > Are you suggesting that this is made to be specific to Hibernate > Search, rather than more generic, for Lucene? As far as Infinispan is > concerned I couldn't care either way since we just depend on HS. Yes, if you check the thread history we've discussed the benefits of tying the DSL to Hibernate Search's metadata. We could basically save a lot of pain points like analyzers and type conversion for users (and we can't if we stay pure Lucene based). ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new
On 28 Sep 2009, at 10:08, Emmanuel Bernard wrote: > > On 28 sept. 09, at 10:48, Manik Surtani wrote: > >> >> On 27 Sep 2009, at 11:26, Sanne Grinovero wrote: >> >>> Next version of Lucene will provide helpers and tools to make it >>> easy >>> to create your own QueryParser, so that everyone can make his one >>> based on business-specific needs >>> (Everybody can already, but is not as easy). >>> So I would avoid reinventing that, and focus on a good API. >>> >>> To make this API very cool IMHO it should integrate with Hibernate >>> Search to exploit all knowledge about object mapping, declarative >>> Analyzer and Filter definitions, and so on... >>> We concluded in another thread that to make best use of this >>> information we need to give the type of what you're searching for >>> as a >>> parameter, so that from the specific >>> mapping the analyzers, fields and fieldbridges can be matched. >>> Should be fine, it means we can provide typesafe results? >> >> Are you suggesting that this is made to be specific to Hibernate >> Search, rather than more generic, for Lucene? As far as Infinispan >> is >> concerned I couldn't care either way since we just depend on HS. > > Yes, if you check the thread history we've discussed the benefits of > tying the DSL to Hibernate Search's metadata. We could basically save > a lot of pain points like analyzers and type conversion for users (and > we can't if we stay pure Lucene based). Yup, makes sense. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
Very elegant. I'm generally a big fan of 'builder' patterns like
this, but this really isn't a DSL, is it? :) When you first
mentioned a DSL I had visions of defining a new grammar and an ANTLR
parser, etc. But that is overkill.
This approach certainly works, and will almost certainly perform
better too. One question: for the sake of brevity, why
SealedQueryBuilder instead of QueryBuilder ? :)
Also, I still think that if this is a generic helper factory that
helps you build Lucene queries - and has no knowledge of how and where
the query is used (why should it?) - then this should be something
people can use outside of HS or Infinispan. E.g., directly with Lucene.
On 26 Aug 2009, at 12:39, Emmanuel Bernard wrote:
I've been thinking about a DSL to build Lucene queries in the last
day.
What do you think of this proposal?
A few remarks:
- it asks the analyzer so that we correctly apply the analyzer on
terms
- it has a few query factory methods
- it contains a few orthogonal operations
- I am not quite satisfied with how boolean is handled, any idea?
Examples
SealedQueryBuilder qb =
searchFactory.withEntityAnalyzer(Address.class);
Query luceneQuery =
qb.must(Occurs.MUST)
.add(
qb.boolean(Occurs.Should)
.add( qb.term("city",
"Atlanta").boostedTo(4).createQuery() )
.add( qb.term("address1",
"Peachtree").fuzzy().createQuery() )
)
.add(
qb.from("movingDate",
"200604").to("201201").exclusive().createQuery()
)
.createQuery();
Analyzer choice
queryBuilder.withAnalyzer(Analyzer)
queryBuilder.withEntityAnalyzer(Class)
queryBuilder.basedOnEntityAnalyzer(Class)
.overridesForField(String field, Analyzer)
.overridesForField(String field, Analyzer)
.build() //sucky name
returns a SealedQueryBuilder //sucky name
SealedQueryBuilder contains the factory methods
Factory methods
Hosted onSealedQueryBuilder
.term(String field, String text) //define a new query
.term(String field, String text) //define a new query
.ignoreAnalyzer() //ignore the analyzer, optional
.fuzzy() //API prevent wildcard calls, optional
.threshold() //optional
.prefixLengh() //optional
.term(String field, String value)
.wildcard() //API prevent fuzzy calls, optional
//range query
.from(String field, String text)
.exclusive() //optional
.to(String text)
.exclusive() //optional
.constantScore() //optional, due to constantScoreRangeQuery but
in practice inherited from the common operations
//match all docs
.all()
//phrase query
.phrase(String field)
.ignoreAnalyzer() //ignore the analyzer, optional
.addWord(String text) //at least one
.addWord(String text)
.sentence(String text) //do we need that?
.slop() //optional
//search multiple fields for same value
.searchInMultipleFields()
.onField(String field)
.boostedTo(float) //optional
.ignoreAnalyzer() //optional
.onField(String field)
.forWords(String) //do we need that?
.forWord(String)
Boolean operations
SealedQueryBuilder contains the boolean methods
.boolean(Occurs occurs)
.add( qb.from().to() )
.add( ... )
Works on all queries
.boostedTo()
.constantScore()
.filter(Filter) //filter the current query
.scoreMultipliedByField(field) //FieldScoreQuery +
FunctionQuery?? //Not backed
.createQuery()
Todo
Span*Queries
MultiPhraseQuery - needs to fillup all accepted terms
FieldScoreQuery
ValueSourceQuery
FuzzyLikeThis
MoreLikeThis
On 25 août 09, at 16:43, Manik Surtani wrote:
On 25 Aug 2009, at 13:34, Emmanuel Bernard wrote:
On 25 août 09, at 14:27, Manik Surtani wrote:
A DSL would work, but I'd rather not define our own language here.
Which is why I asked for a standard. Perhaps something based on
SQL/
JPA-QL? Or are you thinking DSL specific to Lucene - which could
be used by any/all of {Lucene, Hibernate Search, Infinispan}? In
which case the DSL should ideally be a Lucene project.
Yes I was thinking about a DSL used for Hibernate Search and maybe
all
of Lucene if the HS integration benefits offer no value towards
simplicity (but I think i can offer value).
Ok, this should be interesting. Lets chat about this some more -
have
you drafted any thoughts around this DSL somewhere?
___
infinispan-dev mailing list
infinispan-...@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
ma...@jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
On 27 août 09, at 13:07, Manik Surtani wrote: > Very elegant. I'm generally a big fan of 'builder' patterns like > this, but this really isn't a DSL, is it? :) When you first > mentioned a DSL I had visions of defining a new grammar and an ANTLR > parser, etc. But that is overkill. This is called an internal DSL, ie you use Java as the language, not some external representation. > > This approach certainly works, and will almost certainly perform > better too. One question: for the sake of brevity, why > SealedQueryBuilder instead of QueryBuilder ? :) The name is not right yet There are two things: - the query builder that lets you define the analyzer - the query builder that has an analyzer assigned and lets you build query What name is best for each of them. > > Also, I still think that if this is a generic helper factory that > helps you build Lucene queries - and has no knowledge of how and > where the query is used (why should it?) - then this should be > something people can use outside of HS or Infinispan. E.g., > directly with Lucene. As of today this code is technically pure Lucene but to be honest the idea of passing an analyzer multiplexer (like the one we receive from searchFactory.getAnalyzer)) is not wildly spread in Lucene and potentially cumbersome wo the declarative approach of HSearch. The second problem is that some potential improvements will require inner knowledge of HSearch: - object parameters (and not string params) do require to know the FieldBridge of the property. This is a pure HSearch notion. - "property literal" like JPA is introducing could be added to replace the String-based field approach in some situations. Though I don't think that it would be a perfect fit. - spell checker (the old idea we had) That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL. ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
On 27 Aug 2009, at 13:06, Emmanuel Bernard wrote: On 27 août 09, at 13:07, Manik Surtani wrote: Very elegant. I'm generally a big fan of 'builder' patterns like this, but this really isn't a DSL, is it? :) When you first mentioned a DSL I had visions of defining a new grammar and an ANTLR parser, etc. But that is overkill. This is called an internal DSL, ie you use Java as the language, not some external representation. This approach certainly works, and will almost certainly perform better too. One question: for the sake of brevity, why SealedQueryBuilder instead of QueryBuilder ? :) The name is not right yet There are two things: - the query builder that lets you define the analyzer - the query builder that has an analyzer assigned and lets you build query What name is best for each of them. I thought this stuff you mentioned made sense: queryBuilder.withAnalyzer(Analyzer) queryBuilder.withEntityAnalyzer(Class) queryBuilder.basedOnEntityAnalyzer(Class) .overridesForField(String field, Analyzer) .overridesForField(String field, Analyzer) .build() //sucky name Perhaps rename the static factory methods to something like: QueryBuilder.getQueryBuilder(Analyzer) QueryBuilder.getQueryBuilder(Class) and QueryBuilder instances have overrideAnalyzerForField(String, Analyzer). Why do you need the build() method at the end? Also, I still think that if this is a generic helper factory that helps you build Lucene queries - and has no knowledge of how and where the query is used (why should it?) - then this should be something people can use outside of HS or Infinispan. E.g., directly with Lucene. As of today this code is technically pure Lucene but to be honest the idea of passing an analyzer multiplexer (like the one we receive from searchFactory.getAnalyzer)) is not wildly spread in Lucene and potentially cumbersome wo the declarative approach of HSearch. The second problem is that some potential improvements will require inner knowledge of HSearch: - object parameters (and not string params) do require to know the FieldBridge of the property. This is a pure HSearch notion. - "property literal" like JPA is introducing could be added to replace the String-based field approach in some situations. Though I don't think that it would be a perfect fit. - spell checker (the old idea we had) That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL. Not it doesn't have to be either ASL or even hosted at Apache. I guess what I am suggesting is perhaps even a separate project - LuceneQueryBuilder or something - which plain-old-Lucene users could use as well. Doesn't matter where it's hosted or what the license is - as long as its ASL or LGPL :) -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
queryBuilder.withAnalyzer(Analyzer) queryBuilder.withEntityAnalyzer(Class) queryBuilder.basedOnEntityAnalyzer(Class) .overridesForField(String field, Analyzer) .overridesForField(String field, Analyzer) .build() //sucky name Perhaps rename the static factory methods to something like: QueryBuilder.getQueryBuilder(Analyzer) QueryBuilder.getQueryBuilder(Class) and QueryBuilder instances have overrideAnalyzerForField(String, Analyzer). Why do you need the build() method at the end? if you do that, all of the sudden, a QB can change it's analyzer on the fly making it immutable. Also the overridesForField methods would pollute the API when it's time to create a query. One of the advantages of a fluent API in a strongly typed environment is that we can hide methods that are meaningless in a given context. That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL. Not it doesn't have to be either ASL or even hosted at Apache. I guess what I am suggesting is perhaps even a separate project - LuceneQueryBuilder or something - which plain-old-Lucene users could use as well. Doesn't matter where it's hosted or what the license is - as long as its ASL or LGPL :) Let's start it under the Hibernate Search umbrella due to potential synergies and spin it out if needed.___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
On 27 Aug 2009, at 16:10, Emmanuel Bernard wrote: queryBuilder.withAnalyzer(Analyzer) queryBuilder.withEntityAnalyzer(Class) queryBuilder.basedOnEntityAnalyzer(Class) .overridesForField(String field, Analyzer) .overridesForField(String field, Analyzer) .build() //sucky name Perhaps rename the static factory methods to something like: QueryBuilder.getQueryBuilder(Analyzer) QueryBuilder.getQueryBuilder(Class) and QueryBuilder instances have overrideAnalyzerForField(String, Analyzer). Why do you need the build() method at the end? if you do that, all of the sudden, a QB can change it's analyzer on the fly making it immutable. Also the overridesForField methods would pollute the API when it's time to create a query. One of the advantages of a fluent API in a strongly typed environment is that we can hide methods that are meaningless in a given context. That been said, if the API ends up being pure Lucene and once we stabilize it, we can contribute it back even though I am not necessarily a huge fan of ASL. Not it doesn't have to be either ASL or even hosted at Apache. I guess what I am suggesting is perhaps even a separate project - LuceneQueryBuilder or something - which plain-old-Lucene users could use as well. Doesn't matter where it's hosted or what the license is - as long as its ASL or LGPL :) Let's start it under the Hibernate Search umbrella due to potential synergies and spin it out if needed. Ok. Just make sure we use a different maven module or something so that there are no dependencies on the rest of HS or Hibernate. Otherwise spinning out will be a PITA. Lucene should be the only dependencies of this code. Cheers -- Manik Surtani ma...@jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
That's what I said as well. Separate maven module. However, one can feel a disturbance in the force when one mentions MAVEN MODULE within the Hibernate Team :) On Thu, 27 Aug 2009 17:18:23 +0200, Manik Surtani wrote: > Ok. Just make sure we use a different maven module or something so that > there are no dependencies on the rest of HS or Hibernate. Otherwise > spinning out will be a PITA. Lucene should be the only dependencies of > this code. ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
I've nothing against a separate maven module, still Hibernate Search already has lots of "goodies" to work with Lucene which are not necessarily linked to Hibernate (e.g. Analyzer definition helpers, pojo mapping through annotations, enhanced filtering, IndexReader pooling, nice Infinispan Directory...) so this new query builder is not much different. Just a thought. So even if Emmanuel has shown this builder to be useful even with this limited features, it could become even more useful when strongly combined with the other features; 2 come to mind, may be more later: A) adding filters to the builders; I don't think it would be easy to have named filters without the full Search package B) Letting the users forget about the Analyzer matches complexity (optionally), as by using the mapping information we could default to a reasonable Analyzer for each field. Most users on the forum are in trouble because they select the wrong analyzer/ forget to use one when building the F.T.Query. IMHO these are good reasons to couple it to the rest of the code; Maybe it would be possible in future to have Hibernate optional. Sanne 2009/8/27 Manik Surtani : > > On 27 Aug 2009, at 16:10, Emmanuel Bernard wrote: > > > queryBuilder.withAnalyzer(Analyzer) > queryBuilder.withEntityAnalyzer(Class) > queryBuilder.basedOnEntityAnalyzer(Class) > .overridesForField(String field, Analyzer) > .overridesForField(String field, Analyzer) > .build() //sucky name > > Perhaps rename the static factory methods to something like: > QueryBuilder.getQueryBuilder(Analyzer) > QueryBuilder.getQueryBuilder(Class) > and QueryBuilder instances have overrideAnalyzerForField(String, Analyzer). > Why do you need the build() method at the end? > > if you do that, all of the sudden, a QB can change it's analyzer on the fly > making it immutable. > Also the overridesForField methods would pollute the API when it's time to > create a query. > One of the advantages of a fluent API in a strongly typed environment is > that we can hide methods that are meaningless in a given context. > > That been said, if the API ends up being pure Lucene and once we stabilize > it, we can contribute it back even though I am not necessarily a huge fan of > ASL. > > Not it doesn't have to be either ASL or even hosted at Apache. I guess what > I am suggesting is perhaps even a separate project - LuceneQueryBuilder or > something - which plain-old-Lucene users could use as well. Doesn't matter > where it's hosted or what the license is - as long as its ASL or LGPL :) > > Let's start it under the Hibernate Search umbrella due to potential > synergies and spin it out if needed. > > Ok. Just make sure we use a different maven module or something so that > there are no dependencies on the rest of HS or Hibernate. Otherwise > spinning out will be a PITA. Lucene should be the only dependencies of this > code. > Cheers > -- > Manik Surtani > ma...@jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > ___ > infinispan-dev mailing list > infinispan-...@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
Sure I like it! I'm in the swamp of old mails, so I give you my first
impression only:
Even if it's fluent it's not (yet) intuitive to me which methods I should call;
Query luceneQuery =
qb.must(Occurs.MUST)
.add(
qb.boolean(Occurs.Should)
.add( qb.term("city", "Atlanta").boostedTo(4).createQuery() )
.add( qb.term("address1", "Peachtree").fuzzy().createQuery() )
)
.add(
qb.from("movingDate", "200604").to("201201").exclusive().createQuery()
)
.createQuery();
I guess there is a typo? As "must(MUST)" is a bit confusing to me.
why not
qb.booleanQuery()
.Must( qb.otherQuery(...).. )
.Should( qb.secondQuery(..).. )
.build();
and
qb.termQuery("city", "Atlanta").boostedTo(4).createQuery())
or even overloading
qb.termQuery("city", "Atlanta").createQuery())
with
qb.termQuery("city", "Atlanta", 4f).createQuery())
is not as readable as "boostedTo" method but more immediate;
intelligent IDEs should propose the options to devs while typing, even
guessing the parameter name and making it's meaning self-evident.
qb.rangeQuery could be either
rangeQuery("field", "fromX", "toY")
or
rangeQuery("field").from("x").to("y")
so why are you choosing ("field","from").to("to") ?
Thinking about the RangeQuery on dates, it would be cool to accept any
type for which we have Bridges, like accepting Date type or even a
user-defined FieldBridge together with an Object.
I like the Analyzer choices, it would be very cool if we could by
default guess the correct one from the searched-for entity types.
We could even consider a Query-By-Example query builder, reading
indexed fields from an instance of an indexed type, or something like
HSEARCH-119 proposal (for termvectors similatory).
cheers,
Sanne
2009/8/28 Emmanuel Bernard :
> Hey Sanne,
> What do you think of the PAI proposal itself?
> Like it? See improvements?
>
> On 28 août 09, at 10:37, Sanne Grinovero wrote:
>
>> I've nothing against a separate maven module, still Hibernate Search
>> already has lots of "goodies" to work with Lucene which are not
>> necessarily linked to Hibernate (e.g. Analyzer definition helpers,
>> pojo mapping through annotations, enhanced filtering, IndexReader
>> pooling, nice Infinispan Directory...) so this new query builder is
>> not much different. Just a thought.
>>
>> So even if Emmanuel has shown this builder to be useful even with this
>> limited features, it could become even more useful when strongly
>> combined with the other features; 2 come to mind, may be more later:
>>
>> A) adding filters to the builders; I don't think it would be easy to
>> have named filters without the full Search package
>>
>> B) Letting the users forget about the Analyzer matches complexity
>> (optionally), as by using the mapping information we could default to
>> a reasonable Analyzer for each field. Most users on the forum are in
>> trouble because they select the wrong analyzer/ forget to use one when
>> building the F.T.Query.
>>
>> IMHO these are good reasons to couple it to the rest of the code;
>> Maybe it would be possible in future to have Hibernate optional.
>>
>> Sanne
>>
>>
>> 2009/8/27 Manik Surtani :
>>>
>>> On 27 Aug 2009, at 16:10, Emmanuel Bernard wrote:
>>>
>>>
>>> queryBuilder.withAnalyzer(Analyzer)
>>> queryBuilder.withEntityAnalyzer(Class)
>>> queryBuilder.basedOnEntityAnalyzer(Class)
>>> .overridesForField(String field, Analyzer)
>>> .overridesForField(String field, Analyzer)
>>> .build() //sucky name
>>>
>>> Perhaps rename the static factory methods to something like:
>>> QueryBuilder.getQueryBuilder(Analyzer)
>>> QueryBuilder.getQueryBuilder(Class)
>>> and QueryBuilder instances have overrideAnalyzerForField(String,
>>> Analyzer).
>>> Why do you need the build() method at the end?
>>>
>>> if you do that, all of the sudden, a QB can change it's analyzer on the
>>> fly
>>> making it immutable.
>>> Also the overridesForField methods would pollute the API when it's time
>>> to
>>> create a query.
>>> One of the advantages of a fluent API in a strongly typed environment is
>>> that we can hide methods that are meaningless in a given context.
>>>
>>> That been said, if the API ends up being pure Lucene and once we
>>> stabilize
>>> it, we can contribute it back even though I am not necessarily a huge fan
>>> of
>>> ASL.
>>>
>>> Not it doesn't have to be either ASL or even hosted at Apache. I guess
>>> what
>>> I am suggesting is perhaps even a separate project - LuceneQueryBuilder
>>> or
>>> something - which plain-old-Lucene users could use as well. Doesn't
>>> matter
>>> where it's hosted or what the license is - as long as its ASL or LGPL :)
>>>
>>> Let's start it under the Hibernate Search umbrella due to potential
>>> synergies and spin it out if needed.
>>>
>>> Ok. Just make sure we use a different maven module or something so that
>>> there are no dependencies on the rest of HS or Hibernate. Otherwise
>>
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
On 28 août 09, at 07:54, Sanne Grinovero wrote:
> Sure I like it! I'm in the swamp of old mails, so I give you my first
> impression only:
> Even if it's fluent it's not (yet) intuitive to me which methods I
> should call;
>
> Query luceneQuery =
> qb.must(Occurs.MUST)
>.add(
>qb.boolean(Occurs.Should)
>.add( qb.term("city",
> "Atlanta").boostedTo(4).createQuery() )
>.add( qb.term("address1",
> "Peachtree").fuzzy().createQuery() )
>)
>.add(
>qb.from("movingDate",
> "200604").to("201201").exclusive().createQuery()
>)
>.createQuery();
>
> I guess there is a typo? As "must(MUST)" is a bit confusing to me.
yep typo.
> why not
>
> qb.booleanQuery()
> .Must( qb.otherQuery(...).. )
> .Should( qb.secondQuery(..).. )
> .build();
good idea,
.must()
.add( qb.from().to() )
.add( ... )
.must().not()
.should()
>
> and
> qb.termQuery("city", "Atlanta").boostedTo(4).createQuery())
> or even overloading
> qb.termQuery("city", "Atlanta").createQuery())
> with
> qb.termQuery("city", "Atlanta", 4f).createQuery())
> is not as readable as "boostedTo" method but more immediate;
> intelligent IDEs should propose the options to devs while typing, even
> guessing the parameter name and making it's meaning self-evident.
No I find it hard and confusing even in IntelliJ IDEA to use
overloaded methods and a set of parameters.
The idea behing boostedTo() is to mimick named parameters.
Actually ideally we would want term().on("city").matches("Atlanta"),
WDYT?
Or even qb.on("city").matches("atlanta")
which could also be used to unify range queries and multi term queries
qb.on("date").from("2009").to("2010")
qb.on("a").on("b").matches("word")
>
> qb.rangeQuery could be either
> rangeQuery("field", "fromX", "toY")
> or
> rangeQuery("field").from("x").to("y")
> so why are you choosing ("field","from").to("to") ?
I think
range("field").from("x").to("y")
or
on("field").from("x").to("y")
are better, I've move to range and if we decide on is better, I will
move.
>
> Thinking about the RangeQuery on dates, it would be cool to accept any
> type for which we have Bridges, like accepting Date type or even a
> user-defined FieldBridge together with an Object.
That would be cool but realistically we need to accept Object unless
somehow we can convey the "expected" type(s) of a field bridge using
generics or something. SOmebody has an idea on how the API would look
like?
probably something like
on(String field, Class) ...
on("field", Date.class).from(new Date()).
>
> I like the Analyzer choices, it would be very cool if we could by
> default guess the correct one from the searched-for entity types.
Define guessing. Guessing the targeted entity? How?
>
> We could even consider a Query-By-Example query builder, reading
> indexed fields from an instance of an indexed type, or something like
> HSEARCH-119 proposal (for termvectors similatory).
+1 but let's start (not so) modest :)
>
> cheers,
> Sanne
>
> 2009/8/28 Emmanuel Bernard :
>> Hey Sanne,
>> What do you think of the PAI proposal itself?
>> Like it? See improvements?
>>
>> On 28 août 09, at 10:37, Sanne Grinovero wrote:
>>
>>> I've nothing against a separate maven module, still Hibernate Search
>>> already has lots of "goodies" to work with Lucene which are not
>>> necessarily linked to Hibernate (e.g. Analyzer definition helpers,
>>> pojo mapping through annotations, enhanced filtering, IndexReader
>>> pooling, nice Infinispan Directory...) so this new query builder is
>>> not much different. Just a thought.
>>>
>>> So even if Emmanuel has shown this builder to be useful even with
>>> this
>>> limited features, it could become even more useful when strongly
>>> combined with the other features; 2 come to mind, may be more later:
>>>
>>> A) adding filters to the builders; I don't think it would be easy to
>>> have named filters without the full Search package
>>>
>>> B) Letting the users forget about the Analyzer matches complexity
>>> (optionally), as by using the mapping information we could default
>>> to
>>> a reasonable Analyzer for each field. Most users on the forum are in
>>> trouble because they select the wrong analyzer/ forget to use one
>>> when
>>> building the F.T.Query.
>>>
>>> IMHO these are good reasons to couple it to the rest of the code;
>>> Maybe it would be possible in future to have Hibernate optional.
>>>
>>> Sanne
>>>
>>>
>>> 2009/8/27 Manik Surtani :
On 27 Aug 2009, at 16:10, Emmanuel Bernard wrote:
queryBuilder.withAnalyzer(Analyzer)
queryBuilder.withEntityAnalyzer(Class)
queryBuilder.basedOnEntityAnalyzer(Class)
.overridesForField(String field, Analyzer)
.overridesForField(String field, Analyzer)
.build() //sucky name
Perhaps rename the static factory methods to something like:
QueryBuilder.getQueryBuilder(Analyzer)
QueryBu
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
answers and ideas inline:
2009/9/3 Emmanuel Bernard :
>
> On 28 août 09, at 07:54, Sanne Grinovero wrote:
>
>> Sure I like it! I'm in the swamp of old mails, so I give you my first
>> impression only:
>> Even if it's fluent it's not (yet) intuitive to me which methods I should
>> call;
>>
>> Query luceneQuery =
>> qb.must(Occurs.MUST)
>> .add(
>> qb.boolean(Occurs.Should)
>> .add( qb.term("city", "Atlanta").boostedTo(4).createQuery() )
>> .add( qb.term("address1", "Peachtree").fuzzy().createQuery() )
>> )
>> .add(
>> qb.from("movingDate",
>> "200604").to("201201").exclusive().createQuery()
>> )
>> .createQuery();
>>
>> I guess there is a typo? As "must(MUST)" is a bit confusing to me.
>
> yep typo.
>
>> why not
>>
>> qb.booleanQuery()
>> .Must( qb.otherQuery(...).. )
>> .Should( qb.secondQuery(..).. )
>> .build();
>
> good idea,
>
> .must()
> .add( qb.from().to() )
> .add( ... )
> .must().not()
> .should()
>
>>
>> and
>> qb.termQuery("city", "Atlanta").boostedTo(4).createQuery())
>> or even overloading
>> qb.termQuery("city", "Atlanta").createQuery())
>> with
>> qb.termQuery("city", "Atlanta", 4f).createQuery())
>> is not as readable as "boostedTo" method but more immediate;
>> intelligent IDEs should propose the options to devs while typing, even
>> guessing the parameter name and making it's meaning self-evident.
>
> No I find it hard and confusing even in IntelliJ IDEA to use overloaded
> methods and a set of parameters.
> The idea behing boostedTo() is to mimick named parameters.
>
> Actually ideally we would want term().on("city").matches("Atlanta"), WDYT?
> Or even qb.on("city").matches("atlanta")
yes much better
>
> which could also be used to unify range queries and multi term queries
> qb.on("date").from("2009").to("2010")
nice
> qb.on("a").on("b").matches("word")
this is not really fluent, I only understand what you mean because of
your comment
>
>>
>> qb.rangeQuery could be either
>> rangeQuery("field", "fromX", "toY")
>> or
>> rangeQuery("field").from("x").to("y")
>> so why are you choosing ("field","from").to("to") ?
>
> I think
> range("field").from("x").to("y")
> or
> on("field").from("x").to("y")
> are better, I've move to range and if we decide on is better, I will move.
>
I agree with your choice
>>
>> Thinking about the RangeQuery on dates, it would be cool to accept any
>> type for which we have Bridges, like accepting Date type or even a
>> user-defined FieldBridge together with an Object.
>
> That would be cool but realistically we need to accept Object unless somehow
> we can convey the "expected" type(s) of a field bridge using generics or
> something. SOmebody has an idea on how the API would look like?
> probably something like
> on(String field, Class) ...
>
> on("field", Date.class).from(new Date()).
I didn't mean that the method signature should be aware of the type,
Object is fine;
from( Object lowerBoundary ); //automatically finds out the
StringBridge from the BridgeFactory
from( Object lowerBoundary, StringBridge ); //overrides ths
StringBridge selection
Even better making StringBridge generic we get typesafety:
void from(T lowerBoundary, StringBridge bridge)
>
>>
>> I like the Analyzer choices, it would be very cool if we could by
>> default guess the correct one from the searched-for entity types.
>
> Define guessing. Guessing the targeted entity? How?
Ha right; I was assuming you were going to select the targeted entity, like
with Criteria?
>
>>
>> We could even consider a Query-By-Example query builder, reading
>> indexed fields from an instance of an indexed type, or something like
>> HSEARCH-119 proposal (for termvectors similatory).
>
>
> +1 but let's start (not so) modest :)
>
>>
>> cheers,
>> Sanne
>>
>> 2009/8/28 Emmanuel Bernard :
>>>
>>> Hey Sanne,
>>> What do you think of the PAI proposal itself?
>>> Like it? See improvements?
>>>
>>> On 28 août 09, at 10:37, Sanne Grinovero wrote:
>>>
I've nothing against a separate maven module, still Hibernate Search
already has lots of "goodies" to work with Lucene which are not
necessarily linked to Hibernate (e.g. Analyzer definition helpers,
pojo mapping through annotations, enhanced filtering, IndexReader
pooling, nice Infinispan Directory...) so this new query builder is
not much different. Just a thought.
So even if Emmanuel has shown this builder to be useful even with this
limited features, it could become even more useful when strongly
combined with the other features; 2 come to mind, may be more later:
A) adding filters to the builders; I don't think it would be easy to
have named filters without the full Search package
B) Letting the users forget about the Analyzer matches complexity
(optionally), as by using the mapping information we could default to
a reasonable Analyzer for each field. Most users on the forum are in
trouble because they select the wrong ana
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
I don't see how people would nicely have access to the right bridge
implementation
On 3 sept. 09, at 17:30, Sanne Grinovero wrote:
>>
>> That would be cool but realistically we need to accept Object
>> unless somehow
>> we can convey the "expected" type(s) of a field bridge using
>> generics or
>> something. SOmebody has an idea on how the API would look like?
>> probably something like
>> on(String field, Class) ...
>>
>> on("field", Date.class).from(new Date()).
>
> I didn't mean that the method signature should be aware of the type,
> Object is fine;
> from( Object lowerBoundary ); //automatically finds out the
> StringBridge from the BridgeFactory
> from( Object lowerBoundary, StringBridge ); //overrides ths
> StringBridge selection
>
> Even better making StringBridge generic we get typesafety:
> void from(T lowerBoundary, StringBridge bridge)
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [infinispan-dev] [HSearch] DSL for Lucene queries (was: Re: Query module new API and configurations)
Just wanted to get this topic re-started again. Essentially what I think this project/DSL/module/thingy-bob is thought to become: - A simple package where a user can build Lucene queries without having to know too much about Lucene itself. If I'm headed down the wrong thought path then just thwack me. On 26 Aug 2009, at 21:08, Hardy Ferentschik wrote: > On Wed, 2009-08-26 at 13:39 +0200, Emmanuel Bernard wrote: >> I've been thinking about a DSL to build Lucene queries in the last >> day. >> What do you think of this proposal? > > What do you really gain compared to native Lucene queries? What's gained I believe is the fact that people can build complex lucene queries easier. Currently, it's a bit clunky imo so if we provide a cleaner way to build them it can prove beneficial to any lucene user (myself included for querying on Infinispan). Any other thoughts? > If your API achieves exactly the same as what's possible with Lucene > it is just a 'useless' wrapper. > > A wrapper around native Lucene queries would make sense if it could > somehow use some of the Hibernate Search specific meta data. As an > extreme example one could generate some meta classes a la JPA2. This > way > one could ensure that you can get help with which field names are > available. > > --Hardy > > ___ > infinispan-dev mailing list > infinispan-...@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev Navin Surtani Intern Infinispan Intern JBoss Cache Searchable ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
