Re: Hierarchical faceting
Check out project Bobo: http://sna-projects.com/bobo/ A lucene based faceted search library. Now with solr plugin: http://snaprojects.jira.com/wiki/display/BOBO/Bobo+Solr+Plugin -John On Fri, Aug 13, 2010 at 8:56 PM, Jayendra Patil < jayendra.patil@gmail.com> wrote: > Multiple values are probably same as Multiple Tokens with a high position > increment gap. > Would still prefer to go with the multivalued field approach, as it is > inbuilt and easier to get back the individual facets with the count in the > response. > > Regards, > Jayendra > > On Fri, Aug 13, 2010 at 7:57 AM, Mats Bolstad wrote: > > > Thank you for your answer. I sure will implement something in that > > direction. > > > > But couldn't multiple tokens be used instead of multiple values? > > > > > > > >// some tokenizer and filters that generates "0//Europe", > > "1//Europe//Norway", "2//Europe//Norway//Oslo" > > > > > > > > > > > > Wouldn't that work in just the same way? Or is it some difference > > between multiple tokens and multiple values that I'm missing? > > > > -- > > Regards, > > Mats Bolstad > > > > > > > > On Fri, Aug 13, 2010 at 2:21 AM, Jayendra Patil > > wrote: > > > We were able to get the hierarchy faceting working with a work around > > > approach. > > > > > > e.g. if you have Europe//Norway//Oslo as an entry > > > > > > 1. Create a new multivalued field with string type > > > > > > > > multiValued="true"/> > > > > > > 2. Index the field for "Europe//Norway//Oslo" with values > > > > > > 0//Europe > > > 1//Europe//Norway > > > 2//Europe//Norway//Oslo > > > > > > 3. The Facet can now be used in the Queries :- > > > > > > 1st Level - Would return all entries @ 1st level e.g. 0//USA, 0//Europe > > > > > > fq= > > > > > > f.country_facet.facet.prefix=0// > > > > > > facet.field=country_facet > > > > > > > > > 2nd Level - Would return all entries @ second level in Europe > > > 1//Europe//Norway, 1//Europe//Sweden > > > > > > fq=country_facet:0//Europe > > > > > > f.country_facet.facet.prefix=1//Europe > > > > > > facet.field=country_facet > > > > > > > > > > > > 3rd Level - Would return 1//Europe//Norway entries > > > > > > fq=country_facet:1//Europe//Norway > > > > > > f.country_facet.facet.prefix=2//Europe//Norway > > > > > > facet.field=country_facet > > > > > > Increment the facet.prefix by 1 so that you limit the facet results to > to > > > that prefix. > > > Also works for any depth. > > > > > > Regards, > > > Jayendra > > > > > > > > > On Thu, Aug 12, 2010 at 6:01 PM, Mats Bolstad > > wrote: > > > > > >> Hey all, > > >> > > >> I am doing a search on hierarchical data, and I have a hard time > > >> getting my head around the following problem. > > >> > > >> I want a result as follows, in one single query only: > > >> > > >> USA (3) > > >> > California (2) > > >> > Arizona (1) > > >> Europe (4) > > >> > Norway (3) > > >> >> Oslo (3) > > >> > Sweden (1) > > >> > > >> How it looks in the XML/JSON response is not really important, this is > > >> more a presentation issue. I guess I could store the values "USA", > > >> "USA/California", "Europe/Norway/Oslo" as strings for each document, > > >> and do some JavaScript-ing to show the hierarchies appropriately. When > > >> a specific item in the facet is selected, for example "Norway", Solr > > >> could be queries with a filter query on "Europe/Norway*"? > > >> > > >> Do anyone have some experiences they could please share with me? > > >> > > >> I have tried out SOLR-64, and it gives me the results I look for. > > >> However, I do not have the opportunity to use a patch in the > > >> production environment ... > > >> > > >> -- > > >> Thanks, > > >> Mats Bolstad > > >> > > > > > >
Re: Can query boosting be used with a custom request handlers?
Thanks Marc! -John On Mon, Jun 21, 2010 at 2:08 PM, Marc Sturlese wrote: > > Maybe this helps: > http://wiki.apache.org/solr/SolrPlugins#QParserPlugin > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Can-query-boosting-be-used-with-a-custom-request-handlers-tp884499p912691.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Can query boosting be used with a custom request handlers?
Hi Chris: Can you please elaborate on how to use the QParser framework? Thanks! -John On Fri, Jun 11, 2010 at 10:56 AM, Chris Hostetter wrote: > > : So it's possible to use both dismax and custom request handler in the > same query? > > it *really* depends on the request handler ... if it uses the QParser > framework for query parsing, then yes it should work fine -- but the > request handler has to be written to work that way. > > > -Hoss > >
Re: Faceted Search Slows Down as index gets larger
Using the Zoie/Bobo combination gives you realtime faceting. (Lucene based) http://sna-projects.com/zoie/ http://sna-projects.com/bobo/ wiki write-up: http://snaprojects.jira.com/wiki/display/BOBO/Realtime+Faceting+with+Zoie We can take this over to the zoie/bobo mailing list if you have questions. We are doing realtime faceting in production here at LinkedIn serving tens of millions of queries a day with over 70 million user profiles in the index. -John On Sun, Jun 6, 2010 at 10:54 AM, Yonik Seeley wrote: > On Sun, Jun 6, 2010 at 1:12 PM, Furkan Kuru wrote: > > We try to provide real-time search. So the index is changing almost in > every > > minute. > > > > We commit for every 100 documents received. > > > > The facet search is executed every 5 mins. > > OK, that's the problem - pretty much every facet search is rebuilding > the facet cache, which takes most of the time (and facet.fc is more > expensive than facet.enum in this regard). > > One strategy is to use distributed search... have some big cores that > don't change often, and then small cores for the new stuff that > changes rapidly. > > -Yonik > http://www.lucidimagination.com >
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Sorry, have completely missed this thread until someone pointed out to me just now. Some clarifications: commit=true is for the plugin only. I added it to make it consistent with Solr's behavior. Stand-alone Zoie does not require this, it is in more of an autocommit mode since it assumes a constant and busy stream of indexing events. So I think what we will do is add an autocommit parameter to the ZoieUpdateHandler and default it to true. With this, you don't need to have commit=true to see the new docs. -John On Wed, Apr 14, 2010 at 5:25 PM, qaz wrote: > > do you mean that the plugin for solr doesn't have all the functionalities > of > the standalone zoie or do you mean zoie just simply cannot handle large > indexes? by really really small, what exactly are we talking about here? > are > there any better ways for NRT? maybe in solr 1.5? > > thanks > -- > View this message in context: > http://n3.nabble.com/ANN-Zoie-Solr-Plugin-Zoie-Solr-Plugin-enables-real-time-update-functionality-for-Apache-Solr-1-4-tp506099p719893.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: q.alt matching no documents
patch created for lucene: https://issues.apache.org/jira/browse/LUCENE-1931 I am not sure what the right thing to do here is to hook it into QueryParser.java. Maybe the Solr people can comment on how to hook it into Solr. -John On Mon, Sep 28, 2009 at 6:31 AM, John Wang wrote: > You can actually write a NoHitsQuery implementation,it is rather simple. If > you like, I can create a issue and attach a patch. > > -John > > > On Mon, Sep 28, 2009 at 5:17 AM, Øystein F. Steimler < > oyst...@easyconnect.no> wrote: > >> Hi, list! >> >> I want to add a q.alt matching no documents in my dismax handler to serve >> a >> consistent reply to a client application. >> >> Without a q.alt, a missing q from the client will cause an "missing query >> string" error. With a q.alt matching no document I will be able to respond >> with an empty result set when no query is supplied. >> >> I've been using q.alt=-*:* because *:* is said to be the most efficient >> way of >> querying for every document. is -*:* the most efficient way of querying >> for >> no document? >> >> .øs >> >> -- >> Øystein Steimler, Produktans, EasyConnect AS - http://1890.no >> oystein.steim...@easyconnect.no - GPG: 0x784a7dea - Mob: 90010882 >> > >
Re: q.alt matching no documents
You can actually write a NoHitsQuery implementation,it is rather simple. If you like, I can create a issue and attach a patch. -John On Mon, Sep 28, 2009 at 5:17 AM, Øystein F. Steimler wrote: > Hi, list! > > I want to add a q.alt matching no documents in my dismax handler to serve a > consistent reply to a client application. > > Without a q.alt, a missing q from the client will cause an "missing query > string" error. With a q.alt matching no document I will be able to respond > with an empty result set when no query is supplied. > > I've been using q.alt=-*:* because *:* is said to be the most efficient way > of > querying for every document. is -*:* the most efficient way of querying for > no document? > > .øs > > -- > Øystein Steimler, Produktans, EasyConnect AS - http://1890.no > oystein.steim...@easyconnect.no - GPG: 0x784a7dea - Mob: 90010882 >
Re: How to leverage the LogMergePolicy "calibrateSizeByDeletes" patch in Solr ?
Yonik: It would be great if Solr can be configured through some sort of dependency injection framework like Spring! A big +1 from me! -John On Fri, Sep 18, 2009 at 11:10 PM, Yonik Seeley wrote: > On Thu, Sep 17, 2009 at 4:30 PM, Shalin Shekhar Mangar > wrote: > >> I was wondering if there is a way I can modify calibrateSizeByDeletes > just > >> by configuration ? > >> > > > > Alas, no. The only option that I see for you is to sub-class > > LogByteSizeMergePolicy and set calibrateSizeByDeletes to true in the > > constructor. However, please open a Jira issue and so we don't forget > about > > it. > > It's the continuing stuff like this that makes me feel like we should > be Spring (or equivalent) based someday... I'm just not sure how we're > going to get there. > > -Yonik > http://www.lucidimagination.com >
Re: How to leverage the LogMergePolicy "calibrateSizeByDeletes" patch in Solr ?
The current default Lucene MergePolicy does not handle frequent updates well. We have done some performance analysis with that and a custom merge policy: http://code.google.com/p/zoie/wiki/ZoieMergePolicy -John On Mon, Sep 21, 2009 at 1:08 PM, Jason Rutherglen < jason.rutherg...@gmail.com> wrote: > I opened SOLR-1447 for this > > 2009/9/18 Noble Paul നോബിള് नोब्ळ् : > > We can use a simple reflection based implementation to simplify > > reading too many parameters. > > > > What I wish to emphasize is that Solr should be agnostic of xml > > altogether. It should only be aware of specific Objects and > > interfaces. If users wish to plugin something else in some other way , > > it should be fine > > > > > > There is a huge learning involved in learning the current > > solrconfig.xml . Let us not make people throw away that . > > > > On Sat, Sep 19, 2009 at 1:59 AM, Jason Rutherglen > > wrote: > >> Over the weekend I may write a patch to allow simple reflection based > >> injection from within solrconfig. > >> > >> On Fri, Sep 18, 2009 at 8:10 AM, Yonik Seeley > >> wrote: > >>> On Thu, Sep 17, 2009 at 4:30 PM, Shalin Shekhar Mangar > >>> wrote: > > I was wondering if there is a way I can modify calibrateSizeByDeletes > just > > by configuration ? > > > > Alas, no. The only option that I see for you is to sub-class > LogByteSizeMergePolicy and set calibrateSizeByDeletes to true in the > constructor. However, please open a Jira issue and so we don't forget > about > it. > >>> > >>> It's the continuing stuff like this that makes me feel like we should > >>> be Spring (or equivalent) based someday... I'm just not sure how we're > >>> going to get there. > >>> > >>> -Yonik > >>> http://www.lucidimagination.com > >>> > >> > > > > > > > > -- > > - > > Noble Paul | Principal Engineer| AOL | http://aol.com > > >
Re: Aggregating/Grouping Document Search Results on a Field
Hi Brad: We have since (Bobo) added some perf tests which allows you to do some benchmarking very quickly: http://code.google.com/p/bobo-browse/wiki/BoboPerformance Let me know if you need help setting up. -John On Mon, Jul 13, 2009 at 10:41 AM, Jason Rutherglen < jason.rutherg...@gmail.com> wrote: > SOLR 1.4 has a new feature > https://issues.apache.org/jira/browse/SOLR-475that speeds up faceting > on fields with many terms by adding > an UnInvertedField. > Bobo uses a custom field cache as well. It may be useful to benchmark the 3 > different approaches (bitsets, SOLR-475, Bobo). This could be a good wiki > page explaining the differences between them? > > On Mon, Jul 13, 2009 at 9:49 AM, Bradford Stephens < > bradfordsteph...@gmail.com> wrote: > > > Thanks for this -- we're also trying out bobo-browse for Lucene, and > > early results look pretty enticing. They greatly sped up how fast you > > read in documents from disk, among other things: > > http://bobo-browse.wiki.sourceforge.net/ > > > > On Sat, Jul 11, 2009 at 12:10 AM, Shalin Shekhar > > Mangar wrote: > > > On Sat, Jul 11, 2009 at 12:01 AM, Bradford Stephens < > > > bradfordsteph...@gmail.com> wrote: > > > > > >> Does the facet aggregation take place on the Solr search server, or > > >> the Solr client? > > >> > > >> It's pretty slow for me -- on a machine with 8 cores/ 8 GB RAM, 50 > > >> million document index (about 36M unique values in the "author" > > >> field), a query that returns 131,000 hits takes about 20 seconds to > > >> calculate the top 50 authors. The query I'm running is this: > > >> > > >> > > >> > > > http://dttest10:8983/solr/select/select?q=java&facet=true&facet.field=authorname > > >> : > > >> > > >> > > > Is the author field tokenized? Is it multi-valued? It is best to have > > > untokenized fields. > > > > > > Solr 1.4 has huge improvements in faceting performance so you can try > > that > > > and see if it helps. See Yonik's blog post about this - > > > > > > http://yonik.wordpress.com/2008/11/25/solr-faceted-search-performance-improvements/ > > > > > > -- > > > Regards, > > > Shalin Shekhar Mangar. > > > > > > > > > > > -- > > http://www.roadtofailure.com -- The Fringes of Scalability, Social > > Media, and Computer Science > > >
Re: Solr spring application context error
Ah, my bad, this is the same as null. Sorry for the stupid comment. -John On Thu, Jul 9, 2009 at 1:55 PM, John Wang wrote: > Hi guys: > Could it be in method:load in class CoreContainer: > > libLoader = SolrResourceLoader.createClassLoader(f, null); > > >The parent ClassLoader is set to null, shouldn't it be the current > classloader? e.g." > > > libLoader = > SolrResourceLoader.createClassLoader(f,Thread.currentThread().getContextClassLoader( > )); > > > Thanks > > > -John > > On Sun, Jul 5, 2009 at 11:38 PM, John Wang wrote: > >> Hi Chris: >> >> The problem is that (at least for me, not a expert spring'er) The >> beans are being loaded (and when the class loading issue occurs) at >> construction time of FileSystemXmlApplicationContex, the setClassLoader >> method by the time can be called is too late. >> >> -John >> >> >> On Thu, Jul 2, 2009 at 2:26 PM, Chris Hostetter > > wrote: >> >>> >>> : I did try that. The problem is that you can't tell >>> : FileSystemXmlApplicationContext to load with a different ClassLoader. >>> >>> why not? >>> >>> it subclasses DefaultResourceLoader which has the setClassLoader method >>> Mark pointed out. >>> >>> >>> >>> -Hoss >>> >>> >> >
Re: Solr spring application context error
Hi guys: Could it be in method:load in class CoreContainer: libLoader = SolrResourceLoader.createClassLoader(f, null); The parent ClassLoader is set to null, shouldn't it be the current classloader? e.g." libLoader = SolrResourceLoader.createClassLoader(f,Thread.currentThread().getContextClassLoader( )); Thanks -John On Sun, Jul 5, 2009 at 11:38 PM, John Wang wrote: > Hi Chris: > > The problem is that (at least for me, not a expert spring'er) The > beans are being loaded (and when the class loading issue occurs) at > construction time of FileSystemXmlApplicationContex, the setClassLoader > method by the time can be called is too late. > > -John > > > On Thu, Jul 2, 2009 at 2:26 PM, Chris Hostetter > wrote: > >> >> : I did try that. The problem is that you can't tell >> : FileSystemXmlApplicationContext to load with a different ClassLoader. >> >> why not? >> >> it subclasses DefaultResourceLoader which has the setClassLoader method >> Mark pointed out. >> >> >> >> -Hoss >> >> >
Re: Solr spring application context error
Hi Chris: The problem is that (at least for me, not a expert spring'er) The beans are being loaded (and when the class loading issue occurs) at construction time of FileSystemXmlApplicationContex, the setClassLoader method by the time can be called is too late. -John On Thu, Jul 2, 2009 at 2:26 PM, Chris Hostetter wrote: > > : I did try that. The problem is that you can't tell > : FileSystemXmlApplicationContext to load with a different ClassLoader. > > why not? > > it subclasses DefaultResourceLoader which has the setClassLoader method > Mark pointed out. > > > > -Hoss > >
Re: Solr spring application context error
Hi Mark: I did try that. The problem is that you can't tell FileSystemXmlApplicationContext to load with a different ClassLoader. -John On Wed, Jul 1, 2009 at 4:20 PM, Mark Miller wrote: > I havn't done much a spring in a while, and I've never done anything > hardcore, but could you: > > in your plugin code, grab the classloader of the plugin class and init the > spring context with it? > > org.springframework.core.io.DefaultResourceLoader#setClassLoader > > Or you could use the solr classloader to load the spring context? > > Or perhaps we could provide access to the classLoader from the core? > > > -- > -- > - Mark > > http://www.lucidimagination.com > > > On Wed, Jul 1, 2009 at 6:52 PM, John Wang wrote: > > > Hi guys: > > > >What is the plan with this issue? Should there be a bug created? > > > >I am having a similar issue from a different angle: > > > > 1) using spring which is instantiating beans when the plugins are loaded > > 2) classloader mismatch. > > 3) only way to resolve, copy my jars to solr.war's WEB-INF/lib > > > > -John > > > > On Thu, Jun 18, 2009 at 6:53 PM, Mark Miller > > wrote: > > > > > Chris Hostetter wrote: > > > > > >> : Yeah, I actually looked at the code and saw that later. I was > > forgetting > > >> the > > >> : issue that bugged me (and confusing it with the trouble this guy was > > >> having) - > > >> : which is that plugins in the solr/lib folder cannot load from other > > jars > > >> in > > >> : that folder. I think that was the actual issue. > > >> > > >> WTF?!? ... seriously? > > >> I don't think i've ever tried it, but if that's really true then it > > seems > > >> like it must be a bug in URLClassLoader ... we're iterating over the > > file > > >> list to generate a URL[] before calling URLClassLoader.newInstance. > > >> > > >> > > >> -Hoss > > >> > > >> > > >> > > > I have a fairly strong memory that its true - but its been almost a > year > > > now. I'll check when I get a chance. I also seem to remember mentioning > > it > > > to Erik and him already knowing of it... > > > > > > Or my mind is playing tricks on me. I'll check. > > > > > > > > > -- > > > - Mark > > > > > > http://www.lucidimagination.com > > > > > > > > > > > > > > >
Re: Solr spring application context error
Hi guys: What is the plan with this issue? Should there be a bug created? I am having a similar issue from a different angle: 1) using spring which is instantiating beans when the plugins are loaded 2) classloader mismatch. 3) only way to resolve, copy my jars to solr.war's WEB-INF/lib -John On Thu, Jun 18, 2009 at 6:53 PM, Mark Miller wrote: > Chris Hostetter wrote: > >> : Yeah, I actually looked at the code and saw that later. I was forgetting >> the >> : issue that bugged me (and confusing it with the trouble this guy was >> having) - >> : which is that plugins in the solr/lib folder cannot load from other jars >> in >> : that folder. I think that was the actual issue. >> >> WTF?!? ... seriously? >> I don't think i've ever tried it, but if that's really true then it seems >> like it must be a bug in URLClassLoader ... we're iterating over the file >> list to generate a URL[] before calling URLClassLoader.newInstance. >> >> >> -Hoss >> >> >> > I have a fairly strong memory that its true - but its been almost a year > now. I'll check when I get a chance. I also seem to remember mentioning it > to Erik and him already knowing of it... > > Or my mind is playing tricks on me. I'll check. > > > -- > - Mark > > http://www.lucidimagination.com > > > >
Re: Dynamic range Facets
I think another challenge here is in distributed mode, how would you merge the ranges from subsearchers? -John On Fri, Apr 3, 2009 at 3:03 PM, Chris Hostetter wrote: > > : my documents (products) have a price field, and I want to have > : a "dynamically" calculated range facet for that in the response. > ... > : So the question is how to get the dynamic facets response from solr. > : > : This is same question as previously posted back in 2007. But still waits > an > : answer?? > : Is there any solution on this?? > > I'm not sure what you mean by "still waits an answer" ... the email thread > you cut/pasted your example from gots lots of answers. My comments today > are exactly the same as they were then (and in the 2006 threads i > linked to back in 2007)... > > > http://www.nabble.com/Dynamically-calculated-range-facet-to11314725.html#a11314725 > http://www.nabble.com/faceted-browsing-to3655780.html#a3753053 > > in summary... > * this is a hard problem to implement in a generic manner > * it's much easier to do when you have some domain knowledge (ie: field is > in dollars) > > What's changed since then is... > 1) we have SearchComponents now > 2) we have a stats component which does the hard work of finding > min/max/avg/etc... > > It would be fairly easy to write a custom component that expected to run > after the stats component to then get the counts for whatever ranges it > thought were appropriate -- but knowing when to use ranges of "20" vs > ranges of "100" is a very subejctive thing. > > > > > > > > -Hoss > >
Re: Realtime Searching..
Hi Jon: We are running various LinkedIn search systems on Zoie in production. -John On Thu, Feb 19, 2009 at 9:11 AM, Jon Baer wrote: > This part: > > The part of Zoie that enables real-time searchability is the fact that > ZoieSystem contains three IndexDataLoader objects: > >* a RAMLuceneIndexDataLoader, which is a simple wrapper around a > RAMDirectory, >* a DiskLuceneIndexDataLoader, which can index directly to the > FSDirectory (followed by an optimize() call if a specified optimizeDuration > has been exceeded) in batches via an intermediary >* BatchedIndexDataLoader, whose primary job is to queue up and batch > DataEvents that need to be flushed to disk > > Sounds like it (might) be / (can) be layered into Solr somehow, has anyone > been using this project or testing it? > > - Jon > > > On Feb 19, 2009, at 9:44 AM, Genta Kaneyama wrote: > > Michael, >> >> I think you might be get interested in "zoie". >> >> zoie: real-time search and indexing system built on Apache Lucene >> http://code.google.com/p/zoie/ >> >> Zoie is realtime search project for lucene by Linkedin. >> Basically, I think it is similar technique to a Otis's trick. >> >> In the mean time you can use the trick of one large and less frequently updated core and one small and more frequently >>updated core + distributed search across them. Otis >>> >> Genta >> >> >> On Sat, Feb 7, 2009 at 3:02 AM, Michael Austin >> wrote: >> >>> I need to find a solution for our current social application. It's low >>> traffic now because we are early on.. However I'm expecting and want to >>> be >>> prepaired to grow. We have messages of different "types" that are >>> aggregated into one stream. Each of these message types have much >>> different >>> data so that our main queries have a few unions and many joins. I know >>> that >>> Solr would work great for searching but we need a realtime system >>> (twitter-like) to view user updates. I'm not interested in a few minutes >>> delay; I need something that will be fast updating and searchable and >>> have n >>> columns per record/document. Can solor do this? what is Ocean? >>> >>> Thanks >>> >>> >
Re: jira 243
Thanks Chris. -John On 9/23/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > > :was wondering if there are any plans of merging this patch into svn. > > Hey John: I'm going to leave this message in my inbox as a reminder to > look at the latest patch when i get a chance ... but at this point i'm > having a hard enough time keeping up with geniune bugs, and feature > patche people have voted on in Jira (or that have a lot of discussion) > > -Hoss > >
jira 243
hi guys: was wondering if there are any plans of merging this patch into svn. thanks -john
Re: Dynamically calculated range facet
Hi Martin: I just started a wiki on this: http://browseengine.pbwiki.com/ Has references to the source code. I have written a solr plugin, but it depends on SOLR-243. The implmentation is different from the solr facetted search implementation. Look at the demo: http://www.browseengine.com/democar/demo-car.html it exhibts dynamic range calculation. Thanks -John On 6/27/07, Martin Grotzke <[EMAIL PROTECTED]> wrote: On Tue, 2007-06-26 at 19:53 -0700, John Wang wrote: > www.browseengine.com has facet search that handles this. You are calculating range facets dynamically? Do you have any code I can have a look at? I had a look at c.b.solr. BoboRequestHandler, but this does not seem to calculate ranges. Cheers, Martin > > We are working on a solr plugin. > > -John > > On 6/26/07, Mike Klaas <[EMAIL PROTECTED]> wrote: > > > > On 26-Jun-07, at 3:01 PM, Martin Grotzke wrote: > > > AFAICS I do not have the possibility to specify range queries in my > > > application, as I do not have a clue what's the lowest and highest > > > price in the search result and what are "good" ranges according > > > to the (statistical) distribution of prices in the search result. > > > > > > So if it would be possible to go over each item in the search result > > > I could check the price field and define my ranges for the specific > > > query on solr side and return the price ranges as a facet. > > > > > > Has anybody done s.th. like this before, or is there s.th. that I'm > > > missing and why this approach does not make sense at all? > > > > > > Otherwise, what would be a good starting point to plug in such > > > functionality into solr? > > > > Easy: facet based on fixed ranges (say, every 10 dollars for x < 100, > > 100 dollars for x < 1000, etc)., and combine them sensically on the > > client-side. Requires no solr-side modification. > > > > A bit harder: define your own request handler that loops over the > > documents after a search and samples the values of (say) the first 20 > > docs (or more, but be sure to use the FieldCache if so). Calculate > > your range queries, facets (code will be almost identical to the code > > in the builtin request handlers), and return the results. > > > > cheers, > > -Mike > > -- Martin Grotzke http://www.javakaffee.de/blog/
Re: Dynamically calculated range facet
www.browseengine.com has facet search that handles this. We are working on a solr plugin. -John On 6/26/07, Mike Klaas <[EMAIL PROTECTED]> wrote: On 26-Jun-07, at 3:01 PM, Martin Grotzke wrote: > AFAICS I do not have the possibility to specify range queries in my > application, as I do not have a clue what's the lowest and highest > price in the search result and what are "good" ranges according > to the (statistical) distribution of prices in the search result. > > So if it would be possible to go over each item in the search result > I could check the price field and define my ranges for the specific > query on solr side and return the price ranges as a facet. > > Has anybody done s.th. like this before, or is there s.th. that I'm > missing and why this approach does not make sense at all? > > Otherwise, what would be a good starting point to plug in such > functionality into solr? Easy: facet based on fixed ranges (say, every 10 dollars for x < 100, 100 dollars for x < 1000, etc)., and combine them sensically on the client-side. Requires no solr-side modification. A bit harder: define your own request handler that loops over the documents after a search and samples the values of (say) the first 20 docs (or more, but be sure to use the FieldCache if so). Calculate your range queries, facets (code will be almost identical to the code in the builtin request handlers), and return the results. cheers, -Mike
SolrRequestHandler implementations
Solr developers: There are different SolrRequestHandler implementations in Solr, e.g. DisMaxRequestHandler, StandardRequestHandler and they overlap in functionality. And we are thinking of writing our own SolrRequestHandler implementation, which means to also support standard functionality, we'd have to copy over most of the code from existing implementations. This sort of code replication is usually not good. Are there plans to rethink the plugin architecture, e.g. to break into phases or modules where other components/plugins can extend? Or what are some other suggestions you guys may have? Thanks -John
Re: solr plugins
Hi Yonik: I updated with a patch. Is it possible to get this in soon cuz I have a client waiting on this. Thanks again -John On 5/22/07, John Wang <[EMAIL PROTECTED]> wrote: Hi Yonik: Thank you again for your help! I created an improvement item in jira (SOLR-243) on this. -John On 5/19/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: > On 5/19/07, John Wang < [EMAIL PROTECTED]> wrote: > > Hi Yonik: > > > > Thanks for the info! > > > > This solves my problem, but not elegantly. > > > > I have a custome implementation where I derived from the > > IndexReader class to store some custome data. Now I am trying to write > > a Solr plugin for my search implementation but I want to be able to > > use my IndexReader implementation. > > > > Is there a way to overwrite the IndexReader instantiation? e..g > > IndexReader newReader() etc. > > Not currently, but it might be a useful feature. > > -Yonik >
Re: solr plugins
Hi Yonik: Thank you again for your help! I created an improvement item in jira (SOLR-243) on this. -John On 5/19/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 5/19/07, John Wang <[EMAIL PROTECTED]> wrote: > Hi Yonik: > > Thanks for the info! > > This solves my problem, but not elegantly. > > I have a custome implementation where I derived from the > IndexReader class to store some custome data. Now I am trying to write > a Solr plugin for my search implementation but I want to be able to > use my IndexReader implementation. > > Is there a way to overwrite the IndexReader instantiation? e..g > IndexReader newReader() etc. Not currently, but it might be a useful feature. -Yonik
Re: solr plugins
Hi Yonik: Thanks for the info! This solves my problem, but not elegantly. I have a custome implementation where I derived from the IndexReader class to store some custome data. Now I am trying to write a Solr plugin for my search implementation but I want to be able to use my IndexReader implementation. Is there a way to overwrite the IndexReader instantiation? e..g IndexReader newReader() etc. Thanks -John On 5/18/07, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 5/17/07, John Wang <[EMAIL PROTECTED]> wrote: >Is there a way to hook in loading and initialization of the index while > writing a solr plugin? Do you mean a hook to do something when an IndexReader/Searcher is first opened on an index? If so, see newSearcher/firstSearcher hooks in solrconfig.xml -Yonik
solr plugins
Hi: Is there a way to hook in loading and initialization of the index while writing a solr plugin? Thanks -John
