Faceted Search
Hi all, Can someone of you tell me how to implement a faceted search? Thanks, Regards, Sajith Vimukthi Weerakoon.
Faceted Search!
Hi all, I'm couple of days old with Solr so I'm very new to this. However, I'm trying to implemented Faceted search somewhat close to CNET shopper.com. Instead of using some items (like "camera"), I want to search for documents. I'm planning to use Nutch to crawl that website and use Solr to cluster my search results. I tried integrating Nutch with Solr following FooFactory.com's blog ..but I could not follow few of the steps as I'm very new to both of them. If anyone of you have implemented, can you please give me suggestion or code snippets so that I can implemented them to achieve the "faceted search". Any help would be appericated. Thanks, Niraj - You snooze, you lose. Get messages ASAP with AutoCheck in the all-new Yahoo! Mail Beta.
Re: Faceted Search
if you are querying using a http request you can add these two parameters: facet=true facet.field=field_for_faceting and optionally this one to set the max number of facets: facet.limit=facet_limit I don't know if it's what you need... On Fri, Apr 17, 2009 at 6:17 AM, Sajith Weerakoon wrote: > Hi all, > > Can someone of you tell me how to implement a faceted search? > > > > Thanks, > > Regards, > > Sajith Vimukthi Weerakoon. > > > >
Faceted search outofmemory
Hi, I try to make a faceted search on a very large index (around 200GB with 200M doc). I have an out of memory error. With no facet it works fine. There are quite many questions around this but I could not find the answer. How can we know the required memory when facets are used so that I try to scale my server/index correctly to handle it. Thanks Olivier
Faceted search/browse
Hi, I am currently exploring the possibility of using faceted browsing using Solr, and got few questions. I am currently indexing some fields that has more than one value per record or e.g., a field name called metaKeyword, and the value for this contains more than one value per record (e.g., java, programming, object oriented), is it possible to include this field on the faceted search? or this metaKeyword needs to have only unique value per record to use a faceted? Any help in this regard is appreciated Thanks! Giri
Faceted search problem
Hi all, I'm trying this solr ruby DSL called Flare/solrb and I don't really
know how the faceted search works because I cant add whatever fields I want
to to the index. This is currently not working:
conn = Solr::Connection.new('http://localhost:8983/solr')
doc = {:id => 1, :cat => 'eletronics', :features => 'video, music', :product
=> 'iPod'}
conn.send(Solr::Request::AddDocument.new(doc))
=> #ERROR:unknown field 'cat'", @doc= ...
>
In case that if it was working, what I'd like to do is:
(pseudo-code)
request = Solr::Request::Standard.new(
:query => 'ipod',
:facets => {
:fields => :cat
}
)
Any help would be appreciated.
Peter
_
The MSN Entertainment Guide to Golden Globes is here. Get all the scoop.
http://tv.msn.com/tv/globes2007/?icid=nctagline2
Faceted search glitch???
On Solr, I'm indexing a facet field which contains different values on each
record (of course) but it doesn't seem to be working the way I had expected.
For instance:
When looking for:
http://localhost:8983/solr/select/?q=sales&facet=true&facet.field=education_facet
it returns the facet:
'education_facet'=>{
'Bachelor'=>20,
'Associate'=>35,
'Doctorate'=>1,
'High School'=>150,
'Law Degree'=>0,
'Master'=>0}
but when I modify my query to look for High School:
http://localhost:8983/solr/select/?q=sales+AND+education_facet:High+School&facet=true&facet.field=education_facet
it returns nothing. But when I use single-word fields such as Bachelor or
Associate it works fine.
Any idea?
Thanks a lot
_
Check out all that glitters with the MSN Entertainment Guide to the Academy
Awards® http://movies.msn.com/movies/oscars2007/?icid=ncoscartagline2
Re: Faceted Search!
: search for documents. I'm planning to use Nutch to crawl that website : and use Solr to cluster my search results. I tried integrating Nutch : with Solr following FooFactory.com's blog ..but I could not follow : few of the steps as I'm very new to both of them. If anyone of you have : implemented, can you please give me suggestion or code snippets so that : I can implemented them to achieve the "faceted search". Any help would : be appericated. I'm not very familiar with the Nutch/Solr hybrid stuff some people have done, but faceting requires that you have well structured fields containing discreet peices of information ... ie: if you want to facet cameras on manufacturer, megapixels, weight, and battery life, you need sepertate fields for manufacturer, megapixels, weiht, and mattery life ... i'm not sure that nutch is going to be able to do that for you. extracting structured data out of webpages like that without writing customer parser code for each website layout is a pretty weight data harvesting problem. -Hoss
Re: Faceted Search!
Thanks Chris for replying my question. So I'm thinking about using a CMS and when somebody publishes a page in CMS, I would generated this well structure XML file and feed that xml to Solr to generate the index on those data. Then, I can simply do faceted search using the correct Lucene query format, rite? Do you have any other ideas or comment on my CMS approach? Cheers, Niraj Chris Hostetter <[EMAIL PROTECTED]> wrote: : search for documents. I'm planning to use Nutch to crawl that website : and use Solr to cluster my search results. I tried integrating Nutch : with Solr following FooFactory.com's blog ..but I could not follow : few of the steps as I'm very new to both of them. If anyone of you have : implemented, can you please give me suggestion or code snippets so that : I can implemented them to achieve the "faceted search". Any help would : be appericated. I'm not very familiar with the Nutch/Solr hybrid stuff some people have done, but faceting requires that you have well structured fields containing discreet peices of information ... ie: if you want to facet cameras on manufacturer, megapixels, weight, and battery life, you need sepertate fields for manufacturer, megapixels, weiht, and mattery life ... i'm not sure that nutch is going to be able to do that for you. extracting structured data out of webpages like that without writing customer parser code for each website layout is a pretty weight data harvesting problem. -Hoss - Got a little couch potato? Check out fun summer activities for kids.
Re: Faceted Search!
: Thanks Chris for replying my question. So I'm thinking about using a : CMS and when somebody publishes a page in CMS, I would generated this : well structure XML file and feed that xml to Solr to generate the index : on those data. Then, I can simply do faceted search using the correct : Lucene query format, rite? Do you have any other ideas or comment on my : CMS approach? that sounds fine ... as long as you have well structured data and you aren't trying to extract it from unstructured HTML. -Hoss
Re: Faceted Search!
Hi Chris, thank you for the reply. I was reading other posting regarding faceted search and seems like they are using the filtering capability of Lucene for that. If that the case, can we have control over the "label" of categories? For example: in shopper.com when we search for camera gives us the cluster by price, pixal, manufacture and so on. and if we are feeding the xml file to Solr server for faceted search, how can we define the sub-categories. let's say from the above example, the category "price" has different sub-categories like "less than 100" ,"100-200"? I'm guessing, we explicit define this in XML feed file, but I could be very wrong. In any case, can you please give me the short example achieve that implementation. Well, thanks once again. Cheers, Niraj Chris Hostetter <[EMAIL PROTECTED]> wrote: : Thanks Chris for replying my question. So I'm thinking about using a : CMS and when somebody publishes a page in CMS, I would generated this : well structure XML file and feed that xml to Solr to generate the index : on those data. Then, I can simply do faceted search using the correct : Lucene query format, rite? Do you have any other ideas or comment on my : CMS approach? that sounds fine ... as long as you have well structured data and you aren't trying to extract it from unstructured HTML. -Hoss - Food fight? Enjoy some healthy debate in the Yahoo! Answers Food & Drink Q&A.
Re: Faceted Search!
: define the sub-categories. let's say from the above example, the : category "price" has different sub-categories like "less than 100" : ,"100-200"? I'm guessing, we explicit define this in XML feed file, but : I could be very wrong. In any case, can you please give me the short : example achieve that implementation. Well, thanks once again. there's nothing "out of the box" from Solrthat will do this, it's something you would need to implement either in the lcient or in a custom request handler ... Solr's "Simple Faceting" support is esigned to be just that: simple. but the underlying methods/mechanisms of computing DocSet intersetions can be used by any custom requets handler to generate application specific results. I've got 3 or 4 indexes that use the out of the box SimpleFacet support Solr provides, but the major faceting we do (product based facets) all uses custom request handlers so we can have very exact control on all of this kind of stuff driven by our data management tools. -Hoss
RE: Faceted Search!
Niraj: What environment are you using? SQL Server/.NET/Windows? or something else? -Mike -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 20, 2007 4:24 PM To: solr-user@lucene.apache.org Subject: Re: Faceted Search! : define the sub-categories. let's say from the above example, the : category "price" has different sub-categories like "less than 100" : ,"100-200"? I'm guessing, we explicit define this in XML feed file, but : I could be very wrong. In any case, can you please give me the short : example achieve that implementation. Well, thanks once again. there's nothing "out of the box" from Solrthat will do this, it's something you would need to implement either in the lcient or in a custom request handler ... Solr's "Simple Faceting" support is esigned to be just that: simple. but the underlying methods/mechanisms of computing DocSet intersetions can be used by any custom requets handler to generate application specific results. I've got 3 or 4 indexes that use the out of the box SimpleFacet support Solr provides, but the major faceting we do (product based facets) all uses custom request handlers so we can have very exact control on all of this kind of stuff driven by our data management tools. -Hoss
RE: Faceted Search!
Hi Mike, Currently, I'm just running the demo example provided in the Solr web site on my local windows machines. I was purely looking into generating XML feed file and feeding to the Solr server. However, I was also looking into implementing having sub-categories within the categories if that make sense. For example, in the shopper.com we have the categories of by price, manufactures and so on and with in them,they are sub categories (price is sub-cat into <$100, 100-200, 200-300 etc). I don't have constraint in terms of technology. If I have to implement db server I won't mind implementing it. Anyway, plz shine a light on how would you handle this issue. Any suggestion will be appericated. Thanks, Niraj Mike Austin <[EMAIL PROTECTED]> wrote: Niraj: What environment are you using? SQL Server/.NET/Windows? or something else? -Mike -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 20, 2007 4:24 PM To: solr-user@lucene.apache.org Subject: Re: Faceted Search! : define the sub-categories. let's say from the above example, the : category "price" has different sub-categories like "less than 100" : ,"100-200"? I'm guessing, we explicit define this in XML feed file, but : I could be very wrong. In any case, can you please give me the short : example achieve that implementation. Well, thanks once again. there's nothing "out of the box" from Solrthat will do this, it's something you would need to implement either in the lcient or in a custom request handler ... Solr's "Simple Faceting" support is esigned to be just that: simple. but the underlying methods/mechanisms of computing DocSet intersetions can be used by any custom requets handler to generate application specific results. I've got 3 or 4 indexes that use the out of the box SimpleFacet support Solr provides, but the major faceting we do (product based facets) all uses custom request handlers so we can have very exact control on all of this kind of stuff driven by our data management tools. -Hoss - Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games.
RE: Faceted Search!
: generating XML feed file and feeding to the Solr server. However, I was : also looking into implementing having sub-categories within the : categories if that make sense. For example, in the shopper.com we have : the categories of by price, manufactures and so on and with in them,they : are sub categories (price is sub-cat into <$100, 100-200, 200-300 etc). : I don't have constraint in terms of technology. If I have to implement : db server I won't mind implementing it. Anyway, plz shine a light on : how would you handle this issue. Any suggestion will be appericated. the shopper.com solution is very VERY specialized and specific to the datamodel used to manage the category metadata if i had to do it overagain i would do it a lot differnetly. way way back there was a thread about "complex faceting" where i included some ideas on a possible facet configuration xml syntax which could then be parsed by a request handler, with different types of faceting (simple query, ranges, based on terms, prefix) delegated to helper classes. there was also the idea of being able groups facets or make facets depend on other facets (ie: don't show the author facet untill a value has been picked from the author_initial facet) nothing ever really came of it, but it's how i'd probably approach trying to tackle something like the shopper.com functionality if CNET threw away our product metadata data model and started from scratch. http://www.nabble.com/metadata-about-result-sets--t1243321.html#a3334244 -Hoss
Solr faceted search UI
Hi, I am working on a POC, where I have to display faceted search result on web page. can anybody please help me to suggest what all set up I need to configure to display. I would prefer java technologies. Just to mention, I have solr cloud running on remote server. I would like to know: 1. Should I use MVC framework? 2. How will my local interact with remote solr server? 3. How will I send query through java code and what technology I should use to display faceted search result? Please help me on this. Thanks, -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598.html Sent from the Solr - User mailing list archive at Nabble.com.
faceted search field sorting
Hello, Here is a public API that uses facet fields: http://golr.berkeleybop.org/select?qt=standard&fl=*&version=2.2&wt=json&indent=on&rows=0&facet=true&facet.field=type&q=document_category:%22annotation%22 It looks like the default behavior is to sort facet_counts.facet_fields (field called 'type' in this case) descending by frequency count among returned documents. I'd like to find out if this is a default behavior or if this is an extra step that Solr is performing and can be somehow turned off? I'd like to use that data to form a tag cloud and I'd like to avoid the random shuffle step if that is something that can be turned off in Solr query. If this is the default result maybe someone could explain how faceted index works on Solr side, and where does that order come from? I'd like to understand it better either way. Thanks, M
Faceted search question (Tokenizing)
Hey There, We have the following data structure: - Person -- Interest 1 --- Subinterest 1 --- Subinterest 1 Description --- Subinterest 1 ID -- Interest 2 --- Subinterest 2 --- Subinterest 2 Description --- Subinterest 2 ID . -- Interest 99 --- Subinterest 99 --- Subinterest 99 Description --- Subinterest 99 ID Interest, Subinterest, Subinterest Description and Subinterest IDs are all multiavlued fields. A person can have any number of subinterests,descriptions and IDS. How could we faced/search this based on this data structure? Right now we tokenized everything in a seperate multivalued column in the following fasion; |Interest='Interest 1',Subinterest='Subinterest 1',Subinterest='Another Subinterest 1',Description='Interest 1 Description',ID='Interest 1 ID'| |Interest='Interest 2',Description='Interest 2 Description',ID='Interest 2 ID'| I have a feeling like this is a wrong approach to this problem. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-search-question-Tokenizing-tp4012948.html Sent from the Solr - User mailing list archive at Nabble.com.
Faceted search not working?
I'm trying to perform a faceted search without any luck. Result set doesn't return any facet information... http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title I'm getting the result set, but no face information present? Is there something else that needs to happen to turn faceting on? I'm using latest Solr 1.4 release. Data is indexed from the database using dataimporter. Thanks. Ilya Sterin
RE: Faceted search outofmemory
Did you trying paging them? -Original Message- From: olivier sallou [mailto:olivier.sal...@gmail.com] Sent: Tuesday, June 29, 2010 2:04 PM To: solr-user@lucene.apache.org Subject: Faceted search outofmemory Hi, I try to make a faceted search on a very large index (around 200GB with 200M doc). I have an out of memory error. With no facet it works fine. There are quite many questions around this but I could not find the answer. How can we know the required memory when facets are used so that I try to scale my server/index correctly to handle it. Thanks Olivier
Re: Faceted search outofmemory
How do make paging over facets? 2010/6/29 Ankit Bhatnagar > > Did you trying paging them? > > > -Original Message- > From: olivier sallou [mailto:olivier.sal...@gmail.com] > Sent: Tuesday, June 29, 2010 2:04 PM > To: solr-user@lucene.apache.org > Subject: Faceted search outofmemory > > Hi, > I try to make a faceted search on a very large index (around 200GB with > 200M > doc). > I have an out of memory error. With no facet it works fine. > > There are quite many questions around this but I could not find the answer. > How can we know the required memory when facets are used so that I try to > scale my server/index correctly to handle it. > > Thanks > > Olivier >
RE: Faceted search outofmemory
How much memory have you given the solr jvm? Many servlet containers have small amount by default. -Kal -Original Message- From: olivier sallou [mailto:olivier.sal...@gmail.com] Sent: Tuesday, June 29, 2010 2:04 PM To: solr-user@lucene.apache.org Subject: Faceted search outofmemory Hi, I try to make a faceted search on a very large index (around 200GB with 200M doc). I have an out of memory error. With no facet it works fine. There are quite many questions around this but I could not find the answer. How can we know the required memory when facets are used so that I try to scale my server/index correctly to handle it. Thanks Olivier
Re: Faceted search outofmemory
I have given 6G to Tomcat. Using facet.method=enum and facet.limit seems to fix the issue with a few tests, but I do know that it is not a "final" solution. Will work under certain configurations. Real "issue" is to be able to know what is the required RAM for an index... 2010/6/29 Nagelberg, Kallin > How much memory have you given the solr jvm? Many servlet containers have > small amount by default. > > -Kal > > -Original Message- > From: olivier sallou [mailto:olivier.sal...@gmail.com] > Sent: Tuesday, June 29, 2010 2:04 PM > To: solr-user@lucene.apache.org > Subject: Faceted search outofmemory > > Hi, > I try to make a faceted search on a very large index (around 200GB with > 200M > doc). > I have an out of memory error. With no facet it works fine. > > There are quite many questions around this but I could not find the answer. > How can we know the required memory when facets are used so that I try to > scale my server/index correctly to handle it. > > Thanks > > Olivier >
Re: Faceted search outofmemory
There is memory used for each facet. All of the facets are loaded for any facet query. Your best shot is to limit the number of facets. On Tue, Jun 29, 2010 at 11:42 AM, olivier sallou wrote: > I have given 6G to Tomcat. Using facet.method=enum and facet.limit seems to > fix the issue with a few tests, but I do know that it is not a "final" > solution. Will work under certain configurations. > > Real "issue" is to be able to know what is the required RAM for an index... > > 2010/6/29 Nagelberg, Kallin > >> How much memory have you given the solr jvm? Many servlet containers have >> small amount by default. >> >> -Kal >> >> -Original Message- >> From: olivier sallou [mailto:olivier.sal...@gmail.com] >> Sent: Tuesday, June 29, 2010 2:04 PM >> To: solr-user@lucene.apache.org >> Subject: Faceted search outofmemory >> >> Hi, >> I try to make a faceted search on a very large index (around 200GB with >> 200M >> doc). >> I have an out of memory error. With no facet it works fine. >> >> There are quite many questions around this but I could not find the answer. >> How can we know the required memory when facets are used so that I try to >> scale my server/index correctly to handle it. >> >> Thanks >> >> Olivier >> > -- Lance Norskog goks...@gmail.com
Re: Faceted search/browse
On 12/27/06, Giri <[EMAIL PROTECTED]> wrote: I am currently exploring the possibility of using faceted browsing using Solr, and got few questions. I am currently indexing some fields that has more than one value per record or e.g., a field name called metaKeyword, and the value for this contains more than one value per record (e.g., java, programming, object oriented), is it possible to include this field on the faceted search? or this metaKeyword needs to have only unique value per record to use a faceted? Yes, multivalued fields are supported with faceted search, but the speed is currently proportional to the number of unique terms in that field. You also need to increase the size of the filterCache (see solrconfig.xml) to be larger than the number of terms. -Yonik
Re: Faceted search/browse
Hi Yonik, Thanks for the information and suggestion. Do you have any simple tutorial that explains how to enable the faceted search in solr? I wouold appreciate if you point me to some resource. Thanks much On 12/28/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 12/27/06, Giri <[EMAIL PROTECTED]> wrote: > I am currently exploring the possibility of using faceted browsing using > Solr, and got few questions. I am currently indexing some fields that has > more than one value per record or e.g., a field name called metaKeyword, > and the value for this contains more than one value per record (e.g., java, > programming, object oriented), is it possible to include this field on the > faceted search? or this metaKeyword needs to have only unique value per > record to use a faceted? Yes, multivalued fields are supported with faceted search, but the speed is currently proportional to the number of unique terms in that field. You also need to increase the size of the filterCache (see solrconfig.xml) to be larger than the number of terms. -Yonik
Re: Faceted search/browse
On Dec 28, 2006, at 10:18 PM, Giri wrote: Do you have any simple tutorial that explains how to enable the faceted search in solr? I wouold appreciate if you point me to some resource. The wiki is the best source of information on Solr. For facets, look here: <http://wiki.apache.org/solr/SimpleFacetParameters> If this doesn't answer all your questions, ask here and we'll fill in the blanks on the wiki :) Erik
Re: Faceted search problem
On Jan 16, 2007, at 10:05 PM, Peter McPeterson wrote:
Hi all, I'm trying this solr ruby DSL called Flare/solrb and I
don't really know how the faceted search works because I cant add
whatever fields I want to to the index. This is currently not working:
conn = Solr::Connection.new('http://localhost:8983/solr')
doc = {:id => 1, :cat => 'eletronics', :features => 'video,
music', :product => 'iPod'}
conn.send(Solr::Request::AddDocument.new(doc))
=> #@status_message="ERROR:unknown field 'cat'", @status_code="400",
@raw_response="ERROR:unknown field 'cat'result>", @doc= ... >
In case that if it was working, what I'd like to do is:
(pseudo-code)
request = Solr::Request::Standard.new(
:query => 'ipod',
:facets => {
:fields => :cat
}
)
Any help would be appreciated.
I'm copying in Ed Summers, who may not be on solr-user now, but is a
key contributor to solrb at the moment also.
Good question Peter. Bear with this, as I want to detail lots here
so folks understand what is going on with solrb a bit more clearly
than svn commits and brief allusions.
There are a couple of important things to note here specifically
about Solr itself. It is driven by a schema (see solr/solr/conf/
schema.xml) which defines how fields are handled within Solr/Lucene.
Solr needs to know what to do with field text when it gets it from an
. In the solrb version of Solr's schema, which varies from the
Solr schema that ships with the Solr example application, locks down
two only 3 field naming possiblities: id, *_text, and *_facet). I
intentionally started it as simple as I could for now, knowing that
opening up the schema is inevitable and we want to do it wisely with
a bit more knowledge of how we want Ruby and Solr to interoperate.
Two relatively quick fix options to get you started:
(A) difficulty: easy Rename your non-id fields to *_text and
*_facet. For example:
doc = {:id => 1, :cat_facet => 'eletronics', :features_facet
=> 'video, music', :product_text => 'iPod'}
(B) difficulty: solr experienced only. You're welcome to tweak
the schema.xml and go to town with Request::AddDocument and any field
names you want. Be sure you know what you're doing with faceting,
tokenization, and sorting though.
-- NOTE: If you're familiar with Solr, this will make sense as a
difference to the Solr proper example schema --
id: is mandatory, and is a unique identifier for a document, it
can be any string you like. how searchable this id is depends on
what characters it contains. minimizing special characters makes it
easier to search for a specific id without worrying about query
parser syntax conflicts.
*_text: is tokenized and copied into the "text" field (so the
client doesn't need to/shouldn't send a "text" field, only *_text
field names). the default search field is "text" and includes text
from all *_text fields.
*_facet: is not tokenized, and it is suitable for use with the
faceting features that Solr supports
---
The faceting feature is only starting to come together through the
API, and so its not quite easily exposed. In fact, only earlier
today did the response handling refactoring allow for facets to be
accessed.
*** Sidebar ***
Why does the facet data come back as outside the 'response'
structure? Here's an example:
{
'responseHeader'=>{
'status'=>0,
'QTime'=>3057,
'params'=>{
'wt'=>'ruby',
'facet.limit'=>'2',
'rows'=>'0',
'facet.missing'=>'true',
'start'=>'0',
'facet'=>'true',
'facet.field'=>[
'subject_genre_facet',
'subject_era_facet',
'subject_topic_facet'],
'indent'=>'on',
'q'=>'[* TO *]',
'facet.zeros'=>'true'}},
'response'=>{'numFound'=>4,'start'=>0,'docs'=>[]
},
'facet_counts'=>{
'facet_queries'=>{},
'facet_fields'=>{
'subject_genre_facet'=>{
'Biography.'=>2605,
'Congresses.'=>1837,
''=>38262},
'subject_era_facet'=>{
'20th century.'=>1251,
'20th century'=>1250,
''=>41219},
'subject_topic_facet'=>{
'History.'=>2259,
&#
Re: Faceted search glitch???
On 2/9/07, Peter McPeterson <[EMAIL PROTECTED]> wrote:
On Solr, I'm indexing a facet field which contains different values on each
record (of course) but it doesn't seem to be working the way I had expected.
For instance:
When looking for:
http://localhost:8983/solr/select/?q=sales&facet=true&facet.field=education_facet
it returns the facet:
'education_facet'=>{
'Bachelor'=>20,
'Associate'=>35,
'Doctorate'=>1,
'High School'=>150,
'Law Degree'=>0,
'Master'=>0}
but when I modify my query to look for High School:
http://localhost:8983/solr/select/?q=sales+AND+education_facet:High+School&facet=true&facet.field=education_facet
it returns nothing. But when I use single-word fields such as Bachelor or
Associate it works fine.
education_facet:High+School
Is parsed by the lucene query parser as
education_facet:High OR default_field:School
Surround it by quotes to make it a single term.
Also, for efficiency, you should put the filter in an fq param:
http://localhost:8983/solr/select/?q=sales&fq=education_facet:"High
School"&facet=true&facet.field=education_facet
-Yonik
RE: Faceted search glitch???
Try surrounding you mutli-word query in quotes: education_facet:"High
School". Also do you want to use the filter query param instead
fq=education_facet:"High School"
-Andrew
-Original Message-
From: Peter McPeterson [mailto:[EMAIL PROTECTED]
Sent: Friday, February 09, 2007 4:22 PM
To: solr-user@lucene.apache.org
Subject: Faceted search glitch???
On Solr, I'm indexing a facet field which contains different values on
each
record (of course) but it doesn't seem to be working the way I had
expected.
For instance:
When looking for:
http://localhost:8983/solr/select/?q=sales&facet=true&facet.field=educat
ion_facet
it returns the facet:
'education_facet'=>{
'Bachelor'=>20,
'Associate'=>35,
'Doctorate'=>1,
'High School'=>150,
'Law Degree'=>0,
'Master'=>0}
but when I modify my query to look for High School:
http://localhost:8983/solr/select/?q=sales+AND+education_facet:High+Scho
ol&facet=true&facet.field=education_facet
it returns nothing. But when I use single-word fields such as Bachelor
or
Associate it works fine.
Any idea?
Thanks a lot
_
Check out all that glitters with the MSN Entertainment Guide to the
Academy
Awards(r)
http://movies.msn.com/movies/oscars2007/?icid=ncoscartagline2
Re: Faceted search glitch???
Awesome, thanks that works.
From: "Yonik Seeley" <[EMAIL PROTECTED]>
Reply-To: solr-user@lucene.apache.org
To: solr-user@lucene.apache.org
Subject: Re: Faceted search glitch???
Date: Fri, 9 Feb 2007 16:28:05 -0500
On 2/9/07, Peter McPeterson <[EMAIL PROTECTED]> wrote:
On Solr, I'm indexing a facet field which contains different values on
each
record (of course) but it doesn't seem to be working the way I had
expected.
For instance:
When looking for:
http://localhost:8983/solr/select/?q=sales&facet=true&facet.field=education_facet
it returns the facet:
'education_facet'=>{
'Bachelor'=>20,
'Associate'=>35,
'Doctorate'=>1,
'High School'=>150,
'Law Degree'=>0,
'Master'=>0}
but when I modify my query to look for High School:
http://localhost:8983/solr/select/?q=sales+AND+education_facet:High+School&facet=true&facet.field=education_facet
it returns nothing. But when I use single-word fields such as Bachelor or
Associate it works fine.
education_facet:High+School
Is parsed by the lucene query parser as
education_facet:High OR default_field:School
Surround it by quotes to make it a single term.
Also, for efficiency, you should put the filter in an fq param:
http://localhost:8983/solr/select/?q=sales&fq=education_facet:"High
School"&facet=true&facet.field=education_facet
-Yonik
_
Turn searches into helpful donations. Make your search count.
http://click4thecause.live.com/search/charity/default.aspx?source=hmemtagline_donation&FORM=WLMTAG
Re: Faceted search glitch???
On 2/9/07, Gunther, Andrew <[EMAIL PROTECTED]> wrote: Try surrounding you mutli-word query in quotes: education_facet:"High School". Also do you want to use the filter query param instead fq=education_facet:"High School" Hopefully having the client escape simple term filters won't be necessary in the future. We've discussed a prefix to annotate queries to support other query syntaxes. The simplest form would be a term query with no escaping whatsoever: Something like: fq=High School -Yonik
Re: Faceted search glitch???
On Feb 9, 2007, at 4:45 PM, Yonik Seeley wrote: On 2/9/07, Gunther, Andrew <[EMAIL PROTECTED]> wrote: Try surrounding you mutli-word query in quotes: education_facet:"High School". Also do you want to use the filter query param instead fq=education_facet:"High School" Hopefully having the client escape simple term filters won't be necessary in the future. We've discussed a prefix to annotate queries to support other query syntaxes. The simplest form would be a term query with no escaping whatsoever: Something like: fq=High School This is *definitely* needed. Having an fq parameter analyzed won't work in many cases. Erik
Re: Faceted search glitch???
: The simplest form would be a term query with no escaping whatsoever: : : Something like: : fq=High School I know yonik isn't a big fan of query param proliferation, and there's definitely going to be a need for some kind of markup in the future to denote which query syntax to use for different params, but if facetting and filtering is driving a strong need for a simple way to filter on a facet constraint term then perhaps a special param would make that easier ... in hte same way that "fq" is a filter query, perhaps "ft" can be a filter term: no analysys, just split on the first ":" found, and make a Term query using the LHS as the field name and the RHS as the term... URL: ft=education_facet%3AHigh+School CONF: education_facet:High School ...instead of... URL: fq=education_facet%3A%22High+School%22 CONF: education_facet:"High School" ...or... URL: fq=%3C%21term+f%3D%27education_facet%27%3EHigh+School CONF: High School -Hoss
Re: Faceted search glitch???
On 2/9/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : The simplest form would be a term query with no escaping whatsoever: : : Something like: : fq=High School I know yonik isn't a big fan of query param proliferation, and there's definitely going to be a need for some kind of markup in the future to denote which query syntax to use for different params, but if facetting and filtering is driving a strong need for a simple way to filter on a facet constraint term then perhaps a special param would make that easier ... in hte same way that "fq" is a filter query, perhaps "ft" can be a filter term: no analysys, just split on the first ":" found, and make a Term query using the LHS as the field name and the RHS as the term... Yes, I don't like the proliferation, but... ft=education_facet:High School is both simpler and 11 characters shorter than fq=High School And I think term queries for filters are (or will be) very common in Solr. I think facet-related navigation isn't going to go away so I guess I'm not opposed to ft in this case. Anyone else have thoughts/alternatives? -Yonik
Re: Faceted search glitch???
ft and fq work for me! Erik On Feb 9, 2007, at 5:35 PM, Yonik Seeley wrote: On 2/9/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : The simplest form would be a term query with no escaping whatsoever: : : Something like: : fq=High School I know yonik isn't a big fan of query param proliferation, and there's definitely going to be a need for some kind of markup in the future to denote which query syntax to use for different params, but if facetting and filtering is driving a strong need for a simple way to filter on a facet constraint term then perhaps a special param would make that easier ... in hte same way that "fq" is a filter query, perhaps "ft" can be a filter term: no analysys, just split on the first ":" found, and make a Term query using the LHS as the field name and the RHS as the term... Yes, I don't like the proliferation, but... ft=education_facet:High School is both simpler and 11 characters shorter than fq=High School And I think term queries for filters are (or will be) very common in Solr. I think facet-related navigation isn't going to go away so I guess I'm not opposed to ft in this case. Anyone else have thoughts/alternatives? -Yonik
Re: Faceted search glitch???
Might I suggest: filter.query filter.term filter. -Mike On 2/9/07, Erik Hatcher <[EMAIL PROTECTED]> wrote: ft and fq work for me! Erik On Feb 9, 2007, at 5:35 PM, Yonik Seeley wrote: > On 2/9/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: >> : The simplest form would be a term query with no escaping >> whatsoever: >> : >> : Something like: >> : fq=High School >> >> I know yonik isn't a big fan of query param proliferation, and >> there's >> definitely going to be a need for some kind of markup in the >> future to >> denote which query syntax to use for different params, but if >> facetting >> and filtering is driving a strong need for a simple way to filter >> on a >> facet constraint term then perhaps a special param would make that >> easier >> ... in hte same way that "fq" is a filter query, perhaps "ft" can >> be a >> filter term: no analysys, just split on the first ":" found, and >> make a >> Term query using the LHS as the field name and the RHS as the term... > > Yes, I don't like the proliferation, but... > ft=education_facet:High School > is both simpler and 11 characters shorter than > fq=High School > > And I think term queries for filters are (or will be) very common in > Solr. I think facet-related navigation isn't going to go away so > I guess I'm not opposed to ft in this case. > Anyone else have thoughts/alternatives? > > -Yonik
Re: Faceted search glitch???
: filter.query : filter.term : filter. there's a whole lot of Unicode letters availble thta could fill in the f_ blank, how much more expansion do we really need? :) I suppose filter.term is a wise choice, and adding filter.query as an alias makes sense too ... i'm just a big fan of short param names. (if nothing else, it encourages us not to add too many of them) -Hoss
Re: Faceted search glitch???
On Feb 9, 2007, at 9:16 PM, Mike Klaas wrote: Might I suggest: filter.query filter.term filter. I like it. While Hoss has a point, though descriptive names do make a lot of sense too. It has been a bit confusing to explain facet.query and fq as two different things. However, is there any concern, at all, about the added overhead in passing larger requests around?
Re: Faceted search glitch???
On 2/9/07, Erik Hatcher <[EMAIL PROTECTED]> wrote: On Feb 9, 2007, at 9:16 PM, Mike Klaas wrote: > Might I suggest: > > filter.query > filter.term > filter. I like it. While Hoss has a point, though descriptive names do make a lot of sense too. It has been a bit confusing to explain facet.query and fq as two different things. I like self-documenting parameter names, but if there is concern about verbosity, how 'bout: fq fq.term (also, filter.q could be used in the longer version). As for future expansion, some possibilities are: filter.op/filter.q.op (set operator of filter query parser, which is currently OR regardless of the solrconfig setting), filter.range, and filter.q.noncached (something I would definitely use if it existed). However, is there any concern, at all, about the added overhead in passing larger requests around? I would guess that the difference would be impossible to measure, but I don't really know. -Mike
Re: Faceted search glitch???
: > > filter.query : > > filter.term : > > filter. : I like self-documenting parameter names, but if there is concern about : verbosity, how 'bout: : : fq : fq.term no it's cool ... filter.query and filter.term are definitely better. : As for future expansion, some possibilities are: filter.op/filter.q.op : (set operator of filter query parser, which is currently OR regardless I'm not sure why/how thta would ever make sense... you'd be much better off using seperate filters so they can cache independently - not to mention the reason q.op was added was for dealing with user entered query strings, but are filter queries ever entered manually by a user? : of the solrconfig setting), filter.range, and filter.q.noncached : (something I would definitely use if it existed). filter range could just be done with a filter.query/fq right? a non-cached fq might make sense though. -Hoss
Re: Faceted search glitch???
On 2/10/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : As for future expansion, some possibilities are: filter.op/filter.q.op : (set operator of filter query parser, which is currently OR regardless I'm not sure why/how thta would ever make sense... you'd be much better off using seperate filters so they can cache independently - not to mention the reason q.op was added was for dealing with user entered query strings, but are filter queries ever entered manually by a user? See below. : of the solrconfig setting), filter.range, and filter.q.noncached : (something I would definitely use if it existed). filter range could just be done with a filter.query/fq right? a It sure could, but I was just trying to stretch my imagination as far as possible. filter.term also seems to me like only a marginal improvement over lucene syntax. -Mike
Re: Faceted search glitch???
: It sure could, but I was just trying to stretch my imagination as far : as possible. filter.term also seems to me like only a marginal : improvement over lucene syntax. the only reason i suggested filter.term at all was because it eliminates hte need to worry baout quoting/escaping when dealing with simple term based facets ... right now you have to put the facet term in quotes (inlcuding posisble quote escaping) ... if we add a filter.term param you can write client code that is blindly ignorant of what facet constraint is being filtered on. -Hoss
Re: Solr faceted search UI
It's a pretty subjective and opinionated kinda thing here, as UIs are built with all sorts of technologies and even though I'm quite opinionated about how *I* would build something I work with a lot of folks that have their own preferences or organizational standards/constraints on what they can use. Pragmatically speaking, it's best to use what you or your team are familiar with. That being said... if this is strictly for a PoC and not something you need to put into production as-is, you can leverage the /browse feature powered by Solr's VelocityResponseWriter (wt=velocity) that is in Solr's example configuration. I'm not aware of any Java-based framework out there for Solr - there's so many choices (Struts? Tapestry? JSPs? etc) that any single one of them would be off-putting to others. In Java, the SolrJ library is what you want to use for remote access to Solr. You'll get back a Java response object that you can navigate to pull out the facet information to hand to your view tier. If you're ok with something not Java (but can be deployed in a Java container and can interact with Java) then give projectblacklight.org a try - it's a Ruby on Rails full featured front-end to Solr. There's also solrstrap that looks like a fun place to do some lightweight PoC development. Erik On Apr 24, 2013, at 10:43 , richa wrote: > Hi, > I am working on a POC, where I have to display faceted search result on web > page. can anybody please help me to suggest what all set up I need to > configure to display. I would prefer java technologies. Just to mention, I > have solr cloud running on remote server. > I would like to know: > 1. Should I use MVC framework? > 2. How will my local interact with remote solr server? > 3. How will I send query through java code and what technology I should use > to display faceted search result? > > Please help me on this. > > Thanks, > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr faceted search UI
Hi richa, You can use solrJ (http://wiki.apache.org/solr/Solrj#Reading_Data_from_Solr) to query your solr index. On the wiki page indicated, you will see example of faceted search using solrJ. 2009 article by Yonik available on searchhub<http://searchhub.org/2009/09/02/faceted-search-with-solr/> is a good tutorial on faceted search. Whether you go for MVC framework or not is up to you. It is recommend tough to develop search engine application in a Service Oriented Architecture. Regards, Maj On 24 April 2013 16:43, richa wrote: > Hi, > I am working on a POC, where I have to display faceted search result on web > page. can anybody please help me to suggest what all set up I need to > configure to display. I would prefer java technologies. Just to mention, I > have solr cloud running on remote server. > I would like to know: > 1. Should I use MVC framework? > 2. How will my local interact with remote solr server? > 3. How will I send query through java code and what technology I should use > to display faceted search result? > > Please help me on this. > > Thanks, > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Solr faceted search UI
Thank you very much for your suggestion. This is only for PoC. As you suggested about blacklight, can I run this on windows and to build PoC do I have to have ruby on rails knowledge? Irrespective of any technology and considering the fact that in past I had worked on java, j2ee what would you suggest or how would you have proceeded for this? Blacklight seems to be a good option, not sure without prior knowledge of ruby on rails, will I be able to present in short period of time? any suggestion on this? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058617.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr faceted search UI
I tried previous version of blacklight (on a Mac) and was able to get it to the demo stage without much RoR knowledge. The facet field declarations were all in the config files. You should be able to get a go/nogo decision in under four hours. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Wed, Apr 24, 2013 at 11:23 AM, richa wrote: > Thank you very much for your suggestion. > This is only for PoC. As you suggested about blacklight, can I run this on > windows and to build PoC do I have to have ruby on rails knowledge? > > Irrespective of any technology and considering the fact that in past I had > worked on java, j2ee what would you suggest or how would you have proceeded > for this? > > Blacklight seems to be a good option, not sure without prior knowledge of > ruby on rails, will I be able to present in short period of time? any > suggestion on this? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058617.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr faceted search UI
Hi Maj, Thanks for your suggestion. Tell me one thing, do you have any example on solrj? suppose I decide to use solrj in simple web application, to display faceted search on web page. Where will this fit into? what will be the flow? Please suggest. Thanks On Wed, Apr 24, 2013 at 11:01 AM, Majirus FANSI [via Lucene] < ml-node+s472066n4058610...@n3.nabble.com> wrote: > Hi richa, > You can use solrJ ( > http://wiki.apache.org/solr/Solrj#Reading_Data_from_Solr) > to query your solr index. > On the wiki page indicated, you will see example of faceted search using > solrJ. > 2009 article by Yonik available on > searchhub<http://searchhub.org/2009/09/02/faceted-search-with-solr/> > is > a good tutorial on faceted search. > Whether you go for MVC framework or not is up to you. It is recommend > tough > to develop search engine application in a Service Oriented Architecture. > Regards, > > Maj > > > On 24 April 2013 16:43, richa <[hidden > email]<http://user/SendEmail.jtp?type=node&node=4058610&i=0>> > wrote: > > > Hi, > > I am working on a POC, where I have to display faceted search result on > web > > page. can anybody please help me to suggest what all set up I need to > > configure to display. I would prefer java technologies. Just to mention, > I > > have solr cloud running on remote server. > > I would like to know: > > 1. Should I use MVC framework? > > 2. How will my local interact with remote solr server? > > 3. How will I send query through java code and what technology I should > use > > to display faceted search result? > > > > Please help me on this. > > > > Thanks, > > > > > > > > -- > > View this message in context: > > http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058610.html > To unsubscribe from Solr faceted search UI, click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4058598&code=c3RyaWtldGhlZ29hbEBnbWFpbC5jb218NDA1ODU5OHwxNzIzOTAyMzYx> > . > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058619.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr faceted search UI
Hi Rocha,
In your webapp I guess you have at list a view and a service layers.
The indexing and search modules should preferably be hosted at the service
layer.
I recommend you read the Api doc (
http://lucene.apache.org/solr/4_2_1/solr-solrj/index.html) to get a sense
of what you can do with SolrJ.
Followinf is a Basic Example Facets with SolrJ:
//adding the query keyword to the SolrQuery object
mySolrQuery.setQuery(queryBuilder.toString());
// add a facet field
mySolrQuery.addFacetField("myFieldName")
//add a facet query
validatedFromTheLast7DaysFacetQuery = validationDateField + ":[NOW/DAY-7DAY
TO NOW]"
mySolrQuery.addFacetQuery(validatedFromTheLast7DaysFacetQuery)
//send the request in HTTP POST as with HTTP GET you run into issues when
the request string is too long.
QueryResponse queryResponse = getSolrHttpServer().query(mysolrQuery,
METHOD.POST);
//write a transformer to convert the Solr response to a format
understandable by the caller (the client of the search service)
//List of results to transform
SolrDocumentList responseSolrDocumentList = queryResponse.getResults();
//get the facet fields, interate over the list, parse each FacetField and
extract the information you are intersted in
queryResponse.getFacetFields()
//get the facet query from the response
Map mapOfFacetQueries = queryResponse.getFacetQuery();
The keys of this map are your facet queries. The values are the counts you
display to the user. In general, I have an identifier for each facetQuery.
When I parse the keys of this map of facet queries, I return the identifier
of each facet along with its count (if the count > 0 of course). The caller
is aware of this identifier so it knows what to display to the user.
When the user clicks on a facet, you send it as a search criteria along
with the initial keywords to the search service. The criteria resulting
from the facet is treated as a filter query. That is how faceting search
works. Adding a filter to your query is as simple as this snippet
mySolrQuery.addFilterQuery(myfilterQuery). should you are filtering because
your user click on the previously defined facet query, then the filter
query is the same as the facet query. that is myfilterQuery =
validationDateField + ":[NOW/DAY-7DAY TO NOW]".
I hope this helps.
Cheers,
Maj
On 24 April 2013 17:27, richa wrote:
> Hi Maj,
>
> Thanks for your suggestion.
> Tell me one thing, do you have any example on solrj? suppose I decide to
> use solrj in simple web application, to display faceted search on web page.
> Where will this fit into? what will be the flow?
>
> Please suggest.
>
> Thanks
>
>
> On Wed, Apr 24, 2013 at 11:01 AM, Majirus FANSI [via Lucene] <
> ml-node+s472066n4058610...@n3.nabble.com> wrote:
>
> > Hi richa,
> > You can use solrJ (
> > http://wiki.apache.org/solr/Solrj#Reading_Data_from_Solr)
> > to query your solr index.
> > On the wiki page indicated, you will see example of faceted search using
> > solrJ.
> > 2009 article by Yonik available on
> > searchhub<http://searchhub.org/2009/09/02/faceted-search-with-solr/>
> > is
> > a good tutorial on faceted search.
> > Whether you go for MVC framework or not is up to you. It is recommend
> > tough
> > to develop search engine application in a Service Oriented Architecture.
> > Regards,
> >
> > Maj
> >
> >
> > On 24 April 2013 16:43, richa <[hidden email]<
> http://user/SendEmail.jtp?type=node&node=4058610&i=0>>
> > wrote:
> >
> > > Hi,
> > > I am working on a POC, where I have to display faceted search result on
> > web
> > > page. can anybody please help me to suggest what all set up I need to
> > > configure to display. I would prefer java technologies. Just to
> mention,
> > I
> > > have solr cloud running on remote server.
> > > I would like to know:
> > > 1. Should I use MVC framework?
> > > 2. How will my local interact with remote solr server?
> > > 3. How will I send query through java code and what technology I should
> > use
> > > to display faceted search result?
> > >
> > > Please help me on this.
> > >
> > > Thanks,
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598.html
> > > Sent from the Solr - User mailing list archive at Nabble.com.
> > >
> >
> >
> > --
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058610.html
> >
Re: Solr faceted search UI
Hi Richa Solrstrap is probably the best way to go if you just want to get up a PoC as fast as possible. Solrstrap requires no installation of middleware, you just add in the address of your solr server and open the file in your browser. Regards Fergus On Wed, Apr 24, 2013 at 5:23 PM, richa wrote: > Thank you very much for your suggestion. > This is only for PoC. As you suggested about blacklight, can I run this on > windows and to build PoC do I have to have ruby on rails knowledge? > > Irrespective of any technology and considering the fact that in past I had > worked on java, j2ee what would you suggest or how would you have proceeded > for this? > > Blacklight seems to be a good option, not sure without prior knowledge of > ruby on rails, will I be able to present in short period of time? any > suggestion on this? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058617.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Solr faceted search UI
You also get some smooth UI stuff "for free" F On Tue, May 28, 2013 at 10:58 AM, Fergus McDowall wrote: > Hi Richa > > Solrstrap is probably the best way to go if you just want to get up a PoC > as fast as possible. Solrstrap requires no installation of middleware, you > just add in the address of your solr server and open the file in your > browser. > > Regards > Fergus > > > > On Wed, Apr 24, 2013 at 5:23 PM, richa wrote: > >> Thank you very much for your suggestion. >> This is only for PoC. As you suggested about blacklight, can I run this on >> windows and to build PoC do I have to have ruby on rails knowledge? >> >> Irrespective of any technology and considering the fact that in past I had >> worked on java, j2ee what would you suggest or how would you have >> proceeded >> for this? >> >> Blacklight seems to be a good option, not sure without prior knowledge of >> ruby on rails, will I be able to present in short period of time? any >> suggestion on this? >> >> >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/Solr-faceted-search-UI-tp4058598p4058617.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > >
Re: Faceted search on SOLR
Facets work on the tokenized content of the field. You must be using text_en or similar type. Switch the facet field to use String. If you are using eDisMax and want to be able to search it for substrings (e.g. if somebody searches for Intel), use copyField to create two fields, one for searching, one for faceting. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Wed, Mar 6, 2013 at 1:06 PM, Erol Akarsu wrote: > Hello, > > I am trying to design faceted search with SOLR. > > When I define > > CPU_Type > > I am getting results with http://localhost:8080/solr/browse > > CPU Type > > core (415) > intel (368) > i5 (176) > i3 (104) > i7 (74) > series (73) > pentium (60) > amd (55) > 2 (44) > duo (42) > fx (39) > apu (24) > dual (17) > atom (16) > > But I would like to get this type of faceting: > > CPU Type > Intel Core i3 (109) > Intel Core i5 (184) > Intel Core i7 (82) > AMD FX-Series (36) > A4-Series APU (5) > A6-Series APU (7) > > How Can I get second result? > > I appreciate you help > > Erol Akarsu >
Re: Faceted search on SOLR
Thanks for help, Alexandre. It worked as you described. I have other question. Suppose I have product catalogue that has many sub categories, each one has different group of fields. When a user search the catalogue, we should show corresponding facet fields on left based on result set. That means we should be dynamically pick appropriate facets to show based on result products from search. Can we do it with SOLR? Or SOLR can show only static facets configured? It is better to show only related facets I guess. Also, can we filter out facets that has ,say less than 10 results? I appreciate your answer Erol Akarsu -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-search-on-SOLR-tp4045297p4045563.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceted Search Patent Lawsuit
Le 17 août 2011 à 13:01, Robert Muir a écrit : >> On Tue, Aug 16, 2011 at 03:58:29PM -0400, Grant Ingersoll wrote: >>> I know you mean well and are probably wondering what to do next [...] >> >> Still, a short heads-up like Johnson's would seem OK? >> After all, this is of concern to us all. > > nothing to be concerned about, just a stupid patent troll, and I feel > like this thread feeds him. > > just dont give this "company" any money, do your homework first Robert, I believe, precisely, the objective of such a thread is to be helped by knowledgeable techies into being able to do what you say. If Johnson gave only 3 lines of details, such as claimed patent URLs or dates, we might easily be able to tell him the pointer of a publication that would discourage such patent-trolling. paul
Re: Faceted Search Patent Lawsuit
Paul Libbrecht-4 wrote: > > Robert, > > I believe, precisely, the objective of such a thread is to be helped by > knowledgeable techies into being able to do what you say. > > If Johnson gave only 3 lines of details, such as claimed patent URLs or > dates, we might easily be able to tell him the pointer of a publication > that would discourage such patent-trolling. > > paul > I'm sorry. I assumed the information I provided would lead any resourceful techie to the details. At any rate here is some additional information: Patent Claim: http://www.google.com/patents?id=oFwIEBAJ&printsec=frontcover&dq=6275821&hl=en&ei=57ZLTs7jHs3HsQKWvsjcCA&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCkQ6AEwAA Background: http://www.ndcalblog.com/2011/07/ebay-prevails-in-limiting-patent.html Ebay/Microsoft suit: http://www.whda.com/blog/wp-content/uploads/2011/05/2011.5.12-MO-Ebay-v.-Parts-River.pdf Adobe suit: http://news.priorsmart.com/adobe-systems-v-kelora-systems-l4in/ Consequently, they have been known to act on these threats. I don't think it would be prudent to ignore them. At any rate, lawyers will be involved and they aren't cheap. Until the suits have been played out with the likes of eBay, Microsoft, and Adobe, potentially anyone who uses faceted search systems could potentially be at risk. Take it for what it's worth. Don't shoot the messenger. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-Search-Patent-Lawsuit-Please-Read-tp3259475p3261514.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceted Search Patent Lawsuit
On Wed, Aug 17, 2011 at 8:51 AM, LaMaze Johnson wrote: > > Paul Libbrecht-4 wrote: > > > > Robert, > > > > I believe, precisely, the objective of such a thread is to be helped by > > knowledgeable techies into being able to do what you say. > > > > If Johnson gave only 3 lines of details, such as claimed patent URLs or > > dates, we might easily be able to tell him the pointer of a publication > > that would discourage such patent-trolling. > > > > paul > > > > I'm sorry. I assumed the information I provided would lead any resourceful > techie to the details. At any rate here is some additional information: > > Patent Claim: > > http://www.google.com/patents?id=oFwIEBAJ&printsec=frontcover&dq=6275821&hl=en&ei=57ZLTs7jHs3HsQKWvsjcCA&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCkQ6AEwAA > > Background: > http://www.ndcalblog.com/2011/07/ebay-prevails-in-limiting-patent.html > > Ebay/Microsoft suit: > > http://www.whda.com/blog/wp-content/uploads/2011/05/2011.5.12-MO-Ebay-v.-Parts-River.pdf > > Adobe suit: > http://news.priorsmart.com/adobe-systems-v-kelora-systems-l4in/ > > Consequently, they have been known to act on these threats. I don't think > it would be prudent to ignore them. At any rate, lawyers will be involved > and they aren't cheap. Until the suits have been played out with the likes > of eBay, Microsoft, and Adobe, potentially anyone who uses faceted search > systems could potentially be at risk. > > Take it for what it's worth. Don't shoot the messenger. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Faceted-Search-Patent-Lawsuit-Please-Read-tp3259475p3261514.html > Sent from the Solr - User mailing list archive at Nabble.com. > I, for one, am grateful for you posting this information. Thank you Matthew Shields Owner BeanTown Host - Web Hosting, Domain Names, Dedicated Servers, Colocation, Managed Services www.beantownhost.com www.sysadminvalley.com www.jeeprally.com
Re: Faceted Search Patent Lawsuit
I have no plan to look at the patents, but there is some serious prior art in faceted search. First, faceted classification for libraries was invented by S. R. Ranganathan in 1933. Computer search for libraries dates from the 1960's, probably. Combining the two is obvious, even back then. wunder On Aug 17, 2011, at 7:55 AM, Matt Shields wrote: > On Wed, Aug 17, 2011 at 8:51 AM, LaMaze Johnson wrote: > >> >> Paul Libbrecht-4 wrote: >>> >>> Robert, >>> >>> I believe, precisely, the objective of such a thread is to be helped by >>> knowledgeable techies into being able to do what you say. >>> >>> If Johnson gave only 3 lines of details, such as claimed patent URLs or >>> dates, we might easily be able to tell him the pointer of a publication >>> that would discourage such patent-trolling. >>> >>> paul >>> >> >> I'm sorry. I assumed the information I provided would lead any resourceful >> techie to the details. At any rate here is some additional information: >> >> Patent Claim: >> >> http://www.google.com/patents?id=oFwIEBAJ&printsec=frontcover&dq=6275821&hl=en&ei=57ZLTs7jHs3HsQKWvsjcCA&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCkQ6AEwAA >> >> Background: >> http://www.ndcalblog.com/2011/07/ebay-prevails-in-limiting-patent.html >> >> Ebay/Microsoft suit: >> >> http://www.whda.com/blog/wp-content/uploads/2011/05/2011.5.12-MO-Ebay-v.-Parts-River.pdf >> >> Adobe suit: >> http://news.priorsmart.com/adobe-systems-v-kelora-systems-l4in/ >> >> Consequently, they have been known to act on these threats. I don't think >> it would be prudent to ignore them. At any rate, lawyers will be involved >> and they aren't cheap. Until the suits have been played out with the likes >> of eBay, Microsoft, and Adobe, potentially anyone who uses faceted search >> systems could potentially be at risk. >> >> Take it for what it's worth. Don't shoot the messenger. >> >> >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/Faceted-Search-Patent-Lawsuit-Please-Read-tp3259475p3261514.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > > I, for one, am grateful for you posting this information. Thank you > > Matthew Shields > Owner > BeanTown Host - Web Hosting, Domain Names, Dedicated Servers, Colocation, > Managed Services > www.beantownhost.com > www.sysadminvalley.com > www.jeeprally.com
Re: Faceted Search Patent Lawsuit
patent rights only last 17 years then it is public domain. On 08/17/2011 11:05 AM, Walter Underwood wrote: I have no plan to look at the patents, but there is some serious prior art in faceted search. First, faceted classification for libraries was invented by S. R. Ranganathan in 1933. Computer search for libraries dates from the 1960's, probably. Combining the two is obvious, even back then. wunder On Aug 17, 2011, at 7:55 AM, Matt Shields wrote: On Wed, Aug 17, 2011 at 8:51 AM, LaMaze Johnson wrote: Paul Libbrecht-4 wrote: Robert, I believe, precisely, the objective of such a thread is to be helped by knowledgeable techies into being able to do what you say. If Johnson gave only 3 lines of details, such as claimed patent URLs or dates, we might easily be able to tell him the pointer of a publication that would discourage such patent-trolling. paul I'm sorry. I assumed the information I provided would lead any resourceful techie to the details. At any rate here is some additional information: Patent Claim: http://www.google.com/patents?id=oFwIEBAJ&printsec=frontcover&dq=6275821&hl=en&ei=57ZLTs7jHs3HsQKWvsjcCA&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCkQ6AEwAA Background: http://www.ndcalblog.com/2011/07/ebay-prevails-in-limiting-patent.html Ebay/Microsoft suit: http://www.whda.com/blog/wp-content/uploads/2011/05/2011.5.12-MO-Ebay-v.-Parts-River.pdf Adobe suit: http://news.priorsmart.com/adobe-systems-v-kelora-systems-l4in/ Consequently, they have been known to act on these threats. I don't think it would be prudent to ignore them. At any rate, lawyers will be involved and they aren't cheap. Until the suits have been played out with the likes of eBay, Microsoft, and Adobe, potentially anyone who uses faceted search systems could potentially be at risk. Take it for what it's worth. Don't shoot the messenger. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-Search-Patent-Lawsuit-Please-Read-tp3259475p3261514.html Sent from the Solr - User mailing list archive at Nabble.com. I, for one, am grateful for you posting this information. Thank you Matthew Shields Owner BeanTown Host - Web Hosting, Domain Names, Dedicated Servers, Colocation, Managed Services www.beantownhost.com www.sysadminvalley.com www.jeeprally.com
Re: Faceted Search Patent Lawsuit
Hi, Sorry for the top-quote: On a mobile. A discussion on the evils of patents aside, shirley library catalogues are prior art. I remember such systems giving lists of matches by category, if maybe not counts. Will look at the patent applications, but sheesh, what a waste of time and resources. Regards, Gora On 17-Aug-2011 8:36 PM, "Walter Underwood" wrote: > I have no plan to look at the patents, but there is some serious prior art in faceted search. First, faceted classification for libraries was invented by S. R. Ranganathan in 1933. Computer search for libraries dates from the 1960's, probably. Combining the two is obvious, even back then. > > wunder > > On Aug 17, 2011, at 7:55 AM, Matt Shields wrote: > >> On Wed, Aug 17, 2011 at 8:51 AM, LaMaze Johnson wrote: >> >>> >>> Paul Libbrecht-4 wrote: >>>> >>>> Robert, >>>> >>>> I believe, precisely, the objective of such a thread is to be helped by >>>> knowledgeable techies into being able to do what you say. >>>> >>>> If Johnson gave only 3 lines of details, such as claimed patent URLs or >>>> dates, we might easily be able to tell him the pointer of a publication >>>> that would discourage such patent-trolling. >>>> >>>> paul >>>> >>> >>> I'm sorry. I assumed the information I provided would lead any resourceful >>> techie to the details. At any rate here is some additional information: >>> >>> Patent Claim: >>> >>> http://www.google.com/patents?id=oFwIEBAJ&printsec=frontcover&dq=6275821&hl=en&ei=57ZLTs7jHs3HsQKWvsjcCA&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCkQ6AEwAA >>> >>> Background: >>> http://www.ndcalblog.com/2011/07/ebay-prevails-in-limiting-patent.html >>> >>> Ebay/Microsoft suit: >>> >>> http://www.whda.com/blog/wp-content/uploads/2011/05/2011.5.12-MO-Ebay-v.-Parts-River.pdf >>> >>> Adobe suit: >>> http://news.priorsmart.com/adobe-systems-v-kelora-systems-l4in/ >>> >>> Consequently, they have been known to act on these threats. I don't think >>> it would be prudent to ignore them. At any rate, lawyers will be involved >>> and they aren't cheap. Until the suits have been played out with the likes >>> of eBay, Microsoft, and Adobe, potentially anyone who uses faceted search >>> systems could potentially be at risk. >>> >>> Take it for what it's worth. Don't shoot the messenger. >>> >>> >>> >>> -- >>> View this message in context: >>> http://lucene.472066.n3.nabble.com/Faceted-Search-Patent-Lawsuit-Please-Read-tp3259475p3261514.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >> >> I, for one, am grateful for you posting this information. Thank you >> >> Matthew Shields >> Owner >> BeanTown Host - Web Hosting, Domain Names, Dedicated Servers, Colocation, >> Managed Services >> www.beantownhost.com >> www.sysadminvalley.com >> www.jeeprally.com >
solr faceted search performance reason
Hello List, Please see my question at http://stackoverflow.com/questions/5552919/how-does-lucene-solr-achieve-high-performance-in-multi-field-faceted-search, I would be interested to know some details. Thank you, Robin
Re: faceted search field sorting
If you don't want the facets sorted by decreasing count (the default), you can specify facet.sort=index to get "index order" ((lexicographic by indexed term) -Yonik http://heliosearch.com -- making solr shine On Sun, Dec 15, 2013 at 2:08 PM, MC wrote: > Hello, > Here is a public API that uses facet fields: > http://golr.berkeleybop.org/select?qt=standard&fl=*&version=2.2&wt=json&indent=on&rows=0&facet=true&facet.field=type&q=document_category:%22annotation%22 > > It looks like the default behavior is to sort facet_counts.facet_fields > (field called 'type' in this case) descending by frequency count among > returned documents. > I'd like to find out if this is a default behavior or if this is an extra > step that Solr is performing and can be somehow turned off? I'd like to use > that data to form a tag cloud and I'd like to avoid the random shuffle step > if that is something that can be turned off in Solr query. > If this is the default result maybe someone could explain how faceted index > works on Solr side, and where does that order come from? I'd like to > understand it better either way. > Thanks, > > M >
Re: Faceted search question (Tokenizing)
Here is another simpler example of what I am trying to achieve: Multi-Valued Field 1: Data 1 Data 2 Data 3 Data 4 Multi-Valued Field 2: Data 11 Data 12 Data 13 Data 14 Multi-Valued Field 3: Data 21 Data 22 Data 23 Data 24 How can I specify that Data 1,Data 11 and data 21 are all related? And if I facet Data 1 + Data 11, I only want to see Data 21. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-search-question-Tokenizing-tp4012948p4012956.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Faceted search question (Tokenizing)
What do you want the results to be, persons? And the facets should be interests or subinterests? Why are there two layers of interests anyway? Can there my many subinterests under one interest? Is one of those two a name of the interest which would look nice as a facet? Anyway, have you read these pages yet? These should get you started in the right direction. http://wiki.apache.org/solr/SolrFacetingOverview http://wiki.apache.org/solr/HierarchicalFaceting Hope that helps, Robi -Original Message- From: Grapes [mailto:mkloub...@gmail.com] Sent: Wednesday, October 10, 2012 8:52 AM To: solr-user@lucene.apache.org Subject: Faceted search question (Tokenizing) Hey There, We have the following data structure: - Person -- Interest 1 --- Subinterest 1 --- Subinterest 1 Description --- Subinterest 1 ID -- Interest 2 --- Subinterest 2 --- Subinterest 2 Description --- Subinterest 2 ID . -- Interest 99 --- Subinterest 99 --- Subinterest 99 Description --- Subinterest 99 ID Interest, Subinterest, Subinterest Description and Subinterest IDs are all multiavlued fields. A person can have any number of subinterests,descriptions and IDS. How could we faced/search this based on this data structure? Right now we tokenized everything in a seperate multivalued column in the following fasion; |Interest='Interest 1',Subinterest='Subinterest 1',Subinterest='Another Subinterest 1',Description='Interest 1 Description',ID='Interest 1 ID'| |Interest='Interest 2',Description='Interest 2 Description',ID='Interest |2 ID'| I have a feeling like this is a wrong approach to this problem. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-search-question-Tokenizing-tp4012948.html Sent from the Solr - User mailing list archive at Nabble.com.
Faceted Search on Dynamic Fields?
I'm trying to perform a faceted query with the facet field referencing a field that is not in the schema but matches a dynamicField with its suffix. The query returns results but for some reason the facet list is always empty. When I change the facet field to one that is explicitly named in the schema I get the proper results. Is this expected behavior? I wasn't able to find anything in the docs about dynamic fields wrt faceting. One other thing I thought might have been causing the problem is that the values in this field are mostly distinct (that won't be the case in the actual application, I'm just doing it this way now to see how faceted queries behave). However, when I performed the same query with a static field with lots of distinct values I just got an OutOfMemoryError, which leads me back to my original hypothesis. So, is it the case that faceted queries are not permitted on dynamic facet fields, and if so, is there any workaround? -- View this message in context: http://www.nabble.com/Faceted-Search-on-Dynamic-Fields--tp25612887p25612887.html Sent from the Solr - User mailing list archive at Nabble.com.
Metada document for faceted search
Hi. I'm trying to implement custom faceted search like CNET's approach.<http://www.mail-archive.com/java-u...@lucene.apache.org/msg02646.html> . But i couldn't figure out how to structure and index category metadata document. Thanks. -- Osman İZBAT
faceted search cache and optimisations
Hello, I'm using faceted search (perhaps in a dumb way) to collect some statistics for my index. I have documents in various languages, one of the field is "language" and I simply want to see how many documents I have for each language. I have noticed that the search builds a int[maxDoc] array and then traverses the array to count. If facet.method=enum (discovered later) is used, the things are still counted in a different way. But for this case where all the documents are retrieved, the information is already available in the lucene index. So, I think it would be a good optimization to detect these cases (i.e. no filtering) and just return the number from the index instead of counting the docs again. Another issue: there is no way currently to disable the caching of the int[maxDoc], is there? If there are many fields to be faceted, this can quikly lead to out of memory situations. I think it would be good to give the option (as part of the query) to disable the caching, even if it is slow, at least it works and is useful for non-interactive processing. And another possibe optimization for the int[maxDoc] inspired from the column stored databases: the way they do it is to find the minimum number of bits to represent a value. If for example my language field has 30 possible values (i.e. I have docs in 30 languages), I only need 5 bits for each doc (instead of int=32 bits). Then I can represent the whole int[maxDoc] in less than 1/6 of the space required now. What's even better, sometimes the documents can be partitioned such that not all the values of a field are represented in the same partition. For example let's assume that I have a field called doc_generation_date. If I harverst the documents each three days, and I consider a partition as having the same three days of data, for each partition I will basically have only three possible values for the doc_generation_date. That means that I only need to have 2 bits for each document plus a table for each partition that maps from the partition value id (one of the three values represented on two bits) to the index value id (that is the id stored in the lucene index). Of course, for the language field above, the partitioning would not help unless I index successively only english docs, then only french, etc. And also it wouldn't work just like that for multi-value fields. nicolae
Faceted search in 2 indexes
Hi all. I have 2 indexes with some similar fields and some distinct fields. I need to make a faceted search that returns the union of the same search in these 2 indexes. How can I make it? Thank's "Então aproximaram-se os que estavam no barco, e adoraram-no, dizendo: És verdadeiramente o Filho de Deus." (Mateus 14:33)
2 indexes - 1 Faceted search
Hi all. I have 2 indexes with different structures, but a few equal fields. There is some way to do a faceted search (faceting the equal fields) in the 2 indexes at same time, with only one response (like a UNION in sql server)? Thank's "Então aproximaram-se os que estavam no barco, e adoraram-no, dizendo: És verdadeiramente o Filho de Deus." (Mateus 14:33)
Re: Faceted search not working?
Hi Ilya, Ilya Sterin wrote: I'm trying to perform a faceted search without any luck. Result set doesn't return any facet information... http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title I'm getting the result set, but no face information present? Is there something else that needs to happen to turn faceting on? No. What does http://localhost:8080/solr/select/?q=title:*&fl=title&wt=xml return? -Sascha
Re: Faceted search not working?
it's probably because your query isn't matching anything. try q=title: [* TO *], or maybe even simply *:* if you are trying to match all documents. Erik On May 24, 2010, at 5:11 PM, Ilya Sterin wrote: I'm trying to perform a faceted search without any luck. Result set doesn't return any facet information... http://localhost:8080/solr/select/? q=title:*&facet=on&facet.field=title I'm getting the result set, but no face information present? Is there something else that needs to happen to turn faceting on? I'm using latest Solr 1.4 release. Data is indexed from the database using dataimporter. Thanks. Ilya Sterin
RE: Faceted search not working?
Hi, try http://localhost:8080/solr/select/?q=YOUR-QUERY&facet=true&facet.field=title I don't think the bolean fields is mapped to "on" and "off" :) -birger -Original Message- From: Ilya Sterin [mailto:ster...@gmail.com] Sent: 24. mai 2010 23:11 To: solr-user@lucene.apache.org Subject: Faceted search not working? I'm trying to perform a faceted search without any luck. Result set doesn't return any facet information... http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title I'm getting the result set, but no face information present? Is there something else that needs to happen to turn faceting on? I'm using latest Solr 1.4 release. Data is indexed from the database using dataimporter. Thanks. Ilya Sterin
Re: Faceted search not working?
Hi Birger, Birger Lie wrote: I don't think the bolean fields is mapped to "on" and "off" :) You can use true and on interchangeably. -Sascha -birger -Original Message- From: Ilya Sterin [mailto:ster...@gmail.com] Sent: 24. mai 2010 23:11 To: solr-user@lucene.apache.org Subject: Faceted search not working? I'm trying to perform a faceted search without any luck. Result set doesn't return any facet information... http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title I'm getting the result set, but no face information present? Is there something else that needs to happen to turn faceting on? I'm using latest Solr 1.4 release. Data is indexed from the database using dataimporter. Thanks. Ilya Sterin
Re: Faceted search not working?
Is the FacetComponent loaded at all? query facet On 2010-05-25, at 3:32 AM, Sascha Szott wrote: > Hi Birger, > > Birger Lie wrote: >> I don't think the bolean fields is mapped to "on" and "off" :) > You can use true and on interchangeably. > > -Sascha > >> >> >> -birger >> >> -Original Message- >> From: Ilya Sterin [mailto:ster...@gmail.com] >> Sent: 24. mai 2010 23:11 >> To: solr-user@lucene.apache.org >> Subject: Faceted search not working? >> >> I'm trying to perform a faceted search without any luck. Result set doesn't >> return any facet information... >> >> http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title >> >> I'm getting the result set, but no face information present? Is there >> something else that needs to happen to turn faceting on? >> >> I'm using latest Solr 1.4 release. Data is indexed from the database using >> dataimporter. >> >> Thanks. >> >> Ilya Sterin >
Re: Faceted search not working?
Hi, please note, that the FacetComponent is one of the six search components that are automatically associated with solr.SearchHandler (this holds also for the QueryComponent). Another note: By using name="components" all default components will be replaced by the components you explicitly mentioned (i.e., QueryComponent and FacetComponent in your example). To avoid this, use name="last-components" instead. -Sascha Jean-Sebastien Vachon wrote: Is the FacetComponent loaded at all? query facet On 2010-05-25, at 3:32 AM, Sascha Szott wrote: Hi Birger, Birger Lie wrote: I don't think the bolean fields is mapped to "on" and "off" :) You can use true and on interchangeably. -Sascha -birger -Original Message- From: Ilya Sterin [mailto:ster...@gmail.com] Sent: 24. mai 2010 23:11 To: solr-user@lucene.apache.org Subject: Faceted search not working? I'm trying to perform a faceted search without any luck. Result set doesn't return any facet information... http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title I'm getting the result set, but no face information present? Is there something else that needs to happen to turn faceting on? I'm using latest Solr 1.4 release. Data is indexed from the database using dataimporter. Thanks. Ilya Sterin
Re: Faceted search not working?
Sascha thanks for the response, here is the output... 0 0 xml title:* title Baseball game Soccer game Football game On Mon, May 24, 2010 at 5:39 PM, Sascha Szott wrote: > Hi Ilya, > > Ilya Sterin wrote: >> >> I'm trying to perform a faceted search without any luck. Result set >> doesn't return any facet information... >> >> http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title >> >> I'm getting the result set, but no face information present? Is there >> something else that needs to happen to turn faceting on? > > No. > > What does http://localhost:8080/solr/select/?q=title:*&fl=title&wt=xml > return? > > -Sascha > >
RE: Re: Faceted search outofmemory
http://wiki.apache.org/solr/SimpleFacetParameters#facet.limit -Original message- From: olivier sallou Sent: Tue 29-06-2010 20:11 To: solr-user@lucene.apache.org; Subject: Re: Faceted search outofmemory How do make paging over facets? 2010/6/29 Ankit Bhatnagar > > Did you trying paging them? > > > -Original Message- > From: olivier sallou [mailto:olivier.sal...@gmail.com] > Sent: Tuesday, June 29, 2010 2:04 PM > To: solr-user@lucene.apache.org > Subject: Faceted search outofmemory > > Hi, > I try to make a faceted search on a very large index (around 200GB with > 200M > doc). > I have an out of memory error. With no facet it works fine. > > There are quite many questions around this but I could not find the answer. > How can we know the required memory when facets are used so that I try to > scale my server/index correctly to handle it. > > Thanks > > Olivier >
Re: Re: Faceted search outofmemory
I already use facet.limit in my query. I tried however facet.method=enum and though it does not seem to fix everything, I have some requests without the outofmemory error. Best would be to have a calculation rule of required memory for such type of query. 2010/6/29 Markus Jelsma > http://wiki.apache.org/solr/SimpleFacetParameters#facet.limit > > -Original message- > From: olivier sallou > Sent: Tue 29-06-2010 20:11 > To: solr-user@lucene.apache.org; > Subject: Re: Faceted search outofmemory > > How do make paging over facets? > > 2010/6/29 Ankit Bhatnagar > > > > > Did you trying paging them? > > > > > > -Original Message- > > From: olivier sallou [mailto:olivier.sal...@gmail.com] > > Sent: Tuesday, June 29, 2010 2:04 PM > > To: solr-user@lucene.apache.org > > Subject: Faceted search outofmemory > > > > Hi, > > I try to make a faceted search on a very large index (around 200GB with > > 200M > > doc). > > I have an out of memory error. With no facet it works fine. > > > > There are quite many questions around this but I could not find the > answer. > > How can we know the required memory when facets are used so that I try to > > scale my server/index correctly to handle it. > > > > Thanks > > > > Olivier > > >
faceted search with job title
Hi, I am currently using nutch to crawl some job pages from job boards. They are in my solr index now. I want to do faceted search with the job titles. How? The job titles can be in any locations of the page, e.g. title, header, content... If I use indexfilter in Nutch to search the content for job title, there are hundred of thousands of job titles, I can't hard code them all. Do you have a better idea? I think I need the job title in a separate field in the index to make it work with solr faceted search, am I right? Thanks.
Parsing Faceted Search output in Solr
Hi all, I'm using the faceted search in Solr. I wanted to know if there's a way to display selective records in the sublist. I've tried this query - localhost:8983/solr/select?q=*:*&facet=true&facet.pivot=city,jobtype&facet.field=jobtype&facet.query=jobtype:Tiempo&fl=title I want to display jobs of only Tiempo type. But it doesn't consider the facet.query field. Any other way to do this? Thanks, Mayuri
Faceted Search Patent Lawsuit - Please Read
Just wanted to make people aware of a company called Kolera that is sending threatening letters via their law firm, manatt, phelps, phillips in regard to violation of their patent U.S. Patent No. 6,275,821 titled "Method and System for Executing a Guided Parametric Search. Apparently, they believe that this patent gives them the rights to faceted search technology. Consequently, anyone who uses this technology without paying licensing fees is potentially a target of litigation. They have outstanding suits with companies such as Microsoft and Ebay. Additionally, Adobe filed a counter-suit on August 10, 2011.Unfortunately, I received a letter this afternoon and we are determining how to proceed. Just thought I would make others aware of this. I'd appreciate any insight others might have regarding the issue. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceted-Search-Patent-Lawsuit-Please-Read-tp3259475p3259475.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr faceted search performance reason
Please re-post the question here so others can see the discussion without going to another list. Best Erick On Wed, Apr 6, 2011 at 4:09 AM, Robin Palotai wrote: > Hello List, > > Please see my question at > > http://stackoverflow.com/questions/5552919/how-does-lucene-solr-achieve-high-performance-in-multi-field-faceted-search > , > I would be interested to know some details. > > Thank you, > Robin >
Re: solr faceted search performance reason
Carbon copied: *Context* This is a question mainly about Lucene (or possibly Solr) internals. The main topic is *faceted search*, in which search can happen along multiple independent dimensions (facets) of objects (for example size, speed, price of a car). When implemented with relational database, for a large number of facets multi-field indices are not useful, since facets can be searched in any order, so a specific ordered multi-index is used with low chance, and creating all possible orderings of indices is unbearable. Solr is advertised to cope well with the faceted search task, which if I think correctly has to be connected with Lucene (supposedly) performing well on multi-field queries (where fields of a document relate to facets of an object). *Question* The *inverted index* of Lucene can be stored in a relational database, and naturally taking the intersections of the matching documents can also be trivially achieved with RDBMS using single-field indices. Therefore, Lucene supposedly has some advanced technique for multi-field queries other than just taking the intersection of matching documents based on the inverted index. So the question is, what is this technique/trick? More broadly: Why can Lucene/Solr achieve better faceted search performance theoretically than RDBMS could (if so)? *Note: My first guess would be that Lucene would use some space partitioning method for partitioning a vector space built from the document fields as dimensions, but as I understand Lucene is not purely vector space based.* Thanks, Robin On Wed, Apr 6, 2011 at 3:15 PM, Erick Erickson wrote: > Please re-post the question here so others can see > the discussion without going to another list. > > Best > Erick > > On Wed, Apr 6, 2011 at 4:09 AM, Robin Palotai >wrote: > > > Hello List, > > > > Please see my question at > > > > > http://stackoverflow.com/questions/5552919/how-does-lucene-solr-achieve-high-performance-in-multi-field-faceted-search > > , > > I would be interested to know some details. > > > > Thank you, > > Robin > > >
Re: solr faceted search performance reason
On 4/6/2011 10:55 AM, Robin Palotai wrote: Therefore, Lucene supposedly has some advanced technique for multi-field queries other than just taking the intersection of matching documents based on the inverted index. I don't think so, neccesarily. It's just that Lucene's algorithms to doing this is very fast, with some additional optimizations to make it even faster. There may be some edge cases where the optimizations take some shortcuts on top of this -- ie, if you ask for only the first ten facet values ordered by number of hits, in some cases solr/lucene won't even calculate the hit counts for facet values it already knows aren't going to be in the top 10. The facetting code in 1.4+ is actually kind of tangled, in that several different calculation approaches can be taken depending on the nature of the result set and schema. But anyway, I think you're right that you could set up an rdbms schema to _conceptually_ allow very similar operations to a lucene index. It would be unlikely to perform as well, because the devil is in the details of the storage formats and algorithms, and lucene has been optimized for these particular cases (at the expense of not covering a great many cases that an rdbms can cover). In fact, while I can't find it now on Google, I think someone HAS in the past written an extension to lucene to have it store it's indexes in an rdbms using a schema much like you describe, instead of in the file system. I'm not sure why they would want to do this instead of just using the rdbms -- either lucene's access algorithms still provide a performance benefit even when using an rdbms as the underlying 'file system', or lucene provides convenient functions that you wouldn't want to have to re-implement yourself solely in terms of an rdbms, or both. Ah, here's a brief reference to that approach in the lucene FAQ: http://wiki.apache.org/lucene-java/LuceneFAQ#Can_I_store_the_Lucene_index_in_a_relational_database.3F Jonathan So the question is, what is this technique/trick? More broadly: Why can Lucene/Solr achieve better faceted search performance theoretically than RDBMS could (if so)? *Note: My first guess would be that Lucene would use some space partitioning method for partitioning a vector space built from the document fields as dimensions, but as I understand Lucene is not purely vector space based.* Thanks, Robin On Wed, Apr 6, 2011 at 3:15 PM, Erick Ericksonwrote: Please re-post the question here so others can see the discussion without going to another list. Best Erick On Wed, Apr 6, 2011 at 4:09 AM, Robin Palotai wrote: Hello List, Please see my question at http://stackoverflow.com/questions/5552919/how-does-lucene-solr-achieve-high-performance-in-multi-field-faceted-search , I would be interested to know some details. Thank you, Robin
Re: solr faceted search performance reason
PS: If you want to see how Solr actually computes facetting (the facetting code lives in the 'Solr' codebase, not in the lower level lucene codebase), here's the file to look at, this web snapshot is from 1.4.1 dont' know if it's been changed more recently, but I don't think majorly: http://www.jarvana.com/jarvana/view/org/apache/solr/solr-core/1.4.1/solr-core-1.4.1-sources.jar!/org/apache/solr/request/SimpleFacets.java?format=ok It's kind of confusing, precisely because it takes several different approaches depending on the nature of the result set and schema, trying to pick the most performant approach for the context. I still haven't wrapped my head around it entirely (I am not a Solr/lucene developer, just a user). On 4/6/2011 2:06 PM, Jonathan Rochkind wrote: On 4/6/2011 10:55 AM, Robin Palotai wrote: Therefore, Lucene supposedly has some advanced technique for multi-field queries other than just taking the intersection of matching documents based on the inverted index. I don't think so, neccesarily. It's just that Lucene's algorithms to doing this is very fast, with some additional optimizations to make it even faster. There may be some edge cases where the optimizations take some shortcuts on top of this -- ie, if you ask for only the first ten facet values ordered by number of hits, in some cases solr/lucene won't even calculate the hit counts for facet values it already knows aren't going to be in the top 10. The facetting code in 1.4+ is actually kind of tangled, in that several different calculation approaches can be taken depending on the nature of the result set and schema. But anyway, I think you're right that you could set up an rdbms schema to _conceptually_ allow very similar operations to a lucene index. It would be unlikely to perform as well, because the devil is in the details of the storage formats and algorithms, and lucene has been optimized for these particular cases (at the expense of not covering a great many cases that an rdbms can cover). In fact, while I can't find it now on Google, I think someone HAS in the past written an extension to lucene to have it store it's indexes in an rdbms using a schema much like you describe, instead of in the file system. I'm not sure why they would want to do this instead of just using the rdbms -- either lucene's access algorithms still provide a performance benefit even when using an rdbms as the underlying 'file system', or lucene provides convenient functions that you wouldn't want to have to re-implement yourself solely in terms of an rdbms, or both. Ah, here's a brief reference to that approach in the lucene FAQ: http://wiki.apache.org/lucene-java/LuceneFAQ#Can_I_store_the_Lucene_index_in_a_relational_database.3F Jonathan So the question is, what is this technique/trick? More broadly: Why can Lucene/Solr achieve better faceted search performance theoretically than RDBMS could (if so)? *Note: My first guess would be that Lucene would use some space partitioning method for partitioning a vector space built from the document fields as dimensions, but as I understand Lucene is not purely vector space based.* Thanks, Robin On Wed, Apr 6, 2011 at 3:15 PM, Erick Ericksonwrote: Please re-post the question here so others can see the discussion without going to another list. Best Erick On Wed, Apr 6, 2011 at 4:09 AM, Robin Palotai wrote: Hello List, Please see my question at http://stackoverflow.com/questions/5552919/how-does-lucene-solr-achieve-high-performance-in-multi-field-faceted-search , I would be interested to know some details. Thank you, Robin
Memory Using In Faceted Search (UnInvertedField's)
Hello, I'm using Solr 4.3.1 for faceted search and have 4 fields used for faceting. My question is about memory consumtion. I've set up heap size to use 6Gb of RAM, but I see in resource monitor it uses much more than that - up to 10Gb where 4 Gb is reported as shareable memory. I've calculated the size of cached set of UnInverted fields and it's 2Gb - I'm fine with that, both GC monitor and 'fieldValueCache' stats in 'Plugins/Stats' UI for Solr report that. But I can't understand what's that memory that's being reserved after filling in fieldValueCache with uninverted fields (right in UnInvertedField.uninvert method) and not used (or not released). Is that some memory leak? Or is that something I should tune with garbage collector by making it more aggressive (GC only shows me 2.x Gb in Old Space and I see those UnInvertedField's there in heap dump)? Some info: Index size is 76 Gb. I have 6 shards there. Windows OS. Java 6.0.24. Best regards, Anton.
Re: Faceted Search on Dynamic Fields?
Also, here is the field definition in the schema -- View this message in context: http://www.nabble.com/Faceted-Search-on-Dynamic-Fields--tp25612887p25612936.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceted Search on Dynamic Fields?
Faceting, as of now, can only be done of definitive field names. Faceting on field names matching wildcards (dynamic field being one such scenario) is yet to be supported. There are lot of open issues, aiming to achieve this. Find a similar discussion here - http://www.lucidimagination.com/search/document/787cc8cc9ea095e6/item_facet Cheers Avlesh On Fri, Sep 25, 2009 at 7:47 PM, danben wrote: > > Also, here is the field definition in the schema > > indexed="true" stored="true" multiValued="true"/> > > > -- > View this message in context: > http://www.nabble.com/Faceted-Search-on-Dynamic-Fields--tp25612887p25612936.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Re: Faceted Search on Dynamic Fields?
On Fri, Sep 25, 2009 at 12:19 PM, Avlesh Singh wrote: > Faceting, as of now, can only be done of definitive field names. To further clarify, the fields you can facet on can include those defined by dynamic fields. You just must specify the exact field name when you facet. Did you really mean for the ampersand to be in the dynamic field name? I'd advise against this, and it could be the source of your problems (escaping the ampersand in your request, etc). What is the exact facet request you are sending? -Yonik http://www.lucidimagination.com
Sol Lederman's Review of "Faceted Search"
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, Sol Lederman has written a Review <http://federatedsearchblog.com/2009/05/14/review-faceted-search/> of the almost finished manuscript of Daniel Tunkelang's "Faceted Search" which is set to be published in June. As the text also mentions Solr it might make for an interesting read... Andre -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoNGm8ACgkQ3wuzs9k1icUkfwCbBe4oxzaVmxeQAKTqNH3jWb75 ai8AoI93x9I5EOA/3ZK4uxNCh7KLVJey =2tXU -END PGP SIGNATURE-
Re: Metada document for faceted search
: : I'm trying to implement custom faceted search like CNET's : approach.<http://www.mail-archive.com/java-u...@lucene.apache.org/msg02646.html> : . : But i couldn't figure out how to structure and index category metadata : document. The short answer: "however you want" For CNET, we already had a pretty robust datamodel for expressing category based attributes and how to filter products in those categories by sets or ranges of values and what labels to put on those filters, and what order to display those filters as constraints in the UI ... all i did was come up with a syntax for the metadatadocs that let me express that existing data, and then write a request handler that knew how to parse those metadata docs and interpret the existing filter syntax and convert it into a Lucene Query. AS i've mentioned before: theres nothing special about putting this metadata into a doc in your solr index -- i just did it that way so that the solr index would be entirely self contained (for replication and backup) and refreshing my in memory models would be easy using the cache warming hooks of Solr Caches. -Hoss
Re: Metada document for faceted search
Thank you Chris. I've find out how to implement my faceted search. I don't index any metadata document but i create my in-memory faceting data structure from database at my request handlers init method. Compute facet count on any request and wrrite to response as NamedList of NemedLists. Regards. On Fri, Jul 10, 2009 at 12:01 AM, Chris Hostetter wrote: > : > : I'm trying to implement custom faceted search like CNET's > : approach.< > http://www.mail-archive.com/java-u...@lucene.apache.org/msg02646.html> > : . > : But i couldn't figure out how to structure and index category metadata > : document. > > The short answer: "however you want" > > For CNET, we already had a pretty robust datamodel for expressing category > based attributes and how to filter products in those categories by sets or > ranges of values and what labels to put on those filters, and what order > to display those filters as constraints in the UI ... all i did was come > up with a syntax for the metadatadocs that let me express that existing > data, and then write a request handler that knew how to parse those > metadata docs and interpret the existing filter syntax and convert it into > a Lucene Query. > > > AS i've mentioned before: theres nothing special about putting this > metadata into a doc in your solr index -- i just did it that way so that > the solr index would be entirely self contained (for replication and > backup) and refreshing my in memory models would be easy using the cache > warming hooks of Solr Caches. > > > -Hoss > > -- Osman İZBAT
Re: faceted search cache and optimisations
On Mon, Aug 3, 2009 at 4:45 AM, Nicolae Mihalache wrote: > Hello, > > I'm using faceted search (perhaps in a dumb way) to collect some statistics > for my index. I have documents in various languages, one of the field is > "language" and I simply want to see how many documents I have for each > language. I have noticed that the search builds a int[maxDoc] array and then > traverses the array to count. If facet.method=enum (discovered later) is > used, the things are still counted in a different way. But for this case > where all the documents are retrieved, the information is already available > in the lucene index. > So, I think it would be a good optimization to detect these cases (i.e. no > filtering) and just return the number from the index instead of counting the > docs again. That would require - a base query that matched the entire index - no filters - no deletions in the index If you want those numbers, see the terms component. > Another issue: there is no way currently to disable the caching of the > int[maxDoc], is there? use facet.method=enum... the number of filters cached can be controlled by the filterCache. You can also prevent the filterCache from being used via the facet.enum.cache.minDf param. -Yonik
Re: Faceted search in 2 indexes
: I have 2 indexes with some similar fields and some distinct fields. I need : to make a faceted search that returns the union of the same search in these : 2 indexes. : : How can I make it? Assuming you don't need facets or sorting on the fields that are distinct between the two indexes (which wouldn't make any sense), and assuming the "similar" fields include the uniqueKeyField, you can probably solve this using a distributed search query, where your "coordinator" usees schema.xml that is the union of the other schema.xml files (so that it knows how to format the fields that are distinct to each of the two indexes) -Hoss
Re: Faceted search in 2 indexes
Hi, Even I am looking for a solution for this. case: Index1: has the meta data and the contents of the files (basically read only for the end users) Index2: will have the tags attached to the search results that user may get out of index1 (so read/write). so next time when user searches it again, we have to display the results along with the tags attached by that user (previously). and also display some facets for the tags. please give some ideas/suggestions. Best Regards, Kranti K K Parisa 2010/2/23 André Maldonado > Hi all. > > I have 2 indexes with some similar fields and some distinct fields. I need > to make a faceted search that returns the union of the same search in these > 2 indexes. > > How can I make it? > > Thank's > > "Então aproximaram-se os que estavam no barco, e adoraram-no, dizendo: És > verdadeiramente o Filho de Deus." (Mateus 14:33) >
Re: 2 indexes - 1 Faceted search
Not really SOLR doesn't have a way to implement this kind of thing. You'd have to do some sort of post-processing of the results I think. HTH Erick 2010/3/10 André Maldonado > Hi all. > > I have 2 indexes with different structures, but a few equal fields. There > is some way to do a faceted search (faceting the equal fields) in the 2 > indexes at same time, with only one response (like a UNION in sql server)? > > Thank's > > "Então aproximaram-se os que estavam no barco, e adoraram-no, dizendo: És > verdadeiramente o Filho de Deus." (Mateus 14:33) >
Re: Faceted search not working? (RESOLVED)
Ah, the issue was explicitly specifying components... query I don't remember changing this during default install, commenting this out enabled faceted search component. Thanks all for the help. Ilya On Tue, May 25, 2010 at 10:38 AM, Sascha Szott wrote: > Hi, > > please note, that the FacetComponent is one of the six search components > that are automatically associated with solr.SearchHandler (this holds also > for the QueryComponent). > > Another note: By using name="components" all default components will be > replaced by the components you explicitly mentioned (i.e., QueryComponent > and FacetComponent in your example). To avoid this, use > name="last-components" instead. > > -Sascha > > Jean-Sebastien Vachon wrote: >> >> Is the FacetComponent loaded at all? >> >> >> >> query >> facet >> >> >> >> >> On 2010-05-25, at 3:32 AM, Sascha Szott wrote: >> >>> Hi Birger, >>> >>> Birger Lie wrote: >>>> >>>> I don't think the bolean fields is mapped to "on" and "off" :) >>> >>> You can use true and on interchangeably. >>> >>> -Sascha >>> >>>> >>>> >>>> -birger >>>> >>>> -Original Message- >>>> From: Ilya Sterin [mailto:ster...@gmail.com] >>>> Sent: 24. mai 2010 23:11 >>>> To: solr-user@lucene.apache.org >>>> Subject: Faceted search not working? >>>> >>>> I'm trying to perform a faceted search without any luck. Result set >>>> doesn't return any facet information... >>>> >>>> http://localhost:8080/solr/select/?q=title:*&facet=on&facet.field=title >>>> >>>> I'm getting the result set, but no face information present? Is there >>>> something else that needs to happen to turn faceting on? >>>> >>>> I'm using latest Solr 1.4 release. Data is indexed from the database >>>> using dataimporter. >>>> >>>> Thanks. >>>> >>>> Ilya Sterin >>> >> >> > >
RE: faceted search with job title
You'd probably need to do some post processing on the pages and set up rules for each website to grab that specific bit of data. You could load the html into an xml parser, then use xpath to grab content from a particular tag with a class or id, based on the particular website -Original Message- From: Savannah Beckett [mailto:savannah_becket...@yahoo.com] Sent: 21 July 2010 16:38 To: solr-user@lucene.apache.org Subject: faceted search with job title Hi, I am currently using nutch to crawl some job pages from job boards. They are in my solr index now. I want to do faceted search with the job titles. How? The job titles can be in any locations of the page, e.g. title, header, content... If I use indexfilter in Nutch to search the content for job title, there are hundred of thousands of job titles, I can't hard code them all. Do you have a better idea? I think I need the job title in a separate field in the index to make it work with solr faceted search, am I right? Thanks.
Re: faceted search with job title
mmm...there must be better way...each job board has different format. If there are constantly new job boards being crawled, I don't think I can manually look for specific sequence of tags that leads to job title. Most of them don't even have class or id. There is no guarantee that the job title will be in the title tag, or header tag. Something else can be in the title. Should I do this in a class that extends IndexFilter in Nutch? Thanks. From: Dave Searle To: "solr-user@lucene.apache.org" Sent: Wed, July 21, 2010 8:42:55 AM Subject: RE: faceted search with job title You'd probably need to do some post processing on the pages and set up rules for each website to grab that specific bit of data. You could load the html into an xml parser, then use xpath to grab content from a particular tag with a class or id, based on the particular website -Original Message- From: Savannah Beckett [mailto:savannah_becket...@yahoo.com] Sent: 21 July 2010 16:38 To: solr-user@lucene.apache.org Subject: faceted search with job title Hi, I am currently using nutch to crawl some job pages from job boards. They are in my solr index now. I want to do faceted search with the job titles. How? The job titles can be in any locations of the page, e.g. title, header, content... If I use indexfilter in Nutch to search the content for job title, there are hundred of thousands of job titles, I can't hard code them all. Do you have a better idea? I think I need the job title in a separate field in the index to make it work with solr faceted search, am I right? Thanks.
RE: faceted search with job title
Yeah you should definitely just setup a custom parser for each site.. should be easy to extract title using groovy's xml parsing along with tagsoup for sloppy html. If you can't find the pattern for each site leading to the job title how can you expect solr to? Humans have the advantage here :P -Kallin Nagelberg -Original Message- From: Savannah Beckett [mailto:savannah_becket...@yahoo.com] Sent: Wednesday, July 21, 2010 12:20 PM To: solr-user@lucene.apache.org Cc: dave.sea...@magicalia.com Subject: Re: faceted search with job title mmm...there must be better way...each job board has different format. If there are constantly new job boards being crawled, I don't think I can manually look for specific sequence of tags that leads to job title. Most of them don't even have class or id. There is no guarantee that the job title will be in the title tag, or header tag. Something else can be in the title. Should I do this in a class that extends IndexFilter in Nutch? Thanks. From: Dave Searle To: "solr-user@lucene.apache.org" Sent: Wed, July 21, 2010 8:42:55 AM Subject: RE: faceted search with job title You'd probably need to do some post processing on the pages and set up rules for each website to grab that specific bit of data. You could load the html into an xml parser, then use xpath to grab content from a particular tag with a class or id, based on the particular website -Original Message- From: Savannah Beckett [mailto:savannah_becket...@yahoo.com] Sent: 21 July 2010 16:38 To: solr-user@lucene.apache.org Subject: faceted search with job title Hi, I am currently using nutch to crawl some job pages from job boards. They are in my solr index now. I want to do faceted search with the job titles. How? The job titles can be in any locations of the page, e.g. title, header, content... If I use indexfilter in Nutch to search the content for job title, there are hundred of thousands of job titles, I can't hard code them all. Do you have a better idea? I think I need the job title in a separate field in the index to make it work with solr faceted search, am I right? Thanks.
Re: faceted search with job title
I don't see how it can be done without writing sax or dom code for each job board, it is non-maintainable if there are a lot of new job boards being crawled. Maybe I should use regex match? Then I just need to substitute the regex pattern for each job board without writing any new sax or dom code. But is regex pattern flexible enough for all job boards? Thanks. From: "Nagelberg, Kallin" To: "solr-user@lucene.apache.org" Sent: Wed, July 21, 2010 10:39:32 AM Subject: RE: faceted search with job title Yeah you should definitely just setup a custom parser for each site.. should be easy to extract title using groovy's xml parsing along with tagsoup for sloppy html. If you can't find the pattern for each site leading to the job title how can you expect solr to? Humans have the advantage here :P -Kallin Nagelberg -Original Message- From: Savannah Beckett [mailto:savannah_becket...@yahoo.com] Sent: Wednesday, July 21, 2010 12:20 PM To: solr-user@lucene.apache.org Cc: dave.sea...@magicalia.com Subject: Re: faceted search with job title mmm...there must be better way...each job board has different format. If there are constantly new job boards being crawled, I don't think I can manually look for specific sequence of tags that leads to job title. Most of them don't even have class or id. There is no guarantee that the job title will be in the title tag, or header tag. Something else can be in the title. Should I do this in a class that extends IndexFilter in Nutch? Thanks. From: Dave Searle To: "solr-user@lucene.apache.org" Sent: Wed, July 21, 2010 8:42:55 AM Subject: RE: faceted search with job title You'd probably need to do some post processing on the pages and set up rules for each website to grab that specific bit of data. You could load the html into an xml parser, then use xpath to grab content from a particular tag with a class or id, based on the particular website -Original Message- From: Savannah Beckett [mailto:savannah_becket...@yahoo.com] Sent: 21 July 2010 16:38 To: solr-user@lucene.apache.org Subject: faceted search with job title Hi, I am currently using nutch to crawl some job pages from job boards. They are in my solr index now. I want to do faceted search with the job titles. How? The job titles can be in any locations of the page, e.g. title, header, content... If I use indexfilter in Nutch to search the content for job title, there are hundred of thousands of job titles, I can't hard code them all. Do you have a better idea? I think I need the job title in a separate field in the index to make it work with solr faceted search, am I right? Thanks.
