Re: Slow Phrase Queries
that's what my use case has shown, but i havent done enough experimenting to know for sure. the reason the field is untokenized is because i need the full value of an authors name, example: smith, jones, if it was tokenized and faceted it would be jones and another entry for smith. i am running a lot of facets, 19 of them, some are facet queries and some are fields, not including the author faceting. the performance is great with all of them running. the problem is of those other facets there are only a limited number of possibilities. But with authors, there are over 800,000 separate authors in my data. the next route is to modify the index to index the authors as an integer taken from a database table of all authors, and re-attempt the author facets again on a tokenized field of integers. unfortunately it takes 4-9 days for the index to be built as it is a little over 22 GB's Lance Norskog-2 wrote: Are you saying that faceting is faster on a tokenized field? Is this true? On Tue, Oct 20, 2009 at 2:02 PM, DHast hastings.recurs...@gmail.com wrote: ... , removing that facet worked since the field was untokenizd and slow considering how many values tehre were. ... View this message in context: http://www.nabble.com/Slow-Phrase-Queries-tp2597p25982493.html Sent from the Solr - User mailing list archive at Nabble.com. -- Lance Norskog goks...@gmail.com -- View this message in context: http://www.nabble.com/Slow-Phrase-Queries-tp2597p25999252.html Sent from the Solr - User mailing list archive at Nabble.com.
Slow Phrase Queries
Hello, I have recently installed Solr as an alternative to our home made lucene search servers, and while in most respects the performance is better, i notice that phrase searches are incredibly slow compared to normal lucene, primarily when using facets example: City of New York, Matter of takes 11 seconds City of New York, Matter of takes 1 second the same searches using raw lucene take 5 seconds and 3 seconds respectively. i tried cutting out as much as i could from solrconfig without breaking it, is there anything else i could try doing to make solr perform similarly to raw lucene as far as phrase queries are concerned? thanks -- View this message in context: http://www.nabble.com/Slow-Phrase-Queries-tp2597p2597.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow Phrase Queries
ah, it turns out it was one of my 6 facets, the author. in the data pool tehre are over 1.9 million documents, and about 800,000 authors, removing that facet worked since the field was untokenizd and slow considering how many values tehre were. Solr is definitely faster, and as fast and or faster with facets -- View this message in context: http://www.nabble.com/Slow-Phrase-Queries-tp2597p25982493.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: define index at search time
No, I am talking about having multiple indexes, i want to send the index name to the searcher so it will search that index, rather than use the one defined in the schema/solrconfig. nothing t do with multiple cores, i mean different indexes entirely with completely different content. Avlesh Singh wrote: Are you talking about multiple cores? Cheers Avlesh On Mon, Sep 21, 2009 at 9:15 PM, DHast hastings.recurs...@gmail.com wrote: is there a way i can actually tell solr which index i want it to search against with the query? I know it will cost a bit on performance, but it would be helpful i have many indexes and it would be nice to determine which one should be used by the user. thanks -- View this message in context: http://www.nabble.com/define-index-at-search-time-tp25530378p25530378.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/define-index-at-search-time-tp25530378p25564438.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: define index at search time
well after looking at http://wiki.apache.org/solr/CoreAdmin perhaps multiple cores is what i want, DHast wrote: No, I am talking about having multiple indexes, i want to send the index name to the searcher so it will search that index, rather than use the one defined in the schema/solrconfig. nothing t do with multiple cores, i mean different indexes entirely with completely different content. Avlesh Singh wrote: Are you talking about multiple cores? Cheers Avlesh On Mon, Sep 21, 2009 at 9:15 PM, DHast hastings.recurs...@gmail.com wrote: is there a way i can actually tell solr which index i want it to search against with the query? I know it will cost a bit on performance, but it would be helpful i have many indexes and it would be nice to determine which one should be used by the user. thanks -- View this message in context: http://www.nabble.com/define-index-at-search-time-tp25530378p25530378.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/define-index-at-search-time-tp25530378p25564937.html Sent from the Solr - User mailing list archive at Nabble.com.
define index at search time
is there a way i can actually tell solr which index i want it to search against with the query? I know it will cost a bit on performance, but it would be helpful i have many indexes and it would be nice to determine which one should be used by the user. thanks -- View this message in context: http://www.nabble.com/define-index-at-search-time-tp25530378p25530378.html Sent from the Solr - User mailing list archive at Nabble.com.
solr isnt using default field correctly
hi, if i do a search: text:law order~40 i get this: str name=rawquerystringtext:law order~40/str str name=querystringtext:law order~40/str str name=parsedqueryPhraseQuery(text:law order~40)/str str name=parsedquery_toStringtext:law order~40/str str name=QParserOldLuceneQParser/str However if i do: law order~40 i get this: str name=rawquerystringlaw order~40/str str name=querystringlaw order~40/str str name=parsedquerytext:law order/str str name=parsedquery_toStringtext:law order/str lst name=explain/ str name=QParserOldLuceneQParser/str my Schema xml: field name=text type=string indexed=true stored=false / . defaultSearchFieldtext/defaultSearchField what should i be doing differently to get the second results like the first? -- View this message in context: http://www.nabble.com/solr-isnt-using-default-field-correctly-tp25507985p25507985.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr isnt using default field correctly
well it seems what is happening is solr is not being consistent, DHast wrote: hi, if i do a search: text:law order~40 i get this: str name=rawquerystringtext:law order~40/str str name=querystringtext:law order~40/str str name=parsedqueryPhraseQuery(text:law order~40)/str str name=parsedquery_toStringtext:law order~40/str str name=QParserOldLuceneQParser/str However if i do: law order~40 i get this: str name=rawquerystringlaw order~40/str str name=querystringlaw order~40/str str name=parsedquerytext:law order/str str name=parsedquery_toStringtext:law order/str lst name=explain/ str name=QParserOldLuceneQParser/str my Schema xml: field name=text type=string indexed=true stored=false / . defaultSearchFieldtext/defaultSearchField what should i be doing differently to get the second results like the first? -- View this message in context: http://www.nabble.com/solr-isnt-using-default-field-correctly-tp25507985p25508264.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr isnt using default field correctly
yeah something is definitely strange, i think i know what it is though. im going to make a separate post for it, but it cached the results from when i had field:text as a string, Erik Hatcher-4 wrote: I just tried this on trunk and both with and without a field selector it parses to a PhraseQuery. I have trouble believing even Solr 1.3 behaved like you reported, something seems fishy. Erik On Sep 18, 2009, at 9:02 AM, DHast wrote: well it seems what is happening is solr is not being consistent, DHast wrote: hi, if i do a search: text:law order~40 i get this: str name=rawquerystringtext:law order~40/str str name=querystringtext:law order~40/str str name=parsedqueryPhraseQuery(text:law order~40)/str str name=parsedquery_toStringtext:law order~40/str str name=QParserOldLuceneQParser/str However if i do: law order~40 i get this: str name=rawquerystringlaw order~40/str str name=querystringlaw order~40/str str name=parsedquerytext:law order/str str name=parsedquery_toStringtext:law order/str lst name=explain/ str name=QParserOldLuceneQParser/str my Schema xml: field name=text type=string indexed=true stored=false / . defaultSearchFieldtext/defaultSearchField what should i be doing differently to get the second results like the first? -- View this message in context: http://www.nabble.com/solr-isnt-using-default-field-correctly-tp25507985p25508264.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://www.nabble.com/solr-isnt-using-default-field-correctly-tp25507985p25508691.html Sent from the Solr - User mailing list archive at Nabble.com.
want to features of a 'text' field with the non stemming of a 'string' field
when i have my fieldname: text set as a text field, advanced search queries work very well, but when i have it set as a string it seems to ignore them, like proximity searching and so on. example: text as string: str name=rawquerystringtext:law order~33/str str name=querystringtext:law order~33/str str name=parsedquerytext:law order/str str name=parsedquery_toStringtext:law order/str text as text: str name=rawquerystringtext:law order~32 /str str name=querystringtext:law order~32 /str str name=parsedqueryPhraseQuery(text:law order~32)/str str name=parsedquery_toStringtext:law order~32/str however when i search a single term, it stems it if its text, example: text as text: str name=rawquerystringgoats /str str name=querystringgoats /str str name=parsedquerytext:goat/str str name=parsedquery_toStringtext:goat/str text as string: str name=rawquerystringnuts /str str name=querystringnuts /str str name=parsedquerytext:nuts/str str name=parsedquery_toStringtext:nuts/str OR str name=rawquerystringtext:goats /str str name=querystringtext:goats /str str name=parsedquerytext:goats/str str name=parsedquery_toStringtext:goats/str so what i want/need, is to STOP the stemming/plural killing that is happening on the text field, ideas? also, is tehre a way to wipe the cache while testing? -- View this message in context: http://www.nabble.com/want-to-features-of-a-%27text%27-field-with-the-non-stemming-of-a-%27string%27-field-tp25508780p25508780.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: want to features of a 'text' field with the non stemming of a 'string' field
i have looked, and seem to be running into a dead end every time i try it, but again it may be because of the caching and me not realizing it was doing it till my hair was half pulled. i dont suppose youd be willing to give a hint then? Grant Ingersoll-6 wrote: On Sep 18, 2009, at 6:37 AM, DHast wrote: when i have my fieldname: text set as a text field, advanced search queries work very well, but when i have it set as a string it seems to ignore them, like proximity searching and so on. example: text as string: str name=rawquerystringtext:law order~33/str str name=querystringtext:law order~33/str str name=parsedquerytext:law order/str str name=parsedquery_toStringtext:law order/str text as text: str name=rawquerystringtext:law order~32 /str str name=querystringtext:law order~32 /str str name=parsedqueryPhraseQuery(text:law order~32)/str str name=parsedquery_toStringtext:law order~32/str however when i search a single term, it stems it if its text, example: text as text: str name=rawquerystringgoats /str str name=querystringgoats /str str name=parsedquerytext:goat/str str name=parsedquery_toStringtext:goat/str text as string: str name=rawquerystringnuts /str str name=querystringnuts /str str name=parsedquerytext:nuts/str str name=parsedquery_toStringtext:nuts/str OR str name=rawquerystringtext:goats /str str name=querystringtext:goats /str str name=parsedquerytext:goats/str str name=parsedquery_toStringtext:goats/str so what i want/need, is to STOP the stemming/plural killing that is happening on the text field, ideas? It sounds like you need to dig into your schema.xml a bit more and set your analysis better. See http://wiki.apache.org/solr/SchemaXml also, is tehre a way to wipe the cache while testing? -- View this message in context: http://www.nabble.com/want-to-features-of-a-%27text%27-field-with-the-non-stemming-of-a-%27string%27-field-tp25508780p25508780.html Sent from the Solr - User mailing list archive at Nabble.com. -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search -- View this message in context: http://www.nabble.com/want-to-features-of-a-%27text%27-field-with-the-non-stemming-of-a-%27string%27-field-tp25508780p25509828.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: want to features of a 'text' field with the non stemming of a 'string' field
ok, used the built in fieldtype text_ws that seems to go well DHast wrote: i have looked, and seem to be running into a dead end every time i try it, but again it may be because of the caching and me not realizing it was doing it till my hair was half pulled. i dont suppose youd be willing to give a hint then? Grant Ingersoll-6 wrote: On Sep 18, 2009, at 6:37 AM, DHast wrote: when i have my fieldname: text set as a text field, advanced search queries work very well, but when i have it set as a string it seems to ignore them, like proximity searching and so on. example: text as string: str name=rawquerystringtext:law order~33/str str name=querystringtext:law order~33/str str name=parsedquerytext:law order/str str name=parsedquery_toStringtext:law order/str text as text: str name=rawquerystringtext:law order~32 /str str name=querystringtext:law order~32 /str str name=parsedqueryPhraseQuery(text:law order~32)/str str name=parsedquery_toStringtext:law order~32/str however when i search a single term, it stems it if its text, example: text as text: str name=rawquerystringgoats /str str name=querystringgoats /str str name=parsedquerytext:goat/str str name=parsedquery_toStringtext:goat/str text as string: str name=rawquerystringnuts /str str name=querystringnuts /str str name=parsedquerytext:nuts/str str name=parsedquery_toStringtext:nuts/str OR str name=rawquerystringtext:goats /str str name=querystringtext:goats /str str name=parsedquerytext:goats/str str name=parsedquery_toStringtext:goats/str so what i want/need, is to STOP the stemming/plural killing that is happening on the text field, ideas? It sounds like you need to dig into your schema.xml a bit more and set your analysis better. See http://wiki.apache.org/solr/SchemaXml also, is tehre a way to wipe the cache while testing? -- View this message in context: http://www.nabble.com/want-to-features-of-a-%27text%27-field-with-the-non-stemming-of-a-%27string%27-field-tp25508780p25508780.html Sent from the Solr - User mailing list archive at Nabble.com. -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search -- View this message in context: http://www.nabble.com/want-to-features-of-a-%27text%27-field-with-the-non-stemming-of-a-%27string%27-field-tp25508780p25510178.html Sent from the Solr - User mailing list archive at Nabble.com.
Facet vs searching, field types,
hello, my documents have a field called member_of, this field holds a list, so it will look like this: member_of: fred bob mike journals wsjournals jen steve if 'member_of' is a 'string' and i search: goats AND member_of(fred bob journals) i get the right results, however, the faceting doesnt work right, it groups journals and wsjournals together this is happening because a lot of the entries are in both, but the faceting seems to align one result to one facet value, rather than multiple values, so if ten results have member_of journals wsjournals, they are assigned to wsjournals, not journals. any way to make them show as a result twice? -- View this message in context: http://www.nabble.com/Facet-vs-searching%2C-field-types%2C-tp25491963p25491963.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Facet vs searching, field types,
DHast wrote: hello, my documents have a field called member_of, this field holds a list, so it will look like this: member_of: fred bob mike journals wsjournals jen steve if 'member_of' is a 'string' and i search: goats AND member_of(fred bob journals) i get the right results, however, the faceting doesnt work right, it groups journals and wsjournals together this is happening because a lot of the entries are in both, but the faceting seems to align one result to one facet value, rather than multiple values, so if ten results have member_of journals wsjournals, they are assigned to wsjournals, not journals. any way to make them show as a result twice? Ok, i think i got this, i added multiValued='true' for that field and the counts seem much better, or at least what i was looking for -- View this message in context: http://www.nabble.com/Facet-vs-searching%2C-field-types%2C-tp25491963p25491970.html Sent from the Solr - User mailing list archive at Nabble.com.
copyfield at search time?
is it possible to do somehting like this: Now im wondering how to do something like this: field name=member_of_facet type=string indexed=true stored=true multiValued=true/ field name=member_of type=string indexed=true stored=true multiValued=false/ copyField source=member_of dest=member_of_facet / if so, i dont seem to be making progress thanks -- View this message in context: http://www.nabble.com/copyfield-at-search-time--tp25491979p25491979.html Sent from the Solr - User mailing list archive at Nabble.com.
do NOT want to stem plurals for a particular field, or words
I have a field where there are items that are plurals, and used as very specific locators, so i do a solr search type:articles, and it translates it into : type:article, then into type:articl... is tehre a way to stop it from doing this on either the field type or on a list of words articles, notes, etc i tried enering into the protwords.txt file and dont seem to get any where -- View this message in context: http://www.nabble.com/do-NOT-want-to-stem-plurals-for-a-particular-field%2C-or-words-tp25455570p25455570.html Sent from the Solr - User mailing list archive at Nabble.com.
