Re: Implementing Autocomplete/Query Suggest using Solr

2009-12-22 Thread Shalin Shekhar Mangar
On Wed, Dec 23, 2009 at 6:14 AM, Prasanna R wrote: > > I am curious how an approach that simply uses the wildcard query > functionality on an indexed field would work. It works fine as long as the terms are not repeated across documents. > While Solr does not support > wildcard queries out o

Re: Adaptive search?

2009-12-22 Thread Shalin Shekhar Mangar
On Wed, Dec 23, 2009 at 4:09 AM, Lance Norskog wrote: > Nice! > > Siddhant: Another problem to watch out for is the feedback problem: > someone clicks on a link and it automatically becomes more > interesting, so someone else clicks, and it gets even more > interesting... So you need some kind of

Search both diacritics and non-diacritics

2009-12-22 Thread Olala
Hi all! I am developing a seach engine with Solr, and now I want to search both with and without diacritics, for example: if I query kho, it will response kho, khó, khò,... But if I query khó, it will response only khó. Who anyone have solution? I have used but it is not correct :( -- View thi

Re: suggestions for DIH batchSize

2009-12-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
A bigger batchSize results in increased memory usage. I guess performance should be slightly better with bigger values but not verified. On Wed, Dec 23, 2009 at 2:51 AM, Joel Nylund wrote: > Hi, > > it looks like from looking at the code the default is 500, is the > recommended setting for this?

SCHEMA-INDEX-MISMATCH

2009-12-22 Thread johnson hong
Hi,all. I use Lucene's NumericField to index "price" field,And query with solr.TrieDoubleField. When i use "price:[1 TO 5000]" to search,it can return all results that price is between 1 and 5000. but the price value return is :ERROR:SCHEMA-INDEX-MISMATCH,stringValue=2000.0 anybogy know why? --

Profiling Solr

2009-12-22 Thread Maduranga Kannangara
Hi All, Recently we noticed that some of our heavy load Solr instances are facing memory leak kind situations. It goes onto Full GC and as it was unable to release any memory, the broken pripe and socket errors happen. (This happens both in Solr 1.3 and 1.4 for us.) Is there a good tool (prefe

RE: Solr replication 1.3 issue

2009-12-22 Thread Maduranga Kannangara
Yes, that makes sense Lance. Thanks. For the moment we broke our script to do the master's part (starting rsyncd) on the master server itself. Our problem is that we have so many instances running in different environment and we'd really love to minimize the number of them. :-) Thanks again. M

Implementing Autocomplete/Query Suggest using Solr

2009-12-22 Thread Prasanna R
There seem to be a couple of approaches that people have adopted in implementing a query suggestion / auto completion feature using Solr. Depending on the situation, one might use the terms component or go the way of using EdgeNGramFilters and then creating querying the index on the ngrammed field

Re: solr.RemoveDuplicatesTokenFilterFactory

2009-12-22 Thread Lance Norskog
Looking at the code, it does appear to be what you want. In the analysis.jsp page, you can see exactly how your text is processed. http://localhost;8983/solr/admin/analysis.jsp On Tue, Dec 22, 2009 at 2:33 PM, Jae Joo wrote: > Hi, > > Here is the string to be indexed without duplication. > > Ki

Re: Synonym questions

2009-12-22 Thread darniz
i have a similar use case where i want to preserve original words forexamplei have one way mapping at index time monsoon,sony,levinson => audio system when i index document for example with word sony i want ot index audio system as well as the brand name ie sony. so if some one types category:sony

Re: Field Collapsing - disable cache

2009-12-22 Thread Lance Norskog
To avoid this possible bug, you could change the cache to only have a few entries. On Tue, Dec 22, 2009 at 6:34 AM, Martijn v Groningen wrote: > In the latest patch some changes where made on the configuration side, > but if you add the CollapseComponent to the conf no field collapse > cache shou

Re: Solr - Group search result

2009-12-22 Thread Lance Norskog
Rob: What would you like to know? On Tue, Dec 22, 2009 at 3:10 AM, wrote: > see > http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/ > [1] for a good little howto. >        and also http://wiki.apache.org/solr/FieldCollapsing [2] > On Tue 22/12/09 11:03 , JKnight JKnight

Re: Adaptive search?

2009-12-22 Thread Lance Norskog
Nice! Siddhant: Another problem to watch out for is the feedback problem: someone clicks on a link and it automatically becomes more interesting, so someone else clicks, and it gets even more interesting... So you need some kind of suppression. For example, as individual clicks get older, you can

solr.RemoveDuplicatesTokenFilterFactory

2009-12-22 Thread Jae Joo
Hi, Here is the string to be indexed without duplication. Kitchen Cabinet Utah Kitchen Remodeling Utah Is RemoveDuplicatesTokenFilterFactory for this solution? or for something else? Jae

Re: Solr replication 1.3 issue

2009-12-22 Thread Lance Norskog
This is a Unix security question. "rsyncd' is a system daemon and should be managed in the OS scripts. The rsyncd-* scripts include a security setting for the 'solr' account that limits the account to the solr data directory (and this code does not support multicore). For security reasons, I perso

Re: query ( dynamicField ) in solr

2009-12-22 Thread Rob Casson
sounds like a job for copyField: http://wiki.apache.org/solr/FAQ#How_do_I_use_copyField_with_wildcards.3F add sku_defaultPriceAll to your and then: ...query on sku_defaultPriceAll. hth, rob On Tue, Dec 22, 2009 at 3:36 PM, Gustavo wrote: > Hello All, > >  I have been trying to mak

suggestions for DIH batchSize

2009-12-22 Thread Joel Nylund
Hi, it looks like from looking at the code the default is 500, is the recommended setting for this? Has anyone notice any significant performance/memory tradeoffs by making this much bigger? thanks Joel

Re: query ( dynamicField ) in solr

2009-12-22 Thread AHMET ARSLAN
> I have been trying to make a query at a dynamicField. > for example: > i have a product that can have many sku's. So at my xsd i > have > indexed="true" > stored="true" /> > indexed="true" > stored="true" /> >                 > when a make a query: > http://localhost:8983/solr/productIdx/selec

query ( dynamicField ) in solr

2009-12-22 Thread Gustavo
Hello All, I have been trying to make a query at a dynamicField. for example: i have a product that can have many sku's. So at my xsd i have . .

Re: adding a new slave doesn't trigger replication until the master index rebuilds

2009-12-22 Thread Ryan Kennedy
On Tue, Dec 22, 2009 at 9:30 AM, Ryan Kennedy wrote: > I'm trying to automate setting up new slave instances. I've noticed > when I use the core admin to bring a new slave core online that the > slave won't replicate the master index. The issue appears to be that > when the slave core is created,

Re: query parsing ( expansion ) in solr

2009-12-22 Thread AHMET ARSLAN
> Hello All, >             I have been > trying to find out the right place to parse > the query submitted. To be brief, I need to expand the > query. For > example.. let the query be >        city:paris > then I would like to expand the query as .. follows >     city:paris OR place:paris OR town:

RE: java.lang.OutOfMemoryError while autowarming

2009-12-22 Thread Sudarsan, Sithu D.
I'll recommend setting -Xms and -Xmx to the same value. Sincerely, Sithu D Sudarsan -Original Message- From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: Tuesday, December 22, 2009 2:31 PM To: solr-user@lucene.apache.org Subject: Re: java.lang.OutOfMemoryError while auto

Re: java.lang.OutOfMemoryError while autowarming

2009-12-22 Thread Jason Rutherglen
> -Xmx64m JVM parameter. Is going to be far too low. For example I always start at 1 GB and move up from there. On Tue, Dec 22, 2009 at 4:35 AM, c82famo wrote: > > Hi, > > I'm facing some OutOfMemory issues with SOLR. > Tomcat is started with a -Xmx64m JVM parameter. > Tomcat manages 14 SOLR ap

query parsing ( expansion ) in solr

2009-12-22 Thread gudumba l
Hello All, I have been trying to find out the right place to parse the query submitted. To be brief, I need to expand the query. For example.. let the query be city:paris then I would like to expand the query as .. follows city:paris OR place:paris OR town:paris . I gue

adding a new slave doesn't trigger replication until the master index rebuilds

2009-12-22 Thread Ryan Kennedy
I'm trying to automate setting up new slave instances. I've noticed when I use the core admin to bring a new slave core online that the slave won't replicate the master index. The issue appears to be that when the slave core is created, it's given a new (empty) index with a newer timestamp than the

highlighting and external storage

2009-12-22 Thread Thomas Koch
Hi, I'm working on a news crawler with continuous indexing. Thus indexes are merged frequently and older documents aren't as important as recent ones. Therefor I'd like to store the fulltext of documents in an external storage (HBase?) so that merging of indexes isn't as IO intensive. This woul

Re: Field Collapsing - disable cache

2009-12-22 Thread Martijn v Groningen
In the latest patch some changes where made on the configuration side, but if you add the CollapseComponent to the conf no field collapse cache should be enabled. If not let me know. Martijn 2009/12/22 : > > > > > > On Tue 22/12/09 12:28 , Martijn v Groningen wrote: > >> Hi Rob, >> What patch a

Re: Field Collapsing - disable cache

2009-12-22 Thread rob
On Tue 22/12/09 12:28 , Martijn v Groningen wrote: > Hi Rob, > What patch are you actually using from SOLR-236? > Martijn > 2009/12/22 : > > I've tried both, the whole fieldCollapsing tag, and just the > > fieldCollapseCache tag inside it. > >both cause error. > >I guess I c

java.lang.OutOfMemoryError while autowarming

2009-12-22 Thread c82famo
Hi, I'm facing some OutOfMemory issues with SOLR. Tomcat is started with a -Xmx64m JVM parameter. Tomcat manages 14 SOLR applications (not using multi-core : we're using SOLR 1.2). The SOLR application impacted manages an 500Mo index (ie 250 lucene documents). Here is catalina.out log : INF

Re: Field Collapsing - disable cache

2009-12-22 Thread rob
actually i got field-collapse-5.patch, what's the diff? On Tue 22/12/09 12:28 , Martijn v Groningen wrote: > Hi Rob, > What patch are you actually using from SOLR-236? > Martijn > 2009/12/22 : > > I've tried both, the whole fieldCollapsing tag, and just the > > fieldCollapseCache tag inside

Re: Field Collapsing - disable cache

2009-12-22 Thread rob
Yup, that's the one, with a copy of trunk from last week. On Tue 22/12/09 12:28 , Martijn v Groningen wrote: > Hi Rob, > What patch are you actually using from SOLR-236? > Martijn > 2009/12/22 : > > I've tried both, the whole fieldCollapsing tag, and just the > > fieldCollapseCache tag insid

Re: Field Collapsing - disable cache

2009-12-22 Thread Martijn v Groningen
Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 : > I've tried both, the whole fieldCollapsing tag, and just the > fieldCollapseCache tag inside it. >        both cause error. >        I guess I can just set size, initialSize, and autowarmCount to 0 ?? > On Tue 22/12

Re: Field Collapsing - disable cache

2009-12-22 Thread rob
I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that

Re: Field Collapsing - disable cache

2009-12-22 Thread Toby Cole
Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, r...@intelcompute.com wrote: Tha

Re: Solr - Group search result

2009-12-22 Thread rob
see http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/ [1] for a good little howto. and also http://wiki.apache.org/solr/FieldCollapsing [2] On Tue 22/12/09 11:03 , JKnight JKnight wrote:Dear all, I have document doc.xml 3 Love you love me 2 Love story 3

Re: Field Collapsing - disable cache

2009-12-22 Thread rob
That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org.apache.solr.search.fiel

Solr - Group search result

2009-12-22 Thread JKnight JKnight
Dear all, I have document doc.xml 3 Love you love me 2 Love story 3 How deep is your love? 4 I love you 5 How deep is your love? 6 Love story and in schema.xml, I add two line After post doc.xml to solr server and run query http://localhost:8983/solr/selec

Re: Field Collapsing - disable cache

2009-12-22 Thread Toby Cole
If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing#line-63 "If the field collapse cache is not configured then the field collapse logic will not be cached." Regard

Field Collapsing - disable cache

2009-12-22 Thread rob
Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ...

Re: Field Collapsing - disable cache

2009-12-22 Thread rob
my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , r...@intelcompute.com wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches

Re: Adaptive search?

2009-12-22 Thread Siddhant Goel
On Tue, Dec 22, 2009 at 12:01 PM, Ryan Kennedy wrote: > This approach will be limited to applying a "global" rank to all the > documents, which may have some unintended consequences. The most > popular document in your index will be the most popular, even for > queries for which it was never clic

Re: Multi Solr

2009-12-22 Thread Olala
Thanks so much!Ihave done :) Raghuveer Kancherla wrote: > > Based on your need you can choose one of the options listed at > > http://wiki.apache.org/solr/MultipleIndexes > > > - Raghu > > > On Tue, Dec 22, 2009 at 10:46 AM, Olala wrote: > >> >> Hi all! >> >> I have developed Solr on Tom