Re: Multi Solr
Thanks so much!Ihave done :) Raghuveer Kancherla wrote: Based on your need you can choose one of the options listed at http://wiki.apache.org/solr/MultipleIndexes - Raghu On Tue, Dec 22, 2009 at 10:46 AM, Olala hthie...@gmail.com wrote: Hi all! I have developed Solr on Tomcat, but now I want to building many Solr on only one Tomcat server.Is that can be done or not??? -- View this message in context: http://old.nabble.com/Multi-Solr-tp26884086p26884086.html Sent from the Solr - User mailing list archive at Nabble.com. -- View this message in context: http://old.nabble.com/Multi-Solr-tp26884086p26885682.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adaptive search?
On Tue, Dec 22, 2009 at 12:01 PM, Ryan Kennedy rcken...@gmail.com wrote: This approach will be limited to applying a global rank to all the documents, which may have some unintended consequences. The most popular document in your index will be the most popular, even for queries for which it was never clicked on. Right. Makes so much sense. Thanks for sharing. -- - Siddhant
Re: Field Collapsing - disable cache
my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , r...@intelcompute.com wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [2] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ Links: -- [1] http://www.intelcompute.com/solrconfig.xml [2] http://mail.intelcompute.com/parse.php?redirect=a href=
Field Collapsing - disable cache
Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/
Re: Field Collapsing - disable cache
If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, r...@intelcompute.com wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , r...@intelcompute.com wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [2] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ Links: -- [1] http://www.intelcompute.com/solrconfig.xml [2] http://mail.intelcompute.com/parse.php?redirect=a href= -- Toby Cole Senior Software Engineer, Semantico Limited Registered in England and Wales no. 03841410, VAT no. GB-744614334. Registered office Lees House, 21-23 Dyke Road, Brighton BN1 3FE, UK. Check out all our latest news and thinking on the Discovery blog http://blogs.semantico.com/discovery-blog/
Solr - Group search result
Dear all,
I have document doc.xml
adddoc
field name=id3/field
field name=nameLove you love me/field
/doc
doc
field name=id2/field
field name=nameLove story/field
/doc
doc
field name=id3/field
field name=nameHow deep is your love?/field
/doc
doc
field name=id4/field
field name=nameI love you/field
/doc
doc
field name=id5/field
field name=nameHow deep is your love?/field
/doc
doc
field name=id6/field
field name=nameLove story/field
/doc
/add
and in schema.xml, I add two line
field name=name_exact type=string indexed=true stored=false/
copyField source=name dest=name_exact/
After post doc.xml to solr server and run query
http://localhost:8983/solr/select/?wt=jsonindent=onq=love%20youfl=namefacet=truefacet.field=name_exact
I see the response:
{
responseHeader:{
status:0,
QTime:4,
params:{
facet:true,
fl:name,
indent:on,
q:love you,
facet.field:name_exact,
wt:json}},
response:{numFound:5,start:0,docs:[
{
name:I love you},
{
name:Love story},
{
name:Love story},
{
name:How deep is your love?},
{
name:How deep is your love?}]
},
facet_counts:{
facet_queries:{},
facet_fields:{
name_exact:[
How deep is your love?,2,
Love story,2,
I love you,1,
Love you love me,0,
Quyen day ne,0,
day ne,0]},
facet_dates:{}}}
And I have a question. How can I group search result? With my data, I want
to group by name field and see only 3 record
{
name:I love you},
{
name:Love story},
{
name:How deep is your love?}
Could you help me?
Thank a lot for support.
--
Best regards,
JKnight
Re: Field Collapsing - disable cache
That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.executeCollapse(AbstractDocumentCollapser.java:249) at org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java:172) at org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:173) at org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [5] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ [7] target=_blankhttp://atmail.org/ Links: -- [1] http://www.intelcompute.com/solrconfig.xml [8] target=_blankhttp://www.intelcompute.com/solrconfig.xml [2] http://mail.intelcompute.com/parse.php%3Fredirect%3D%26lt%3Ba [9] target=_blankhttp://mail.intelcompute.com/parse.php?redirect=http://blogs.semantico.com/discovery-blog/ - Message sent via Atmail Open - http://atmail.org/ Links: -- [1] http://mail.intelcompute.com/parse.php?redirect=a href= [3] http://mail.intelcompute.com/parse.php?redirect=a href= [5] http://mail.intelcompute.com/parse.php?redirect=a href= [6] http://mail.intelcompute.com/parse.php?redirect=a href= [7] http://mail.intelcompute.com/parse.php?redirect=a href= [8] http://mail.intelcompute.com/parse.php?redirect=a href= [9] http://mail.intelcompute.com/parse.php?redirect=a href= [10] http://mail.intelcompute.com/parse.php?redirect=a href=
Re: Solr - Group search result
see
http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/
[1] for a good little howto.
and also http://wiki.apache.org/solr/FieldCollapsing [2]
On Tue 22/12/09 11:03 , JKnight JKnight wrote:Dear all,
I have document doc.xml
3
Love you love me
2
Love story
3
How deep is your love?
4
I love you
5
How deep is your love?
6
Love story
and in schema.xml, I add two line
After post doc.xml to solr server and run query
http://localhost:8983/solr/select/%3Fwt%3Djson%26amp%3Bindent%3Don%26amp%3Bq%3Dlove%2520you%26amp%3Bfl%3Dname%26amp%3Bfacet%3Dtrue%26amp%3Bfacet.field%3Dname_exact
[3]
target=_blankhttp://localhost:8983/solr/select/?wt=jsonindent=onq=love%20youfl=namefacet=truefacet.field=name_exact
I see the response:
{
responseHeader:{
status:0,
QTime:4,
params:{
facet:true,
fl:name,
indent:on,
q:love you,
facet.field:name_exact,
wt:json}},
response:{numFound:5,start:0,docs:[
{
name:I love you},
{
name:Love story},
{
name:Love story},
{
name:How deep is your love?},
{
name:How deep is your love?}]
},
facet_counts:{
facet_queries:{},
facet_fields:{
name_exact:[
How deep is your love?,2,
Love story,2,
I love you,1,
Love you love me,0,
Quyen day ne,0,
day ne,0]},
facet_dates:{}}}
And I have a question. How can I group search result? With my data, I
want
to group by name field and see only 3 record
{
name:I love you},
{
name:Love story},
{
name:How deep is your love?}
Could you help me?
Thank a lot for support.
--
Best regards,
JKnight
-
Message sent via Atmail Open - http://atmail.org/
Links:
--
[1]
http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/
[2] http://wiki.apache.org/solr/FieldCollapsing
[3] http://mail.intelcompute.com/parse.php?redirect=a href=
Re: Field Collapsing - disable cache
Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, r...@intelcompute.com wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [5] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ [7] target=_blankhttp://atmail.org/ Links: -- [1] http://www.intelcompute.com/solrconfig.xml [8] target=_blankhttp://www.intelcompute.com/solrconfig.xml [2] http://mail.intelcompute.com/parse.php%3Fredirect%3D%26lt%3Ba [9] target=_blankhttp://mail.intelcompute.com/parse.php?redirect=http://blogs.semantico.com/discovery-blog/ - Message sent via Atmail Open - http://atmail.org/ Links: -- [1] http://mail.intelcompute.com/parse.php?redirect=a href= [3] http://mail.intelcompute.com/parse.php?redirect=a href= [5] http://mail.intelcompute.com/parse.php?redirect=a href= [6] http://mail.intelcompute.com/parse.php?redirect=a href= [7] http://mail.intelcompute.com/parse.php?redirect=a href= [8] http://mail.intelcompute.com/parse.php?redirect=a href= [9] http://mail.intelcompute.com/parse.php?redirect=a href= [10] http://mail.intelcompute.com/parse.php?redirect=a href= -- Toby Cole Senior Software Engineer, Semantico Limited toby.c...@semantico.com tel:+44 1273 358 238 Registered in England and Wales no. 03841410, VAT no. GB-744614334. Registered office Lees House, 21-23 Dyke
Re: Field Collapsing - disable cache
I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [6] target=_blankhttp://atmail.org/ [5] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [8] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ [9] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ [10] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [11] target=_blankhttp://atmail.org/ Links: -- [1] http://www.intelcompute.com/solrconfig.xml [12] target=_blankhttp://www.intelcompute.com/solrconfig.xml [8] target=_blankhttp://www.intelcompute.com/solrconfig.xml [13] target=_blankhttp://www.intelcompute.com/solrconfig.xml [2]
Re: Field Collapsing - disable cache
Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 r...@intelcompute.com: I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [6] target=_blankhttp://atmail.org/ [5] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [8] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ [9] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ [10] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [11] target=_blankhttp://atmail.org/ Links: -- [1] http://www.intelcompute.com/solrconfig.xml [12] target=_blankhttp://www.intelcompute.com/solrconfig.xml [8] target=_blankhttp://www.intelcompute.com/solrconfig.xml [13]
Re: Field Collapsing - disable cache
Yup, that's the one, with a copy of trunk from last week. On Tue 22/12/09 12:28 , Martijn v Groningen martijn.is.h...@gmail.com wrote: Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 : I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [6] target=_blankhttp://atmail.org/ [5] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [8] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ [9] target=_blankhttp://atmail.org/ - Message sent via Atmail Open
Re: Field Collapsing - disable cache
actually i got field-collapse-5.patch, what's the diff? On Tue 22/12/09 12:28 , Martijn v Groningen martijn.is.h...@gmail.com wrote: Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 : I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [6] target=_blankhttp://atmail.org/ [5] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [8] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ [9] target=_blankhttp://atmail.org/ - Message sent via Atmail Open -
java.lang.OutOfMemoryError while autowarming
Hi,
I'm facing some OutOfMemory issues with SOLR.
Tomcat is started with a -Xmx64m JVM parameter.
Tomcat manages 14 SOLR applications (not using multi-core : we're using SOLR
1.2).
The SOLR application impacted manages an 500Mo index (ie 250 lucene
documents).
Here is catalina.out log :
INFO: autowarming searc...@23d823d8 main from searc...@1ef21ef2 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=1,evictions=0,size=1,cumulative_lookups=1,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=1,cumulative_evictions=0}
Dec 19, 2009 4:57:44 AM org.apache.solr.core.SolrException log
SEVERE: Error during auto-warming of
key:org.apache.solr.search.queryresult...@e2819933:java.lang.OutOfMemoryError
at
org.apache.lucene.search.FieldCacheImpl$3.createValue(FieldCacheImpl.java:164)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:154)
at
org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:148)
at
org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:204)
at
org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:175)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:155)
at
org.apache.lucene.search.FieldSortedHitQueue.init(FieldSortedHitQueue.java:56)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:856)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:805)
at
org.apache.solr.search.SolrIndexSearcher.access$100(SolrIndexSearcher.java:60)
at
org.apache.solr.search.SolrIndexSearcher$2.regenerateItem(SolrIndexSearcher.java:251)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:193)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1385)
at org.apache.solr.core.SolrCore$1.call(SolrCore.java:488)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
at java.lang.Thread.run(Thread.java:810)
I join the solrconfig.xml file ti this post.
Thank you for your help.
Frédéric http://old.nabble.com/file/p26887639/solrconfig.xml solrconfig.xml
--
View this message in context:
http://old.nabble.com/java.lang.OutOfMemoryError-while-autowarming-tp26887639p26887639.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Field Collapsing - disable cache
On Tue 22/12/09 12:28 , Martijn v Groningen martijn.is.h...@gmail.com wrote: Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 : I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [6] target=_blankhttp://atmail.org/ [5] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/ [8] target=_blankhttp://atmail.org/ [6] target=_blankhttp://atmail.org/ [9] target=_blankhttp://atmail.org/ - Message sent via Atmail Open - http://atmail.org/ [10]
Re: Field Collapsing - disable cache
In the latest patch some changes where made on the configuration side, but if you add the CollapseComponent to the conf no field collapse cache should be enabled. If not let me know. Martijn 2009/12/22 r...@intelcompute.com: On Tue 22/12/09 12:28 , Martijn v Groningen martijn.is.h...@gmail.com wrote: Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 : I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail Open - http://atmail.org/ [6] target=_blankhttp://atmail.org/ [5] target=_blankhttp://atmail.org/ [7] target=_blankhttp://atmail.org/ [2] target=_blankhttp://atmail.org/
highlighting and external storage
Hi, I'm working on a news crawler with continuous indexing. Thus indexes are merged frequently and older documents aren't as important as recent ones. Therefor I'd like to store the fulltext of documents in an external storage (HBase?) so that merging of indexes isn't as IO intensive. This would give me the additional benefit, that I could selectively delete the fulltext of older articles when running out of disc space while keeping the url of the document in the index. Do you know, whether sth. like this would be possible? Best regards, Thomas Koch, http://www.koch.ro
adding a new slave doesn't trigger replication until the master index rebuilds
I'm trying to automate setting up new slave instances. I've noticed when I use the core admin to bring a new slave core online that the slave won't replicate the master index. The issue appears to be that when the slave core is created, it's given a new (empty) index with a newer timestamp than the master index timestamp. When the master finally reindexes, only then does the slave replicate. Short of triggering a reindex on the master, how can I get the newly created slave core to do replication? To create the slave core, I execute the following shell script: curl http://localhost:8080/solr/admin/cores?action=CREATEname=slave0instanceDir=solrhome/slave0config=solrhome/slave0/conf/solrconfig.xmlschema=solrhome/slave0/conf/schema.xmldataDir=solrhome/slave0/data; curl http://localhost:8080/solr/slave0/replication?command=fetchindexmasterUrl=http://localhost:8080/solr/master/replication; curl http://localhost:8080/solr/slave0/replication?command=enablepoll; Ryan
query parsing ( expansion ) in solr
Hello All, I have been trying to find out the right place to parse the query submitted. To be brief, I need to expand the query. For example.. let the query be city:paris then I would like to expand the query as .. follows city:paris OR place:paris OR town:paris . I guess the synonym support is provided only for values but not field names. 1) I could write a custom query parser which directly modifies input query string and then pass it to some default parser provided in solr. But it becomes more complex when its a complicated query and also I think it is not wise to write , because solr may provide few more search operators in the future. keeping that in mind I dont feel its going to be alright to follow this approach. 2) I could first pass the incoming query string to a default parser provided by Solr and then retrieve all the Terms ( and then add synonym terms ) using by calling Query.extractTerms() ..on the returned Query object but I am unable to get how to point out relations among the Terms like.. whether its Term1 OR Term2 AND Term3, or Term1 AND Term2 AND Term3 .. or something else. Can you please suggest me the right way of doing it. Thanks.
Re: java.lang.OutOfMemoryError while autowarming
-Xmx64m JVM parameter.
Is going to be far too low. For example I always start at 1 GB and
move up from there.
On Tue, Dec 22, 2009 at 4:35 AM, c82famo
ext_amouroux.frede...@agora.msa.fr wrote:
Hi,
I'm facing some OutOfMemory issues with SOLR.
Tomcat is started with a -Xmx64m JVM parameter.
Tomcat manages 14 SOLR applications (not using multi-core : we're using SOLR
1.2).
The SOLR application impacted manages an 500Mo index (ie 250 lucene
documents).
Here is catalina.out log :
INFO: autowarming searc...@23d823d8 main from searc...@1ef21ef2 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=1,evictions=0,size=1,cumulative_lookups=1,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=1,cumulative_evictions=0}
Dec 19, 2009 4:57:44 AM org.apache.solr.core.SolrException log
SEVERE: Error during auto-warming of
key:org.apache.solr.search.queryresult...@e2819933:java.lang.OutOfMemoryError
at
org.apache.lucene.search.FieldCacheImpl$3.createValue(FieldCacheImpl.java:164)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:154)
at
org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:148)
at
org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:204)
at
org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:175)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:155)
at
org.apache.lucene.search.FieldSortedHitQueue.init(FieldSortedHitQueue.java:56)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:856)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:805)
at
org.apache.solr.search.SolrIndexSearcher.access$100(SolrIndexSearcher.java:60)
at
org.apache.solr.search.SolrIndexSearcher$2.regenerateItem(SolrIndexSearcher.java:251)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:193)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1385)
at org.apache.solr.core.SolrCore$1.call(SolrCore.java:488)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
at java.lang.Thread.run(Thread.java:810)
I join the solrconfig.xml file ti this post.
Thank you for your help.
Frédéric http://old.nabble.com/file/p26887639/solrconfig.xml solrconfig.xml
--
View this message in context:
http://old.nabble.com/java.lang.OutOfMemoryError-while-autowarming-tp26887639p26887639.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: java.lang.OutOfMemoryError while autowarming
I'll recommend setting -Xms and -Xmx to the same value.
Sincerely,
Sithu D Sudarsan
-Original Message-
From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com]
Sent: Tuesday, December 22, 2009 2:31 PM
To: solr-user@lucene.apache.org
Subject: Re: java.lang.OutOfMemoryError while autowarming
-Xmx64m JVM parameter.
Is going to be far too low. For example I always start at 1 GB and
move up from there.
On Tue, Dec 22, 2009 at 4:35 AM, c82famo
ext_amouroux.frede...@agora.msa.fr wrote:
Hi,
I'm facing some OutOfMemory issues with SOLR.
Tomcat is started with a -Xmx64m JVM parameter.
Tomcat manages 14 SOLR applications (not using multi-core : we're using SOLR
1.2).
The SOLR application impacted manages an 500Mo index (ie 250 lucene
documents).
Here is catalina.out log :
INFO: autowarming searc...@23d823d8 main from searc...@1ef21ef2 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=1,evictions=0,size=1,cumulative_lookups=1,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=1,cumulative_evictions=0}
Dec 19, 2009 4:57:44 AM org.apache.solr.core.SolrException log
SEVERE: Error during auto-warming of
key:org.apache.solr.search.queryresult...@e2819933:java.lang.OutOfMemoryError
at
org.apache.lucene.search.FieldCacheImpl$3.createValue(FieldCacheImpl.java:164)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:154)
at
org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:148)
at
org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:204)
at
org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:175)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
at
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:155)
at
org.apache.lucene.search.FieldSortedHitQueue.init(FieldSortedHitQueue.java:56)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:856)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:805)
at
org.apache.solr.search.SolrIndexSearcher.access$100(SolrIndexSearcher.java:60)
at
org.apache.solr.search.SolrIndexSearcher$2.regenerateItem(SolrIndexSearcher.java:251)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:193)
at
org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1385)
at org.apache.solr.core.SolrCore$1.call(SolrCore.java:488)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
at java.lang.Thread.run(Thread.java:810)
I join the solrconfig.xml file ti this post.
Thank you for your help.
Frédéric http://old.nabble.com/file/p26887639/solrconfig.xml solrconfig.xml
--
View this message in context:
http://old.nabble.com/java.lang.OutOfMemoryError-while-autowarming-tp26887639p26887639.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: query parsing ( expansion ) in solr
Hello All, I have been trying to find out the right place to parse the query submitted. To be brief, I need to expand the query. For example.. let the query be city:paris then I would like to expand the query as .. follows city:paris OR place:paris OR town:paris . I guess the synonym support is provided only for values but not field names. Why not use DisMaxRequestHandler? ...search for the individual words across several fields... http://wiki.apache.org/solr/DisMaxRequestHandler
Re: adding a new slave doesn't trigger replication until the master index rebuilds
On Tue, Dec 22, 2009 at 9:30 AM, Ryan Kennedy rcken...@gmail.com wrote: I'm trying to automate setting up new slave instances. I've noticed when I use the core admin to bring a new slave core online that the slave won't replicate the master index. The issue appears to be that when the slave core is created, it's given a new (empty) index with a newer timestamp than the master index timestamp. When the master finally reindexes, only then does the slave replicate. Figured it out, I needed to add startup to the replicateAfter for this particular use case (I had tried adding the new core after restarting the master): str name=replicateAfterstartup/str Ryan
query ( dynamicField ) in solr
Hello All, I have been trying to make a query at a dynamicField. for example: i have a product that can have many sku's. So at my xsd i have dynamicField name=sku_externalId_* type=text indexed=true stored=true / dynamicField name=sku_defaultPrice_* type=text indexed=true stored=true / . . . when a make a query: http://localhost:8983/solr/productIdx/select/?q=sku_defaultPrice_2:valueOfThePrice or http://localhost:8983/solr/productIdx/select/?q=sku_defaultPrice_1:valueOfThePriceor sku_defaultPrice_2:valueOfThePrice or sku_defaultPrice_3:valueOfThePrice it works !! but i can have many sku_defaultPrice_* but i have no idea in how to make the query with this kind of field thanks for helping -- -- Atenciosamente, Gustavo de Araujo Lima Machado Telefone: +55 +21 *** Celular: +55+21 99034401 E-Mail: g.macha...@gmail.com.br Residência :
Re: query ( dynamicField ) in solr
I have been trying to make a query at a dynamicField. for example: i have a product that can have many sku's. So at my xsd i have dynamicField name=sku_externalId_* type=text indexed=true stored=true / dynamicField name=sku_defaultPrice_* type=text indexed=true stored=true / when a make a query: http://localhost:8983/solr/productIdx/select/?q=sku_defaultPrice_2:valueOfThePrice or http://localhost:8983/solr/productIdx/select/?q=sku_defaultPrice_1:valueOfThePriceor sku_defaultPrice_2:valueOfThePrice or sku_defaultPrice_3:valueOfThePrice it works !! but i can have many sku_defaultPrice_* but i have no idea in how to make the query with this kind of field thanks for helping So you want to query all fields of a dynamic field. I am not sure how to do that automatically but you can dynamically get those field names from : http://localhost:8983/solr/productIdx/admin/luke?numTerms=0 /response/l...@name='fields']/lst/s...@name='dynamicBase'][.='sku_defaultPrice_*' and then construct your query. Hope this helps.
suggestions for DIH batchSize
Hi, it looks like from looking at the code the default is 500, is the recommended setting for this? Has anyone notice any significant performance/memory tradeoffs by making this much bigger? thanks Joel
Re: query ( dynamicField ) in solr
sounds like a job for copyField: http://wiki.apache.org/solr/FAQ#How_do_I_use_copyField_with_wildcards.3F add sku_defaultPriceAll to your fields and then: copyField source=sku_defaultPrice_* dest=sku_defaultPriceAll / ...query on sku_defaultPriceAll. hth, rob On Tue, Dec 22, 2009 at 3:36 PM, Gustavo g.macha...@gmail.com wrote: Hello All, I have been trying to make a query at a dynamicField. for example: i have a product that can have many sku's. So at my xsd i have dynamicField name=sku_externalId_* type=text indexed=true stored=true / dynamicField name=sku_defaultPrice_* type=text indexed=true stored=true / . . . when a make a query: http://localhost:8983/solr/productIdx/select/?q=sku_defaultPrice_2:valueOfThePrice or http://localhost:8983/solr/productIdx/select/?q=sku_defaultPrice_1:valueOfThePriceor sku_defaultPrice_2:valueOfThePrice or sku_defaultPrice_3:valueOfThePrice it works !! but i can have many sku_defaultPrice_* but i have no idea in how to make the query with this kind of field thanks for helping -- -- Atenciosamente, Gustavo de Araujo Lima Machado Telefone: +55 +21 *** Celular: +55+21 99034401 E-Mail: g.macha...@gmail.com.br Residência :
Re: Solr replication 1.3 issue
This is a Unix security question. rsyncd' is a system daemon and should be managed in the OS scripts. The rsyncd-* scripts include a security setting for the 'solr' account that limits the account to the solr data directory (and this code does not support multicore). For security reasons, I personally would not make a sudoer out of a user with an automated remote login; but every site is different. On Mon, Dec 21, 2009 at 7:54 PM, Maduranga Kannangara mkannang...@infomedia.com.au wrote: Hi All, We're trying to replicate indexes on Solr 1.3 across from Dev-QA-Staging-Prod etc. So at each stage other than Dev and Prod, each would live as a master and a slave at a given time. We hit a bottle neck (may be?) when we try to start rsyncd-start on the master from the slave machine. Commands used: ssh -o StrictHostKeyChecking=no ad...@192.168.22.1 /solr/SolrHome/bin/rsyncd-enable ssh -o StrictHostKeyChecking=no ad...@192.168.22.1 /solr / SolrHome /bin/rsyncd-start -p 18003 On slave following error is displayed: @RSYNCD: 29 @ERROR: protocol startup error On master logs following were found: 2009/12/21 22:46:05 enabled by admin 2009/12/21 22:46:05 command: / solr/SolrHome /bin/rsyncd-enable 2009/12/21 22:46:05 ended (elapsed time: 0 sec) 2009/12/21 22:46:09 started by admin 2009/12/21 22:46:09 command: /solr/SolrHome/bin/rsyncd-start -p 18993 2009/12/21 22:46:09 [16964] forward name lookup for devserver002 failed: ai_family not supported 2009/12/21 22:46:09 [16964] connect from UNKNOWN (localhost) 2009/12/21 22:46:29 [16964] rsync: connection unexpectedly closed (0 bytes received so far) [receiver] 2009/12/21 22:46:29 [16964] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8] 2009/12/21 22:46:44 rsyncd not accepting connections, exiting 2009/12/21 22:46:57 enabled by admin 2009/12/21 22:46:57 command: /solr/SolrHome/bin/rsyncd-enable 2009/12/21 22:46:57 rsyncd already currently enabled 2009/12/21 22:46:57 exited (elapsed time: 0 sec) 2009/12/21 22:47:00 started by admin 2009/12/21 22:47:00 command: /solr/SolrHome/bin/rsyncd-start -p 18993 2009/12/21 22:47:00 [17115] forward name lookup for devserver002 failed: ai_family not supported 2009/12/21 22:47:00 [17115] connect from UNKNOWN (localhost) 2009/12/21 22:49:18 rsyncd not accepting connections, exiting Is it not possible to start the rsync daemon on master from the slave? The user that we use is on the sudoers list as well. Thanks Madu -- Lance Norskog goks...@gmail.com
solr.RemoveDuplicatesTokenFilterFactory
Hi, Here is the string to be indexed without duplication. Kitchen Cabinet Utah Kitchen Remodeling Utah Is RemoveDuplicatesTokenFilterFactory for this solution? or for something else? Jae
Re: Adaptive search?
Nice! Siddhant: Another problem to watch out for is the feedback problem: someone clicks on a link and it automatically becomes more interesting, so someone else clicks, and it gets even more interesting... So you need some kind of suppression. For example, as individual clicks get older, you can push them down. Or you can put a cap on the number of clicks used to rank the query. On Tue, Dec 22, 2009 at 2:36 AM, Siddhant Goel siddhantg...@gmail.com wrote: On Tue, Dec 22, 2009 at 12:01 PM, Ryan Kennedy rcken...@gmail.com wrote: This approach will be limited to applying a global rank to all the documents, which may have some unintended consequences. The most popular document in your index will be the most popular, even for queries for which it was never clicked on. Right. Makes so much sense. Thanks for sharing. -- - Siddhant -- Lance Norskog goks...@gmail.com
Re: Solr - Group search result
Rob: What would you like to know?
On Tue, Dec 22, 2009 at 3:10 AM, r...@intelcompute.com wrote:
see
http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/
[1] for a good little howto.
and also http://wiki.apache.org/solr/FieldCollapsing [2]
On Tue 22/12/09 11:03 , JKnight JKnight wrote:Dear all,
I have document doc.xml
3
Love you love me
2
Love story
3
How deep is your love?
4
I love you
5
How deep is your love?
6
Love story
and in schema.xml, I add two line
After post doc.xml to solr server and run query
http://localhost:8983/solr/select/%3Fwt%3Djson%26amp%3Bindent%3Don%26amp%3Bq%3Dlove%2520you%26amp%3Bfl%3Dname%26amp%3Bfacet%3Dtrue%26amp%3Bfacet.field%3Dname_exact
[3]
target=_blankhttp://localhost:8983/solr/select/?wt=jsonindent=onq=love%20youfl=namefacet=truefacet.field=name_exact
I see the response:
{
responseHeader:{
status:0,
QTime:4,
params:{
facet:true,
fl:name,
indent:on,
q:love you,
facet.field:name_exact,
wt:json}},
response:{numFound:5,start:0,docs:[
{
name:I love you},
{
name:Love story},
{
name:Love story},
{
name:How deep is your love?},
{
name:How deep is your love?}]
},
facet_counts:{
facet_queries:{},
facet_fields:{
name_exact:[
How deep is your love?,2,
Love story,2,
I love you,1,
Love you love me,0,
Quyen day ne,0,
day ne,0]},
facet_dates:{}}}
And I have a question. How can I group search result? With my data, I
want
to group by name field and see only 3 record
{
name:I love you},
{
name:Love story},
{
name:How deep is your love?}
Could you help me?
Thank a lot for support.
--
Best regards,
JKnight
-
Message sent via Atmail Open - http://atmail.org/
Links:
--
[1]
http://blog.jteam.nl/2009/10/20/result-grouping-field-collapsing-with-solr/
[2] http://wiki.apache.org/solr/FieldCollapsing
[3] http://mail.intelcompute.com/parse.php?redirect=a href=
--
Lance Norskog
goks...@gmail.com
Re: Field Collapsing - disable cache
To avoid this possible bug, you could change the cache to only have a few entries. On Tue, Dec 22, 2009 at 6:34 AM, Martijn v Groningen martijn.is.h...@gmail.com wrote: In the latest patch some changes where made on the configuration side, but if you add the CollapseComponent to the conf no field collapse cache should be enabled. If not let me know. Martijn 2009/12/22 r...@intelcompute.com: On Tue 22/12/09 12:28 , Martijn v Groningen martijn.is.h...@gmail.com wrote: Hi Rob, What patch are you actually using from SOLR-236? Martijn 2009/12/22 : I've tried both, the whole fieldCollapsing tag, and just the fieldCollapseCache tag inside it. both cause error. I guess I can just set size, initialSize, and autowarmCount to 0 ?? On Tue 22/12/09 11:17 , Toby Cole wrote:Which elements did you comment out? It could be the case that you need to get rid of the entire fieldCollapsing element, not just the fieldCollapsingCache element. (Disclaimer: I've not used field collapsing in anger before :) Toby. On 22 Dec 2009, at 11:09, wrote: That's what I assumed, but I'm getting the following error with it commented out MESSAGE null java.lang.NullPointerException at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .createDocumentCollapseResult(AbstractDocumentCollapser.java:276) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser .executeCollapse(AbstractDocumentCollapser.java:249) at org .apache .solr .search .fieldcollapse .AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java: 172) at org .apache .solr .handler .component.CollapseComponent.doProcess(CollapseComponent.java:173) at org .apache .solr .handler.component.CollapseComponent.process(CollapseComponent.java: 127) at org .apache .solr .handler .component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org .apache .solr .handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org .apache .solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:336) at org .apache .solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:239) at org .apache .catalina .core .ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java: 215) at org .apache .catalina .core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org .apache .catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 210) at org .apache .catalina.core.StandardContextValve.invoke(StandardContextValve.java: 172) at org .apache .catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org .apache .catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org .apache .catalina.core.StandardEngineValve.invoke(StandardEngineValve.java: 108) at org .apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java: 151) at org .apache.coyote.http11.Http11Processor.process(Http11Processor.java: 870) at org.apache.coyote.http11.Http11BaseProtocol $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java: 665) at org .apache .tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java: 528) at org .apache .tomcat .util .net .LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool $ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:636) On Tue 22/12/09 11:02 , Toby Cole wrote:If you take out the fieldCollapsing/fieldCollapseCache element in your config the fieldcollapse component will not use a cache. From http://wiki.apache.org/solr/FieldCollapsing%2523line-63 [2] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [1] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing%23line-63 [3] target=_blankhttp://wiki.apache.org/solr/FieldCollapsing#line-63 If the field collapse cache is not configured then the field collapse logic will not be cached. Regards, Toby. On 22 Dec 2009, at 10:56, wrote: my solconfig can be seen at http://www.intelcompute.com/solrconfig.xml [4] target=_blankhttp://www.intelcompute.com/solrconfig.xml [3] target=_blankhttp://www.intelcompute.com/solrconfig.xml [5] target=_blankhttp://www.intelcompute.com/solrconfig.xml [1] On Tue 22/12/09 10:51 , wrote:Is it possible to disable the field collapsing cache? I'm trying to perform some speed tests, and have managed to comment out the filter, queryResult, and document caches successfully. on 1.5 ... ... collapse facet tvComponent ... - Message sent via Atmail
Re: Synonym questions
i have a similar use case where i want to preserve original words forexamplei have one way mapping at index time monsoon,sony,levinson = audio system when i index document for example with word sony i want ot index audio system as well as the brand name ie sony. so if some one types category:sony i want highlighted resutls. if user types category:audio system i want the same document too. i initially thought there might be some property like preserveOriginal for synonym filter too like we had for worddelimiter tokeniser. please let me know is there any alternative. Thanks darniz hossman wrote: : 1) For some of my synonyms, it seems to make senses to simply replace the : original word with the other (e.g. theatre = theater, so searches for : either will find either). For others, I want to add an alternate term while : preserving the original (e.g. cirque = circus, so searches for circus : find Cirque du Soleil, but searches for cirque only match cirque, not : circus. the simplest way i can think of to achieve something like this off the top of my head is to do synonym injecting when idexing and don't use the synonym filter at all when querying... theatre, theater = theatre, theater circus = circus, cirque (but i haven't tested this to be sure it does what i think it does, and i haven't thought about SynonymFilter very hard for a while now so i may be forgetting something subtle) -Hoss -- View this message in context: http://old.nabble.com/Synonym-questions-tp12078428p26896098.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr.RemoveDuplicatesTokenFilterFactory
Looking at the code, it does appear to be what you want. In the analysis.jsp page, you can see exactly how your text is processed. http://localhost;8983/solr/admin/analysis.jsp On Tue, Dec 22, 2009 at 2:33 PM, Jae Joo jae...@gmail.com wrote: Hi, Here is the string to be indexed without duplication. Kitchen Cabinet Utah Kitchen Remodeling Utah Is RemoveDuplicatesTokenFilterFactory for this solution? or for something else? Jae -- Lance Norskog goks...@gmail.com
Implementing Autocomplete/Query Suggest using Solr
There seem to be a couple of approaches that people have adopted in implementing a query suggestion / auto completion feature using Solr. Depending on the situation, one might use the terms component or go the way of using EdgeNGramFilters and then creating querying the index on the ngrammed field. I also found that there is a bug currently active in JIRA ( http://issues.apache.org/jira/browse/SOLR-1316) for creating an auto suggest component. I am curious how an approach that simply uses the wildcard query functionality on an indexed field would work. While Solr does not support wildcard queries out of the box currently, it will definitely be included in the future and I believe the edismax parser already lets you do that. Would using the wildcard query to implement autocomplete have high overhead and be less efficient than the other approaches? Am I missing anything here? Kindly comment and provide some guidance. Thanks, Prasanna.
RE: Solr replication 1.3 issue
Yes, that makes sense Lance. Thanks. For the moment we broke our script to do the master's part (starting rsyncd) on the master server itself. Our problem is that we have so many instances running in different environment and we'd really love to minimize the number of them. :-) Thanks again. Madu -Original Message- From: Lance Norskog [mailto:goks...@gmail.com] Sent: Wednesday, 23 December 2009 9:33 AM To: solr-user@lucene.apache.org Subject: Re: Solr replication 1.3 issue This is a Unix security question. rsyncd' is a system daemon and should be managed in the OS scripts. The rsyncd-* scripts include a security setting for the 'solr' account that limits the account to the solr data directory (and this code does not support multicore). For security reasons, I personally would not make a sudoer out of a user with an automated remote login; but every site is different. On Mon, Dec 21, 2009 at 7:54 PM, Maduranga Kannangara mkannang...@infomedia.com.au wrote: Hi All, We're trying to replicate indexes on Solr 1.3 across from Dev-QA-Staging-Prod etc. So at each stage other than Dev and Prod, each would live as a master and a slave at a given time. We hit a bottle neck (may be?) when we try to start rsyncd-start on the master from the slave machine. Commands used: ssh -o StrictHostKeyChecking=no ad...@192.168.22.1 /solr/SolrHome/bin/rsyncd-enable ssh -o StrictHostKeyChecking=no ad...@192.168.22.1 /solr / SolrHome /bin/rsyncd-start -p 18003 On slave following error is displayed: @RSYNCD: 29 @ERROR: protocol startup error On master logs following were found: 2009/12/21 22:46:05 enabled by admin 2009/12/21 22:46:05 command: / solr/SolrHome /bin/rsyncd-enable 2009/12/21 22:46:05 ended (elapsed time: 0 sec) 2009/12/21 22:46:09 started by admin 2009/12/21 22:46:09 command: /solr/SolrHome/bin/rsyncd-start -p 18993 2009/12/21 22:46:09 [16964] forward name lookup for devserver002 failed: ai_family not supported 2009/12/21 22:46:09 [16964] connect from UNKNOWN (localhost) 2009/12/21 22:46:29 [16964] rsync: connection unexpectedly closed (0 bytes received so far) [receiver] 2009/12/21 22:46:29 [16964] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8] 2009/12/21 22:46:44 rsyncd not accepting connections, exiting 2009/12/21 22:46:57 enabled by admin 2009/12/21 22:46:57 command: /solr/SolrHome/bin/rsyncd-enable 2009/12/21 22:46:57 rsyncd already currently enabled 2009/12/21 22:46:57 exited (elapsed time: 0 sec) 2009/12/21 22:47:00 started by admin 2009/12/21 22:47:00 command: /solr/SolrHome/bin/rsyncd-start -p 18993 2009/12/21 22:47:00 [17115] forward name lookup for devserver002 failed: ai_family not supported 2009/12/21 22:47:00 [17115] connect from UNKNOWN (localhost) 2009/12/21 22:49:18 rsyncd not accepting connections, exiting Is it not possible to start the rsync daemon on master from the slave? The user that we use is on the sudoers list as well. Thanks Madu -- Lance Norskog goks...@gmail.com
Profiling Solr
Hi All, Recently we noticed that some of our heavy load Solr instances are facing memory leak kind situations. It goes onto Full GC and as it was unable to release any memory, the broken pripe and socket errors happen. (This happens both in Solr 1.3 and 1.4 for us.) Is there a good tool (preferably open source) that we could use to profile on the Solr deployed Tomcat and to figure out what is happening with the situation? If it was a connection keep alive or some wrong queries/bad schema configuration etc? Sorry about the laymen language.. Thanks in advance for all the responses! Madu
SCHEMA-INDEX-MISMATCH
Hi,all. I use Lucene's NumericField to index price field,And query with solr.TrieDoubleField. When i use price:[1 TO 5000] to search,it can return all results that price is between 1 and 5000. but the price value return is :ERROR:SCHEMA-INDEX-MISMATCH,stringValue=2000.0/str anybogy know why? -- View this message in context: http://old.nabble.com/SCHEMA-INDEX-MISMATCH-tp26897605p26897605.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: suggestions for DIH batchSize
A bigger batchSize results in increased memory usage. I guess performance should be slightly better with bigger values but not verified. On Wed, Dec 23, 2009 at 2:51 AM, Joel Nylund jnyl...@yahoo.com wrote: Hi, it looks like from looking at the code the default is 500, is the recommended setting for this? Has anyone notice any significant performance/memory tradeoffs by making this much bigger? thanks Joel -- - Noble Paul | Systems Architect| AOL | http://aol.com
Search both diacritics and non-diacritics
Hi all! I am developing a seach engine with Solr, and now I want to search both with and without diacritics, for example: if I query kho, it will response kho, khó, khò,... But if I query khó, it will response only khó. Who anyone have solution? I have used filter class=solr.ISOLatin1AccentFilterFactory/ but it is not correct :( -- View this message in context: http://old.nabble.com/Search-both-diacritics-and-non-diacritics-tp26897627p26897627.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Adaptive search?
On Wed, Dec 23, 2009 at 4:09 AM, Lance Norskog goks...@gmail.com wrote: Nice! Siddhant: Another problem to watch out for is the feedback problem: someone clicks on a link and it automatically becomes more interesting, so someone else clicks, and it gets even more interesting... So you need some kind of suppression. For example, as individual clicks get older, you can push them down. Or you can put a cap on the number of clicks used to rank the query. We use clicks/views instead of just clicks to avoid this problem. -- Regards, Shalin Shekhar Mangar.
Re: Implementing Autocomplete/Query Suggest using Solr
On Wed, Dec 23, 2009 at 6:14 AM, Prasanna R plistma...@gmail.com wrote: I am curious how an approach that simply uses the wildcard query functionality on an indexed field would work. It works fine as long as the terms are not repeated across documents. While Solr does not support wildcard queries out of the box currently, it will definitely be included in the future and I believe the edismax parser already lets you do that. Solr supports prefix queries and there's a reverse wild card filter in trunk too. We do auto-complete through prefix searches on shingles. -- Regards, Shalin Shekhar Mangar.
