Re: ShardHandler - distribution to non-default request handler doesn't work

2012-10-28 Thread AlexeyK
The only way I succeeded to forward to the right request handler was:
1. shard.qt = /suggest (shards.qt=%2Fsuggest actually) in query
2.handleSelect='true' in solrconfig
3. NO /select handler in solrconfig

Only this combination forces 2 things - shard handler forwards qt=/suggest
parameter to other shards AND qt is handled by filter. (Otherwise qt is
ignored and the query gets forwarded to the /select handler)

Is there a better way of accomplishing this? How else can I retrieve
suggestions using a distinct handler?

Thanks,
Alexey



--
View this message in context: 
http://lucene.472066.n3.nabble.com/ShardHandler-distribution-to-non-default-request-handler-doesn-t-work-tp4015855p4016401.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: ShardHandler - distribution to non-default request handler doesn't work

2012-10-28 Thread AlexeyK
Correction:
shard.qt is sufficient, but you cannot define only spellcheck component in
requestHandler as it doesn't create shard requests, seems like 'query'
handler is a must if you want distributed processing.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/ShardHandler-distribution-to-non-default-request-handler-doesn-t-work-tp4015855p4016409.html
Sent from the Solr - User mailing list archive at Nabble.com.


Delete query puzzle

2012-10-28 Thread Eric Grobler
Hi

I am a bit confused why the server sometimes takes 80 seconds to respond
when I specify an id to delete than does not even exist in the index.

If I loop this query and send a bogus id to delete every minute.
03:27:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:28:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:29:38 69125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:30:38   124 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:31:38 84141 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:33:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:34:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:35:43 55476 ms  deleteidbogusidthatdoesnotexist/id/delete commit
03:36:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete commit
This was at 3am and the server only has about 200,000 documents and is not
busy, average query time is a constant  5ms.

If the server takes 80 seconds when it needs to update the index I would
understand it.
*But in this case the id does not exists, so the server should just return
immediately?*
I then must assume that the delete command must be in some low priority
queue and waits for some exclusive lock?
When I look at the stats it seems that it was only my loop that did
cumulative_deletesById every minute.

What settings in the solrconfig.xml would effect this behaviour?

Thank you  Regards
Ericz


Re: Delete query puzzle

2012-10-28 Thread Erick Erickson
That is very weird. What version of Solr are you using, and is there
any way you could get a stack trace when this is happening?

Best
Erick

On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler impalah...@googlemail.com wrote:
 Hi

 I am a bit confused why the server sometimes takes 80 seconds to respond
 when I specify an id to delete than does not even exist in the index.

 If I loop this query and send a bogus id to delete every minute.
 03:27:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:28:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:29:38 69125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:30:38   124 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:31:38 84141 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:33:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:34:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:35:43 55476 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 03:36:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete commit
 This was at 3am and the server only has about 200,000 documents and is not
 busy, average query time is a constant  5ms.

 If the server takes 80 seconds when it needs to update the index I would
 understand it.
 *But in this case the id does not exists, so the server should just return
 immediately?*
 I then must assume that the delete command must be in some low priority
 queue and waits for some exclusive lock?
 When I look at the stats it seems that it was only my loop that did
 cumulative_deletesById every minute.

 What settings in the solrconfig.xml would effect this behaviour?

 Thank you  Regards
 Ericz


Exception while getting Field info in Lucene

2012-10-28 Thread adityab
Hi
Deployed 4.0 and while investigating the schema Browser for seeing the
unique term count for each field observed following error. The top term
shows 10/-1. its -1 all the time. Any idea what might be wrong. 

thanks
Aditya

2012-10-28 10:48:42,017 SEVERE [org.apache.solr.servlet.SolrDispatchFilter]
(ajp-0.0.0.0-8009-12) null:java.lang.NullPointerException
at
org.apache.solr.handler.admin.LukeRequestHandler.getDetailedFieldInfo(LukeRequestHandler.java:602)
at
org.apache.solr.handler.admin.LukeRequestHandler.getIndexedFieldsInfo(LukeRequestHandler.java:371)
at
org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:159)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
at
org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:662)




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Exception-while-getting-Field-info-in-Lucene-tp4016448.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Delete query puzzle

2012-10-28 Thread Eric Grobler
Hi Erick,

It is Solr 1.41 (a Drupal installation) running on Jetty.
How can one get a stack trace? (there is no exception/error)

Could it be that solr does something like this?
start delete job
   cannot find bogus id to delete
   does some reindex or optimization anyway regardless which takes 80
seconds
end delete job

Anyway, does it sound like Solr is just waiting 80 seconds for some
exclusive lock or is it actually doing something in a background thread?. I
do not know what kind of calls drupal is doing.

Thanks  Regards
Ericz




On Sun, Oct 28, 2012 at 3:08 PM, Erick Erickson erickerick...@gmail.comwrote:

 That is very weird. What version of Solr are you using, and is there
 any way you could get a stack trace when this is happening?

 Best
 Erick

 On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler impalah...@googlemail.com
 wrote:
  Hi
 
  I am a bit confused why the server sometimes takes 80 seconds to respond
  when I specify an id to delete than does not even exist in the index.
 
  If I loop this query and send a bogus id to delete every minute.
  03:27:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:28:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:29:38 69125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:30:38   124 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:31:38 84141 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:33:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:34:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:35:43 55476 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:36:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  This was at 3am and the server only has about 200,000 documents and is
 not
  busy, average query time is a constant  5ms.
 
  If the server takes 80 seconds when it needs to update the index I would
  understand it.
  *But in this case the id does not exists, so the server should just
 return
  immediately?*
  I then must assume that the delete command must be in some low priority
  queue and waits for some exclusive lock?
  When I look at the stats it seems that it was only my loop that did
  cumulative_deletesById every minute.
 
  What settings in the solrconfig.xml would effect this behaviour?
 
  Thank you  Regards
  Ericz



Re: Delete query puzzle

2012-10-28 Thread Erick Erickson
Oh, 1.4.1. You're probably on your own here. I'd be surprised if
people were willing to work on code of that vintage. Are
you sure you can't upgrade at least to 3.6?

Best
Erick

On Sun, Oct 28, 2012 at 12:43 PM, Eric Grobler
impalah...@googlemail.com wrote:
 Hi Erick,

 It is Solr 1.41 (a Drupal installation) running on Jetty.
 How can one get a stack trace? (there is no exception/error)

 Could it be that solr does something like this?
 start delete job
cannot find bogus id to delete
does some reindex or optimization anyway regardless which takes 80
 seconds
 end delete job

 Anyway, does it sound like Solr is just waiting 80 seconds for some
 exclusive lock or is it actually doing something in a background thread?. I
 do not know what kind of calls drupal is doing.

 Thanks  Regards
 Ericz




 On Sun, Oct 28, 2012 at 3:08 PM, Erick Erickson 
 erickerick...@gmail.comwrote:

 That is very weird. What version of Solr are you using, and is there
 any way you could get a stack trace when this is happening?

 Best
 Erick

 On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler impalah...@googlemail.com
 wrote:
  Hi
 
  I am a bit confused why the server sometimes takes 80 seconds to respond
  when I specify an id to delete than does not even exist in the index.
 
  If I loop this query and send a bogus id to delete every minute.
  03:27:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:28:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:29:38 69125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:30:38   124 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:31:38 84141 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:33:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:34:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:35:43 55476 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  03:36:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete
 commit
  This was at 3am and the server only has about 200,000 documents and is
 not
  busy, average query time is a constant  5ms.
 
  If the server takes 80 seconds when it needs to update the index I would
  understand it.
  *But in this case the id does not exists, so the server should just
 return
  immediately?*
  I then must assume that the delete command must be in some low priority
  queue and waits for some exclusive lock?
  When I look at the stats it seems that it was only my loop that did
  cumulative_deletesById every minute.
 
  What settings in the solrconfig.xml would effect this behaviour?
 
  Thank you  Regards
  Ericz



Re: Exception while getting Field info in Lucene

2012-10-28 Thread Erick Erickson
I'm afraid you have to give more details, this works fine for me with
the example docs and the example schema.

Best
Erick

On Sun, Oct 28, 2012 at 11:02 AM, adityab aditya_ba...@yahoo.com wrote:
 Hi
 Deployed 4.0 and while investigating the schema Browser for seeing the
 unique term count for each field observed following error. The top term
 shows 10/-1. its -1 all the time. Any idea what might be wrong.

 thanks
 Aditya

 2012-10-28 10:48:42,017 SEVERE [org.apache.solr.servlet.SolrDispatchFilter]
 (ajp-0.0.0.0-8009-12) null:java.lang.NullPointerException
 at
 org.apache.solr.handler.admin.LukeRequestHandler.getDetailedFieldInfo(LukeRequestHandler.java:602)
 at
 org.apache.solr.handler.admin.LukeRequestHandler.getIndexedFieldsInfo(LukeRequestHandler.java:371)
 at
 org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:159)
 at
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
 at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
 at
 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
 at
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
 at
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at
 org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
 at
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
 at
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at
 org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
 at
 org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
 at
 org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
 at
 org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
 at
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at
 org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
 at
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
 at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
 at
 org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
 at
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
 at java.lang.Thread.run(Thread.java:662)




 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Exception-while-getting-Field-info-in-Lucene-tp4016448.html
 Sent from the Solr - User mailing list archive at Nabble.com.


Re: Delete query puzzle

2012-10-28 Thread Eric Grobler
Hi Erick

 You're probably on your own here. I'd be surprised if  people were
willing to work on code of that vintage.
Yes, this is not a vintage wine!

I just hoped someone would say, ah, we had this issue before and... :-)

I think best is to just upgrade like you suggested.

Thanks for your time
Ericz


On Sun, Oct 28, 2012 at 6:34 PM, Erick Erickson erickerick...@gmail.comwrote:

 Oh, 1.4.1. You're probably on your own here. I'd be surprised if
 people were willing to work on code of that vintage. Are
 you sure you can't upgrade at least to 3.6?

 Best
 Erick

 On Sun, Oct 28, 2012 at 12:43 PM, Eric Grobler
 impalah...@googlemail.com wrote:
  Hi Erick,
 
  It is Solr 1.41 (a Drupal installation) running on Jetty.
  How can one get a stack trace? (there is no exception/error)
 
  Could it be that solr does something like this?
  start delete job
 cannot find bogus id to delete
 does some reindex or optimization anyway regardless which takes 80
  seconds
  end delete job
 
  Anyway, does it sound like Solr is just waiting 80 seconds for some
  exclusive lock or is it actually doing something in a background
 thread?. I
  do not know what kind of calls drupal is doing.
 
  Thanks  Regards
  Ericz
 
 
 
 
  On Sun, Oct 28, 2012 at 3:08 PM, Erick Erickson erickerick...@gmail.com
 wrote:
 
  That is very weird. What version of Solr are you using, and is there
  any way you could get a stack trace when this is happening?
 
  Best
  Erick
 
  On Sun, Oct 28, 2012 at 6:22 AM, Eric Grobler 
 impalah...@googlemail.com
  wrote:
   Hi
  
   I am a bit confused why the server sometimes takes 80 seconds to
 respond
   when I specify an id to delete than does not even exist in the index.
  
   If I loop this query and send a bogus id to delete every minute.
   03:27:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:28:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:29:38 69125 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:30:38   124 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:31:38 84141 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:33:38   125 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:34:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:35:43 55476 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   03:36:38   141 ms  deleteidbogusidthatdoesnotexist/id/delete
  commit
   This was at 3am and the server only has about 200,000 documents and is
  not
   busy, average query time is a constant  5ms.
  
   If the server takes 80 seconds when it needs to update the index I
 would
   understand it.
   *But in this case the id does not exists, so the server should just
  return
   immediately?*
   I then must assume that the delete command must be in some low
 priority
   queue and waits for some exclusive lock?
   When I look at the stats it seems that it was only my loop that did
   cumulative_deletesById every minute.
  
   What settings in the solrconfig.xml would effect this behaviour?
  
   Thank you  Regards
   Ericz
 



Re: Occasional Solr performance issues

2012-10-28 Thread Dotan Cohen
On Fri, Oct 26, 2012 at 11:04 PM, Shawn Heisey s...@elyograg.org wrote:
 Warming doesn't seem to be a problem here -- all your warm times are zero,
 so I am going to take a guess that it may be a heap/GC issue.  I would
 recommend starting with the following additional arguments to your JVM.
 Since I have no idea how solr gets started on your server, I don't know
 where you would add these:

 -Xmx4096M -Xms4096M -XX:NewRatio=1 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
 -XX:+CMSParallelRemarkEnabled


Thanks. I've added those flags to the Solr line that I use to start
Solr. Those are Java flags, not Solr, correct? I'm googling the flags
now, but I find it interesting that I cannot find a canonical
reference for them.


 This allocates 4GB of RAM to java, sets up a larger than normal Eden space
 in the heap, and uses garbage collection options that usually fare better in
 a server environment than the default.Java memory management options are
 like religion to some people ... I may start a flamewar with these
 recommendations. ;)  The best I can tell you about these choices: They made
 a big difference for me.


Thanks. I will experiment with them empirically. The first step is to
learn to read the debug info, though. I've been googing for days, but
I must be missing something. Where is the information that I pasted in
pastebin documented?


 I would also recommend switching to a Sun/Oracle jvm.  I have heard that
 previous versions of Solr were not happy on variants like OpenJDK, I have no
 idea whether that might still be the case with 4.0.  If you choose to do
 this, you probably have package choices in Ubuntu.  I know that in Debian,
 the package is called sun-java6-jre ... Ubuntu is probably something
 similar. Debian has a CLI command 'update-java-alternatives' that will
 quickly switch between different java implementations that are installed.
 Hopefully Ubuntu also has this.  If not, you might need the following
 command instead to switch the main java executable:

 update-alternatives --config java


Thanks, I will take a look at the current Oracle JVM.


-- 
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com


Re: throttle segment merging

2012-10-28 Thread Lance Norskog
1) Do you use compound files (CFS)? This adds a lot of overhead to merging.
2) Does ES use the same merge policy code as Solr?

In solrconfig.xml, here are the lines that control segment merging. You can 
probably set mergeFactor to 20 and cut the amount of disk I/O.

!-- Expert: Merge Policy 
 The Merge Policy in Lucene controls how merging of segments is done.
 The default since Solr/Lucene 3.3 is TieredMergePolicy.
 The default since Lucene 2.3 was the LogByteSizeMergePolicy,
 Even older versions of Lucene used LogDocMergePolicy.
  --
!--
mergePolicy class=org.apache.lucene.index.TieredMergePolicy
  int name=maxMergeAtOnce10/int
  int name=segmentsPerTier10/int
/mergePolicy
  --
   
!-- Merge Factor
 The merge factor controls how many segments will get merged at a time.
 For TieredMergePolicy, mergeFactor is a convenience parameter which
 will set both MaxMergeAtOnce and SegmentsPerTier at once.
 For LogByteSizeMergePolicy, mergeFactor decides how many new segments
 will be allowed before they are merged into one.
 Default is 10 for both merge policies.
  --
!-- 
mergeFactor10/mergeFactor
  --

!-- Expert: Merge Scheduler
 The Merge Scheduler in Lucene controls how merges are
 performed.  The ConcurrentMergeScheduler (Lucene 2.3 default)
 can perform merges in the background using separate threads.
 The SerialMergeScheduler (Lucene 2.2 default) does not.
 --
!-- 
   mergeScheduler 
class=org.apache.lucene.index.ConcurrentMergeScheduler/
   --


- Original Message -
| From: Radim Kolar h...@filez.com
| To: solr-user@lucene.apache.org
| Sent: Saturday, October 27, 2012 7:44:46 PM
| Subject: Re: throttle segment merging
| 
| Dne 26.10.2012 3:47, Tomás Fernández Löbbe napsal(a):
|  Is there way to set-up logging to output something when segment
|  merging
|  runs?
| 
|  I think segment merging is logged when you enable infoStream
|  logging (you
|  should see it commented in the solrconfig.xml)
| no, segment merging is not logged at info level. it needs customized
| log
| config.
| 
| 
|  Can be segment merges throttled?
|   You can change when and how segments are merged with the merge
| policy, maybe it's enough for you changing the initial settings
| (mergeFactor for example)?
| 
| I am now researching elasticsearch, it can do it, its lucene 3.6
| based
| 


Re: org.apache.lucene.queryparser.classic.ParseException - a Bug?

2012-10-28 Thread deniz
hello iorixxx,

i have just tried url encoding because the raw format was also giving the
same error/exception... i was curious if it could fix...

anyone has any ideas on the exception? i still couldnt find a way to
overcome this 



-
Zeki ama calismiyor... Calissa yapar...
--
View this message in context: 
http://lucene.472066.n3.nabble.com/org-apache-lucene-queryparser-classic-ParseException-a-Bug-tp4015763p4016550.html
Sent from the Solr - User mailing list archive at Nabble.com.


Re: Search Suggest for full content with Filter option

2012-10-28 Thread Sujatha Arun
Any Suggestions on this?



On Sun, Oct 28, 2012 at 1:30 PM, Sujatha Arun suja.a...@gmail.com wrote:

 Hello,

 I want  suggestions for full content of several books with a filter
 that restricts suggestions to a single book .However the best options of
  suggester and terms component  do not support filter.

 That leaves the facets and  ngram Indexing.I indexed entire content by
 splitting on white space as the suggestions should work for any words in
 the index, But I find this query

 url/select?q=tesfacets=truefacet.field=ph_sufacet.prefix=tesfacet.limit=5
 extremely time consuming .This could be because of the number of unique
 terms in the full Index.

 For an ngram Indexing ,If were to Index the entire content as tokenized
 into a field and store the same  ,for any token for the document ,I would
 get the entire stored content as suggestion .How can I get the only the
 correct keyword as suggestion using an non-suggester based n gram Indexing
 such that it can be filtered?


 Regards
 Sujatha



Re: Occasional Solr performance issues

2012-10-28 Thread Shawn Heisey

On 10/28/2012 2:28 PM, Dotan Cohen wrote:

On Fri, Oct 26, 2012 at 11:04 PM, Shawn Heisey s...@elyograg.org wrote:

Warming doesn't seem to be a problem here -- all your warm times are zero,
so I am going to take a guess that it may be a heap/GC issue.  I would
recommend starting with the following additional arguments to your JVM.
Since I have no idea how solr gets started on your server, I don't know
where you would add these:

-Xmx4096M -Xms4096M -XX:NewRatio=1 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled

Thanks. I've added those flags to the Solr line that I use to start
Solr. Those are Java flags, not Solr, correct? I'm googling the flags
now, but I find it interesting that I cannot find a canonical
reference for them.


They are indeed Java options.  The first two control the maximum and 
starting heap sizes.  NewRatio controls the relative size of the young 
and old generations, making the young generation considerably larger 
than it is by default.  The others are garbage collector options.  This 
seems to be a good summary:


http://www.petefreitag.com/articles/gctuning/

Here's the official Sun (Oracle) documentation on GC tuning:

http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html

Thanks,
Shawn



Management of solr.xml on master server

2012-10-28 Thread maneesha
Hi,

As suggested by the Solr Enterprise book, I have separate strategies for
updating the solr core (e.g. blah) when I need to do incremental
updates(every day) VS create a fresh index from scratch(once in a 4 months).
Assuming that the dataDir for the core blah is /home/blah/data/defaultData
in the solr.xml.

For creating the full index for my core every quarter from scratch:
 - I create a new core e.g. blahNov2012 using admin url with option
action=CREATE and I give it a new dataDir property e.g.
/home/blah/data/Nov2012.
- I do a full import on blahNov2012 to populate the new core blah-Nov2012
and test it. 
- If all is good, I run the admin url with option action=SWAP to swap (blah
with blahNov2012).
- Since I have persistent=true in the solr.xml, it updates the dataDir for
the  core blah to point to the new directory /home/blah/data/Nov2012.

?xml version=1.0 encoding=UTF-8 ?
solr sharedLib=lib persistent=true
  property name=enable.slave value=false/
  property name=enable.master value=true/
  cores adminPath=/admin/cores
core name=blah instanceDir=blah/ dataDir=/home/blah/data/Nov2012
/core
  /cores
/solr

My first question: is this the pattern most people use to create fresh index
(e.g. create a new tmp core, test, and swap).

My second question is if I need to make further unrelated changes the
solr.xml in next releases and I must update the solr.xml on the production
system, I need to manually change the blah's data directory from
/home/blah/data/defaultData to /home/blah/data/Nov2012. Is that what
needs to be done? Do people automate this step somehow in their production
releases..?

-maneesha




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Management-of-solr-xml-on-master-server-tp4016570.html
Sent from the Solr - User mailing list archive at Nabble.com.