Re: Exceptions in Embedded Solr
I experienced this on an EmbeddedSolrServer which was running behind a
tomcat process. After restarting the tomcat process 2-3 times (implying this
also recreates the SolrServer every time as well) this issue went away but I
don't know why it ever started. It looked like the searcher shutdown was not
clean the previous time and I believe that could have to do with it.
Tony
On Sat, Dec 4, 2010 at 11:44 PM, Tharindu Mathew mcclou...@gmail.comwrote:
Any help on this?
On Thu, Dec 2, 2010 at 7:51 PM, Tharindu Mathew mcclou...@gmail.com
wrote:
Hi everyone,
I get the exception below when using Embedded Solr suddenly. If I
delete the Solr index it goes back to normal, but it obviously has to
start indexing from scratch. Any idea what the cause of this is?
java.lang.RuntimeException: java.io.FileNotFoundException:
/home/evanthika/WSO2/CARBON/GREG/3.6.0/23-11-2010/normal/wso2greg-3.6.0/solr/data/index/segments_2
(No such file or directory)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068)
at org.apache.solr.core.SolrCore.init(SolrCore.java:579)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:137)
at
org.wso2.carbon.registry.indexing.solr.SolrClient.init(SolrClient.java:103)
at
org.wso2.carbon.registry.indexing.solr.SolrClient.getInstance(SolrClient.java:115)
... 44 more
Caused by: java.io.FileNotFoundException:
/home/evanthika/WSO2/CARBON/GREG/3.6.0/23-11-2010/normal/wso2greg-3.6.0/solr/data/index/segments_2
(No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.init(RandomAccessFile.java:212)
at
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.init(SimpleFSDirectory.java:78)
at
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.init(SimpleFSDirectory.java:108)
at
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.init(NIOFSDirectory.java:94)
at
org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:70)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:691)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:236)
at
org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:72)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:403)
at
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1057)
... 48 more
[2010-11-23 14:14:46,568] ERROR {org.apache.solr.core.SolrCore} -
REFCOUNT ERROR: unreferenced org.apache.solr.core.solrc...@58f24b6
(null) has a reference count of 1
[2010-11-23 14:14:46,568] ERROR {org.apache.solr.core.SolrCore} -
REFCOUNT ERROR: unreferenced org.apache.solr.core.solrc...@654dbbf6
(null) has a reference count of 1
[2010-11-23 14:14:46,568] ERROR {org.apache.solr.core.CoreContainer} -
CoreContainer was not shutdown prior to finalize(), indicates a bug --
POSSIBLE RESOURCE LEAK!!!
[2010-11-23 14:14:46,568] ERROR {org.apache.solr.core.CoreContainer} -
CoreContainer was not shutdown prior to finalize(), indicates a bug --
POSSIBLE RESOURCE LEAK!!!
--
Regards,
Tharindu
--
Regards,
Tharindu
--
Regards,
Tharindu
Issues with SolrJ and IndexReader reopening (again)
I saw there had been a previous discussion on commit failing for EmbeddedSolrServer here: http://www.mail-archive.com/solr-user@lucene.apache.org/msg28236.html But it was never resolved. I have an embedded solr server and it does not seem to pick up changes in the index after a commit through Solrj. Looking at the logs, I can see a new searcher was opened - 20100929.141930/162 INFO [pool-1-thread-1] [] core.SolrCore - [] Registered new searcher searc...@8611b5c main 20100929.141930/162 INFO [pool-1-thread-1] [] search.SolrIndexSearcher - Closing searc...@5ff78541 main I'm using a single searcher, autowarming sizes of 0 to make sure no invalid entries get transfered over to the new searcher, I even set the httpCaching max-age=0 (i know pointless but i believe it technically is off then). Am I missing a form of caching or a configuration that will make sure this new searcher is pure or at least after time will be purified once invalid results expire? Thanks, Tony
Re: Embedded Server, Caching, Stats page updates
So you're right i did miss removing the app deployment but removing that still didn't really do that great. The avg request response time is still slower. The bell curve is a lot more streched than it was before but it doesn't seem to give an overall better performance. Thanks for your suggestions, Tony On Wed, May 19, 2010 at 4:37 PM, Chris Hostetter hossman_luc...@fucit.orgwrote: : Switched works for the specific setup i'm using - the server would refer : to itself in the CommonHttpSolrServer request sent, i.e. it would run both : the server and client sides. Removing this and simply using : EmbeddedSolrServer just made the setup a little more sane in that aspect. : Does that make more sense now? not really ... what *exactly* did you change about your setup and your client code? please be specific -- how did you run solr before when you were using CommonsHttpSolrServer? whare are *all* of the steps you did when you switched to EmbeddedSolrServer (specificly: what did the changes to your java client code look like, and what did you hcange about how you run solr) Because if you still have the solr.war running in your servlet container, and all you did is edit your java code to use EmbeddedSolrServer (poiting at the same directory on disk) instead of COmmonsHttpSolrServer, thne you are now running *two* instances of Solr in your VM, both reading from the same indexes. -Hoss
Re: Embedded Server, Caching, Stats page updates
The way you phrased that paragraph makes me think that one of us doesn't understand what exactly you did when you switched ... Switched works for the specific setup i'm using - the server would refer to itself in the CommonHttpSolrServer request sent, i.e. it would run both the server and client sides. Removing this and simply using EmbeddedSolrServer just made the setup a little more sane in that aspect. Does that make more sense now? Now for starters: if the remote server you were running solr on is more powerful then the local machine you are running your java application on, that alone could explain some performance differences (likewise for JVM settings). The machine I'm running it on is exactly the same - the code change was pushed and I had performance before and after. Same load observed (since it's a testing machine i could regulate that). That's why i was so surprised that removing that additional http request didn't cause improvement. Most importantly: when running solr embedded in your application, there is no stats.jsp page for you to look at -- because solr is no longer running in a servlet container. so if you are seeing stats on your solr server that say your caches aren't being hit, the reason is because the server isn't being hit at all. This is nice to know, I didn't look into how the actual page was generated. I expected something like this to be true. Thank you! When running an embedded solr server, the filterCache and queryResultCache will still be used. the settings in the solrconfig.xml you specify when initializing the SolrCore will be honored. you can see use JMX to monitor those cache hit rates (assuming you have JMX enabled for your application, and the appropriate setting is in your solrconfig.xml) I'll look into using JMX, thanks for the suggestion. Tony
Embedded Server, Caching, Stats page updates
I just switched from using CommonHttpSolrServer to EmbeddedSolrServer and the performance surprisingly deteriorated. I was expecting an improvement so in my confusion i went to the stats page and noticed that the caches were no longer getting hit. The embedded server however should still use IndexSearcher from Lucene (which is what the caches are supposed to be related to). Is there some kind of property that needs to be added or adjusted for embedded server to use cache? Should I create my own cache and wipe the rest out entirely? Should I remove the httpcache from the configuration since i'll no longer be accessing the service remotely? How accurate is the stats page and is the error actually coming from it rather than the actual backend? Thank you beforehand, Tony
