Re: Lucene indexWriter update does not affect Solr search
I did some investigation and found out that the retrieving part of
documents works fine while Solr did not restarted. But the searching part
of documents did not work. After I restarted Solr it seems that the core
corrupted and failed to start! Here is the corresponding log:
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.(SolrCore.java:896)
at org.apache.solr.core.SolrCore.(SolrCore.java:662)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:513)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:278)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:272)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1604)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1716)
at org.apache.solr.core.SolrCore.(SolrCore.java:868)
... 9 more
Caused by: org.apache.lucene.index.IndexNotFoundException: no
segments* file found in
NRTCachingDirectory(MMapDirectory@C:\Users\Ali\workspace\lucene_solr_5_0_0\solr\server\solr\document\data\index
lockFactory=org.apache.lucene.store.SimpleFSLockFactory@3bf76891;
maxCacheMB=48.0 maxMergeSizeMB=4.0): files: [_2_Lucene50_0.doc,
write.lock, _2_Lucene50_0.pos, _2.nvd, _2.fdt, _2_Lucene50_0.tim]
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:821)
at org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:78)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:272)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:115)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1573)
... 11 more
4/7/2015, 6:53:26 PM
ERROR
SolrIndexWriter
SolrIndexWriter was not closed prior to finalize(), indicates a bug
-- POSSIBLE RESOURCE LEAK!!!
4/7/2015, 6:53:26 PM
ERROR
SolrIndexWriter
Error closing IndexWriter
java.lang.NullPointerException
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:2959)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2927)
at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1010)
at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:130)
at org.apache.solr.update.SolrIndexWriter.finalize(SolrIndexWriter.java:183)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
There for my guess would be problem with indexing the keywordField and also
problem related to closing the IndexWriter.
On Tue, Apr 7, 2015 at 6:13 PM, Ali Nazemian wrote:
> Dear Upayavira,
> Hi,
> It is just the part of my code in which caused the problem. I know
> searchComponent is not for changing the index, but for the purpose of
> extracting document keywords I was forced to hack searchComponent for
> extracting keywords and putting them into index.
> For more information about why I chose searchComponent at the first place
> please follow this link:
>
> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201503.mbox/browser
>
> Best regards.
>
>
> On Tue, Apr 7, 2015 at 5:30 PM, Upayavira wrote:
>
>> What are you trying to do? A search component is not intended for
>> updating the index, so it really doesn’t surprise me that you aren’t
>> seeing updates.
>>
>> I’d suggest you describe the problem you are trying to solve before
>> proposing solutions.
>>
>> Upayavira
>>
>>
>> On Tue, Apr 7, 2015, at 01:32 PM, Ali Nazemian wrote:
>> > I implement a small code for the purpose of extracting some keywords out
>> > of
>> > Lucene index. I did implement that using search component. My problem is
>> > when I tried to update Lucene IndexWriter, Solr index which is placed on
>> > top of that, does not affect. As you can see I did the commit part.
>> >
>> > BooleanQuery query = new BooleanQuery();
>> > for (String fieldName : keywordSourceFields) {
>> > TermQuery termQuery = new TermQuery(new
>> Term(fieldName,"N/A"));
>> > query.add(termQuery, Occur.MUST_NOT);
>> > }
>> > TermQuery termQuery=new TermQuery(new Term(keywordField,
>> "N/A"));
>> > query.add(termQuery, Occur.MUST);
>> > try {
>> > //Query q=
Re: Lucene indexWriter update does not affect Solr search
Dear Upayavira,
Hi,
It is just the part of my code in which caused the problem. I know
searchComponent is not for changing the index, but for the purpose of
extracting document keywords I was forced to hack searchComponent for
extracting keywords and putting them into index.
For more information about why I chose searchComponent at the first place
please follow this link:
https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201503.mbox/browser
Best regards.
On Tue, Apr 7, 2015 at 5:30 PM, Upayavira wrote:
> What are you trying to do? A search component is not intended for
> updating the index, so it really doesn’t surprise me that you aren’t
> seeing updates.
>
> I’d suggest you describe the problem you are trying to solve before
> proposing solutions.
>
> Upayavira
>
>
> On Tue, Apr 7, 2015, at 01:32 PM, Ali Nazemian wrote:
> > I implement a small code for the purpose of extracting some keywords out
> > of
> > Lucene index. I did implement that using search component. My problem is
> > when I tried to update Lucene IndexWriter, Solr index which is placed on
> > top of that, does not affect. As you can see I did the commit part.
> >
> > BooleanQuery query = new BooleanQuery();
> > for (String fieldName : keywordSourceFields) {
> > TermQuery termQuery = new TermQuery(new Term(fieldName,"N/A"));
> > query.add(termQuery, Occur.MUST_NOT);
> > }
> > TermQuery termQuery=new TermQuery(new Term(keywordField, "N/A"));
> > query.add(termQuery, Occur.MUST);
> > try {
> > //Query q= new QueryParser(keywordField, new
> > StandardAnalyzer()).parse(query.toString());
> > TopDocs results = searcher.search(query,
> > maxNumDocs);
> > ScoreDoc[] hits = results.scoreDocs;
> > IndexWriter writer = getLuceneIndexWriter(searcher.getPath());
> > for (int i = 0; i < hits.length; i++) {
> > Document document = searcher.doc(hits[i].doc);
> > List keywords = keyword.getKeywords(hits[i].doc);
> > if(keywords.size()>0) document.removeFields(keywordField);
> > for (String word : keywords) {
> > document.add(new StringField(keywordField, word,
> > Field.Store.YES));
> > }
> > String uniqueKey =
> > searcher.getSchema().getUniqueKeyField().getName();
> > writer.updateDocument(new Term(uniqueKey,
> > document.get(uniqueKey)),
> > document);
> > }
> > writer.commit();
> > writer.forceMerge(1);
> > writer.close();
> > } catch (IOException | SyntaxError e) {
> > throw new RuntimeException();
> > }
> >
> > Please help me through solving this problem.
> >
> > --
> > A.Nazemian
>
--
A.Nazemian
Re: Lucene indexWriter update does not affect Solr search
What are you trying to do? A search component is not intended for
updating the index, so it really doesn’t surprise me that you aren’t
seeing updates.
I’d suggest you describe the problem you are trying to solve before
proposing solutions.
Upayavira
On Tue, Apr 7, 2015, at 01:32 PM, Ali Nazemian wrote:
> I implement a small code for the purpose of extracting some keywords out
> of
> Lucene index. I did implement that using search component. My problem is
> when I tried to update Lucene IndexWriter, Solr index which is placed on
> top of that, does not affect. As you can see I did the commit part.
>
> BooleanQuery query = new BooleanQuery();
> for (String fieldName : keywordSourceFields) {
> TermQuery termQuery = new TermQuery(new Term(fieldName,"N/A"));
> query.add(termQuery, Occur.MUST_NOT);
> }
> TermQuery termQuery=new TermQuery(new Term(keywordField, "N/A"));
> query.add(termQuery, Occur.MUST);
> try {
> //Query q= new QueryParser(keywordField, new
> StandardAnalyzer()).parse(query.toString());
> TopDocs results = searcher.search(query,
> maxNumDocs);
> ScoreDoc[] hits = results.scoreDocs;
> IndexWriter writer = getLuceneIndexWriter(searcher.getPath());
> for (int i = 0; i < hits.length; i++) {
> Document document = searcher.doc(hits[i].doc);
> List keywords = keyword.getKeywords(hits[i].doc);
> if(keywords.size()>0) document.removeFields(keywordField);
> for (String word : keywords) {
> document.add(new StringField(keywordField, word,
> Field.Store.YES));
> }
> String uniqueKey =
> searcher.getSchema().getUniqueKeyField().getName();
> writer.updateDocument(new Term(uniqueKey,
> document.get(uniqueKey)),
> document);
> }
> writer.commit();
> writer.forceMerge(1);
> writer.close();
> } catch (IOException | SyntaxError e) {
> throw new RuntimeException();
> }
>
> Please help me through solving this problem.
>
> --
> A.Nazemian
Lucene indexWriter update does not affect Solr search
I implement a small code for the purpose of extracting some keywords out of
Lucene index. I did implement that using search component. My problem is
when I tried to update Lucene IndexWriter, Solr index which is placed on
top of that, does not affect. As you can see I did the commit part.
BooleanQuery query = new BooleanQuery();
for (String fieldName : keywordSourceFields) {
TermQuery termQuery = new TermQuery(new Term(fieldName,"N/A"));
query.add(termQuery, Occur.MUST_NOT);
}
TermQuery termQuery=new TermQuery(new Term(keywordField, "N/A"));
query.add(termQuery, Occur.MUST);
try {
//Query q= new QueryParser(keywordField, new
StandardAnalyzer()).parse(query.toString());
TopDocs results = searcher.search(query,
maxNumDocs);
ScoreDoc[] hits = results.scoreDocs;
IndexWriter writer = getLuceneIndexWriter(searcher.getPath());
for (int i = 0; i < hits.length; i++) {
Document document = searcher.doc(hits[i].doc);
List keywords = keyword.getKeywords(hits[i].doc);
if(keywords.size()>0) document.removeFields(keywordField);
for (String word : keywords) {
document.add(new StringField(keywordField, word,
Field.Store.YES));
}
String uniqueKey =
searcher.getSchema().getUniqueKeyField().getName();
writer.updateDocument(new Term(uniqueKey,
document.get(uniqueKey)),
document);
}
writer.commit();
writer.forceMerge(1);
writer.close();
} catch (IOException | SyntaxError e) {
throw new RuntimeException();
}
Please help me through solving this problem.
--
A.Nazemian
