It looks like your code has a leak and does not close all
IndexReaders/Writers that you use during your custom code in Solr. It is
impossible to review this from outside.
You shuld use the Solr provided SolrIndexWriter and SolrIndexSearcher to
do your custom stuff and let Solr manage them.
U
Uwe,
Thanks for the response. I have openSearcher=false in autoCommit, but I do
have an autoSoftCommit interval of 5 minutes configured as well which
should open a searcher.
In vanilla Solr, without my code, I see that if I completely reindex all
documents in a segment (via a client call), the segm
Hi,
in Solr the empty segment keeps open as long as there is a Searcher
still open. At some point the empty segment (100% deletions) will be
deleted, but you have to wait until SolIndexSearcher has restarted.
Maybe check your solrconfig.xml and check if openSearcher is enabled
after autoSoftC
Stefan, Mike,
Appreciate your responses! I spent some time analyzing your inputs and
going further down the rabbit hole.
Stefan,
I looked at the IndexRearranger code you referenced where it tries to drop
the segment. I see that it eventually gets handled via
IndexFileDeleter.checkpoint() through f
Hi Rahul,
Please do not pursue Approach 2 :) ReadersAndUpdates.release is not
something the application should be calling. This path can only lead to
pain.
It sounds to me like something in Solr is holding an old reader (maybe the
last commit point, or reader prior to the refresh after you re-i
Hi Rahul,
What you're describing sounds similar to index rearranging [1], although in
that case the reindexing is done in a new index. The last commit in the
IndexRearranger class added support for reading and reindexing deletes -
maybe
having a look at that and at the Javadoc would help?
Stefan
Thanks for the response Mikhail. I don't think I am looking for
forceMergeDeletes() though since it could be more expensive than I would
like and I only want to see the unreferenced segments with 0 live docs to
be deleted. Just the way they get deleted with a commit=true option or even
softDelete.
Hi Rahul.
Are you looking for
https://lucene.apache.org/core/9_0_0/core/org/apache/lucene/index/IndexWriter.html#forceMergeDeletes()
?
On Tue, Aug 29, 2023 at 5:20 AM Rahul Goswami wrote:
> Hello,
> I am trying to execute a program to read documents segment-by-segment and
> reindex to the same i
Hello,
I am trying to execute a program to read documents segment-by-segment and
reindex to the same index. I am reading using Lucene apis and indexing
using solr api (in a core that is currently loaded).
What I am observing is that even after a segment has been fully processed
and an autoCommit (
