Re: question about an entry in the log file
Thanks Shalin, that was very helpfull.
On 09/20/2017 01:02 PM, Shalin Shekhar Mangar wrote:
That log shows that the searcher being opened is the "realtime"
searcher as opposed to the "main" searcher. The realtime searcher is
quite lightweight. It causes a flush of built index segments from the
memory to the disk and opens a new searcher over them. No autowarming
or fsync happens for realtime searcher. It is used for realtime gets,
atomic updates and some other internal housekeeping tasks. This
searcher is essential and unavoidable.
The "main" searcher, on the other hand, is used for searching and is
heavy-weight. This is controlled by the "openSearcher" request
parameter and configuration.
On Wed, Sep 20, 2017 at 12:07 PM, kaveh minooie <ka...@plutoz.com> wrote:
Hi Erick
Thanks for your response. I understand your point, but what I was asking was
does solr reopen searchers after a commit call even if the commit was called
with openSearcher=false since this is what seems to be happening based on
these log entries?
Also, it seems that if autocommit is configured with
false and softAutoCommit is set to -1, absence
of any other commit call, the new updates, although committed, would remain
invisible forever. my problem here is that I am going for maximum indexing
performance, millions of document per each batch job, couple of batch jobs
per day. their immediate visibility or even delayed visibility is not a
priority, but they have to become visible at some point. preferably at the
end of each batch job. what do you think would be the best way to go about
this?
thanks,
On 09/20/2017 10:17 AM, Erick Erickson wrote:
First, I would not recommend you call commit from the client. It's
usually far better to let your autocommit settings in solrconfig.xml
deal with it. When you need to search, you either need to configure
with true
or set to something other than -1.
https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
Best,
Erick
On Tue, Sep 19, 2017 at 4:53 PM, kaveh minooie <ka...@plutoz.com> wrote:
Hi eveyone
I am trying to figure out why calling commit from my client takes a very
long time in an environment with concurrent updates, and I see the
following
snippet in the solr log files when client calls for commit. my question
is
regarding the third info. what is it opening? and how can make solr to
stop
doing that?
INFO - 2017-09-19 16:42:20.557; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.DirectUpdateHandler2; start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2017-09-19 16:42:20.557; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.SolrIndexWriter; Calling
setCommitData
with IW:org.apache.solr.update.SolrIndexWriter@3ee73284
INFO - 2017-09-19 16:42:20.660; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.search.SolrIndexSearcher; Opening
[Searcher@644a8d33[dosweb2016] realtime]
INFO - 2017-09-19 16:42:20.668; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.DirectUpdateHandler2;
end_commit_flush
thanks,
--
Kaveh Minooie
--
Kaveh Minooie
--
Kaveh Minooie
Re: question about an entry in the log file
Hi Erick
Thanks for your response. I understand your point, but what I was asking
was does solr reopen searchers after a commit call even if the commit
was called with openSearcher=false since this is what seems to be
happening based on these log entries?
Also, it seems that if autocommit is configured with
false and softAutoCommit is set to -1,
absence of any other commit call, the new updates, although committed,
would remain invisible forever. my problem here is that I am going for
maximum indexing performance, millions of document per each batch job,
couple of batch jobs per day. their immediate visibility or even delayed
visibility is not a priority, but they have to become visible at some
point. preferably at the end of each batch job. what do you think would
be the best way to go about this?
thanks,
On 09/20/2017 10:17 AM, Erick Erickson wrote:
First, I would not recommend you call commit from the client. It's
usually far better to let your autocommit settings in solrconfig.xml
deal with it. When you need to search, you either need to configure
with true
or set to something other than -1.
https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
Best,
Erick
On Tue, Sep 19, 2017 at 4:53 PM, kaveh minooie <ka...@plutoz.com> wrote:
Hi eveyone
I am trying to figure out why calling commit from my client takes a very
long time in an environment with concurrent updates, and I see the following
snippet in the solr log files when client calls for commit. my question is
regarding the third info. what is it opening? and how can make solr to stop
doing that?
INFO - 2017-09-19 16:42:20.557; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.DirectUpdateHandler2; start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2017-09-19 16:42:20.557; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.SolrIndexWriter; Calling setCommitData
with IW:org.apache.solr.update.SolrIndexWriter@3ee73284
INFO - 2017-09-19 16:42:20.660; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.search.SolrIndexSearcher; Opening
[Searcher@644a8d33[dosweb2016] realtime]
INFO - 2017-09-19 16:42:20.668; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
thanks,
--
Kaveh Minooie
--
Kaveh Minooie
question about an entry in the log file
Hi eveyone
I am trying to figure out why calling commit from my client takes a very
long time in an environment with concurrent updates, and I see the
following snippet in the solr log files when client calls for commit. my
question is regarding the third info. what is it opening? and how can
make solr to stop doing that?
INFO - 2017-09-19 16:42:20.557; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.DirectUpdateHandler2; start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2017-09-19 16:42:20.557; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.SolrIndexWriter; Calling
setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3ee73284
INFO - 2017-09-19 16:42:20.660; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.search.SolrIndexSearcher; Opening
[Searcher@644a8d33[dosweb2016] realtime]
INFO - 2017-09-19 16:42:20.668; [c:dosweb2016 s:shard2 r:core_node5
x:dosweb2016] org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
thanks,
--
Kaveh Minooie
using spell check on phrases
Hi everyone I am using solr 6 and DirectSolrSpellChecker, and edismax parser. the problem that I am having is that when the query is a phrase, every single word in the phrase need to be misspelled for the spell checker to gets activated and gives suggestions. if only one of the word is misspelled then it just says that spelling is correct: true I was wondering if anyone has encountered this situation before and knows how to solve it? thanks, -- Kaveh Minooie
from 1.4 to 3.6
HI I hope this doesn't turn out to be a very stupid question. I have upgraded from solr 1.4 to 3.6 and now in the response that I am getting from solr maxScore field in the [response] is missing. I am doing something wrong? how can I get it back? thanks, -- Kaveh Minooie www.plutoz.com
