[
https://issues.apache.org/jira/browse/SOLR-18626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated SOLR-18626:
--------------------------------
Description:
*Problem*: It's common for Solr clients to need to search for everything
already sent to Solr. Such a client could issue a commit (softCommit ideally)
to do this, albeit this can lead to situations with too many commits happening,
which makes Solr very busy and is generally an anti-pattern. Assuming the
client is willing to wait up to some amount of time, it could issue
commitWithin... albeit frustratingly it's not a good match for this use-case
because the client can't be sure exactly when the searcher is open, so it may
wastefully wait a while more than needed.
*Proposal*: {{commitWithin}} with {{waitSearcher=true}}
Today, the expert waitSearcher option is only applicable on the synchronous
"commit"/"softCommit" options. Imagine generalizing it to apply to
commitWithin as well, albeit simply with a different default. On commitWithin,
it should default to false to match the current semantics. What's newly
proposed is setting this to true on commitWithin (i.e. used with commitWithin),
thus having synchronous semantics. This means the update request will wait for
any pending data to be searchable.
Worst case, the update requests waits the full commitWithin provided plus
however long it takes to open the searcher. Best case, it returns immediately
when there's no pending data to be made searchable.
In the bigger picture, the searches that come after much either reach NRT
replicas or TLOG leaders to find what was made searchable just prior. But
that's standard no matter how you commit today (isn't a new concern).
was:
*Problem*: It's common for Solr clients to need to search for everything
already sent to Solr. Such a client could issue a commit (softCommit ideally)
to do this, albeit this can lead to situations with too many commits happening,
which makes Solr very busy and is generally an anti-pattern. Assuming the
client is willing to wait up to some amount of time, it could issue
commitWithin... albeit frustratingly it's not a good match for this use-case
because the client can't be sure exactly when the searcher is open, so it may
wastefully wait a while more than needed.
*Proposal*: {{commitWithin}} with {{waitSearcher=true}}
Today, the expert waitSearcher option is only applicable on the synchronous
"commit"/"softCommit" options. Imagine generalizing it to apply to
commitWithin as well, albeit simply with a different default. On commitWithin,
it should default to false to match the current semantics. What's newly
proposed is setting this to true on commitWithin (i.e. used with commitWithin),
thus having synchronous semantics. This means the update request will wait for
any pending data to be searchable.
> Want a synchronous commitWithin, maybe by using waitSearcher=true
> -----------------------------------------------------------------
>
> Key: SOLR-18626
> URL: https://issues.apache.org/jira/browse/SOLR-18626
> Project: Solr
> Issue Type: New Feature
> Components: update
> Reporter: David Smiley
> Priority: Major
>
> *Problem*: It's common for Solr clients to need to search for everything
> already sent to Solr. Such a client could issue a commit (softCommit
> ideally) to do this, albeit this can lead to situations with too many commits
> happening, which makes Solr very busy and is generally an anti-pattern.
> Assuming the client is willing to wait up to some amount of time, it could
> issue commitWithin... albeit frustratingly it's not a good match for this
> use-case because the client can't be sure exactly when the searcher is open,
> so it may wastefully wait a while more than needed.
> *Proposal*: {{commitWithin}} with {{waitSearcher=true}}
> Today, the expert waitSearcher option is only applicable on the synchronous
> "commit"/"softCommit" options. Imagine generalizing it to apply to
> commitWithin as well, albeit simply with a different default. On
> commitWithin, it should default to false to match the current semantics.
> What's newly proposed is setting this to true on commitWithin (i.e. used with
> commitWithin), thus having synchronous semantics. This means the update
> request will wait for any pending data to be searchable.
> Worst case, the update requests waits the full commitWithin provided plus
> however long it takes to open the searcher. Best case, it returns
> immediately when there's no pending data to be made searchable.
> In the bigger picture, the searches that come after much either reach NRT
> replicas or TLOG leaders to find what was made searchable just prior. But
> that's standard no matter how you commit today (isn't a new concern).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
