There are some additional things I'm curious about.
When I tested it in the development environment, I found that setting the
commit interval to 15 seconds was sufficient to select a leader.(Thank you.)
But if you commit every 15 seconds, doesn't it remake the "Searcher" each
time and clear the query result cache, document cache and filter cache more
often?
Each replica has about 10,000 read requests per minute, so I'm worried
about whether the latency gets worse quickly.
For now, I try to autowarm it.
The settings look like this.
<filterCache class="solr.FastLRUCache"
size="${solr.query.filterCache.size:50000}"
initialSize="${solr.query.filterCache.initialSize:10000}"
autowarmCount="${solr.query.filterCache.autowarmCount:2000}"/>
<queryResultCache class="solr.LRUCache"
size="${solr.query.queryResultCache.size:40000}"
initialSize="${solr.query.queryResultCache.initialSize:20000}"
autowarmCount="${solr.query.queryResultCache.autowarmCount:2000}"/>
<documentCache class="solr.LRUCache"
size="${solr.query.documentCache.size:200000}"
initialSize="${solr.query.documentCache.initialSize:200000}"
autowarmCount="${solr.query.documentCache.autowarmCount:200000}"/>
<fieldValueCache class="solr.FastLRUCache"
size="512"
autowarmCount="128"
showItems="32" />
<cache name="perSegFilter"
class="solr.search.LRUCache"
size="10"
initialSize="0"
autowarmCount="10"
regenerator="solr.NoOpRegenerator" />
Is there a way for a leader's re-election to be over in 20 minutes or so,
but without worsening the latency?
Translated with www.DeepL.com/Translator (free version)
2020年4月15日(水) 21:27 Erick Erickson <erickerick...@gmail.com>:
> There’s no way leader election, even with tlog replay should take a day.
> 10,000 docs/minute doesn’t sound like enough to clog up
> replay either, so something’s definitely not what I’d expect.
>
> What is your hard commit interval? That controls how big the tlog
> is and thus how long it’d take to replay. Here’s more than you want to
> know about that:
>
>
> https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> I’d set it to, say, 15 seconds (openSearcher=false). This is entirely
> independent of the soft commit interval which governs the ability
> to search the docs…
>
> Best,
> Erick
>
> > On Apr 15, 2020, at 1:31 AM, Taisuke Miyazaki <
> miyazakitais...@lifull.com> wrote:
> >
> > Hi,
> >
> > Made Of: tlog replicas + pull replicas
> > Writing: leader and tlog replicas
> > Loading: pull replica only
> >
> > Solr version: 7.5
> > Number of shards: 1
> > Write throughput: 10000 docs/minutes
> > Number of documents: 4,500,000
> > Size per document: about 4KB
> >
> > During verification, the replay of the transaction log took a lot of
> time,
> > and it took 1 day and 6 hours to select a leader.
> > The tlog files are a few GB at best, so I don't think IO is the
> bottleneck,
> > but it's taking a lot longer than I imagined!
> >
> > I'd like to make the leader election quicker because I can't write until
> > the leader is elected.
> > Is there a faster way to do it?
> >
> > There is a mechanism to re-write records that have failed to write.
> > So, if I can't do leader selection quickly, I'm thinking of losing the
> > replica in the tlog and starting it automatically when the leader goes
> down
> > (if it's faster to recover that way).
> >
> > Thank you to everyone in the community for always being so supportive.
> >
> > Translated with www.DeepL.com/Translator (free version)
>
>
