I know that. The question is if a new directory is populated sourced from the current/live directory for existing segments... OR is it populated in-place (same directory).
~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Wed, Nov 30, 2022 at 12:37 AM Noble Paul <noble.p...@gmail.com> wrote: > > For such older segments, after replication, are they at the same exact > file > path or does replication create a new path? > > Replication only downloads the delta. If it already has the files, they are > not downloaded > > > On Wed, Nov 30, 2022 at 4:41 AM David Smiley <dsmi...@apache.org> wrote: > > > It could be interesting to explore optimizing "openNewSearcher" when > there > > is an existing searcher open over some of the same segments. > > For such older segments, after replication, are they at the same exact > file > > path or does replication create a new path? I forget this detail; I > recall > > seeing some index data path shuffling going on. ReplicationHandler etc. > is > > overly complex. > > > > ~ David Smiley > > Apache Lucene/Solr Search Developer > > http://www.linkedin.com/in/davidwsmiley > > > > > > On Wed, Nov 23, 2022 at 1:50 PM Patson Luk <pat...@fullstory.com> wrote: > > > > > Hi all! > > > > > > We are testing multiple replica setup here (1 NRT + 1 PULL) and noticed > > > that CPU consumption for replication is unreasonably high. Profiling > > shows > > > that `SolrCore#openNewSearcher` triggered from `IndexFetcher` takes > much > > > more CPU time than the same method triggered from regular commits. > > > > > > Debugging shows that when `SolrCore#openNewSearcher` is triggered from > > > `IndexFetcher`, it opens a new `SegmentReader` for every single > fragment > > > for the updated collection. As a new `IndexWriter`, which keeps a > > > `ReaderPool`, is instantiated for each replication. And such pool is > not > > > reused nor previous segment readers are carried over. > > > > > > Details in this ticket > https://issues.apache.org/jira/browse/SOLR-16560. > > > > > > Since I'm pretty new to this area, I would love to get some thoughts > from > > > the community! > > > > > > Many thanks! > > > Patsn > > > > > > > > -- > ----------------------------------------------------- > Noble Paul >
