Hi Patrick,
This is an interesting question, and from what I understood, I see
correctness problems in what you're trying to implement. Let me make sure I
understand correctly...
So indexer-1 created segments 1,2,3,4 and indexer-2 created segments 1',
2', 3', 4' independently (they just have the
Hi Robert, got it, thanks!
Hi Uwe, yes we do have a way to detect whether the segment is created by
node A or B even if they share the same name, however, lucene does not
allow such situation (same name but generated by different writer) when
calling `openIfChanged` to try to incrementally load
No, you can't control them. And we must not open up anything to try to
support this.
On Fri, Dec 16, 2022 at 7:28 PM Patrick Zhai wrote:
>
> Hi Mike, Robert
>
> Thanks for replying, the system is almost like what Mike has described: one
> writer is primary,
> and the other is trying to catch up
Hi,
have you thought about storing additional metadata in the commits? If
you want to have custom information in each commit, just save some
internal tracking identifiers so you can figure out if node A or node B
is primary by checking their latest commit metadata.
Generally I do not
Hi Mike, Robert
Thanks for replying, the system is almost like what Mike has described: one
writer is primary,
and the other is trying to catch up and wait, but in our internal
discussion we found there might
be small chances where the secondary mistakenly think itself as primary
(due to errors
+1 trying to coordinate multiple writers running independently will
not work. My 2c for availability: you can have a single primary active
writer with a backup one waiting, receiving all the segments from the
primary. Then if the primary goes down, the secondary one has the most
recent commit
You are still talking "Multiple writers". Like i said, going down this
path (playing tricks with filenames) isn't going to work out well.
On Fri, Dec 16, 2022 at 2:48 AM Patrick Zhai wrote:
>
> Hi Robert,
>
> Maybe I didn't explain it clearly but we're not going to constantly switch
> between
Hi Robert,
Maybe I didn't explain it clearly but we're not going to constantly switch
between writers or share effort between writers, it's purely for
availability: the second writer only kicks in when the first writer is not
available for some reason.
And as far as I know the replicator/nrt
This multiple-writer isn't going to work and customizing names won't
allow it anyway. Each file also contains a unique identifier tied to
its commit so that we know everything is intact.
I would look at the segment replication in lucene/replicator and not
try to play games with files and mixing
Hi Folks,
We're trying to build a search architecture using segment replication
(indexer and searcher are separated and indexer shipping new segments to
searchers) right now and one of the problems we're facing is: for
availability reason we need to have multiple indexers running, and when the
10 matches
Mail list logo
