[
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943615#comment-14943615
]
Ishan Chattopadhyaya commented on SOLR-5944:
--------------------------------------------
Thanks for looking at it.
bq. Whew... this seems really tricky. I've been diving into the Chaos* fails
recently, and at first blush it seems like this would add more complexity to
recovery as well (log replays, peer sync, etc?) What are the implications there?
I need to do the due diligence and write some tests to verify that things will
work with log replays and peer sync.
Actually, since the following comment, things changed a bit (maybe be simpler?):
bq. If, upon receiving the update on a replica, the doc version on index/tlog
is not the "old version" (that means we've missed in update in between to the
doc, because of reordering), then we can write this update to tlog (and mark it
somehow as something we're waiting on) but not actually update the doc in the
index until we receive the update whose update "old version" is what we are
expecting. After doing this (for all pending updates for the doc), we could
unmark the documents.
Changed the above to the following:
{noformat}
If, upon receiving the update on a replica, the doc version on index/tlog is
not the "old version" (that means we've missed in update in between to the doc,
because of reordering), then we write this in-place update to a temporary
in-memory buffer and not actually write this to the tlog/index until we receive
the update whose "old version" is what we are expecting for the buffered
updates. As buffered updates get written to the tlog/index, they are removed
from the in-memory buffer.
{noformat}
This ensures that the tlog entries are always exactly in the order in which the
documents were written.
bq. It seems like we can't return success on an update until that update has
actually been applied?
Good point, I haven't thought about this. Is it okay to return success if it
was written to (at least) the in-memory buffer (which holds these reordered
updates)? Of course, that would entail the risk of queries to this replica to
return the updated document till before the point at which reordering started.
bq. Also, what happens to this prevPointer you are writing to the tlog if
there is a commit in-between?
This prevPointer is just used (in the patch) for RTGs. In the
{{InPlaceUpdateDistribTest}}, I've introduced commits (with 1/3 probability) in
between the re-ordered updates, and the RTG seems to work fine.
bq. Another approach would be to get rid of update reordering... i.e. ensure
that updates are not reordered when sending from leader to replicas.
Sounds interesting. How do you suggest can this be achieved?
> Support updates of numeric DocValues
> ------------------------------------
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
> Issue Type: New Feature
> Reporter: Ishan Chattopadhyaya
> Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be
> really nice to have Solr support this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
