[
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323784#comment-15323784
]
Hoss Man commented on SOLR-5944:
--------------------------------
bq. When update2 (say a partial update) arrives before update1 (say a full
update, on which update2 depends), *then the call for indexing update2 is a
blocking call* (which finishes either after update1 is indexed, or timeout is
reached).
Ahhh... now it makes sense to me. The part I wasn't getting before was that
update2 blocks on the replica until it sees the update1 it is dependent on.
I feel like there is probably a way we could write a more sophisticate "grey
box" type test for this leveraging callbacks in the DebugFilter, but I'm having
trouble working out what that would really look like.
I think the hueristic approach you're taking here is generall fine for now (as
a way to _try_ to run the updates in a given order even though we know there
are no garuntees) but i have a few suggestions to improve things:
* lots more comments in the test code to make it clear that we use multiple
threads because each update may block if it depends on another update
* replace the comments on the sleep calls to make it clear that while we can't
garuntee/trust what order the updates are executed in since multiple threads
are involved, we're trying to bias the thread scheduling to run them in the
order submitted
** (the wording right now seems definitive and makes the code look clearly
suspicious)
* create {{atLeast(3)}} updates instead of just a fixed set of "3" so we
increase our odds of finding potential bugs when more then one update is out of
order.
* loop over multiple (random) permutations of orderings of the updates
** don't worry about wether a given ordering is actually correct, that's a
valid random ordering for the purposes of the test
** a simple comment saying we know it's possible but it doesn't affect any
assumptions/assertions in the test is fine
* for each random permutation, execute it (and check the results) multiple times
** this will help increase the odds that the thread scheduling actaully winds
up running our updates in the order we were hoping for.
* essentially this should be a a micro "stress test" of updates in arbitrary
order
Something like...
{code}
final String ID = "0";
final int numUpdates = atLeast(3);
final int numPermutationTotest = atLeast(5);
for (int p = 0; p < numPermutationTotest; p++) {
del("*:*);
commit();
index("id",ID, ...); // goes to all replicas
commit();
long version = assertExpectedValuesViaRTG(LEADER, ID, ...);
List<UpdateRequest> updates = makeListOfSequentialSimulatedUpdates(ID,
version, numUpdates);
for (UpdateRequest req : updates) {
assertEquals(0, REPLICA_1.requets(req).getStatus());
}
Collections.shuffle(updates, random());
// this method is where you'd comment the hell out of why we use threads for
this,
// and can be re-used in the other place where a threadpool is used...
assertSendUpdatesInThreadsWithDelay(REPLICA_0, updates, 100ms);
for (SolrClient client : NONLEADERS) [
// assert value on replica matches original value + numUpdates
}
}
{code}
----
As a related matter -- if we are expecting a replica to "block & eventually
time out" when it sees an out of order update, then there should be a white box
test asserting the expected failure situation as well -- something like...
{code}
final String ID = "0";
del("*:*);
commit();
index("id",ID, ...);
UpdateRequest req = simulatedUpdateRequest(version + 1, ID, ...);
Timer timer = new Timer();
timer.start();
SolrServerException e = expectThrows(() -> { REPLICA_0.request(req); });
timer.stop();
assert( /* elapsed time of timer is at least the X that we expect it to block
for */ )
assert(e.getgetHttpStatusMesg().contains("something we expect it to say if the
update was out of order"))
assertEquls(/* whatever we expect in this case */, e.getHttpStatusCode());
{code}
> Support updates of numeric DocValues
> ------------------------------------
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
> Issue Type: New Feature
> Reporter: Ishan Chattopadhyaya
> Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch,
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt,
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt,
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz,
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt,
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt,
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be
> really nice to have Solr support this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
