[ https://issues.apache.org/jira/browse/SOLR-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031011#comment-16031011 ]
Amrit Sarkar commented on SOLR-10734: ------------------------------------- The testmethod::testMultipleThreads() is faulty, the multithreaded support is intact until a certain limit. If I bring down the number of threads spawned b/w range 10-20, it works swiftly. As I debugged through the threads, there are multiple threads executing at the same time and some threads (10-20%) are not able to catch up ever with the version_id for maximum number of attempts. expected_is: X, actual_is: Y and when it fetches the current version from _vinfo_, it becomes Z (updated by some other thread), always behind. For MAX_ATTEMPTS=100, out of total 1000 threads, if I spawn, around 200-350 threads are failing to execute resulting in failure of poorly written test scenarios. But some threads do get indexed after multiple retries and populates the document successfully. e.g. {code} <response> <lst name="responseHeader"><int name="status">0</int><int name="QTime">3</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">10</int><arr name="range_facet_l"><long>10</long></arr><arr name="range_facet_l_dv"><long>10</long></arr><arr name="range_facet_i_dv"><int>10</int></arr><arr name="cat"><str>b0iuxlwrlloc7408ialh</str><str>b0iuxlwrlloc7408ialh</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>b0iuxlwrlloc7408ialh</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>b0iuxlwrlloc7408ialh</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>b0iuxlwrlloc7408ialh</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>b0iuxlwrlloc7408ialh</str><str>b0iuxlwrlloc7408ialh</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>b0iuxlwrlloc7408ialh</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>rcj1nr5whgo6bw31e4qa</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>gy3vfjzqbn41ir5tljdq</str><str>l5z00axtj727yv2dsz38</str><str>uwho9ivymrn3ewkrncgs</str><str>l5z00axtj727yv2dsz38</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>gy3vfjzqbn41ir5tljdq</str><str>uwho9ivymrn3ewkrncgs</str><str>b0iuxlwrlloc7408ialh</str><str>l5z00axtj727yv2dsz38</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>b0iuxlwrlloc7408ialh</str><str>rcj1nr5whgo6bw31e4qa</str><str>rcj1nr5whgo6bw31e4qa</str></arr><arr name="int_i"><int>432</int></arr><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2017-05-31T10:53:41.564Z</date><long name="_version_">1568908793938444289</long><int name="intDvoDefault">42</int></doc></result> </response> {code} Like the below scenario, I am expecting all the threads will be executed and and the field int_i will be populated correctly. If some of the threads fail, which is a valid scenario in real world, it can happen if the number of threads are many (here 1000), the test scenario fails. e.g error: {code} Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1] xml response was: <?xml version="1.0" encoding="UTF-8"?> <response> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result> </response> request was:q=int_i:2089&wt=xml at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:891) ... 40 more {code} I believe the multithreading support is broken but it has its breakpoint. Seeking suggestions on how to write test cases for AUP multithreading scenario as the current one is faulty, I can bring the number of threads down to 10-20 or even if some of the threads fail to execute and index atomic updates, we write test such that atleast document is present in index, with updates from successfully executed threads. Thank you. > Multithreaded test/support for AtomicURP broken > ----------------------------------------------- > > Key: SOLR-10734 > URL: https://issues.apache.org/jira/browse/SOLR-10734 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Reporter: Ishan Chattopadhyaya > Attachments: log-snippet > > > The multithreaded test doesn't actually start the threads, but only invokes > the run directly. The join afterwards doesn't do anything, hence. > {code} > diff --git > a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java > > b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java > index f3f833d..10b7770 100644 > --- > a/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java > +++ > b/solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java > @@ -238,7 +238,7 @@ public class AtomicUpdateProcessorFactoryTest extends > SolrTestCaseJ4 { > } > } > }; > - t.run(); > + t.run(); // red flag, shouldn't this be t.start? > threads.add(t); > finalCount += index; //int_i > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org