The test.seed will cause the randomized testing to choose the same locale/timezone/encoding/etc so that if you're hitting a bug that only shows up with swahili or for the kiribati timezone it will reproduce reliably.
This only works if your test is tripped by something influenced by that seed. Otherwise: A) it's key to run cleanTest before running that "reproduce with" command so that the build doesn't pass thinking the test is up to date (the reproduce line probably ought to include this by default). B) The bad news is what you are probably seeing is flakey test issues. Unfortunately, there is something wrong either with the test harness or the server wherein at least two or maybe more things seem to happen. Test shut down leaks threads relating to zkclient/httpclient, or collections that don't seem to exist when they should. There may be more failure modes than that, but those are two that I've noticed. These issues resolve when the tests are run individually, and probably relate to something being slow due to the high load of the tests. The thread leak version fails a test called "class method" and the reproduce string always fails with a message about junit not finding tests, because there isn't any test named .classMethod I recently started a thread titled "unit tests" where I so far have documented how bad this is but have not yet found any solution. -Gus On Mon, Nov 22, 2021 at 7:30 AM Eric Pugh <ep...@opensourceconnections.com> wrote: > Can someone explain how the -Ptest.seed works, and how I use it to figure > out if a bug exists? > > I ran the full set of tests, and had this message: > ERROR: The following test(s) have failed: > - > org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test > (:solr:core) > Test output: > /Users/epugh/Documents/projects/solr-epugh-2/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.txt > Reproduce with: gradlew :solr:core:test --tests > "org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test" > -Ptests.jvms=4 -Ptests.jvmargs=-XX:TieredStopAtLevel=1 > -Ptests.seed=5B595A296E3623CE -Ptests.file.encoding=US-ASCII > > > I then tried the same tests, but with out the -Ptests.seed, and it > passed! Does this mean there is a bug or not? > > I did go and check > > > http://fucit.org/solr-jenkins-reports/history-trend-of-recent-failures.html#series/org.apache.solr.handler.component.DistributedQueryComponentCustomSortTest.test > > And it appears that this test has been failing more since July. > > Eric > > > > _______________________ > *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 > | http://www.opensourceconnections.com | My Free/Busy > <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless > of whether attachments are marked as such. > > -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
