(sorry for writing so much -- not AI slop but 100% me here)
The tests in solr/packaging/tests[1] were originally for testing
Solr's bin / shell scripts. Since then, its scope has grown in scope
to tests related to Jetty installation/configuration/behavior and
probably other things that don't have to do with bin/shell scripts
[4]. These tests would be less trustworthy, if even possible, when
emulated in JUnit, so they make sense as BATS. There's also an unusual
one -- a SolrCloud multi-version rolling upgrade test (using
Docker).[3] The GHA workflow[2] has naming and path activation
reflective of it's original scope but not it's expanded scope.
note: test_rolling_upgrade.bats is slow & resource intensive -- it has
been the cause of some GHA flaky test runs. *If* it were a JUnit test
(which I don't want BTW), we clearly would have labelled it as
@Nightly but we have no similar mechanism for BATS. Also, note this
isn't located in solr/docker/tests because Docker isn't the *point* of
the test, only a practical choice of how to implement the
compatibility test goal.
We could do a number of things, **not mutually exclusive**:
(A) Rebrand the entire suite as either "testIntegration" or "testBats"
(verb first, more consistent with other Gradle commands, noting that
testDocker exists too). Update the GHA workflow file & title / naming
accordingly. Same for Jenkins jobs if needed. Update the GHA
activation paths to be any Jetty configuration.
(C) use a naming convention to identify the nature of the bats test?
Or use subfolders to identify the nature of the bats test? "scripts"
vs "integration"? Or don't bother.
(D) if BATS infra was made generic and easy to add to any module that
wishes to have them ("packaging, "docker", "server") then we could
move the ones relating to Jetty into "server".
Side topic: solr/docker/tests doesn't use BATS but clearly should be
ported to that framework
RE test_rolling_upgrade:
(R) *somehow*(?) disable test_rolling_upgrade.bats from being run
under normal circumstances. (R.1) The Jenkins Nightly job could add a
step to run it. But could flaky tests would likely be overlooked.
(R.2) The Jenkins Smoketester job could add a step to run it. Or
better -- expand The Smoketester script itself to do so.
(R.3) The Jenkins Docker job could add a step to run it. Or move &
port the test to solr/docker/tests. Note: HOWEVER, this test's
fundamental purpose isn't to test Docker!
References:
[1]
https://github.com/apache/solr/blob/4f40e977a337d440c598b6f37156045e5bd8a926/solr/packaging/test
[2]
https://github.com/apache/solr/blob/f5da9a62d4c66a25289b340011b852be61693977/.github/workflows/bin-solr-test.yml
[3]
https://github.com/apache/solr/blob/4f40e977a337d440c598b6f37156045e5bd8a926/solr/packaging/test/test_rolling_upgrade.bats
[4] Tests not for bin/solr shell scripts
- test_extraction.bats - Tests Tika extraction functionality with
ExtractingRequestHandler and Docker-based Tika server
- test_opennlp.bats - Tests OpenNLP sentiment classification and NLP
features (described as exploratory, not a "true" integration test)
- test_modules.bats - Tests SQL module functionality (uses CLI to
start Solr, but tests the SQL module feature)
- test_placement_plugin.bats - Tests affinity/random placement
plugin functionality
- test_security_manager.bats - Tests security manager and allowPaths
functionality
- test_adminconsole_urls.bats - Tests that the admin console web UI
is accessible
... coming soon, a test_compression.bats
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
