janhoy commented on PR #4703: URL: https://github.com/apache/solr/pull/4703#issuecomment-5190025000
> Wouldn't we start with a slow test and see what that test needs to be better This PR is backward on purpose. I did not attempt to delve into a formal process of assessing and improving the slowest tests (like I did a few months ago). Rather I decided to attempt LLM assisted test improvement along two axis - this one where mocks could help simplify complex failure scenarios (which is one good use of mocks). And the other where we spin up the entire sol(a)r system to test something that won't need a big cluster. So from a random set of 50 test classes I found two candidates. Which is a hint that we may have perhaps hundreds of such opportunities across the codebase. I believe we have grown numb to writing full integration tests for every single use case. Integration tests has their place and we should not stop writing them. But we should always go with lower level tests first. I'm hoping we can get to a place some day where `gradle test` runs in a few minutes and has great coverage and stability for everyday commit work. While invoking integration tests can take longer (but we should strive to keep it way below the hour) and be invoked in nightly, before releases and conditionally in PRs depending on what code is touched. Take ExtractingRequestHandler as an example. Currently we spin up a real TikaServer with Testcontainers to test the component. This is integration testing and should ideally be opt-in. For everyday test runs it would be sufficient to mock `TikaServerExtractionBackend.callTikaServer()` to test the key logic without a real Tika server. I did not do that but probably should have to shave some 10-15s off of every normal test run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
