dsmiley commented on PR #4280: URL: https://github.com/apache/solr/pull/4280#issuecomment-4236821552
A failure here only needs to be caught at release time, so I'm okay with the risk of us collectively not noticing till then. Even in saying that, I'm implying that absolutely nobody ever reviews the state of our jenkins jobs. While somewhat true, and a freaking crying shame, I think that should be solved independently out of scope of this conversation. @iamsanjay and I are discussing ideas to address that. I don't think we should expect GHA/PR to be our total coverage of tests, albeit certainly the vast majority -- yes. > How could we make the test faster? It might be interesting to try and understand why it feels slow... Maybe identify some things we need to work on? We'll always have some tests that are not fast so we need to deal with it. Personally I'm not interested in optimizing this specific test further but go for it if you are so inclined. > What about moving it to run as part of the Docker tests? That way we could test rolling upgrade against a freshly built Docker image which might be useful. Did you see my [dev-list thread](https://lists.apache.org/thread/8jz3b2n0wlc7ldjzk8mhlskpsm52xs6l)?: For this rolling upgrade test, Docker isn't the point but is a practical implementation choice. > We are moving towards more Docker based testing infrastructure, like for Tika. Maybe we just declare that Docker is required for the iTest? Shrug; I'm not moving this out because it uses Docker. I'm moving it out both because it's expensive and because a failure here is very unlikely and it's acceptible to be caught at release time. But it's very important for an eventual failure here to be noticed by us before a release is made. I think "smoke" fits the bill. I was looking at our smoke tester, and I wonder if much of it could be converted to bats tests. So that also was a factor for me choosing to create a new "testSmokeBats" target & directory, even though it only has one test there now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
