+1. Ability to run tests in Github actions should help prevent a ton of build breakages.
Thanks for leading this, Houston! On Fri, 18 Sep 2020 at 21:24, Houston Putman <houstonput...@gmail.com> wrote: > Good point on the reference_impl branch. Eventually that's the goal, but > given there's not a timeline for that to be merged yet I think this is a > good stop-gap. It's a few minutes of work to get these PR actions written, > so I feel like there is little downside. And we can always remove them when > the reference_imp branch gets merged. > > This may block my ability to merge any PR whatsoever. > > > The docker tests will not be run on any PRs that don't touch bin/solr, > solr/packaging or solr/docker. > > There are complications around running integration tests in a > non-containerized environment as well. And if the docker image tests are > failing on the PR, wouldn't you rather know before committing? Even if you > don't want to install docker, you can call in someone else that has it to > help debug. Much like PRs that affect solr.cmd, for committers without > access to a windows machine. > > On Thu, Sep 17, 2020 at 6:23 PM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> > It would be great to run all the tests every time, but clearly that is >> too expensive. >> >> The reference_impl branch requires around 30 seconds to run all solr-core >> tests. That's where we should all put our collective efforts. >> Also, I have reservations against docker based tests blocking PRs. If I >> don't have docker running on my dev machine, I wouldn't be able to make >> those tests pass. This may block my ability to merge any PR whatsoever. >> Why can't we have integration tests that do not rely on docker? >> >> On Thu, Sep 17, 2020 at 9:26 PM Houston Putman <houstonput...@gmail.com> >> wrote: >> >>> Thought I'd make this a thread instead of a discussion on a single JIRA >>> ticket. >>> >>> Currently we have gradle precommit run on PRs for master, which is very >>> useful and gives people confidence in approving PRs. But precommit is >>> obviously not the only thing we care about before committing. It would be >>> great to run all the tests every time, but clearly that is too expensive. >>> >>> In SOLR-14856 <https://issues.apache.org/jira/browse/SOLR-14856>, I >>> proposed adding a github action to build and test the solr docker image for >>> PRs that affected relevant parts of the repo (solr/docker, solr/bin, >>> solr/packaging and solr/contrib/prometheus-exporter/bin). Running the >>> docker tests currently takes roughly 12 minutes in the github action, which >>> would be costly if it ran on every PR. But when running on the small >>> percentage of PRs that affect those code paths, I think the benefit >>> outweighs the cost. >>> >>> Beyond just the docker tests, I think we can leverage this ability for >>> other features that are limited to certain code paths. For example running >>> tests for contrib modules, testing solr/examples, and many of >>> the independent lucene modules. The SolrJ tests just ran in 3 minutes >>> locally for me, maybe that'd be a good candidate as well. >>> >>> Anyways I'm sure there are other good candidates out there, but I just >>> wanted to start the discussion and hear other opinions before diving any >>> deeper. >>> >>> >>> >> >> > > -- Regards, Atri Apache Concerted
