Thanks for the feedback everyone! I'll get us started off with the docker and SolrJ tests. We can revisit after a few weeks and see how it has gone.
- Houston On Fri, Sep 18, 2020 at 2:28 PM Atri Sharma <a...@apache.org> wrote: > +1. > > Ability to run tests in Github actions should help prevent a ton of build > breakages. > > Thanks for leading this, Houston! > > On Fri, 18 Sep 2020 at 21:24, Houston Putman <houstonput...@gmail.com> > wrote: > >> Good point on the reference_impl branch. Eventually that's the goal, but >> given there's not a timeline for that to be merged yet I think this is a >> good stop-gap. It's a few minutes of work to get these PR actions written, >> so I feel like there is little downside. And we can always remove them when >> the reference_imp branch gets merged. >> >> This may block my ability to merge any PR whatsoever. >> >> >> The docker tests will not be run on any PRs that don't touch bin/solr, >> solr/packaging or solr/docker. >> >> There are complications around running integration tests in a >> non-containerized environment as well. And if the docker image tests are >> failing on the PR, wouldn't you rather know before committing? Even if you >> don't want to install docker, you can call in someone else that has it to >> help debug. Much like PRs that affect solr.cmd, for committers without >> access to a windows machine. >> >> On Thu, Sep 17, 2020 at 6:23 PM Ishan Chattopadhyaya < >> ichattopadhy...@gmail.com> wrote: >> >>> > It would be great to run all the tests every time, but clearly that is >>> too expensive. >>> >>> The reference_impl branch requires around 30 seconds to run all >>> solr-core tests. That's where we should all put our collective efforts. >>> Also, I have reservations against docker based tests blocking PRs. If I >>> don't have docker running on my dev machine, I wouldn't be able to make >>> those tests pass. This may block my ability to merge any PR whatsoever. >>> Why can't we have integration tests that do not rely on docker? >>> >>> On Thu, Sep 17, 2020 at 9:26 PM Houston Putman <houstonput...@gmail.com> >>> wrote: >>> >>>> Thought I'd make this a thread instead of a discussion on a single JIRA >>>> ticket. >>>> >>>> Currently we have gradle precommit run on PRs for master, which is very >>>> useful and gives people confidence in approving PRs. But precommit is >>>> obviously not the only thing we care about before committing. It would be >>>> great to run all the tests every time, but clearly that is too expensive. >>>> >>>> In SOLR-14856 <https://issues.apache.org/jira/browse/SOLR-14856>, I >>>> proposed adding a github action to build and test the solr docker image for >>>> PRs that affected relevant parts of the repo (solr/docker, solr/bin, >>>> solr/packaging and solr/contrib/prometheus-exporter/bin). Running the >>>> docker tests currently takes roughly 12 minutes in the github action, which >>>> would be costly if it ran on every PR. But when running on the small >>>> percentage of PRs that affect those code paths, I think the benefit >>>> outweighs the cost. >>>> >>>> Beyond just the docker tests, I think we can leverage this ability for >>>> other features that are limited to certain code paths. For example running >>>> tests for contrib modules, testing solr/examples, and many of >>>> the independent lucene modules. The SolrJ tests just ran in 3 minutes >>>> locally for me, maybe that'd be a good candidate as well. >>>> >>>> Anyways I'm sure there are other good candidates out there, but I just >>>> wanted to start the discussion and hear other opinions before diving any >>>> deeper. >>>> >>>> >>>> >>> >>> >> >> -- > Regards, > > Atri > Apache Concerted >
