I think this is a good idea. In general, I'm +1 on improving PR validations as much as possible, and as Houston says, we can always remove them later if it's not helping. I also agree with David in his Jira comment that even more important than this is to have the tests running on Jenkins, but I don't see why we can't have both.
Regards, Tomás On Fri, Sep 18, 2020 at 9:05 AM Atri Sharma <a...@apache.org> wrote: > +1 to not depending on Docker for local tests. > > I do not wish to derail this thread — but re: reference branch, doesn’t it > have a bunch of tests disabled? > > On Fri, 18 Sep 2020 at 03:53, Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> > It would be great to run all the tests every time, but clearly that is >> too expensive. >> >> The reference_impl branch requires around 30 seconds to run all solr-core >> tests. That's where we should all put our collective efforts. >> Also, I have reservations against docker based tests blocking PRs. If I >> don't have docker running on my dev machine, I wouldn't be able to make >> those tests pass. This may block my ability to merge any PR whatsoever. >> Why can't we have integration tests that do not rely on docker? >> >> On Thu, Sep 17, 2020 at 9:26 PM Houston Putman <houstonput...@gmail.com> >> wrote: >> >>> Thought I'd make this a thread instead of a discussion on a single JIRA >>> ticket. >>> >>> Currently we have gradle precommit run on PRs for master, which is very >>> useful and gives people confidence in approving PRs. But precommit is >>> obviously not the only thing we care about before committing. It would be >>> great to run all the tests every time, but clearly that is too expensive. >>> >>> In SOLR-14856 <https://issues.apache.org/jira/browse/SOLR-14856>, I >>> proposed adding a github action to build and test the solr docker image for >>> PRs that affected relevant parts of the repo (solr/docker, solr/bin, >>> solr/packaging and solr/contrib/prometheus-exporter/bin). Running the >>> docker tests currently takes roughly 12 minutes in the github action, which >>> would be costly if it ran on every PR. But when running on the small >>> percentage of PRs that affect those code paths, I think the benefit >>> outweighs the cost. >>> >>> Beyond just the docker tests, I think we can leverage this ability for >>> other features that are limited to certain code paths. For example running >>> tests for contrib modules, testing solr/examples, and many of >>> the independent lucene modules. The SolrJ tests just ran in 3 minutes >>> locally for me, maybe that'd be a good candidate as well. >>> >>> Anyways I'm sure there are other good candidates out there, but I just >>> wanted to start the discussion and hear other opinions before diving any >>> deeper. >>> >>> >>> >> >> -- > Regards, > > Atri > Apache Concerted >
