Yeah I'm definitely in favor of adding tests to GitHub actions. Even if the action reports a failure, presumably we could opt to commit anyway if, for example, we know that the test that failed is totally unrelated to the work being committed and/or the failing test is related to some other known issue.
~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Fri, Sep 18, 2020 at 1:00 PM Tomás Fernández Löbbe <tomasflo...@gmail.com> wrote: > I think this is a good idea. In general, I'm +1 on improving PR > validations as much as possible, and as Houston says, we can always remove > them later if it's not helping. I also agree with David in his Jira comment > that even more important than this is to have the tests running on Jenkins, > but I don't see why we can't have both. > > Regards, > > Tomás > > On Fri, Sep 18, 2020 at 9:05 AM Atri Sharma <a...@apache.org> wrote: > >> +1 to not depending on Docker for local tests. >> >> I do not wish to derail this thread — but re: reference branch, doesn’t >> it have a bunch of tests disabled? >> >> On Fri, 18 Sep 2020 at 03:53, Ishan Chattopadhyaya < >> ichattopadhy...@gmail.com> wrote: >> >>> > It would be great to run all the tests every time, but clearly that is >>> too expensive. >>> >>> The reference_impl branch requires around 30 seconds to run all >>> solr-core tests. That's where we should all put our collective efforts. >>> Also, I have reservations against docker based tests blocking PRs. If I >>> don't have docker running on my dev machine, I wouldn't be able to make >>> those tests pass. This may block my ability to merge any PR whatsoever. >>> Why can't we have integration tests that do not rely on docker? >>> >>> On Thu, Sep 17, 2020 at 9:26 PM Houston Putman <houstonput...@gmail.com> >>> wrote: >>> >>>> Thought I'd make this a thread instead of a discussion on a single JIRA >>>> ticket. >>>> >>>> Currently we have gradle precommit run on PRs for master, which is very >>>> useful and gives people confidence in approving PRs. But precommit is >>>> obviously not the only thing we care about before committing. It would be >>>> great to run all the tests every time, but clearly that is too expensive. >>>> >>>> In SOLR-14856 <https://issues.apache.org/jira/browse/SOLR-14856>, I >>>> proposed adding a github action to build and test the solr docker image for >>>> PRs that affected relevant parts of the repo (solr/docker, solr/bin, >>>> solr/packaging and solr/contrib/prometheus-exporter/bin). Running the >>>> docker tests currently takes roughly 12 minutes in the github action, which >>>> would be costly if it ran on every PR. But when running on the small >>>> percentage of PRs that affect those code paths, I think the benefit >>>> outweighs the cost. >>>> >>>> Beyond just the docker tests, I think we can leverage this ability for >>>> other features that are limited to certain code paths. For example running >>>> tests for contrib modules, testing solr/examples, and many of >>>> the independent lucene modules. The SolrJ tests just ran in 3 minutes >>>> locally for me, maybe that'd be a good candidate as well. >>>> >>>> Anyways I'm sure there are other good candidates out there, but I just >>>> wanted to start the discussion and hear other opinions before diving any >>>> deeper. >>>> >>>> >>>> >>> >>> -- >> Regards, >> >> Atri >> Apache Concerted >> >
