We’re kind of in limbo at this point. Mark, Ishan and Noble are pushing (along
with others) to make a much faster test suite, the “reference_impl”. Once
that’s done, and assuming it live up to expectations, it should then be more
reasonable to run all the test for each PR. We’ll have to wait a
Hi everyone,
I was wondering why we don't run every test on every PR.
I saw the earlier comment which said it was too expensive but it would be
good if we had that feature.
As an experiment I ran all tests in a Github PR action here
Thanks for the feedback everyone!
I'll get us started off with the docker and SolrJ tests. We can revisit
after a few weeks and see how it has gone.
- Houston
On Fri, Sep 18, 2020 at 2:28 PM Atri Sharma wrote:
> +1.
>
> Ability to run tests in Github actions should help prevent a ton of build
+1.
Ability to run tests in Github actions should help prevent a ton of build
breakages.
Thanks for leading this, Houston!
On Fri, 18 Sep 2020 at 21:24, Houston Putman
wrote:
> Good point on the reference_impl branch. Eventually that's the goal, but
> given there's not a timeline for that to
> The docker tests will not be run on any PRs that don't touch bin/solr,
solr/packaging or solr/docker.
Sounds good, then!
On Fri, Sep 18, 2020 at 11:44 PM Anshum Gupta
wrote:
> I like the idea as I really feel Github actions provide a ton of value.
>
> It doesn't have to be a blocker for all
I like the idea as I really feel Github actions provide a ton of value.
It doesn't have to be a blocker for all cases but for it to just run would
be great. We don't really lose anything there.
On Thu, Sep 17, 2020 at 8:56 AM Houston Putman
wrote:
> Thought I'd make this a thread instead of a
Yeah I'm definitely in favor of adding tests to GitHub actions. Even if
the action reports a failure, presumably we could opt to commit anyway if,
for example, we know that the test that failed is totally unrelated to the
work being committed and/or the failing test is related to some other known
I think this is a good idea. In general, I'm +1 on improving PR validations
as much as possible, and as Houston says, we can always remove them later
if it's not helping. I also agree with David in his Jira comment that even
more important than this is to have the tests running on Jenkins, but I
+1 to not depending on Docker for local tests.
I do not wish to derail this thread — but re: reference branch, doesn’t it
have a bunch of tests disabled?
On Fri, 18 Sep 2020 at 03:53, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:
> > It would be great to run all the tests every time,
Good point on the reference_impl branch. Eventually that's the goal, but
given there's not a timeline for that to be merged yet I think this is a
good stop-gap. It's a few minutes of work to get these PR actions written,
so I feel like there is little downside. And we can always remove them when
> It would be great to run all the tests every time, but clearly that is
too expensive.
The reference_impl branch requires around 30 seconds to run all solr-core
tests. That's where we should all put our collective efforts.
Also, I have reservations against docker based tests blocking PRs. If I
Thought I'd make this a thread instead of a discussion on a single JIRA
ticket.
Currently we have gradle precommit run on PRs for master, which is very
useful and gives people confidence in approving PRs. But precommit is
obviously not the only thing we care about before committing. It would be
12 matches
Mail list logo
