Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-04-06 Thread Robin Chan
Thank you David & Brian on driving this plan forward. I am happy we will release a Pulp 3.0 Core Beta with a known % coverage for unit tests, a policy in place and mechanisms in place to ensure that number only goes up. In the future, I would like for us to flush out more the definition of valuab

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-04-06 Thread Brian Bouterse
There was a lot of +1 support on the PR. The basic Pulp3 unit test policy can be seen here: https://docs.pulpproject.org/en/3.0/nightly/contributing/unit_tests.html On Thu, Apr 5, 2018 at 1:06 PM, Brian Bouterse wrote: > This is a wrap-up update with the last next steps (for now) on the Pulp3 >

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-04-05 Thread Brian Bouterse
This is a wrap-up update with the last next steps (for now) on the Pulp3 unit test discussion. 1. Here is a docs PR adopting simple but specific policy language recommending unit tests for Pulp3: https://github.com/pulp/pulp/pull/3411 2. We need to move our existing unit tests into their "forever

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-26 Thread Brian Bouterse
I want to summarize what I've heard to facilitate some next steps and further discussion. There seems to be broad support and no -1 votes to the idea of a soft check that tracks unit test coverage, so we wanted to get that out of the way. Daviddavis enabled unit test coverage reporting for all Pul

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-26 Thread Jeremy Audet
> I'm also generally -1 against trying to pick a number (100%, 80%, 60%) > up-front. We should unit test what makes sense to unit test, push that > number as high as reasonable, and otherwise focus on pulp-smash, which I > think has historically been more useful. QE is flattered by the focus o

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-26 Thread Daniel Alley
We would still block on failing tests, yes. I'm also -1 blocking on coverage, and -1 against attempting 100%. I'm also generally -1 against trying to pick a number (100%, 80%, 60%) up-front. We should unit test what makes sense to unit test, push that number as high as reasonable, and otherwise

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-26 Thread Bryan Kearney
I assume you would still gate (hard fail) if unit tests fail? -- bk On 03/26/2018 05:55 AM, Ina Panova wrote: -1 for hard check, -1 for 100% coverage. Unittests are good but integration tests are better. I totally agree with what Austin said. We should add tests where it makes sense. +1 soft

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-26 Thread Ina Panova
-1 for hard check, -1 for 100% coverage. Unittests are good but integration tests are better. I totally agree with what Austin said. We should add tests where it makes sense. +1 soft check. I would not like finding myself banging my head [0] (just because of 100% coverage policy) against one line

Re: [Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-23 Thread Austin Macdonald
-1 For a blocking check, -1 for attempting 100% coverage. There is a *lot* of code in Pulp 3 that simply involves inheriting from parents classes and defining attributes. If we add a ViewSet for instance, there is nothing to test other than "asserting that we did what we did". I have written lots

[Pulp-dev] Pulp 3 Unit Test Plan Proposal

2018-03-23 Thread David Davis
We haven't had a unit test policy in Pulp 3, and the community and core committers would all like one. The general desire we've heard so far is to change course and encourage developers to add unit tests to their changes to Pulp 3. The policy we're suggesting is to add a coveralls[0] check for Pul