On July 30, 2024 4:27:48 AM UTC, "Louis-Philippe Véronneau" <po...@debian.org>
wrote:
>On 2024-07-30 12:58, Scott Kitterman wrote:
>>
>>
>> On July 30, 2024 2:47:08 AM UTC, "Louis-Philippe Véronneau"
>> <po...@debian.org> wrote:
>>> On 2024-07-30 11:09, Scott Kitterman wrote:
>>>>
>>>>
>>>> On July 30, 2024 12:49:50 AM UTC, "Louis-Philippe Véronneau"
>>>> <po...@debian.org> wrote:
>>>>> On 2024-07-29 21:07, Scott Kitterman wrote:
>>>>>>
>>>>>>
>>>>>> On July 29, 2024 8:53:11 AM UTC, "Louis-Philippe Véronneau"
>>>>>> <po...@debian.org> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> As discussed during the DebConf24 Python BoF, I'm submitting this
>>>>>>> change to the policy to require the use of the upstream test suite,
>>>>>>> both during the build process and as an autopkgtest.
>>>>>>>
>>>>>>> You can find the MR here:
>>>>>>>
>>>>>>> https://salsa.debian.org/python-team/tools/python-modules/-/merge_requests/24
>>>>>>>
>>>>>>> People present at the BoF seemed to think this was a good idea. If you
>>>>>>> don't please reply to this message and make yourself heard :)
>>>>>>
>>>>>> I understand the theory and why it's a good idea to run the test suite.
>>>>>> I don't think it ought to be a hard requirement. I have several
>>>>>> packages where there's a test suite, but I don't run it:
>>>>>>
>>>>>> 1. The largest set is packages that need test only dependencies which
>>>>>> are not packaged. When I am packaging something new which has a test
>>>>>> suite, then I generally package any needed test depends. If those test
>>>>>> depends also need test depends packaged, I generally stop and don't
>>>>>> enable tests for things that are only in the archives to support tests.
>>>>>> Noseofyeti is an example of this.
>>>>>
>>>>> That sounds like a valid technical reason not to run the tests to me :)
>>>>>
>>>>>> 2. There's at least one case where Debian has customizations that cause
>>>>>> the test suite to fail, but the failures don't seem to cause any real
>>>>>> problems. If anyone wants to make it so the weasyprint test suite works
>>>>>> on Debian, please knock yourself out.
>>>>>
>>>>> Again, as long as you document that, I don't think it would go against
>>>>> the proposed policy change.
>>>>>
>>>>>> 3. I also maintain multiple packages which require network access to
>>>>>> run their test suite, so they can't run tests during build, only
>>>>>> autopkgtests.
>>>>>
>>>>> Same.
>>>>>
>>>>
>>>> Except for #3, I don't get that from the wording in the MR. I don't think
>>>> not worth the trouble is a technical reason. I think the real rule that's
>>>> being proposed is that packages must run the test suit or document why
>>>> not. I don't have a problem with that, but I don't think that's what it
>>>> actually says now. I think if you were to change must to should and
>>>> strike the word technical before reason, it would accomplish the same
>>>> thing and be clearer. I could support that.
>>>
>>> Language is hard and I'm not a native English speaker :)
>>>
>>> What I want to prevent is people not running tests because they don't feel
>>> like it, even though it would not take them a large amount of efforts to do
>>> so.
>>>
>>> I'll strike "technical", as it seems others also interpreted this word they
>>> way you have.
>>>
>>> As for "MUST" vs "SHOULD", I believe the exception clause provides enough
>>> leeway to justify a reasonable way out in case of $VALID_REASON.
>>>
>>> "SHOULD" is not particularly strong and again, I fear it would let people
>>> get away with not running tests although it wouldn't be much work to do
>>> so...
>>>
>>
>> I guess it depends on how you look at the word. In the context of technical
>> specifications, I tend to think of terms as defined in RFC 2119 [1]. I
>> think that's pretty close to what you are suggesting:
>>
>> 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
>> may exist valid reasons in particular circumstances to ignore a
>> particular item, but the full implications must be understood and
>> carefully weighed before choosing a different course.
>>
>> The way RFC 2119 defines MUST doesn't leave much room for exceptions. If
>> you want to be clear, I suggest SHOULD with a reference to RFC 2119. While
>> not universal, it is a widely used reference for definitions of these terms
>> in technical specifications.
>>
>> Scott K
>>
>> [1] https://datatracker.ietf.org/doc/html/rfc2119#section-3
>>
>
>Fair enough. I also made that change.
Thanks,
Scott K
