Hi Sławomir and Morgan, Thanks for this interesting discussion.
Sławomir Grochowski <[email protected]> writes: > I do see the tension with what Ihor wrote in that thread, and with the > paragraph he added to testing/README in e1ef98202: tests in Org are > sometimes the only reference for how the code *should* behave, so a > passing assertion of awkward behavior can be misread as "this oddity is > intentional". That is a fair worry. Where I would push back gently is > that the worry is about how the test *reads*, not about whether the > knowledge it captures is worth having -- and reading can be fixed with a > comment or docstring ("this pins current behavior that surprised the > author; see <thread>; the desired behavior is X"), in a way that > ":expected-result :failed" cannot, because :failed silently drops out of > a green run and gives the next reader nothing to grab onto. Tests either succeed or fail based on the expected behaviour. I can see how characterisation "tests" could be useful for mapping the existing behaviours, but I find it confusing to call them "tests". I'd call them "records" because they just record a behavior. Adding characterisation records that describe the current behaviour would probably create a maintenance burden because you would end up maintaining code that is not so useful: failed characterisation "tests" do not indicate what needs to be fixed (the behaviour or the test). I'm not saying it's completely useless, I see your point, but the usefulness-to-maintenance ratio seems too low to me. So I agree with the approach from testing/README: "The tests are usually designed aiming to ensure the *expected* Org mode behavior." Also, we had a test suite that we used to run every few hours against main/maint, and we stopped it because it is unmaintained - if someone wants to maintain this, we can set it up again. -- Bastien
