On Fri, Sep 5, 2014 at 4:27 PM, Daniel Manrique <[email protected]> wrote: > I'm looking back at the tests we had in old checkbox :) > > We even had some tests for packaging and setup, but I don't think > those would apply to plainbox. > > - Test that all the job (.txt.in) files are declared in setup.cfg
Correct me if I'm wrong but we don't have anything to do here. We don't have setup.cfg's anymore > - Test that all the job files are added to POTFILES.in This is a bit harder to validate. I'll think about what to do to address it. One thing I wanted to add was a automatic FileUnit so that we can advise on the new file extension and deprecate .txt and .txt.in. If this goes far enough we can actually generate POTFILES.in ourselves and cut the need to validate but I'm not sure if we really should always generate it (are there corner cases that we're missing if we do) > - Test that all the job files are included by local.txt.in (to avoid > non-categorized jobs, plus checkbox wouldn't see them otherwise). We'll start recommending category_id soon. > For the message/job files themselves: > > - Test all jobs have name (id now) Check and far more :-) > - Test all jobs have either a command or a description (this may be > obsolete now) Check and far more :-) > - Test all shell jobs have a description Actually we check that all manual jobs have a description and recommend having a description in general. > - Test all shell jobs with user: root have a proper environ for any > env variables referenced in their command: We're currently not checking that. It's blocked by heredoc-aware parser. I'll open a bug on that. [https://bugs.launchpad.net/plainbox/+bug/1366192] > - Test jobs comply with schema (it's to ensure they don't contain any > unknown attributes but since plainbox is more free-form this is not > needed now) That's true but I think we will have some way to improve the situation. First we cannot catch typos this way. We should still look at all the unknown fields to see if they are fishy. Second, with the provider unit we can statically declare which version of the de-facto schema we're using and strictly validate that schema if it is known to plainbox. If the schema refers to a more recent version than plainbox (old copy of) supports we can disable this feature. > - Test user-interact and user-verify jobs have a command Check and more :-) > - Test manual, user-verify and user-interact jobs have a parsable > description (again, not needed since we don't do the > PURPOSE/STEPS/VERIFICATION parsing stuff anymore). We don't check that. > A lot of them are obsolete now but it may be a good guideline on what > to test for. Most of them were inspired by actual problems we > encountered in the field, so the tests were added to save some time. Thanks Sending CC to the mailing list for reference ZK -- Mailing list: https://launchpad.net/~checkbox-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~checkbox-dev More help : https://help.launchpad.net/ListHelp
