On Thu, 26 Jun 2025 at 10:32, Dmitry Vyukov <dvyu...@google.com> wrote: > > On Wed, 25 Jun 2025 at 17:55, Sasha Levin <sas...@kernel.org> wrote: > > > > On Wed, Jun 25, 2025 at 10:52:46AM +0200, Dmitry Vyukov wrote: > > >On Tue, 24 Jun 2025 at 22:04, Sasha Levin <sas...@kernel.org> wrote: > > > > > >> >6. What's the goal of validation of the input arguments? > > >> >Kernel code must do this validation anyway, right. > > >> >Any non-trivial validation is hard, e.g. even for open the validation > > >> >function > > >> >for file name would need to have access to flags and check file > > >> >precense for > > >> >some flags combinations. That may add significant amount of non-trivial > > >> >code > > >> >that duplicates main syscall logic, and that logic may also have bugs > > >> >and > > >> >memory leaks. > > >> > > >> Mostly to catch divergence from the spec: think of a scenario where > > >> someone added a new param/flag/etc but forgot to update the spec - this > > >> will help catch it. > > > > > >How exactly is this supposed to work? > > >Even if we run with a unit test suite, a test suite may include some > > >incorrect inputs to check for error conditions. The framework will > > >report violations on these incorrect inputs. These are not bugs in the > > >API specifications, nor in the test suite (read false positives). > > > > Right now it would be something along the lines of the test checking for > > an expected failure message in dmesg, something along the lines of: > > > > > > https://github.com/linux-test-project/ltp/blob/0c99c7915f029d32de893b15b0a213ff3de210af/testcases/commands/sysctl/sysctl02.sh#L67 > > > > I'm not opposed to coming up with a better story...
If the goal of validation is just indirectly validating correctness of the specification itself, then I would look for other ways of validating correctness of the spec. Either removing duplication between specification and actual code (i.e. generating it from SYSCALL_DEFINE, or the other way around) , then spec is correct by construction. Or, cross-validating it with info automatically extracted from the source (using clang/dwarf/pahole). This would be more scalable (O(1) work, rather than thousands more manually written tests). > Oh, you mean special tests for this framework (rather than existing tests). > I don't think this is going to work in practice. Besides writing all > these specifications, we will also need to write dozens of tests per > each specification (e.g. for each fd arg one needs at least 3 tests: > -1, valid fd, inclid fd; an enum may need 5 various inputs of > something; let alone netlink specifications).
