On Wed, 25 Jun 2025 at 17:55, Sasha Levin <sas...@kernel.org> wrote: > > On Wed, Jun 25, 2025 at 10:52:46AM +0200, Dmitry Vyukov wrote: > >On Tue, 24 Jun 2025 at 22:04, Sasha Levin <sas...@kernel.org> wrote: > > > >> >6. What's the goal of validation of the input arguments? > >> >Kernel code must do this validation anyway, right. > >> >Any non-trivial validation is hard, e.g. even for open the validation > >> >function > >> >for file name would need to have access to flags and check file precense > >> >for > >> >some flags combinations. That may add significant amount of non-trivial > >> >code > >> >that duplicates main syscall logic, and that logic may also have bugs and > >> >memory leaks. > >> > >> Mostly to catch divergence from the spec: think of a scenario where > >> someone added a new param/flag/etc but forgot to update the spec - this > >> will help catch it. > > > >How exactly is this supposed to work? > >Even if we run with a unit test suite, a test suite may include some > >incorrect inputs to check for error conditions. The framework will > >report violations on these incorrect inputs. These are not bugs in the > >API specifications, nor in the test suite (read false positives). > > Right now it would be something along the lines of the test checking for > an expected failure message in dmesg, something along the lines of: > > > https://github.com/linux-test-project/ltp/blob/0c99c7915f029d32de893b15b0a213ff3de210af/testcases/commands/sysctl/sysctl02.sh#L67 > > I'm not opposed to coming up with a better story...
Oh, you mean special tests for this framework (rather than existing tests). I don't think this is going to work in practice. Besides writing all these specifications, we will also need to write dozens of tests per each specification (e.g. for each fd arg one needs at least 3 tests: -1, valid fd, inclid fd; an enum may need 5 various inputs of something; let alone netlink specifications).
