On Sat, 13 Jan 2018 22:28:29 +0900 Akinobu Mita <akinobu.m...@gmail.com> wrote:
> 2018-01-13 2:56 GMT+09:00 Masami Hiramatsu <mhira...@kernel.org>: > > Support in-kernel fault-injection framework via debugfs. > > This allows you to inject a conditional error to specified > > function using debugfs interfaces. > > > > Here is the result of test script described in > > Documentation/fault-injection/fault-injection.txt > > > > =========== > > # ./test_fail_function.sh > > 1+0 records in > > 1+0 records out > > 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227404 s, 46.1 MB/s > > btrfs-progs v4.4 > > See http://btrfs.wiki.kernel.org for more information. > > > > Label: (null) > > UUID: bfa96010-12e9-4360-aed0-42eec7af5798 > > Node size: 16384 > > Sector size: 4096 > > Filesystem size: 1001.00MiB > > Block group profiles: > > Data: single 8.00MiB > > Metadata: DUP 58.00MiB > > System: DUP 12.00MiB > > SSD detected: no > > Incompat features: extref, skinny-metadata > > Number of devices: 1 > > Devices: > > ID SIZE PATH > > 1 1001.00MiB /dev/loop2 > > > > mount: mount /dev/loop2 on /opt/tmpmnt failed: Cannot allocate memory > > SUCCESS! > > =========== > > > > > > Signed-off-by: Masami Hiramatsu <mhira...@kernel.org> > > Reviewed-by: Josef Bacik <jba...@fb.com> > > --- > > Changes in v3: > > - Check and adjust error value for each target function > > - Clear kporbe flag for reuse > > - Add more documents and example > > Changes in v5: > > - Support multi-function error injection > > --- > > Documentation/fault-injection/fault-injection.txt | 68 ++++ > > kernel/Makefile | 1 > > kernel/fail_function.c | 349 > > +++++++++++++++++++++ > > lib/Kconfig.debug | 10 + > > 4 files changed, 428 insertions(+) > > create mode 100644 kernel/fail_function.c > > > > diff --git a/Documentation/fault-injection/fault-injection.txt > > b/Documentation/fault-injection/fault-injection.txt > > index 918972babcd8..f4a32463ca48 100644 > > --- a/Documentation/fault-injection/fault-injection.txt > > +++ b/Documentation/fault-injection/fault-injection.txt > > @@ -30,6 +30,12 @@ o fail_mmc_request > > injects MMC data errors on devices permitted by setting > > debugfs entries under /sys/kernel/debug/mmc0/fail_mmc_request > > > > +o fail_function > > + > > + injects error return on specific functions, which are marked by > > + ALLOW_ERROR_INJECTION() macro, by setting debugfs entries > > + under /sys/kernel/debug/fail_function. No boot option supported. > > + > > Configure fault-injection capabilities behavior > > ----------------------------------------------- > > > > @@ -123,6 +129,29 @@ configuration of fault-injection capabilities. > > default is 'N', setting it to 'Y' will disable failure injections > > when dealing with private (address space) futexes. > > > > +- /sys/kernel/debug/fail_function/inject: > > + > > + Format: { 'function-name' | '!function-name' | '' } > > + specifies the target function of error injection by name. > > + If the function name leads '!' prefix, given function is > > + removed from injection list. If nothing specified ('') > > + injection list is cleared. > > + > > +- /sys/kernel/debug/fail_function/injectable: > > + > > + (read only) shows error injectable functions and what type of > > + error values can be specified. The error type will be one of > > + below; > > + - NULL: retval must be 0. > > + - ERRNO: retval must be -1 to -MAX_ERRNO (-4096). > > + - ERR_NULL: retval must be 0 or -1 to -MAX_ERRNO (-4096). > > + > > +- /sys/kernel/debug/fail_function/<functiuon-name>/retval: > > + > > + specifies the "error" return value to inject to the given > > + function for given function. This will be created when > > + user specifies new injection entry. > > + > > o Boot option > > > > In order to inject faults while debugfs is not available (early boot time), > > @@ -268,6 +297,45 @@ trap "echo 0 > > > /sys/kernel/debug/$FAILTYPE/probability" SIGINT SIGTERM EXIT > > echo "Injecting errors into the module $module... (interrupt to stop)" > > sleep 1000000 > > > > +------------------------------------------------------------------------------ > > + > > +o Inject open_ctree error while btrfs mount > > + > > +#!/bin/bash > > + > > +rm -f testfile.img > > +dd if=/dev/zero of=testfile.img bs=1M seek=1000 count=1 > > +DEVICE=$(losetup --show -f testfile.img) > > +mkfs.btrfs -f $DEVICE > > +mkdir -p tmpmnt > > + > > +FAILTYPE=fail_function > > +FAILFUNC=open_ctree > > +echo $FAILFUNC > /sys/kernel/debug/$FAILTYPE/inject > > +echo -12 > /sys/kernel/debug/$FAILTYPE/$FAILFUNC/retval > > +echo N > /sys/kernel/debug/$FAILTYPE/task-filter > > +echo 100 > /sys/kernel/debug/$FAILTYPE/probability > > +echo 0 > /sys/kernel/debug/$FAILTYPE/interval > > +echo -1 > /sys/kernel/debug/$FAILTYPE/times > > +echo 0 > /sys/kernel/debug/$FAILTYPE/space > > +echo 1 > /sys/kernel/debug/$FAILTYPE/verbose > > I expected that the fault_attr is created for each target function. > (i.e. /sys/kernel/debug/fail_function/<func>/ directory contains not > only 'retval', but also 'probability', 'interval', ...) > > Or is there any good reason to share a single fault_attr for all > error injected functions? Good question. I actually considered that. But since fault-attr only considers no relationship between 2 fault points (in time too), this means each time should_fail() will be done as independent trial. So, in most case user will just copy the setting for those 2 points. (another reason is not to change the document so much :), all interface are still under /sys/kernel/debug/fail_*/ ...) Thank you, -- Masami Hiramatsu <mhira...@kernel.org>