Hello, This series removes fork-based fuzzing. How does fork-based fuzzing work? * A single parent process initializes QEMU * We identify the devices we wish to fuzz (fuzzer-dependent) * Use QTest to PCI enumerate the devices * After that we start a fork-server which forks the process and executes fuzzer inputs inside the disposable children.
In a normal fuzzing process, everything happens in a single process. Pros of fork-based fuzzing: * We only need to do common configuration once (e.g. PCI enumeration). * Fork provides a strong guarantee that fuzzer inputs will not interfere with each-other * The fuzzing process can continue even after a child-process crashes * We can apply our-own timers to child-processes to exit slow inputs, early Cons of fork-based fuzzing: * Fork-based fuzzing is not supported by libfuzzer. We had to build our own fork-server and rely on tricks using linker-scripts and shared-memory to support fuzzing. ( https://physics.bu.edu/~alxndr/libfuzzer-forkserver/ ) * Fork-based fuzzing is currently the main blocker preventing us from enabling other fuzzers such as AFL++ on OSS-Fuzz * Fork-based fuzzing may be a reason why coverage-builds are failing on OSS-Fuzz. Coverage is an important fuzzing metric which would allow us to find parts of the code that are not well-covered. * Fork-based fuzzing has high overhead. fork() is an expensive system-call, especially for processes running ASAN (with large/complex) VMA layouts. * Fork prevents us from effectively fuzzing devices that rely on threads (e.g. qxl). These patches remove fork-based fuzzing and replace it with reboot-based fuzzing for most cases. Misc notes about this change: * libfuzzer appears to be no longer in active development. As such, the current implementation of fork-based fuzzing (while having some nice advantages) is likely to hold us back in the future. If these changes are approved and appear to run successfully on OSS-Fuzz, we should be able to easily experiment with other fuzzing engines (AFL++). * Some device do not completely reset their state. This can lead to non-reproducible crashes. However, in my local tests, most crashes were reproducible. OSS-Fuzz shouldn't send us reports unless it can consistently reproduce a crash. * In theory, the corpus-format should not change, so the existing corpus-inputs on OSS-Fuzz will transfer to the new reset()-able fuzzers. * Each fuzzing process will now exit after a single crash is found. To continue the fuzzing process, use libfuzzer flags such as -jobs=-1 * We no long control input-timeouts (those are handled by libfuzzer). Since timeouts on oss-fuzz can be many seconds long, I added a limit on the number of DMA bytes written. Alexander Bulekov (10): hw/sparse-mem: clear memory on reset fuzz: add fuzz_reboot API fuzz/generic-fuzz: use reboots instead of forks to reset state fuzz/generic-fuzz: add a limit on DMA bytes written fuzz/virtio-scsi: remove fork-based fuzzer fuzz/virtio-net: remove fork-based fuzzer fuzz/virtio-blk: remove fork-based fuzzer fuzz/i440fx: remove fork-based fuzzer fuzz: remove fork-fuzzing scaffolding docs/fuzz: remove mentions of fork-based fuzzing docs/devel/fuzzing.rst | 22 +----- hw/mem/sparse-mem.c | 13 +++- meson.build | 4 - tests/qtest/fuzz/fork_fuzz.c | 41 ---------- tests/qtest/fuzz/fork_fuzz.h | 23 ------ tests/qtest/fuzz/fork_fuzz.ld | 56 -------------- tests/qtest/fuzz/fuzz.c | 6 ++ tests/qtest/fuzz/fuzz.h | 2 +- tests/qtest/fuzz/generic_fuzz.c | 111 +++++++--------------------- tests/qtest/fuzz/i440fx_fuzz.c | 27 +------ tests/qtest/fuzz/meson.build | 6 +- tests/qtest/fuzz/virtio_blk_fuzz.c | 51 ++----------- tests/qtest/fuzz/virtio_net_fuzz.c | 54 ++------------ tests/qtest/fuzz/virtio_scsi_fuzz.c | 51 ++----------- 14 files changed, 72 insertions(+), 395 deletions(-) delete mode 100644 tests/qtest/fuzz/fork_fuzz.c delete mode 100644 tests/qtest/fuzz/fork_fuzz.h delete mode 100644 tests/qtest/fuzz/fork_fuzz.ld -- 2.39.0
