Hi Janardhan, instead of continuing the discussion on the other thread of google/oss-fuzz, let us arrive at a project-wide conclusion first. As I said before, I don't think that creating fuzz tests for our native kernels is a pressing issue right now because they only receive internally constructed intermediates. Instead it would be better to harden our external entry points (read/programmatic APIs) which are all Java.
However, I don't want to hold you back. If you're interested in this project and you're willing to do the work, please come up with a concrete plan of action items that we can discuss here. If I don't here back in 3 days, I'll recommend to close the issue at google/oss-fuzz. Regards, Matthias On Mon, May 21, 2018 at 5:29 PM, Matthias Boehm <[email protected]> wrote: > Well, in general this can be interesting. Apart from our default > testsuite, we occasionally ran static code analysis tools. Having > additional tests for partially valid scripts and inputs can help to > find more issues. > > That being said, I don't think we currently qualify as a project with > "significant user base and/or be critical to the global IT > infrastructure". Also, without Java support these tests would only > apply to our native and GPU operations, which do not directly deal > with external inputs. > > So Janardhan, which fuzz targets to you have in mind? Looking over the > existing projects we would have to provide build scripts that > reference C/C++ entry points for fuzz testing. Although I can see > applications (e.g., corrupted column indexes in sparse matrices), I'm > not sure if it's a good idea to perform checks for valid inputs on > every operation instead of simply hardening the code path for external > inputs. > > Regards, > Matthias > > On Mon, May 21, 2018 at 10:41 AM, Janardhan <[email protected]> wrote: >> They accepted( google/oss-fuzz ), SystemML project for fuzz testing. >> >> PR link: https://github.com/google/oss-fuzz/pull/1429 >> >> - Janardhan >> >> On Mon, May 21, 2018 at 11:46 AM, Janardhan <[email protected]> wrote: >> >>> Hi all, >>> >>> ---- >>> To find various programming errors (mostly detectable such as buffer >>> overflow), a fuzz testing can be of great help. >>> >>> ---- >>> Merits: >>> 1. It will easily detects common programming errors, which we might have >>> missed or not unit tested. >>> 2. Improves the quality of our code. >>> >>> --- >>> Demerits: >>> 1. If a bug is found, it will be made public after 90 + 15 (grace period) >>> days. So, we must fix it before three months, if there is bug. >>> 2. For now only C and C++ are supported, Java will be supported soon. >>> >>> Please use this PR for discussion https://github.com/ >>> google/oss-fuzz/pull/1429 , for adding our project's CPP part for fuzzing. >>> >>> Once you approve, I will try to build a docker image of SystemML and >>> configure with help. The results of the test will be CC'ed to private >>> mailing list, only. >>> >>> >>> Thank you, >>> Janardhan >>>
