Hi Mark, I think the best way forward is to collaborate and have a short feedback loop.
Did you mean build failures by “Invalid due to broken test”? If yes, I am not sure what we can do about the broken tests since those are already executed and tested by check build scripts locally and in a CI/CD pipeline. Build and Coverage failures are sometimes supposed to happen if there are changes in the target repository itself or if there are infrastructure issues in OSS-Fuzz. We will investigate those issues in more detail. Maybe some filter in the apache mailing list is helpful for you in the short term, Fuzzing and Coverage build issues have a "build failure" string in the subject. That would enable you to focus on the reports only. In order to make sure that we get high-quality tests and results, maintainer feedback from apache will be very valuable and welcome. You have the best domain knowledge about your code base and can give valuable hints on which APIs to tackle best. There was already some valuable feedback for Apache Tomcat in https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53153. Let us extend this collaboration. We can discuss and agree on the attack vectors in apache-commons components. Best regards Roman On Thu, Nov 10, 2022 at 10:29 AM Mark Thomas <ma...@apache.org> wrote: > Oliver, > > My requirements regarding configuration are: > > - secur...@commons.apache.org MUST be notified of all security > vulnerability reports for all Apache Commons components > > - a mechanism MUST be provided for the secur...@commons.apache.org > Google user to view all historical reports that were not previously > notified to that address > > - if any further Apache Commons components get added to oss-fuzz > then secur...@commons.apache.org MUST receive notification of any > issues as they are found > > - more generally, if *any* Apache Software Foundation project is added > to oss-fuzz then the notifications for that project MUST include the > relevant security team for that project > > If you can achieve the above with the current structure then great. > > > Separately, there is a concern regarding the false positive rate. With > the oss-fuzz integration provided by Code Intelligence we have seen the > following with Apache Tomcat in a little under 3 months. > > Total "vulnerability" reports: 39 > > Invalid due to broken test: 31% > False positive: 52% > Bugs: 18% > Valid security issues: 0% > > To add some commentary: > - the bugs were minor / extreme edge cases users were unlikely to hit > - false positives were all due to the tests being based on invalid > assumptions regarding whether input was expected to be trusted or not > > > If those statistics were repeated across multiple Apache Commons > components, the volume of invalid reports would be more than the > volunteers of the Apache Commons security team could handle. > > I have no objection to being overwhelmed with valid security > vulnerability reports. If that ever happened, we would find a way to > deal with it. > > I do have very strong objections to being overwhelmed with invalid > security vulnerability reports. If we see the same false positive rate > repeated across the Apache Commons components that has been observed > with Apache Tomcat then I don't see that Apache Commons would have any > choice but to request the removal of all Apache Commons Components from > oss-fuzz. > > Mark > > > > > > On 10/11/2022 04:19, Oliver Chang wrote: > > Hi Mark, > > > > In addition to the reasons Roman listed, the current structure also > > allows us to allocate more compute resources to all of these Apache > > packages, rather than all of them sharing the CPUs allocated for a > > single OSS-Fuzz "project". > > > > We can definitely ensure that secur...@commons.apache.org > > <mailto:secur...@commons.apache.org> is included on all relevant Apache > > projects going forward, and other than that I believe there's not much > > other difference in terms of the end result (i.e. bug reports) that end > > up getting filed. > > > > Does that sound OK to you? Or did you have other concerns around the > > current directory structure? > > > > Best regards, > > -- > > Oliver > > > > > > On Wed, 9 Nov 2022 at 21:31, Roman Wagner <wag...@code-intelligence.com > > <mailto:wag...@code-intelligence.com>> wrote: > > > > Hi Mark, > > > > I have added @Oliver Chang <mailto:och...@google.com> from the > > Google OSS-Fuzz to the thread. > > > > I had a short discussion with Oliver. There could be different > > issues in OSS-Fuzz by design If all apache-commons components will > > move under apache-commons directory: > > > > * it is not scalable and will slow down both fuzzing and triage > > (e.g. automated bisections, fix verification) > > * changing the structure this way will invalidate all existing > > open testcases, and cause new ones to be filed, which will > > result in a fair bit of spam. > > > > My proposal would be that "secur...@commons.apache.org > > <mailto:secur...@commons.apache.org>" is added to all individual > > apache-commons components. > > I am not sure how it is possible to ensure that future onboardings > > of apache-commons components will automatically have > > "secur...@commons.apache.org <mailto:secur...@commons.apache.org>" > > as primary contact. OSS-Fuzz could have some additional > > documentation for that. @Oliver Chang <mailto:och...@google.com> do > > you have any ideas here? > > > > Best regards > > Roman > > > > On Tue, Nov 8, 2022 at 5:56 PM Mark Thomas <ma...@apache.org > > <mailto:ma...@apache.org>> wrote: > > > > Thanks for the update. > > > > I'll wait for that PR to be resolved before taking any further > > action. > > > > Mark > > > > > > On 08/11/2022 16:42, Roman Wagner wrote: > > > Hi Mark, > > > > > > there is a PR open in oss-fuzz > > https://github.com/google/oss-fuzz/pull/8933 > > <https://github.com/google/oss-fuzz/pull/8933> > > > . > > > > > > Best regards > > > Roman > > > > > > On Tue, Nov 8, 2022 at 4:15 PM Gary Gregory > > <garydgreg...@gmail.com <mailto:garydgreg...@gmail.com>> wrote: > > > > > >> Sounds good. > > >> > > >> Gary > > >> > > >> On Tue, Nov 8, 2022, 10:07 Mark Thomas <ma...@apache.org > > <mailto:ma...@apache.org>> wrote: > > >> > > >>> There has been no response to this email from anyone from > Code > > >>> Intelligence. > > >>> > > >>> Unless there are objections from the Apache Commons > > Community my next > > >>> step will be to submit a PR to have the following modules > > removed from > > >>> oss-fuzz: > > >>> > > >>> apache-commons-bcel > > >>> apache-commons-beanutils > > >>> apache-commons-cli > > >>> apache-commons-codec > > >>> apache-commons-collections > > >>> apache-commons-configuration > > >>> apache-commons-io > > >>> apache-commons-jxpath > > >>> apache-commons-lang > > >>> apache-commons-logging > > >>> > > >>> Code Intelligence (or anyone else) will remain free to add > > them back in > > >>> the right place - under apache-commons should they wish to > > do so. > > >>> > > >>> Mark > > >>> > > >>> > > >>> > > >>> On 19/10/2022 10:56, Mark Thomas wrote: > > >>>> Hi, > > >>>> > > >>>> You are receiving this email as you are currently > > configured as the > > >>>> recipients for oss-fuzz reports for Apache Commons JXPath. > > >>>> > > >>>> As per the discussion on the Apache Commons dev list[1], > > please make > > >>> the > > >>>> following configuration changes to the oss-fuzz > > integrations with > > >>>> immediate effect: > > >>>> > > >>>> - Move all oss-fuzz integrations added for *ALL* Apache > > Commons > > >>>> components to the oss-fuzz module for Apache-Commons: > > >>>> > > >>>> > > >>> > > > https://github.com/google/oss-fuzz/tree/master/projects/apache-commons < > https://github.com/google/oss-fuzz/tree/master/projects/apache-commons> > > >>>> > > >>>> There should *NOT* be separate oss-fuzz modules for > > each component > > >>>> > > >>>> > > >>>> - Add the Google account for "secur...@commons.apache.org > > <mailto:secur...@commons.apache.org>" to > > >>>> - the notifications for these issues > > >>>> - the ACL to enable this account to access the details > > for each > > >>> report > > >>>> > > >>>> > > >>>> Please notify dev@commons.apache.org > > <mailto:dev@commons.apache.org> and secur...@commons.apache.org > > <mailto:secur...@commons.apache.org> > > >>>> when these changes have been completed. > > >>>> > > >>>> Thanks, > > >>>> > > >>>> Mark > > >>>> > > >>>> > > >>>> > > >>>> [1] > > https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln > > < > https://lists.apache.org/thread/53vwy3g8w3f8nydz7jvxm8snrqx7msln> > > >>>> > > >>>> > > > --------------------------------------------------------------------- > > >>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > <mailto:dev-unsubscr...@commons.apache.org> > > >>>> For additional commands, e-mail: > > dev-h...@commons.apache.org <mailto:dev-h...@commons.apache.org> > > >>>> > > >>> > > >>> > > > --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > <mailto:dev-unsubscr...@commons.apache.org> > > >>> For additional commands, e-mail: > > dev-h...@commons.apache.org <mailto:dev-h...@commons.apache.org> > > >>> > > >>> > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > <mailto:dev-unsubscr...@commons.apache.org> > > For additional commands, e-mail: dev-h...@commons.apache.org > > <mailto:dev-h...@commons.apache.org> > > > > > > > > -- > > > > Roman Wagner > > Application Security Engineer > > > > Code Intelligence > > Rheinwerkallee 6 > > 53227 Bonn > > > > Amtsgericht Bonn > > HRB 23408 > > > > Geschäftsführer: Sergej Dechand, Dr. Khaled Yakdan > > > -- Roman Wagner Application Security Engineer Code Intelligence Rheinwerkallee 6 53227 Bonn Amtsgericht Bonn HRB 23408 Geschäftsführer: Sergej Dechand, Dr. Khaled Yakdan
