Re: Correctly configuring Apache Commons components for oss-fuzz

[Oliver Chang]

Thanks Mark.

Please let us know how we can help make this fuzzing experience better for
you. We're also happy to jump on a call to walk through your concerns and
reach a good outcome.

Best regards,
--
Oliver


On Thu, 17 Nov 2022 at 06:56, Mark Thomas  wrote:

> I haven't forgotten about this. I am currently working through the open
> issues. I want to complete first that so feedback isn't skewed by a
> single project.
>
> Mark
>
>
> On 11/11/2022 14:45, Roman Wagner wrote:
> > Hi Mark,
> >
> > I think the best way forward is to collaborate and have a short feedback
> > loop.
> >
> > Did you mean build failures by “Invalid due to broken test”? If yes, I am
> > not sure what we can do about the broken tests since those are already
> > executed and tested by check build scripts locally and in a CI/CD
> pipeline.
> > Build and Coverage failures are sometimes supposed to happen if there are
> > changes in the target repository itself or if there are infrastructure
> > issues in OSS-Fuzz. We will investigate those issues in more detail.
> Maybe
> > some filter in the apache mailing list is helpful for you in the short
> > term, Fuzzing and Coverage build issues have a "build failure" string in
> > the subject. That would enable you to focus on the reports only.
> >
> > In order to make sure that we get high-quality tests and results,
> > maintainer feedback from apache will be very valuable and welcome. You
> have
> > the best domain knowledge about your code base and can give valuable
> hints
> > on which APIs to tackle best. There was already some valuable feedback
> for
> > Apache Tomcat in
> https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53153.
> > Let us extend  this collaboration. We can discuss and agree on the attack
> > vectors in apache-commons components.
> >
> > Best regards
> > Roman
> >
> > On Thu, Nov 10, 2022 at 10:29 AM Mark Thomas  wrote:
> >
> >> Oliver,
> >>
> >> My requirements regarding configuration are:
> >>
> >> - secur...@commons.apache.org MUST be notified of all security
> >> vulnerability reports for all Apache Commons components
> >>
> >> - a mechanism MUST be provided for the secur...@commons.apache.org
> >> Google user to view all historical reports that were not previously
> >> notified to that address
> >>
> >> - if any further Apache Commons components get added to oss-fuzz
> >> then secur...@commons.apache.org MUST receive notification of any
> >> issues as they are found
> >>
> >> - more generally, if *any* Apache Software Foundation project is added
> >> to oss-fuzz then the notifications for that project MUST include the
> >> relevant security team for that project
> >>
> >> If you can achieve the above with the current structure then great.
> >>
> >>
> >> Separately, there is a concern regarding the false positive rate. With
> >> the oss-fuzz integration provided by Code Intelligence we have seen the
> >> following with Apache Tomcat in a little under 3 months.
> >>
> >> Total "vulnerability" reports: 39
> >>
> >> Invalid due to broken test: 31%
> >> False positive: 52%
> >> Bugs:   18%
> >> Valid security issues:   0%
> >>
> >> To add some commentary:
> >> - the bugs were minor / extreme edge cases users were unlikely to hit
> >> - false positives were all due to the tests being based on invalid
> >> assumptions regarding whether input was expected to be trusted or
> not
> >>
> >>
> >> If those statistics were repeated across multiple Apache Commons
> >> components, the volume of invalid reports would be more than the
> >> volunteers of the Apache Commons security team could handle.
> >>
> >> I have no objection to being overwhelmed with valid security
> >> vulnerability reports. If that ever happened, we would find a way to
> >> deal with it.
> >>
> >> I do have very strong objections to being overwhelmed with invalid
> >> security vulnerability reports. If we see the same false positive rate
> >> repeated across the Apache Commons components that has been observed
> >> with Apache Tomcat then I don't see that Apache Commons would have any
> >> choice but to request the removal of all Apache Commons Components from
> >> oss-fuzz.
> >>
> >> Mark
> >>
> >>
> >>
> >>
> >>
> >> On 10/11/2022 04:19, Oliver Chang wrote:
> >>> Hi Mark,
> >>>
> >>> In addition to the reasons Roman listed, the current structure also
> >>> allows us to allocate more compute resources to all of these Apache
> >>> packages, rather than all of them sharing the CPUs allocated for a
> >>> single OSS-Fuzz "project".
> >>>
> >>> We can definitely ensure that secur...@commons.apache.org
> >>>  is included on all relevant
> Apache
> >>> projects going forward, and other than that I believe there's not much
> >>> other difference in terms of the end result (i.e. bug reports) that end
> >>> up getting filed.
> >>>
> >>> Does that sound OK to you? Or did you have other concerns around the
> >>> 

RE: [BCEL] Invalid test or bug?

[Mark Roberts]

I will try to take a look at this later today.

Mark

-Original Message-
From: Gary D. Gregory [mailto:ggreg...@apache.org]
Sent: Thursday, November 17, 2022 7:14 AM
To: dev@commons.apache.org
Subject: Re: [BCEL] Invalid test or bug?

More specifically, javap says:

21: invokevirtual #68 // Method
"[B".clone:()Ljava/lang/Object;

So calling a method on an array with invokevirtual is ok and we have a bug.

Thoughts?

Gary

On 2022/11/17 14:45:41 "Gary D. Gregory" wrote:
> Hm, I'm thinking bug when I see javap output like:
>
> #68 = Methodref  #901.#902//
> "[B".clone:()Ljava/lang/Object;
>
> Thoughts?
>
> Gary
>
> On 2022/11/17 13:04:32 "Gary D. Gregory" wrote:
> > Actually: VerifyJavaMathTestCase and VerifyJavaUtilTestCase
> >
> > Gary
> >
> > On 2022/11/17 13:00:21 "Gary D. Gregory" wrote:
> > > Hi All & Mark Roberts:
> > >
> > > I added JavaMathTestCase as a disabled test as it fails.
> > >
> > > Is this a legal test to try or do we have a bug?
> > >
> > > Gary
> > >
> > > --
> > > --- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > >
> > >
> >
> > 
> > - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [BCEL] Invalid test or bug?

[Gary D. Gregory]

More specifically, javap says:

21: invokevirtual #68 // Method 
"[B".clone:()Ljava/lang/Object;

So calling a method on an array with invokevirtual is ok and we have a bug.

Thoughts?

Gary

On 2022/11/17 14:45:41 "Gary D. Gregory" wrote:
> Hm, I'm thinking bug when I see javap output like:
> 
> #68 = Methodref  #901.#902// "[B".clone:()Ljava/lang/Object;
> 
> Thoughts?
> 
> Gary
> 
> On 2022/11/17 13:04:32 "Gary D. Gregory" wrote:
> > Actually: VerifyJavaMathTestCase and VerifyJavaUtilTestCase
> > 
> > Gary
> > 
> > On 2022/11/17 13:00:21 "Gary D. Gregory" wrote:
> > > Hi All & Mark Roberts:
> > > 
> > > I added JavaMathTestCase as a disabled test as it fails.
> > > 
> > > Is this a legal test to try or do we have a bug?
> > > 
> > > Gary
> > > 
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > 
> > > 
> > 
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> > 
> > 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [BCEL] Invalid test or bug?

[Gary D. Gregory]

Hm, I'm thinking bug when I see javap output like:

#68 = Methodref  #901.#902// "[B".clone:()Ljava/lang/Object;

Thoughts?

Gary

On 2022/11/17 13:04:32 "Gary D. Gregory" wrote:
> Actually: VerifyJavaMathTestCase and VerifyJavaUtilTestCase
> 
> Gary
> 
> On 2022/11/17 13:00:21 "Gary D. Gregory" wrote:
> > Hi All & Mark Roberts:
> > 
> > I added JavaMathTestCase as a disabled test as it fails.
> > 
> > Is this a legal test to try or do we have a bug?
> > 
> > Gary
> > 
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> > 
> > 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [BCEL] Invalid test or bug?

[Gary D. Gregory]

Actually: VerifyJavaMathTestCase and VerifyJavaUtilTestCase

Gary

On 2022/11/17 13:00:21 "Gary D. Gregory" wrote:
> Hi All & Mark Roberts:
> 
> I added JavaMathTestCase as a disabled test as it fails.
> 
> Is this a legal test to try or do we have a bug?
> 
> Gary
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[BCEL] Invalid test or bug?

[Gary D. Gregory]

Hi All & Mark Roberts:

I added JavaMathTestCase as a disabled test as it fails.

Is this a legal test to try or do we have a bug?

Gary

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org