Re: [Vote] Create a "machine learning" component

[Paul King]

Hi Avijit Basak,

+1 to thanking you for your offer. Just a couple of comments from
someone who is only a marginal contributor to the commons project.

I would be keen to see a new commons component incorporating various
machine learning/data science components. The other main contenders
that seem to be reasonably actively developed are Smile[1] and Weka[2]
which are licensed under GPL or LGPL. Such a component would be a
natural fit for the algorithm you propose. If you look at Apache
Spark[3] and Apache Ignite[4], they both offer some "machine learning"
offerings but they tend to only support algorithms which are either
"embarrassingly" parallel or inherently parallel. They tend not to
include sequential by nature algorithms. Even "embarrassingly"
parallel algorithms are often not included since they can typically
already be used already by Spark, Ignite, Beam, Wayang, or home-grown
threads/fibres.

There has been previous research into PGA with Hadoop, Spark and
Ignite[5][6] but so far, none of that has made it into those
distributions as far as I know. I don't know how customisable the
Ignite GA algorithm[7] is but it might be worth looking into.

With respect to component naming, you either go very broad with "math"
or something like "datascience", or potentially too narrow with
something like "ml" or "machinelearning". Of the latter two, "ml" is
most common when bundled into some other framework. The other
alternative is to simply come up with another name but the typical
convention within commons is to use a descriptive to purpose name.
Numerous "ml" libraries also bundle things like regression into them,
so there is precedence for such libraries to be algorithms broadly in
the topic space. On the commons math front, I think regression is
currently earmarked for statistics but not sure it has made the jump
as of yet. An "ml" home would be equally suitable in my mind.

Having said all of that, as others have pointed out, the volunteer
space in commons is somewhat lean at the moment. I would be happy to
help a little from the ASF side of things but machine learning/data
science isn't my principal area of expertise nor a major aspect in my
"day job" activities, it probably takes others with interest to fully
give this the effort it deserves. But sometimes someone has to get the
ball rolling before other interested parties show up.

Cheers, Paul

[1] https://haifengl.github.io/
[2] https://www.cs.waikato.ac.nz/ml/weka/
[3] https://spark.apache.org/mllib/
[4] https://ignite.apache.org/docs/latest/machine-learning/machine-learning
[5] https://hajirajabeen.github.io/publications/SGA.pdf
[6] https://dzone.com/articles/genetic-algorithms-with-apache-ignite
[7] 
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/ml/util/genetic/GeneticAlgorithm.html

On Sun, Feb 14, 2021 at 6:06 PM Avijit Basak  wrote:
>
> Hi
>
>I would like to mention a few points here. Genetic Algorithm has a
> vast range of applications in optimization and search problems. Machine
> learning is only one of those.
>If we couple the new GA library with any specific domain like ml it
> would be meaningless for people working in other domains. They have to
> incorporate the entire ml library which may be completely unrelated to
> their project. Coupling it with any technology like spark might also limit
> it's usability.
>If a separate component is not approved for this change then we can
> incorporate the changes as part of *commons.math* library.
>The same library can be reused in ml or neural network libraries as
> a dependency.
>Kindly share further views on this.
>
> Thanks & Regards
> --Avijit Basak
>
> On Wed, 10 Feb 2021 at 19:49, Gilles Sadowski  wrote:
>
> > Le mer. 10 févr. 2021 à 13:19, sebb  a écrit :
> > >
> > > Likewise, commons-ml is too cryptic.
> > >
> > > Also, the Spark project has a machine-learning library:
> > >
> > > https://spark.apache.org/mllib/
> >
> > Thanks for the pointer.
> >
> > >
> > > Maybe that would be better home?
> >
> > On the face of it, probably.
> > [For sure, Avijit should comment on the suggestion.]
> >
> > On the other hand, "Commons" is the place where one can pick "bare
> > bone" implementations, and add the functionality to one's application
> > without necessarily comply with an overarching framework.
> > [I don't mean that framework compliance is bad; quite the contrary, it is
> > hopefully the result of a thorough reflection by experts.  But ... cf. the
> > numerous "no-dependency" discussions ...]
> >
> > Actually, concerning Avijit's proposed contribution, didn't I say:[1]
> > ---CUT---
> > Thus, I think that we must assess whether the "genetic algorithms"
> > functionality has a reasonable future within "Apache Commons" (i.e.
> > potential users and contributors) while there exist other libraries that
> > seem much more advanced for any serious usage.
> > ---CUT---
> >
> > > I'm also a bit concerned as to whether there are 

Re: [geometry] 1.0-beta2

[Matt Juntunen]

Hi Alex,

First of all, thanks for all of the work you've done here! Second, I vote to go 
with the fast dot2s implementation for LinearCombination and use it as a static 
method in commons-geometry. The reason is that the scale of accuracy 
improvements we're talking about here is not going to mean much for practical 
purposes, whereas a performance hit in a critical piece of code like this 
definitely is. Also, I'd prefer to keep the choice of LinearCombination 
algorithm internal to the library and not make it configurable. The vast 
majority of users are not going to want to have to think about it and it will 
keep the API cleaner if we don't explicitly allow it to be changed. If users 
want to use a different algorithm for something in commons-geometry, they can 
write a utility method that leverages other parts of commons-numbers.

Regards,
Matt J

From: Alex Herbert 
Sent: Tuesday, April 20, 2021 12:25 PM
To: Commons Developers List 
Subject: Re: [geometry] 1.0-beta2

On Tue, 20 Apr 2021 at 13:51, Gilles Sadowski  wrote:

> Hi Alex.
>
> Le mar. 20 avr. 2021 à 11:17, Alex Herbert  a
> écrit :
> >
> > [...]
>
> I'm a bit lost in all these bits... ;-)
>

I also had to remind myself of the work since it was a long time ago.


>
> > Any opinions on how changing LinearComination may affect Geometry? Either
> > we clean up the implementation to use the fast dot2s algorithm with
> correct
> > support for splitting 53-bit mantissas, or switch to the extended
> precision
> > version. But I do not think we should leave it as the current
> > implementation which has disadvantages against either of the
> alternatives.
>
> What is your suggestion?
>

Some background...

The dot2s is a specialisation of the DotK algorithm where the K represents
the k-fold increase in precision over a standard scalar product and denotes
qualitatively the level of robustness for a sum with massive cancellation
(e.g. 1e-100 + 1e200 - 2e-100 - 1e200 = -1e-100). Any floating-point sum is
limited by the representation of a 64-bit double as a binary floating point
number with only 53-bits of precision and an exponent.  When adding numbers
the result is computed and any extra bits that cannot fit into the mantissa
are lost. Whether this occurs is dependent on the input numbers, but in
general as the difference in magnitude becomes greater then the result will
contain progressively less of the true result due to the 53-bit mantissa.
If two numbers are different by more than 2^53 then adding them makes no
difference to the bigger number. In the example above a standard precision
result is 0. Only with extended precision can you hold the result of 1e-100
+ 1e200, etc.

In a sum of magnitude this only really matters when cancellation (opposite
signs) may occur. Take the recently discussed code to compute the 3D vector
length:

len = sqrt(x^2 + y^2 + z^2)

This uses a sum where the sign of the terms is the same. There will be no
cancellation and thus the answer will be close to the true result (within a
few ulps).

But if the sum of multiple values has different signs then the answer may
be in between the magnitudes of the terms. So the intermediate sum should
store knowledge of more bits of the intermediate result than the 64-bits of
a double.

The LinearCombination is actually providing two operations, multiplication
and addition:

sum_i {a_i * b_i}

The multiplication has a potential 106-bit mantissa for the result with a
single exponent. Any addition may require a longer mantissa with a single
exponent which is how BigDecimal would store the result. An extended
precision double number would represent the result as an array of two
doubles of very different magnitudes. Adding or multiplying the extended
precision result can increase the length of the result further. The use of
arrays of double to represent a number in extended precision is well
described in Shewchuk (1997): Arbitrary Precision Floating-Point Arithmetic
[1], on which a lot of the work in Numbers-142 is based.

There could be a case for a class to implement this generically:

public class ExtendedDouble {
  public static ExtendedDouble of(double a);
  public ExtendedDouble add(double a);
  public ExtendedDouble add(ExtendedDouble a);
  public ExtendedDouble multiply(double a);
  public ExtendedDouble multiply(ExtendedDouble a);
  public double toDouble();
}

That could be in another module in numbers. Shewchuk does not describe
division but it may be in references he provides. His paper is based upon
developing exact arithmetic for geometry applications so addition and
multiplication are all that are used.

As for LinearCombination then I would suggest that the use case is for any
dot product where cancellation may be expected. If handling cancellation is
more important than speed then using the most accurate method would seem to
be a better choice.

I do recall seeing LinearCombination used in the Geometry code inside a
loop with the result 

Re: [all] OSS Fuzz

[Bruno P. Kinoshita]

 Done in imaging! Thanks

https://github.com/apache/commons-imaging/pull/130

Bruno



On Wednesday, 21 April 2021, 2:27:38 am NZST, Fabian Meumertzheim 
 wrote:  
 
 The first OSS-Fuzz build passed and some bugs have already been created.
Everything looks good from my side, but let me know if you have any
questions.

One more thing: Could you perhaps add the following line to the READMEs of
compress and imaging?

[![Fuzzing Status](
https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
)

This will give you both an indicator of the fuzzer build status as well as
a convenient link to the bugs on the OSS-Fuzz issue tracker.

On Mon, Apr 19, 2021, 12:16 sebb  wrote:

> On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig  wrote:
> >
> > On 2021-04-18, Stefan Bodewig wrote:
> >
> > > I've created https://issues.apache.org/jira/browse/INFRA-21741 if you
> > > want to lend a hand moderating, you may want to add yourself to the
> > > ticket before the list is created.
> >
> > The list has been created, so if you want to receive the fuzz reports
> > please subscribe to fuzz-testing@commons and one of the intial
> > moderators will accept the subscription (if we can recognize the email
> > address :-).
>
> Best to ensure that you use an address that is listed in your LDAP account.
> Visit https://whimsy.apache.org/roster/committer/__self__ to maintain the
> list.
> Moderators can then use Whimsy to search committers by email address
> to find you.
>
> Also, it looks like the content will appear on lists.a.o (requires
> login to view):
> https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
>
> There's nothing there at present, so the URL will report an error
> > Stefan
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>
  

Re: [all] OSS Fuzz

[Matt Sicker]

I see that now. Thanks for handling the ticket!

On Tue, 20 Apr 2021 at 11:38, sebb  wrote:
>
> On Tue, 20 Apr 2021 at 17:22, Matt Sicker  wrote:
> >
> > I've tried adding that email to the allow-subscribe list for that
> > mailing list. Let's see if the next messages get through without
> > moderation now.
>
> As already noted, the Return-Path is different for each email - have a
> look at the headers in the email source.
>
> I have raised INFRA-21752 with a potential solution.
>
> > On Tue, 20 Apr 2021 at 10:46, Matt Sicker  wrote:
> > >
> > > I've accepted all the moderation requests so far, though I also get
> > > CC'd on the same emails, so my inbox is a little messy at the moment
> > > to verify that it's going through the mailing list, too. There's
> > > already been like 20 alerts found, so good call on the separate
> > > mailing list! :)
> > >
> > > On Tue, 20 Apr 2021 at 10:43, sebb  wrote:
> > > >
> > > > On Tue, 20 Apr 2021 at 16:34, Fabian Meumertzheim
> > > >  wrote:
> > > > >
> > > > > I think that the sender address has been 
> > > > > monorail+v2.382749...@chromium.org for
> > > > > me since February, so it might be more stable than it looks.
> > > >
> > > > Yes, but ezmlm uses the envelope sender, i.e. the Return-Path.
> > > >
> > > > At least the current ones all have the same domain name.
> > > >
> > > > If this is sufficiently unique it should be possible to filter all
> > > > such emails without opening the floodgates.
> > > > (the filter just removes the variable part, so normal moderation rules
> > > > can be applied)
> > > >
> > > > > On Tue, Apr 20, 2021, 17:30 Matt Sicker  wrote:
> > > > >
> > > > > > Guess we'll have to ask infra then. They probably have a way to 
> > > > > > filter
> > > > > > based on regex or something.
> > > > > >
> > > > > > On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> > > > > > >
> > > > > > > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Looks like we need to add the bot email as an allowed sender to 
> > > > > > > > the
> > > > > > list.
> > > > > > > >
> > > > > > >
> > > > > > > Easier said than done, as email appears to use a dynamic address.
> > > > > > >
> > > > > > >
> > > > > > > > Otherwise, I’ve seen the alerts start already
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > > > > > > meumertzh...@code-intelligence.com> wrote:
> > > > > > > >
> > > > > > > > > The first OSS-Fuzz build passed and some bugs have already 
> > > > > > > > > been
> > > > > > created.
> > > > > > > > > Everything looks good from my side, but let me know if you 
> > > > > > > > > have any
> > > > > > > > > questions.
> > > > > > > > >
> > > > > > > > > One more thing: Could you perhaps add the following line to 
> > > > > > > > > the
> > > > > > READMEs
> > > > > > > > of
> > > > > > > > > compress and imaging?
> > > > > > > > >
> > > > > > > > > [![Fuzzing Status](
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > > > > > > )
> > > > > > > > >
> > > > > > > > > This will give you both an indicator of the fuzzer build 
> > > > > > > > > status as
> > > > > > well
> > > > > > > > as
> > > > > > > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > > > > > > >
> > > > > > > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > > > > > > >
> > > > > > > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > > > > > > > > 
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I've created 
> > > > > > > > > > > > https://issues.apache.org/jira/browse/INFRA-21741
> > > > > > if
> > > > > > > > > you
> > > > > > > > > > > > want to lend a hand moderating, you may want to add 
> > > > > > > > > > > > yourself
> > > > > > to the
> > > > > > > > > > > > ticket before the list is created.
> > > > > > > > > > >
> > > > > > > > > > > The list has been created, so if you want to receive the 
> > > > > > > > > > > fuzz
> > > > > > reports
> > > > > > > > > > > please subscribe to fuzz-testing@commons and one of the 
> > > > > > > > > > > intial
> > > > > > > > > > > moderators will accept the subscription (if we can 
> > > > > > > > > > > recognize the
> > > > > > > > email
> > > > > > > > > > > address :-).
> > > > > > > > > >
> > > > > > > > > > Best to ensure that you use an address that is listed in 
> > > > > > > > > > your LDAP
> > > > > > > > > account.
> > > > > > > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to
> > > > > > maintain
> > > > > > > > > the
> > > > > > > > > > list.
> > > > > > > > > > Moderators can then use Whimsy to search committers by email
> > > > > > address
> > > > > > > > > > to find you.
> > > > > > > > > >
> > > > > > > > 

Re: [all] OSS Fuzz

[sebb]

On Tue, 20 Apr 2021 at 17:22, Matt Sicker  wrote:
>
> I've tried adding that email to the allow-subscribe list for that
> mailing list. Let's see if the next messages get through without
> moderation now.

As already noted, the Return-Path is different for each email - have a
look at the headers in the email source.

I have raised INFRA-21752 with a potential solution.

> On Tue, 20 Apr 2021 at 10:46, Matt Sicker  wrote:
> >
> > I've accepted all the moderation requests so far, though I also get
> > CC'd on the same emails, so my inbox is a little messy at the moment
> > to verify that it's going through the mailing list, too. There's
> > already been like 20 alerts found, so good call on the separate
> > mailing list! :)
> >
> > On Tue, 20 Apr 2021 at 10:43, sebb  wrote:
> > >
> > > On Tue, 20 Apr 2021 at 16:34, Fabian Meumertzheim
> > >  wrote:
> > > >
> > > > I think that the sender address has been 
> > > > monorail+v2.382749...@chromium.org for
> > > > me since February, so it might be more stable than it looks.
> > >
> > > Yes, but ezmlm uses the envelope sender, i.e. the Return-Path.
> > >
> > > At least the current ones all have the same domain name.
> > >
> > > If this is sufficiently unique it should be possible to filter all
> > > such emails without opening the floodgates.
> > > (the filter just removes the variable part, so normal moderation rules
> > > can be applied)
> > >
> > > > On Tue, Apr 20, 2021, 17:30 Matt Sicker  wrote:
> > > >
> > > > > Guess we'll have to ask infra then. They probably have a way to filter
> > > > > based on regex or something.
> > > > >
> > > > > On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> > > > > >
> > > > > > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
> > > > > >
> > > > > > > Looks like we need to add the bot email as an allowed sender to 
> > > > > > > the
> > > > > list.
> > > > > > >
> > > > > >
> > > > > > Easier said than done, as email appears to use a dynamic address.
> > > > > >
> > > > > >
> > > > > > > Otherwise, I’ve seen the alerts start already
> > > > > > >
> > > > > >
> > > > > >
> > > > > > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > > > > > meumertzh...@code-intelligence.com> wrote:
> > > > > > >
> > > > > > > > The first OSS-Fuzz build passed and some bugs have already been
> > > > > created.
> > > > > > > > Everything looks good from my side, but let me know if you have 
> > > > > > > > any
> > > > > > > > questions.
> > > > > > > >
> > > > > > > > One more thing: Could you perhaps add the following line to the
> > > > > READMEs
> > > > > > > of
> > > > > > > > compress and imaging?
> > > > > > > >
> > > > > > > > [![Fuzzing Status](
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > > > > > )
> > > > > > > >
> > > > > > > > This will give you both an indicator of the fuzzer build status 
> > > > > > > > as
> > > > > well
> > > > > > > as
> > > > > > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > > > > > >
> > > > > > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > > > > > >
> > > > > > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > > > > > > > 
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > > > > > >
> > > > > > > > > > > I've created 
> > > > > > > > > > > https://issues.apache.org/jira/browse/INFRA-21741
> > > > > if
> > > > > > > > you
> > > > > > > > > > > want to lend a hand moderating, you may want to add 
> > > > > > > > > > > yourself
> > > > > to the
> > > > > > > > > > > ticket before the list is created.
> > > > > > > > > >
> > > > > > > > > > The list has been created, so if you want to receive the 
> > > > > > > > > > fuzz
> > > > > reports
> > > > > > > > > > please subscribe to fuzz-testing@commons and one of the 
> > > > > > > > > > intial
> > > > > > > > > > moderators will accept the subscription (if we can 
> > > > > > > > > > recognize the
> > > > > > > email
> > > > > > > > > > address :-).
> > > > > > > > >
> > > > > > > > > Best to ensure that you use an address that is listed in your 
> > > > > > > > > LDAP
> > > > > > > > account.
> > > > > > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to
> > > > > maintain
> > > > > > > > the
> > > > > > > > > list.
> > > > > > > > > Moderators can then use Whimsy to search committers by email
> > > > > address
> > > > > > > > > to find you.
> > > > > > > > >
> > > > > > > > > Also, it looks like the content will appear on lists.a.o 
> > > > > > > > > (requires
> > > > > > > > > login to view):
> > > > > > > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > > > > > > >
> > > > > > > > > There's nothing there at present, so the URL will report an 
> > > > > > > > > error
> > > > > > > > > > Stefan
> > > > > > > > > >
> > > > > > > > > >
> > > > 

Re: [geometry] 1.0-beta2

[Alex Herbert]

On Tue, 20 Apr 2021 at 13:51, Gilles Sadowski  wrote:

> Hi Alex.
>
> Le mar. 20 avr. 2021 à 11:17, Alex Herbert  a
> écrit :
> >
> > [...]
>
> I'm a bit lost in all these bits... ;-)
>

I also had to remind myself of the work since it was a long time ago.


>
> > Any opinions on how changing LinearComination may affect Geometry? Either
> > we clean up the implementation to use the fast dot2s algorithm with
> correct
> > support for splitting 53-bit mantissas, or switch to the extended
> precision
> > version. But I do not think we should leave it as the current
> > implementation which has disadvantages against either of the
> alternatives.
>
> What is your suggestion?
>

Some background...

The dot2s is a specialisation of the DotK algorithm where the K represents
the k-fold increase in precision over a standard scalar product and denotes
qualitatively the level of robustness for a sum with massive cancellation
(e.g. 1e-100 + 1e200 - 2e-100 - 1e200 = -1e-100). Any floating-point sum is
limited by the representation of a 64-bit double as a binary floating point
number with only 53-bits of precision and an exponent.  When adding numbers
the result is computed and any extra bits that cannot fit into the mantissa
are lost. Whether this occurs is dependent on the input numbers, but in
general as the difference in magnitude becomes greater then the result will
contain progressively less of the true result due to the 53-bit mantissa.
If two numbers are different by more than 2^53 then adding them makes no
difference to the bigger number. In the example above a standard precision
result is 0. Only with extended precision can you hold the result of 1e-100
+ 1e200, etc.

In a sum of magnitude this only really matters when cancellation (opposite
signs) may occur. Take the recently discussed code to compute the 3D vector
length:

len = sqrt(x^2 + y^2 + z^2)

This uses a sum where the sign of the terms is the same. There will be no
cancellation and thus the answer will be close to the true result (within a
few ulps).

But if the sum of multiple values has different signs then the answer may
be in between the magnitudes of the terms. So the intermediate sum should
store knowledge of more bits of the intermediate result than the 64-bits of
a double.

The LinearCombination is actually providing two operations, multiplication
and addition:

sum_i {a_i * b_i}

The multiplication has a potential 106-bit mantissa for the result with a
single exponent. Any addition may require a longer mantissa with a single
exponent which is how BigDecimal would store the result. An extended
precision double number would represent the result as an array of two
doubles of very different magnitudes. Adding or multiplying the extended
precision result can increase the length of the result further. The use of
arrays of double to represent a number in extended precision is well
described in Shewchuk (1997): Arbitrary Precision Floating-Point Arithmetic
[1], on which a lot of the work in Numbers-142 is based.

There could be a case for a class to implement this generically:

public class ExtendedDouble {
  public static ExtendedDouble of(double a);
  public ExtendedDouble add(double a);
  public ExtendedDouble add(ExtendedDouble a);
  public ExtendedDouble multiply(double a);
  public ExtendedDouble multiply(ExtendedDouble a);
  public double toDouble();
}

That could be in another module in numbers. Shewchuk does not describe
division but it may be in references he provides. His paper is based upon
developing exact arithmetic for geometry applications so addition and
multiplication are all that are used.

As for LinearCombination then I would suggest that the use case is for any
dot product where cancellation may be expected. If handling cancellation is
more important than speed then using the most accurate method would seem to
be a better choice.

I do recall seeing LinearCombination used in the Geometry code inside a
loop with the result stored as a double on each iteration. This would
result in loss of the extended precision intermediate sum and so could
cause error due to cancellation. So there is a case for going through
Geometry to find use cases for LinearCombination and potentially find
places where changes should be made to compute all the terms in advance and
call LinearCombination once with the two input arrays.


> My impression is that [Geometry] emphasizes accuracy over ultimate
> speed (as opposed to libraries used for real-time rendering, I guess).
>
> However, could it be possible to leave this as a user's decision?
> Quoting from Matt's tutorial:
> ---CUT---
> Typically, users of Commons Geometry will construct a single instance
> of this type for use by multiple objects throughout an entire
> operation, or even application. Since we don't want our class to
> assume such a heavy responsibility, we will simply accept a
> DoublePrecisionContext in the constructor.
> ---CUT---
> Would it be conceivable that the choice of the 

Re: [all] OSS Fuzz

[Matt Sicker]

I've tried adding that email to the allow-subscribe list for that
mailing list. Let's see if the next messages get through without
moderation now.

On Tue, 20 Apr 2021 at 10:46, Matt Sicker  wrote:
>
> I've accepted all the moderation requests so far, though I also get
> CC'd on the same emails, so my inbox is a little messy at the moment
> to verify that it's going through the mailing list, too. There's
> already been like 20 alerts found, so good call on the separate
> mailing list! :)
>
> On Tue, 20 Apr 2021 at 10:43, sebb  wrote:
> >
> > On Tue, 20 Apr 2021 at 16:34, Fabian Meumertzheim
> >  wrote:
> > >
> > > I think that the sender address has been 
> > > monorail+v2.382749...@chromium.org for
> > > me since February, so it might be more stable than it looks.
> >
> > Yes, but ezmlm uses the envelope sender, i.e. the Return-Path.
> >
> > At least the current ones all have the same domain name.
> >
> > If this is sufficiently unique it should be possible to filter all
> > such emails without opening the floodgates.
> > (the filter just removes the variable part, so normal moderation rules
> > can be applied)
> >
> > > On Tue, Apr 20, 2021, 17:30 Matt Sicker  wrote:
> > >
> > > > Guess we'll have to ask infra then. They probably have a way to filter
> > > > based on regex or something.
> > > >
> > > > On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> > > > >
> > > > > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
> > > > >
> > > > > > Looks like we need to add the bot email as an allowed sender to the
> > > > list.
> > > > > >
> > > > >
> > > > > Easier said than done, as email appears to use a dynamic address.
> > > > >
> > > > >
> > > > > > Otherwise, I’ve seen the alerts start already
> > > > > >
> > > > >
> > > > >
> > > > > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > > > > meumertzh...@code-intelligence.com> wrote:
> > > > > >
> > > > > > > The first OSS-Fuzz build passed and some bugs have already been
> > > > created.
> > > > > > > Everything looks good from my side, but let me know if you have 
> > > > > > > any
> > > > > > > questions.
> > > > > > >
> > > > > > > One more thing: Could you perhaps add the following line to the
> > > > READMEs
> > > > > > of
> > > > > > > compress and imaging?
> > > > > > >
> > > > > > > [![Fuzzing Status](
> > > > > > >
> > > > > > >
> > > > > >
> > > > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > > > > )
> > > > > > >
> > > > > > > This will give you both an indicator of the fuzzer build status as
> > > > well
> > > > > > as
> > > > > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > > > > >
> > > > > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > > > > >
> > > > > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > > > > > > 
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > > > > >
> > > > > > > > > > I've created 
> > > > > > > > > > https://issues.apache.org/jira/browse/INFRA-21741
> > > > if
> > > > > > > you
> > > > > > > > > > want to lend a hand moderating, you may want to add yourself
> > > > to the
> > > > > > > > > > ticket before the list is created.
> > > > > > > > >
> > > > > > > > > The list has been created, so if you want to receive the fuzz
> > > > reports
> > > > > > > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > > > > > > moderators will accept the subscription (if we can recognize 
> > > > > > > > > the
> > > > > > email
> > > > > > > > > address :-).
> > > > > > > >
> > > > > > > > Best to ensure that you use an address that is listed in your 
> > > > > > > > LDAP
> > > > > > > account.
> > > > > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to
> > > > maintain
> > > > > > > the
> > > > > > > > list.
> > > > > > > > Moderators can then use Whimsy to search committers by email
> > > > address
> > > > > > > > to find you.
> > > > > > > >
> > > > > > > > Also, it looks like the content will appear on lists.a.o 
> > > > > > > > (requires
> > > > > > > > login to view):
> > > > > > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > > > > > >
> > > > > > > > There's nothing there at present, so the URL will report an 
> > > > > > > > error
> > > > > > > > > Stefan
> > > > > > > > >
> > > > > > > > >
> > > > -
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > -
> > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > 

Re: [all] OSS Fuzz

[Matt Sicker]

I've accepted all the moderation requests so far, though I also get
CC'd on the same emails, so my inbox is a little messy at the moment
to verify that it's going through the mailing list, too. There's
already been like 20 alerts found, so good call on the separate
mailing list! :)

On Tue, 20 Apr 2021 at 10:43, sebb  wrote:
>
> On Tue, 20 Apr 2021 at 16:34, Fabian Meumertzheim
>  wrote:
> >
> > I think that the sender address has been monorail+v2.382749...@chromium.org 
> > for
> > me since February, so it might be more stable than it looks.
>
> Yes, but ezmlm uses the envelope sender, i.e. the Return-Path.
>
> At least the current ones all have the same domain name.
>
> If this is sufficiently unique it should be possible to filter all
> such emails without opening the floodgates.
> (the filter just removes the variable part, so normal moderation rules
> can be applied)
>
> > On Tue, Apr 20, 2021, 17:30 Matt Sicker  wrote:
> >
> > > Guess we'll have to ask infra then. They probably have a way to filter
> > > based on regex or something.
> > >
> > > On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> > > >
> > > > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
> > > >
> > > > > Looks like we need to add the bot email as an allowed sender to the
> > > list.
> > > > >
> > > >
> > > > Easier said than done, as email appears to use a dynamic address.
> > > >
> > > >
> > > > > Otherwise, I’ve seen the alerts start already
> > > > >
> > > >
> > > >
> > > > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > > > meumertzh...@code-intelligence.com> wrote:
> > > > >
> > > > > > The first OSS-Fuzz build passed and some bugs have already been
> > > created.
> > > > > > Everything looks good from my side, but let me know if you have any
> > > > > > questions.
> > > > > >
> > > > > > One more thing: Could you perhaps add the following line to the
> > > READMEs
> > > > > of
> > > > > > compress and imaging?
> > > > > >
> > > > > > [![Fuzzing Status](
> > > > > >
> > > > > >
> > > > >
> > > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > > > )
> > > > > >
> > > > > > This will give you both an indicator of the fuzzer build status as
> > > well
> > > > > as
> > > > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > > > >
> > > > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > > > >
> > > > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > > > wrote:
> > > > > > > >
> > > > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > > > >
> > > > > > > > > I've created https://issues.apache.org/jira/browse/INFRA-21741
> > > if
> > > > > > you
> > > > > > > > > want to lend a hand moderating, you may want to add yourself
> > > to the
> > > > > > > > > ticket before the list is created.
> > > > > > > >
> > > > > > > > The list has been created, so if you want to receive the fuzz
> > > reports
> > > > > > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > > > > > moderators will accept the subscription (if we can recognize the
> > > > > email
> > > > > > > > address :-).
> > > > > > >
> > > > > > > Best to ensure that you use an address that is listed in your LDAP
> > > > > > account.
> > > > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to
> > > maintain
> > > > > > the
> > > > > > > list.
> > > > > > > Moderators can then use Whimsy to search committers by email
> > > address
> > > > > > > to find you.
> > > > > > >
> > > > > > > Also, it looks like the content will appear on lists.a.o (requires
> > > > > > > login to view):
> > > > > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > > > > >
> > > > > > > There's nothing there at present, so the URL will report an error
> > > > > > > > Stefan
> > > > > > > >
> > > > > > > >
> > > -
> > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > -
> > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org

Re: [all] OSS Fuzz

[sebb]

On Tue, 20 Apr 2021 at 16:34, Fabian Meumertzheim
 wrote:
>
> I think that the sender address has been monorail+v2.382749...@chromium.org 
> for
> me since February, so it might be more stable than it looks.

Yes, but ezmlm uses the envelope sender, i.e. the Return-Path.

At least the current ones all have the same domain name.

If this is sufficiently unique it should be possible to filter all
such emails without opening the floodgates.
(the filter just removes the variable part, so normal moderation rules
can be applied)

> On Tue, Apr 20, 2021, 17:30 Matt Sicker  wrote:
>
> > Guess we'll have to ask infra then. They probably have a way to filter
> > based on regex or something.
> >
> > On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> > >
> > > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
> > >
> > > > Looks like we need to add the bot email as an allowed sender to the
> > list.
> > > >
> > >
> > > Easier said than done, as email appears to use a dynamic address.
> > >
> > >
> > > > Otherwise, I’ve seen the alerts start already
> > > >
> > >
> > >
> > > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > > meumertzh...@code-intelligence.com> wrote:
> > > >
> > > > > The first OSS-Fuzz build passed and some bugs have already been
> > created.
> > > > > Everything looks good from my side, but let me know if you have any
> > > > > questions.
> > > > >
> > > > > One more thing: Could you perhaps add the following line to the
> > READMEs
> > > > of
> > > > > compress and imaging?
> > > > >
> > > > > [![Fuzzing Status](
> > > > >
> > > > >
> > > >
> > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > > )
> > > > >
> > > > > This will give you both an indicator of the fuzzer build status as
> > well
> > > > as
> > > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > > >
> > > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > > >
> > > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > > wrote:
> > > > > > >
> > > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > > >
> > > > > > > > I've created https://issues.apache.org/jira/browse/INFRA-21741
> > if
> > > > > you
> > > > > > > > want to lend a hand moderating, you may want to add yourself
> > to the
> > > > > > > > ticket before the list is created.
> > > > > > >
> > > > > > > The list has been created, so if you want to receive the fuzz
> > reports
> > > > > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > > > > moderators will accept the subscription (if we can recognize the
> > > > email
> > > > > > > address :-).
> > > > > >
> > > > > > Best to ensure that you use an address that is listed in your LDAP
> > > > > account.
> > > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to
> > maintain
> > > > > the
> > > > > > list.
> > > > > > Moderators can then use Whimsy to search committers by email
> > address
> > > > > > to find you.
> > > > > >
> > > > > > Also, it looks like the content will appear on lists.a.o (requires
> > > > > > login to view):
> > > > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > > > >
> > > > > > There's nothing there at present, so the URL will report an error
> > > > > > > Stefan
> > > > > > >
> > > > > > >
> > -
> > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > -
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] OSS Fuzz

[sebb]

On Tue, 20 Apr 2021 at 16:30, Matt Sicker  wrote:
>
> Guess we'll have to ask infra then. They probably have a way to filter
> based on regex or something.

I just remembered we had to do something similar for Travis and
Twitter, see INFRA-18843 and INFRA-19360

Are you going to accept the moderation requests for now?

> On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> >
> > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
> >
> > > Looks like we need to add the bot email as an allowed sender to the list.
> > >
> >
> > Easier said than done, as email appears to use a dynamic address.
> >
> >
> > > Otherwise, I’ve seen the alerts start already
> > >
> >
> >
> > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > meumertzh...@code-intelligence.com> wrote:
> > >
> > > > The first OSS-Fuzz build passed and some bugs have already been created.
> > > > Everything looks good from my side, but let me know if you have any
> > > > questions.
> > > >
> > > > One more thing: Could you perhaps add the following line to the READMEs
> > > of
> > > > compress and imaging?
> > > >
> > > > [![Fuzzing Status](
> > > >
> > > >
> > > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > )
> > > >
> > > > This will give you both an indicator of the fuzzer build status as well
> > > as
> > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > >
> > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > >
> > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > wrote:
> > > > > >
> > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > >
> > > > > > > I've created https://issues.apache.org/jira/browse/INFRA-21741 if
> > > > you
> > > > > > > want to lend a hand moderating, you may want to add yourself to 
> > > > > > > the
> > > > > > > ticket before the list is created.
> > > > > >
> > > > > > The list has been created, so if you want to receive the fuzz 
> > > > > > reports
> > > > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > > > moderators will accept the subscription (if we can recognize the
> > > email
> > > > > > address :-).
> > > > >
> > > > > Best to ensure that you use an address that is listed in your LDAP
> > > > account.
> > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to maintain
> > > > the
> > > > > list.
> > > > > Moderators can then use Whimsy to search committers by email address
> > > > > to find you.
> > > > >
> > > > > Also, it looks like the content will appear on lists.a.o (requires
> > > > > login to view):
> > > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > > >
> > > > > There's nothing there at present, so the URL will report an error
> > > > > > Stefan
> > > > > >
> > > > > > -
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > >
> > > > >
> > > > > -
> > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > >
> > > > >
> > > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] OSS Fuzz

[Fabian Meumertzheim]

I think that the sender address has been monorail+v2.382749...@chromium.org for
me since February, so it might be more stable than it looks.

On Tue, Apr 20, 2021, 17:30 Matt Sicker  wrote:

> Guess we'll have to ask infra then. They probably have a way to filter
> based on regex or something.
>
> On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
> >
> > On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
> >
> > > Looks like we need to add the bot email as an allowed sender to the
> list.
> > >
> >
> > Easier said than done, as email appears to use a dynamic address.
> >
> >
> > > Otherwise, I’ve seen the alerts start already
> > >
> >
> >
> > > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > > meumertzh...@code-intelligence.com> wrote:
> > >
> > > > The first OSS-Fuzz build passed and some bugs have already been
> created.
> > > > Everything looks good from my side, but let me know if you have any
> > > > questions.
> > > >
> > > > One more thing: Could you perhaps add the following line to the
> READMEs
> > > of
> > > > compress and imaging?
> > > >
> > > > [![Fuzzing Status](
> > > >
> > > >
> > >
> https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > > )
> > > >
> > > > This will give you both an indicator of the fuzzer build status as
> well
> > > as
> > > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > > >
> > > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > > >
> > > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > > wrote:
> > > > > >
> > > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > > >
> > > > > > > I've created https://issues.apache.org/jira/browse/INFRA-21741
> if
> > > > you
> > > > > > > want to lend a hand moderating, you may want to add yourself
> to the
> > > > > > > ticket before the list is created.
> > > > > >
> > > > > > The list has been created, so if you want to receive the fuzz
> reports
> > > > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > > > moderators will accept the subscription (if we can recognize the
> > > email
> > > > > > address :-).
> > > > >
> > > > > Best to ensure that you use an address that is listed in your LDAP
> > > > account.
> > > > > Visit https://whimsy.apache.org/roster/committer/__self__ to
> maintain
> > > > the
> > > > > list.
> > > > > Moderators can then use Whimsy to search committers by email
> address
> > > > > to find you.
> > > > >
> > > > > Also, it looks like the content will appear on lists.a.o (requires
> > > > > login to view):
> > > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > > >
> > > > > There's nothing there at present, so the URL will report an error
> > > > > > Stefan
> > > > > >
> > > > > >
> -
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > > >
> > > > >
> > > > >
> -
> > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > >
> > > > >
> > > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>

Re: [all] OSS Fuzz

[Matt Sicker]

Guess we'll have to ask infra then. They probably have a way to filter
based on regex or something.

On Tue, 20 Apr 2021 at 10:05, sebb  wrote:
>
> On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:
>
> > Looks like we need to add the bot email as an allowed sender to the list.
> >
>
> Easier said than done, as email appears to use a dynamic address.
>
>
> > Otherwise, I’ve seen the alerts start already
> >
>
>
> > On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> > meumertzh...@code-intelligence.com> wrote:
> >
> > > The first OSS-Fuzz build passed and some bugs have already been created.
> > > Everything looks good from my side, but let me know if you have any
> > > questions.
> > >
> > > One more thing: Could you perhaps add the following line to the READMEs
> > of
> > > compress and imaging?
> > >
> > > [![Fuzzing Status](
> > >
> > >
> > https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > > )
> > >
> > > This will give you both an indicator of the fuzzer build status as well
> > as
> > > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> > >
> > > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> > >
> > > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> > wrote:
> > > > >
> > > > > On 2021-04-18, Stefan Bodewig wrote:
> > > > >
> > > > > > I've created https://issues.apache.org/jira/browse/INFRA-21741 if
> > > you
> > > > > > want to lend a hand moderating, you may want to add yourself to the
> > > > > > ticket before the list is created.
> > > > >
> > > > > The list has been created, so if you want to receive the fuzz reports
> > > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > > moderators will accept the subscription (if we can recognize the
> > email
> > > > > address :-).
> > > >
> > > > Best to ensure that you use an address that is listed in your LDAP
> > > account.
> > > > Visit https://whimsy.apache.org/roster/committer/__self__ to maintain
> > > the
> > > > list.
> > > > Moderators can then use Whimsy to search committers by email address
> > > > to find you.
> > > >
> > > > Also, it looks like the content will appear on lists.a.o (requires
> > > > login to view):
> > > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > > >
> > > > There's nothing there at present, so the URL will report an error
> > > > > Stefan
> > > > >
> > > > > -
> > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > >
> > > >
> > >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Vote] Create a "machine learning" component

[Gilles Sadowski]

Le mar. 20 avr. 2021 à 16:09, Avijit Basak  a écrit :
>
> Hi
>
>   > Did you ask "Spark" people about their opinion about it?
> -- Not yet. I am not sure what would be the right option for
> this communication. It will be good if you can approach them.

You are the one who proposes a functionality that might be of interest
to the "Spark" project, perhaps on some condition on their part which
*you* are going to have to accept (or not).

In other words: It would be useless that *I* go and tell them there exist
some code in Commons Math which they could take an adapt for their
project (they can always do that).
What might be of value to them (as to the Commons project, too), is a
contributor willing to do the necessary work to create or improve a
community-supported feature.

>   > where it can be used in real-life (performance-wise)
> applications, then you should demonstrate it
> -- Do we have any kind of performance benchmark or use case
> regarding this?

Please assume that *you* are the person with the most GA expertise
in this forum.
There certainly are unit tests for the GA functionality, but I don't think
there are benchmarks; certainly, one task would be to set up a module
for (JMH-based) experimentation.

> Once that is decided,

One mantra of ASF communities is that "those who do the work get
to decide".
[The PMC can decide (by vote) whether to accept a new component;
but it's up to you to show that it's worth it (with the risk that the PMC
won't accurately judge the contribution, unfortunately)...]

> then I can proceed with this.

There is already a long list of things that can be done.

You don't *have* to contact "Spark" if you don't feel that it's the
right project for your work.  You could just hope for the best, and
start somewhere else (modularization of Commons Math, a fork
on GitHub of of CM ML-related codes, and so on).

The one thing which I won't be helping with is merging ad-hoc
GA-related changes into the current CM codebase.
This doesn't preclude that other committers might want to do that
for you; however judging by the last 5 years, I wouldn't count too
much on it. ;-)

Regards,
Gilles

>
>
> Thanks & Regards
> --Avijit Basak
>
> On Mon, 19 Apr 2021 at 18:51, Gilles Sadowski  wrote:
>
> > Hello.
> >
> > Le lun. 19 avr. 2021 à 08:35, Avijit Basak  a
> > écrit :
> > >
> > > Hi
> > >
> > > >Isn't a GA inherently parallel?
> > > >If so, why not take advantage of the concurrency tools provided by the
> > JDK?
> > >   -- Are we planning to implement multi-threading for GA operations even
> > as
> > > part of a single population
> >
> > This seems an obvious improvement to our current implementation
> > (in case a chromosome's evaluation is not population-dependent).
> >
> > > or only for multi-population parallel GA.
> > >   -- We can implement different types of co-evolution as part of parallel
> > > GA. Need to decide on the corresponding strategies we are going to
> > > incorporate.
> >
> > The discussion is still about the "administrative" question of whether
> > any of this should be implemented in the "Commons" project...
> >
> > Did you ask "Spark" people about their opinion about it?
> >
> > As I said, if you are confident that you can bring our implementation to
> > a state where it can be used in real-life (performance-wise) applications,
> > then you should demonstrate it (in order to convince other people from
> > the Commons PMC that it is worth engaging in long-term maintenance).
> > AFAICT, a way to do it would be to create a GitHub project (aimed at
> > becoming a new "machine learning" component, or a maven/JPMS
> > module within Commons Math).
> >
> > Best regards,
> > Gilles

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] OSS Fuzz

[sebb]

On Tue, 20 Apr 2021 at 15:53, Matt Sicker  wrote:

> Looks like we need to add the bot email as an allowed sender to the list.
>

Easier said than done, as email appears to use a dynamic address.


> Otherwise, I’ve seen the alerts start already 
>


> On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
> meumertzh...@code-intelligence.com> wrote:
>
> > The first OSS-Fuzz build passed and some bugs have already been created.
> > Everything looks good from my side, but let me know if you have any
> > questions.
> >
> > One more thing: Could you perhaps add the following line to the READMEs
> of
> > compress and imaging?
> >
> > [![Fuzzing Status](
> >
> >
> https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> > )
> >
> > This will give you both an indicator of the fuzzer build status as well
> as
> > a convenient link to the bugs on the OSS-Fuzz issue tracker.
> >
> > On Mon, Apr 19, 2021, 12:16 sebb  wrote:
> >
> > > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig 
> wrote:
> > > >
> > > > On 2021-04-18, Stefan Bodewig wrote:
> > > >
> > > > > I've created https://issues.apache.org/jira/browse/INFRA-21741 if
> > you
> > > > > want to lend a hand moderating, you may want to add yourself to the
> > > > > ticket before the list is created.
> > > >
> > > > The list has been created, so if you want to receive the fuzz reports
> > > > please subscribe to fuzz-testing@commons and one of the intial
> > > > moderators will accept the subscription (if we can recognize the
> email
> > > > address :-).
> > >
> > > Best to ensure that you use an address that is listed in your LDAP
> > account.
> > > Visit https://whimsy.apache.org/roster/committer/__self__ to maintain
> > the
> > > list.
> > > Moderators can then use Whimsy to search committers by email address
> > > to find you.
> > >
> > > Also, it looks like the content will appear on lists.a.o (requires
> > > login to view):
> > > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> > >
> > > There's nothing there at present, so the URL will report an error
> > > > Stefan
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > > For additional commands, e-mail: dev-h...@commons.apache.org
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > >
> > >
> >
>

Re: [all] OSS Fuzz

[Matt Sicker]

Looks like we need to add the bot email as an allowed sender to the list.
Otherwise, I’ve seen the alerts start already 

On Tue, Apr 20, 2021 at 09:27 Fabian Meumertzheim <
meumertzh...@code-intelligence.com> wrote:

> The first OSS-Fuzz build passed and some bugs have already been created.
> Everything looks good from my side, but let me know if you have any
> questions.
>
> One more thing: Could you perhaps add the following line to the READMEs of
> compress and imaging?
>
> [![Fuzzing Status](
>
> https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
> )
>
> This will give you both an indicator of the fuzzer build status as well as
> a convenient link to the bugs on the OSS-Fuzz issue tracker.
>
> On Mon, Apr 19, 2021, 12:16 sebb  wrote:
>
> > On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig  wrote:
> > >
> > > On 2021-04-18, Stefan Bodewig wrote:
> > >
> > > > I've created https://issues.apache.org/jira/browse/INFRA-21741 if
> you
> > > > want to lend a hand moderating, you may want to add yourself to the
> > > > ticket before the list is created.
> > >
> > > The list has been created, so if you want to receive the fuzz reports
> > > please subscribe to fuzz-testing@commons and one of the intial
> > > moderators will accept the subscription (if we can recognize the email
> > > address :-).
> >
> > Best to ensure that you use an address that is listed in your LDAP
> account.
> > Visit https://whimsy.apache.org/roster/committer/__self__ to maintain
> the
> > list.
> > Moderators can then use Whimsy to search committers by email address
> > to find you.
> >
> > Also, it looks like the content will appear on lists.a.o (requires
> > login to view):
> > https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
> >
> > There's nothing there at present, so the URL will report an error
> > > Stefan
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > > For additional commands, e-mail: dev-h...@commons.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
> >
>

Re: [all] OSS Fuzz

[Fabian Meumertzheim]

The first OSS-Fuzz build passed and some bugs have already been created.
Everything looks good from my side, but let me know if you have any
questions.

One more thing: Could you perhaps add the following line to the READMEs of
compress and imaging?

[![Fuzzing Status](
https://oss-fuzz-build-logs.storage.googleapis.com/badges/apache-commons.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened=1=proj:apache-commons
)

This will give you both an indicator of the fuzzer build status as well as
a convenient link to the bugs on the OSS-Fuzz issue tracker.

On Mon, Apr 19, 2021, 12:16 sebb  wrote:

> On Mon, 19 Apr 2021 at 07:54, Stefan Bodewig  wrote:
> >
> > On 2021-04-18, Stefan Bodewig wrote:
> >
> > > I've created https://issues.apache.org/jira/browse/INFRA-21741 if you
> > > want to lend a hand moderating, you may want to add yourself to the
> > > ticket before the list is created.
> >
> > The list has been created, so if you want to receive the fuzz reports
> > please subscribe to fuzz-testing@commons and one of the intial
> > moderators will accept the subscription (if we can recognize the email
> > address :-).
>
> Best to ensure that you use an address that is listed in your LDAP account.
> Visit https://whimsy.apache.org/roster/committer/__self__ to maintain the
> list.
> Moderators can then use Whimsy to search committers by email address
> to find you.
>
> Also, it looks like the content will appear on lists.a.o (requires
> login to view):
> https://lists.apache.org/list.html?fuzz-test...@commons.apache.org
>
> There's nothing there at present, so the URL will report an error
> > Stefan
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>

Re: [Vote] Create a "machine learning" component

[Avijit Basak]

Hi

  > Did you ask "Spark" people about their opinion about it?
-- Not yet. I am not sure what would be the right option for
this communication. It will be good if you can approach them.
  > where it can be used in real-life (performance-wise)
applications, then you should demonstrate it
-- Do we have any kind of performance benchmark or use case
regarding this? Once that is decided, then I can proceed with this.


Thanks & Regards
--Avijit Basak

On Mon, 19 Apr 2021 at 18:51, Gilles Sadowski  wrote:

> Hello.
>
> Le lun. 19 avr. 2021 à 08:35, Avijit Basak  a
> écrit :
> >
> > Hi
> >
> > >Isn't a GA inherently parallel?
> > >If so, why not take advantage of the concurrency tools provided by the
> JDK?
> >   -- Are we planning to implement multi-threading for GA operations even
> as
> > part of a single population
>
> This seems an obvious improvement to our current implementation
> (in case a chromosome's evaluation is not population-dependent).
>
> > or only for multi-population parallel GA.
> >   -- We can implement different types of co-evolution as part of parallel
> > GA. Need to decide on the corresponding strategies we are going to
> > incorporate.
>
> The discussion is still about the "administrative" question of whether
> any of this should be implemented in the "Commons" project...
>
> Did you ask "Spark" people about their opinion about it?
>
> As I said, if you are confident that you can bring our implementation to
> a state where it can be used in real-life (performance-wise) applications,
> then you should demonstrate it (in order to convince other people from
> the Commons PMC that it is worth engaging in long-term maintenance).
> AFAICT, a way to do it would be to create a GitHub project (aimed at
> becoming a new "machine learning" component, or a maven/JPMS
> module within Commons Math).
>
> Best regards,
> Gilles
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>

-- 
Avijit Basak

Re: [geometry] 1.0-beta2

[Gilles Sadowski]

Hi Alex.

Le mar. 20 avr. 2021 à 11:17, Alex Herbert  a écrit :
>
> [...]

I'm a bit lost in all these bits... ;-)

> Any opinions on how changing LinearComination may affect Geometry? Either
> we clean up the implementation to use the fast dot2s algorithm with correct
> support for splitting 53-bit mantissas, or switch to the extended precision
> version. But I do not think we should leave it as the current
> implementation which has disadvantages against either of the alternatives.

What is your suggestion?

My impression is that [Geometry] emphasizes accuracy over ultimate
speed (as opposed to libraries used for real-time rendering, I guess).

However, could it be possible to leave this as a user's decision?
Quoting from Matt's tutorial:
---CUT---
Typically, users of Commons Geometry will construct a single instance
of this type for use by multiple objects throughout an entire
operation, or even application. Since we don't want our class to
assume such a heavy responsibility, we will simply accept a
DoublePrecisionContext in the constructor.
---CUT---
Would it be conceivable that the choice of the implementation
activated by a call to the "LinearCombination" facility is also
encapsulated in the "DoublePrecisionContext"?

Regards,
Gilles

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

JDK 17 Early Acces build 18 is available

[Rory O'Donnell]

*Hi Benedikt, *

*OpenJDK 17 Early Access build 18is now available at 
**https://jdk.java.net/17 *


 * These early-access , open-source builds are provided under the
 o GNU General Public License, version 2, with the Classpath
   Exception 
 * Release Notes are available at http://jdk.java.net/17/release-notes
   


**G1 pauses may be extremely long with EA build JDK-17+18*

*During performance testing we noticed that due to a recent change 
(JDK-8262068) GC pauses after a G1 full GC may be extremely slow. The 
problem has been fixed with JDK-8264987 and that has already been 
integrated. This change will be available with the following EA build  
JDK-17+19.  For more technical info please see [1]



*JEP 382 [2]**  - Starting with build 19, **JDK 17 for macOS is 
*temporarily* switched from using OpenGL**to using Apple's Metal 
API**for Java 2D rendering.*


Heads up to anyone who is testing JDK 17 for running apps on macOS. 
Starting with build 19, JDK 17 for macOS is *temporarily* switched from 
using OpenGL to using Apple's Metal API for Java 2D rendering.


If you are running any kind of 2D / Swing/ AWT UI application on macOS, 
and see any rendering related problems
starting with JDK 17 b19, please do report them to us along with a test 
case and screen shots.


You may also set "-Dsun.java2d.opengl=true" to re-enable OpenGL - which  
implicitly disables Metal - to confirm that it is a Metal related 
rendering glltch.



Rgds,Rory

[1] 
https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2021-April/034745.html

[2] https://openjdk.java.net/jeps/382

--
Rgds, Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin, Ireland

Re: [geometry] 1.0-beta2

[Alex Herbert]

On Mon, 19 Apr 2021 at 22:30, Gilles Sadowski  wrote:

> Le lun. 19 avr. 2021 à 20:26, Matt Juntunen
>
> > What needs to be done on numbers before we're ready for
> > 1.0 (aside from moving over some code from geometry)?
>
> The most basic utilities haven't fundamentally changed.  It
> will be nice to increase the visibility of the many consistency
> and performance improvements.
> Modules to perhaps be left out are also TBD (in another thread).
>

I did a lot of work investigating improving the accuracy of
LinearCombination (see Numbers 142 [1]). This ended by improving the
accuracy of the linear combination that is used inside the Complex class in
a very special use case summing terms close to zero (i.e. where large
cancellations of terms are significant). However that is private to the
class. The main LinearCombination class remains the same which uses an
approximate quad level accuracy (128-bit) sum of linear terms. This could
be changed to improve accuracy at the expense of runtime speed.

Even if we keep the quad level accuracy the method to split the upper and
lower parts of the number before multiplication can be improved to retain
an extra bit of information and support sub-normal numbers. All the code
for variations and the performance benchmarks are in
'o.a.c.numbers.examples.jmh.arrays'. The main LinearCombinations class in
that package has all variants.

Currently the LinearCombination class is implementing a variant of the
dot2s method of Ogita et al (2005). The method for array inputs uses array
allocation in the computation. It also uses a split of the 53-bit double
mantissa (including the leading 1-bit) to two 26-bit doubles. This can be
changed to the reference dot2s method to avoid array allocation and use
Dekker's split to extract a 27-bit double and a 26-bit double with support
for sub-normal numbers. This would be the implementation in the
LinearCominations.Dot2s class.

Alternatively there is a version which computes an exact summation without
using BigDecimal (LinearCominations.ExtendedPrecision using ExactSum for
the summation). It is slower than the current 2-fold precision method but
much faster than using BigDecimal. Changing to this implementation may have
a big impact on the performance of Geometry which uses LinearCombination
extensively.

The Jira ticket has a lot of performance information. This post [2] seems
to summarise the speed relative to the current implementation as:

Value Method Relative
twoD dot2s 0.73
twoD extended 1.13
threeD dot2s 0.75
threeD extended 1.48
fourD dot2s 0.75
fourD extended 1.68
nD dot2s 0.56
nD extended 2.95

So an exact dot2s implementation is faster and the extended precision
implementation is about 3x slower on long arrays but not much slower on
small combinations. Here the extended precision sum is accurate to 1 ULP.
Making it exact to 0 ULP (extended2) takes a bit more time and the
performance is summarised later as:

"Using the extended2 method versus the fast dot2s will have a performance
impact dependent on the condition number of the dot product and the length
of the dot product. For short lengths (2 to 4) the method is about 1.2 to
2.5x slower. Long dot products are much slower (8x). The runtime is
insignificant compared to using BigDecimal which is a magnitude slower."

Any opinions on how changing LinearComination may affect Geometry? Either
we clean up the implementation to use the fast dot2s algorithm with correct
support for splitting 53-bit mantissas, or switch to the extended precision
version. But I do not think we should leave it as the current
implementation which has disadvantages against either of the alternatives.

Alex


[1] https://issues.apache.org/jira/browse/NUMBERS-142
[2]
https://issues.apache.org/jira/browse/NUMBERS-142?focusedCommentId=17040111=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17040111