[jira] [Commented] (MATH-1373) In LogNormalDistribution.java, it appears shape & scale are reversed/mis-labelled.
[ https://issues.apache.org/jira/browse/MATH-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333561#comment-16333561 ] Gilles commented on MATH-1373: -- Discussion should be moved to the issue tracker of the new "Commons Statistics" component (see STATISTICS-2). > In LogNormalDistribution.java, it appears shape & scale are > reversed/mis-labelled. > -- > > Key: MATH-1373 > URL: https://issues.apache.org/jira/browse/MATH-1373 > Project: Commons Math > Issue Type: Bug >Affects Versions: 3.6.1 >Reporter: Karl D. Gierach >Priority: Minor > Fix For: 4.0 > > Attachments: MATH-1373.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > When I compute the logshape and log scale based on the formulas on > wikipedia's lognormal distribution page that use empirical mean and variance, > I found that the getNumericalMean() method was not returning the empirical > mean. > However, upon just trying to reverse the shape and scale parameters in the > constructor proved to fix the problem, and the object then returns the > correct empirical mean. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MATH-1373) In LogNormalDistribution.java, it appears shape & scale are reversed/mis-labelled.
[
https://issues.apache.org/jira/browse/MATH-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338637#comment-15338637
]
Gilles commented on MATH-1373:
--
I think that it's a documentation issue.
{quote}
* MathWorld's M being equivalent to Commons Math's scale
* MathWorld's S being equivalent to Commons Math's shape
{quote}
In [MathWorld|http://mathworld.wolfram.com/LogNormalDistribution.html], {{M}}
and {{S}} above are respectively named {{mu}} and {{sigma}}.
But CM uses the names {{scale}} and {{shape}} (in that order), whereas
[Wikipedia|https://en.wikipedia.org/wiki/Log-normal_distribution#Location_and_scale]
refers to them as {{location}} and {{scale}} (in that order).
IIUC, CM uses the [NIST convention
|http://www.itl.nist.gov/div898/handbook/eda/section3/eda3669.htm] (where
{{sigma}} is referred to as {{shape}}) but the Javadoc links to sites that use
other conventions.
The API would be less confusing if we'd use {{meanLog}} (a.k.a. {{mu}} or
{{scale}}) and {{standardDeviationLog}} (a.k.a. {{sigma}} or {{shape}}).
Do you agree?
> In LogNormalDistribution.java, it appears shape & scale are
> reversed/mis-labelled.
> --
>
> Key: MATH-1373
> URL: https://issues.apache.org/jira/browse/MATH-1373
> Project: Commons Math
> Issue Type: Bug
>Affects Versions: 3.6.1
>Reporter: Karl D. Gierach
>Priority: Minor
> Attachments: MATH-1373.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When I compute the logshape and log scale based on the formulas on
> wikipedia's lognormal distribution page that use empirical mean and variance,
> I found that the getNumericalMean() method was not returning the empirical
> mean.
> However, upon just trying to reverse the shape and scale parameters in the
> constructor proved to fix the problem, and the object then returns the
> correct empirical mean.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MATH-1373) In LogNormalDistribution.java, it appears shape & scale are reversed/mis-labelled.
[
https://issues.apache.org/jira/browse/MATH-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338359#comment-15338359
]
Brent Worden commented on MATH-1373:
I am not completely convinced this change is correct.
Referencing on of the citations,
http://mathworld.wolfram.com/LogNormalDistribution.html, the density function
is parameterized the same way LogNormalDistribution is parameterized with
* MathWorld's M being equivalent to Commons Math's scale
* MathWorld's S being equivalent to Commons Math's shape
The distribution mean according to MathWorld is Exp(M + S^2 / 2) which
corresponds to Exp(scale + shape^2 / 2) and is how it is coded in
LogNormalDistribution.
Likewise, MathWorld states the distribution variance Exp(S^2 + 2 M) * (Exp(S^2
- 1) which is Exp(shape^2 + 2 scale) * (Exp(shape^2 - 1). Again, this matches
the implementation.
Furthermore, generating a large sample from the distribution results in sample
means and variances that are pretty close to the expected values returned from
the getNumericalMean and getNumericalVariance methods. Here is the code I am
using to make that claim:
{code}
@Test
public void testMeanAndVariance() {
LogNormalDistribution dist = new LogNormalDistribution(5.375, 1.125);
double[] x = new double[10];
for (int i = 0; i < x.length; ++i) {
x[i] = dist.inverseCumulativeProbability(Math.random());
}
double actualMean = new Mean().evaluate(x);
double actualVariance = new Variance().evaluate(x);
double expectedMean = dist.getNumericalMean();
double expectedVariance = dist.getNumericalVariance();
System.out.println(String.format("Mean: %f vs %f (actual vs expected)",
actualMean, expectedMean));
System.out.println(String.format("Variance: %f vs %f (actual vs
expected)", actualVariance, expectedVariance));
}
{code}
> In LogNormalDistribution.java, it appears shape & scale are
> reversed/mis-labelled.
> --
>
> Key: MATH-1373
> URL: https://issues.apache.org/jira/browse/MATH-1373
> Project: Commons Math
> Issue Type: Bug
>Affects Versions: 3.6.1
>Reporter: Karl D. Gierach
>Priority: Minor
> Attachments: MATH-1373.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When I compute the logshape and log scale based on the formulas on
> wikipedia's lognormal distribution page that use empirical mean and variance,
> I found that the getNumericalMean() method was not returning the empirical
> mean.
> However, upon just trying to reverse the shape and scale parameters in the
> constructor proved to fix the problem, and the object then returns the
> correct empirical mean.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MATH-1373) In LogNormalDistribution.java, it appears shape & scale are reversed/mis-labelled.
[ https://issues.apache.org/jira/browse/MATH-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336901#comment-15336901 ] Karl D. Gierach commented on MATH-1373: --- I will come up with a patch based on github: http://git-wip-us.apache.org/repos/asf/commons-math.git Also, it appears the parameters are reversed in getNumericalVariance() as well. Sound OK? Thanks, Karl > In LogNormalDistribution.java, it appears shape & scale are > reversed/mis-labelled. > -- > > Key: MATH-1373 > URL: https://issues.apache.org/jira/browse/MATH-1373 > Project: Commons Math > Issue Type: Bug >Affects Versions: 3.6.1 >Reporter: Karl D. Gierach >Priority: Minor > Original Estimate: 1h > Remaining Estimate: 1h > > When I compute the logshape and log scale based on the formulas on > wikipedia's lognormal distribution page that use empirical mean and variance, > I found that the getNumericalMean() method was not returning the empirical > mean. > However, upon just trying to reverse the shape and scale parameters in the > constructor proved to fix the problem, and the object then returns the > correct empirical mean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MATH-1373) In LogNormalDistribution.java, it appears shape & scale are reversed/mis-labelled.
[
https://issues.apache.org/jira/browse/MATH-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336549#comment-15336549
]
Karl D. Gierach commented on MATH-1373:
---
The problem is that in the method shape & scale are simply reversed:
@Override
public double getNumericalMean() {
double s = shape;
return FastMath.exp(scale + (s * s / 2));
}
It should be:FastMath.exp( shape + (scale * scale / 2 ) );
In any case here is a unit test code snippet that illustrates the problem:
var defaultScale = 1.0; // aka variance
var mean = 12.2;
var meanSquared = mean * mean
// compute sigma (log scale) parameter of the lognormal distribution.
// according to formulas on Wikipedia
var logScale =
Math.sqrt(
Math.log( 1.0 + (defaultScale / meanSquared) )
)
var logShape = Math.log( mean /
Math.sqrt( 1.0 + ( defaultScale / meanSquared
) )
)
println( "verifyLogNormalParms(): initializing with: scale/shape=" +
logScale + ", " + logShape )
// parameter order according to api docs: scale shape/location
// here, parameters are reversed, and produce the correct result
var dist = new LogNormalDistribution( logShape, logScale );
var numMean = dist.getNumericalMean()
if( Math.abs( numMean - mean ) > 0.01 ) {
println( "verifyLogNormalParms(): mean is NOT OK: " + numMean )
assertTrue( false )
}
else {
println( "verifyLogNormalParms(): mean is OK: " + numMean )
}
> In LogNormalDistribution.java, it appears shape & scale are
> reversed/mis-labelled.
> --
>
> Key: MATH-1373
> URL: https://issues.apache.org/jira/browse/MATH-1373
> Project: Commons Math
> Issue Type: Bug
>Affects Versions: 3.6.1
>Reporter: Karl D. Gierach
>Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When I compute the logshape and log scale based on the formulas on
> wikipedia's lognormal distribution page that use empirical mean and variance,
> I found that the getNumericalMean() method was not returning the empirical
> mean.
> However, upon just trying to reverse the shape and scale parameters in the
> constructor proved to fix the problem, and the object then returns the
> correct empirical mean.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MATH-1373) In LogNormalDistribution.java, it appears shape & scale are reversed/mis-labelled.
[ https://issues.apache.org/jira/browse/MATH-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321625#comment-15321625 ] Gilles commented on MATH-1373: -- Since there are many tests for this class, I wonder what the problem is. Is it a documentation bug? Would you provide a patch that would fix the issue? > In LogNormalDistribution.java, it appears shape & scale are > reversed/mis-labelled. > -- > > Key: MATH-1373 > URL: https://issues.apache.org/jira/browse/MATH-1373 > Project: Commons Math > Issue Type: Bug >Affects Versions: 3.6.1 >Reporter: Karl D. Gierach >Priority: Minor > Original Estimate: 1h > Remaining Estimate: 1h > > When I compute the logshape and log scale based on the formulas on > wikipedia's lognormal distribution page that use empirical mean and variance, > I found that the getNumericalMean() method was not returning the empirical > mean. > However, upon just trying to reverse the shape and scale parameters in the > constructor proved to fix the problem, and the object then returns the > correct empirical mean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
