[jira] [Comment Edited] (STATISTICS-31) Add survival probability function to continuous distributions
[
https://issues.apache.org/jira/browse/STATISTICS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383324#comment-17383324
]
Gilles Sadowski edited comment on STATISTICS-31 at 7/19/21, 12:56 PM:
--
{quote}JUnit 5 invok[es] tests only on the class where they are annotated.
{quote}
I didn't know. Is that a _feature_?
If so, it would probably be good to refactor so that we don't rely on
inheritance (which is now confusing and otherwise not recommend practice IIRC
for unit tests).
was (Author: erans):
bq. I JUnit 5 invok[es] tests only on the class where they are annotated.
I didn't know. Is that a _feature_?
If so, it would probably be good to refactor so that we don't rely on
inheritance (which is now confusing and otherwise not recommend practice IIRC
for unit tests).
> Add survival probability function to continuous distributions
> -
>
> Key: STATISTICS-31
> URL: https://issues.apache.org/jira/browse/STATISTICS-31
> Project: Apache Commons Statistics
> Issue Type: New Feature
>Reporter: Benjamin W Trent
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> It is useful to know the [survival
> function|[https://en.wikipedia.org/wiki/Survival_function]] of a number given
> a continuous distribution.
> While this can be approximated with
> {noformat}
> 1 - cdf(x){noformat}
> , there is an opportunity for greater accuracy in certain distributions.
>
> A good example of this is the gamma distribution. The survival function for
> that distribution would probably look similar to:
>
> ```java
> @Override
> public double survivalProbability(double x) {
> if (x <= SUPPORT_LO)
> { return 1; }
> else if (x >= SUPPORT_HI)
> { return 0; }
> return RegularizedGamma.Q.value(shape, x / scale);
> }
> ```
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (STATISTICS-31) Add survival probability function to continuous distributions
[
https://issues.apache.org/jira/browse/STATISTICS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383292#comment-17383292
]
Alex Herbert edited comment on STATISTICS-31 at 7/19/21, 12:02 PM:
---
I've not looked over the PR in detail but I did notice that the new survival
probability functions are tested with probabilities in the range 1e-7 to 1e-9
or so with a default absolute tolerance of 1e-15. Given the limit of 1 - CDF(
x) is 2^-53 (or approximately 1.11e-16) did you try pushing the values and the
resulting probabilities further out into the long tail?
was (Author: alexherbert):
I've not looked over the PR in detail but I did notice that the new survival
probability functions are tested with probabilities in the range 1e-7 to 1e-9
or so with a default absolute tolerance of 1e-15. Given the limit of 1 - CDF(x)
is 2^-53 (or approximately 1.11e-16) did you try pushing the values and the
resulting probabilities further out into the long tail?
> Add survival probability function to continuous distributions
> -
>
> Key: STATISTICS-31
> URL: https://issues.apache.org/jira/browse/STATISTICS-31
> Project: Apache Commons Statistics
> Issue Type: New Feature
>Reporter: Benjamin W Trent
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> It is useful to know the [survival
> function|[https://en.wikipedia.org/wiki/Survival_function]] of a number given
> a continuous distribution.
> While this can be approximated with
> {noformat}
> 1 - cdf(x){noformat}
> , there is an opportunity for greater accuracy in certain distributions.
>
> A good example of this is the gamma distribution. The survival function for
> that distribution would probably look similar to:
>
> ```java
> @Override
> public double survivalProbability(double x) {
> if (x <= SUPPORT_LO)
> { return 1; }
> else if (x >= SUPPORT_HI)
> { return 0; }
> return RegularizedGamma.Q.value(shape, x / scale);
> }
> ```
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (STATISTICS-31) Add survival probability function to continuous distributions
[
https://issues.apache.org/jira/browse/STATISTICS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382322#comment-17382322
]
Alex Herbert edited comment on STATISTICS-31 at 7/16/21, 8:35 PM:
--
This should also apply to the DiscreteDistribution. Of the implementations the
following marked with a * can have a high precision or alternate implementation
for the survival function:
* Binomial *
* Geometric *
* Hypergeometric *
* Pascal *
* Poisson *
* Uniform
* Zipf
The Hypergeometric even has a method to do this called
{{upperCumulativeProbability}}.
[~benwtrent] does Wolfram have survival probability for these distributions
where we can obtain test data?
was (Author: alexherbert):
This should also apply to the DiscreteDistribution. Of the implementations the
following marked with a * can have a high precision or alternate implementation
for the survival function:
* Binomial *
* Geometric *
* Hypergeometric *
* Pascal
* Poisson *
* Uniform
* Zipf
The Hypergeometric even has a method to do this called
{{upperCumulativeProbability}}.
[~benwtrent] does Wolfram have survival probability for these distributions
where we can obtain test data?
> Add survival probability function to continuous distributions
> -
>
> Key: STATISTICS-31
> URL: https://issues.apache.org/jira/browse/STATISTICS-31
> Project: Apache Commons Statistics
> Issue Type: New Feature
>Reporter: Benjamin W Trent
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> It is useful to know the [survival
> function|[https://en.wikipedia.org/wiki/Survival_function]] of a number given
> a continuous distribution.
> While this can be approximated with
> {noformat}
> 1 - cdf(x){noformat}
> , there is an opportunity for greater accuracy in certain distributions.
>
> A good example of this is the gamma distribution. The survival function for
> that distribution would probably look similar to:
>
> ```java
> @Override
> public double survivalProbability(double x) {
> if (x <= SUPPORT_LO)
> { return 1; }
> else if (x >= SUPPORT_HI)
> { return 0; }
> return RegularizedGamma.Q.value(shape, x / scale);
> }
> ```
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (STATISTICS-31) Add survival probability function to continuous distributions
[
https://issues.apache.org/jira/browse/STATISTICS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382322#comment-17382322
]
Alex Herbert edited comment on STATISTICS-31 at 7/16/21, 8:27 PM:
--
This should also apply to the DiscreteDistribution. Of the implementations the
following marked with a * can have a high precision or alternate implementation
for the survival function:
* Binomial *
* Geometric *
* Hypergeometric *
* Pascal
* Poisson *
* Uniform
* Zipf
The Hypergeometric even has a method to do this called
{{upperCumulativeProbability}}.
[~benwtrent] does Wolfram have survival probability for these distributions
where we can obtain test data?
was (Author: alexherbert):
This should also apply to the DiscreteDistribution. Of the implementations the
following marked with a * can have a high precision or alternate implementation
for the survival function:
* Binomial *
* Geometric *
* Hypergeometric *
* Pascal
* Poisson *
* Uniform *
* Zipf
The Hypergeometric even has a method to do this called
{{upperCumulativeProbability}}.
[~benwtrent] does Wolfram have survival probability for these distributions
where we can obtain test data?
> Add survival probability function to continuous distributions
> -
>
> Key: STATISTICS-31
> URL: https://issues.apache.org/jira/browse/STATISTICS-31
> Project: Apache Commons Statistics
> Issue Type: New Feature
>Reporter: Benjamin W Trent
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> It is useful to know the [survival
> function|[https://en.wikipedia.org/wiki/Survival_function]] of a number given
> a continuous distribution.
> While this can be approximated with
> {noformat}
> 1 - cdf(x){noformat}
> , there is an opportunity for greater accuracy in certain distributions.
>
> A good example of this is the gamma distribution. The survival function for
> that distribution would probably look similar to:
>
> ```java
> @Override
> public double survivalProbability(double x) {
> if (x <= SUPPORT_LO)
> { return 1; }
> else if (x >= SUPPORT_HI)
> { return 0; }
> return RegularizedGamma.Q.value(shape, x / scale);
> }
> ```
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (STATISTICS-31) Add survival probability function to continuous distributions
[
https://issues.apache.org/jira/browse/STATISTICS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380794#comment-17380794
]
Alex Herbert edited comment on STATISTICS-31 at 7/14/21, 6:31 PM:
--
Support can be added to the ContinuousDistribution interface using a default
method:
{code:java}
/**
* For a random variable {@code X} whose values are distributed according
* to this distribution, this method returns {@code P(X >= x)}.
* In other words, this method represents the complementary cumulative
distribution
* function for this distribution, also known as the survival function or
* reliability function.
*
* @param x Point at which the complementary CDF is evaluated.
* @return the probability that a random variable with this
* distribution takes a value more than or equal to {@code x}.
*/
default double survivalProbability(double x) {
return 1 - cumulativeProbability(x);
}
{code}
A similar method can be added to DiscreteDistribution interface too.
Distributions where a more accurate result can be obtained as the CDF -> 1 can
override the method. These would exploit the availability of far more double
values for the result as it approaches 0, rather than the default
implementation which is limited to a minimum of 2^-53 (i.e. 1.0 -
nextDown(1.0)).
Is this what you are suggesting?
was (Author: alexherbert):
Support can be added to the ContinuousDistribution interface using a default
method:
{code:java}
/**
* For a random variable {@code X} whose values are distributed according
* to this distribution, this method returns {@code P(X >= x)}.
* In other words, this method represents the complementary cumulative
distribution
* function for this distribution, also known as the survival function or
* reliability function.
*
* @param x Point at which the CDF is evaluated.
* @return the probability that a random variable with this
* distribution takes a value more than or equal to {@code x}.
*/
default double survivalProbability(double x) {
return 1 - cumulativeProbability(x);
}
{code}
A similar method can be added to DiscreteDistribution interface too.
Distributions where a more accurate result can be obtained as the CDF -> 1 can
override the method. These would exploit the availability of far more double
values for the result as it approaches 0, rather than the default
implementation which is limited to a minimum of 2^-53 (i.e. 1.0 -
nextDown(1.0)).
Is this what you are suggesting?
> Add survival probability function to continuous distributions
> -
>
> Key: STATISTICS-31
> URL: https://issues.apache.org/jira/browse/STATISTICS-31
> Project: Apache Commons Statistics
> Issue Type: New Feature
>Reporter: Benjamin W Trent
>Priority: Major
>
> It is useful to know the [survival
> function|[https://en.wikipedia.org/wiki/Survival_function]] of a number given
> a continuous distribution.
> While this can be approximated with
> {noformat}
> 1 - cdf(x){noformat}
> , there is an opportunity for greater accuracy in certain distributions.
>
> A good example of this is the gamma distribution. The survival function for
> that distribution would probably look similar to:
>
> ```java
> @Override
> public double survivalProbability(double x) {
> if (x <= SUPPORT_LO)
> { return 1; }
> else if (x >= SUPPORT_HI)
> { return 0; }
> return RegularizedGamma.Q.value(shape, x / scale);
> }
> ```
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
