[jira] [Commented] (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095682#comment-13095682
]
Gilles commented on MATH-413:
-
{{PowellOptimizer}} updated in revision 1164303.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095677#comment-13095677
]
Gilles commented on MATH-413:
-
{{SimplexOptimizer}} updated in revision 1164300.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095264#comment-13095264
]
Gilles commented on MATH-413:
-
{{NonLinearConjugateGradientOptimizer}} updated in revision 1164044.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090885#comment-13090885
]
Gilles commented on MATH-413:
-
{{LevenbergMarquardtOptimizer}} updated in revision 1161463.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087026#comment-13087026
]
Gilles commented on MATH-413:
-
{{GaussNewtonOptimizer}} updated in revision 1159233.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068356#comment-13068356
]
Luc Maisonobe commented on MATH-413:
Coming back to this old issue.
I'm fine with previous Gilles comment about passing the checker as argument to
constructor.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935350#action_12935350
]
Gilles commented on MATH-413:
-
About point 6 (initialization of the {{ConvergenceChecker}}):
I think the safest would be to pass it as argument to the constructor(s). So,
we would have, on the one hand, constructors that take the arguments for the
"standard" convergence criterion of the algorithm and, on the other,
constructors that take a {{ConvergenceChecker}} argument. That way, the same
kind of information (how to stop) is passed at the same place (the
constructors).
And I'd remove the {{setConvergenceChecker}} (another step towards
immutability).
This procedure will also "force" implementations that use "line search" to take
the necessary precaution of syncing the tolerances of the internal optimizer or
solver. It is a problem for {{PowellOptimizer}} and, I just noticed, for
{{NonLinearConjugateGradientOptimizer}}: A user unaware of the workings of the
algorithm might use unnecessarily low tolerances thresholds for the line
search, so that the line search computation will be take a needlessly high
number iterations.
When given the "main" algorithm's tolerance levels, the constructor should
adapt the tolerance of its internal optimizer.
The would also entail the removal of the {{setLineSearchSolver}} method in
{{NonLinearConjugateGradientOptimizer}}.
What do you think?
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
>
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913995#action_12913995
]
Gilles commented on MATH-413:
-
Point 13 resolved in revision 1000422.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for confusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906465#action_12906465
]
Gilles commented on MATH-413:
-
Revision 992976 resolves the problems indicated in point 1, 2 and 10 for
{{BrentOptimizer}}, {{PowellOptimizer}} and {{LevenbergMarquardtOptimizer}}:
there is a default (always applied) convergence check and a additional
(optional) one defined with a sub-class of {{ConvergenceChecker}}.
The other optimizer classes must still be upgraded to this 2-level check.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in order to reduce the number of classes and thus limit the potential
> for c
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905071#action_12905071
]
Luc Maisonobe commented on MATH-413:
Your two-step feature seems fine to me.
The callback can even be exactly the existing convergence checker we already
have as its sole method returns a boolean which is basically equivalent to a
SUCCESS/CONTINUE alternative and it would be simpler for users to upgrade to
the new design.
Documentation should state that first level double threshold must always be
passed and only the checker is optional and used if not null.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-guess value while in this class, it is on the search interval. I
> think that here also we should randomly choose the start value (within the
> user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I
> couldn't figure out how to correct.
> # Some previously existing classes and interfaces have become no more than a
> specialisation of new "generics" classes; it might be interesting to remove
> them in
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905010#action_12905010
]
Gilles commented on MATH-413:
-
The main problem is not whether it is deemed complicated or not to initialize
the checker.
The fact is that the convergence checker procedure is _not_ (always)
independent of the algorithm (as in the cases of {{BrentOptimizer}} and
{{LevenbergMarquardtOptimizer}}).
Currently it is very dangerous to assume that convergence check is independent
and will always work correctly. This severely impacts the robustness of CM;
thus it is a big problem, and it seems that the current approach cannot solve
it.
* As I have suggested, we could try to _modify_ the algorithms in such a way
that an independent checking procedure is always valid. The drawback is that
we'll have non-standard behaviour of otherwise well-known algorithms and some
users won't like it (cf. request by the reporter of issue
[MATH-362|https://issues.apache.org/jira/browse/MATH-362]).
* An alternative is to go back to a "simple", algorithm-specific convergence
check (i.e. what most users are already used to), with tolerances parameters
passed through the constructor.
* To enhance the checking (in effect, trying to recover the flexibility of
{{ConvergenceChecker}} functionality), I'd propose to optionally enable a
"callback" to be called after each iteration with two arguments (the previous
and current best point/value pairs) and that should return "SUCCESS" (an
{{enum}}) if the current best point is good enough and "CONTINUE" if it is not.
It is indeed very much like the existing {{converged}} method but the main idea
is that the default convergence check (the one tied with the optimization
algorithm) will always have a higher priority: if it passes, we exit from the
iteration loop. The callback is a _second_ check that gives the opportunity to
exit earlier if some additional, user-defined, criteria are met.
Combining the last two points, we
# always provide a self-consistent behaviour in the simple use-cases
# make it clear that the user can optionally define an "out-of-band" checking
criterion: it is clearly independent in that any parameter (apart from the
previous and current points) used to check convergence must be separately
collected and passed to the constructor of the callback.
What do you think?
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOp
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904971#action_12904971
]
Luc Maisonobe commented on MATH-413:
The ConvergenceChecker principle is inherited from the mantissa library that
was merged with commons-math in 2006-2007.
I introduced it in mantissa because it leveraged very simply the two most
important cases I encountered : sometimes one wants to check convergence on the
free variable (i.e. the input) and sometimes one wants to check convergence on
the cost (i.e. the output). I think even one time I add to check on both input
and output using a custom checker but am not really sure. These two cases are
real one and depend on the user problem.
I don't agree with the word "complex" about the interface, even if quoted.
Furthermore, the user is not forced to implement it by himself, there are two
implementations that fits the two cases above: the SimpleXxxChecker. So a user
typically has to call:
optimizer.setConvergenceChecker(new SimpleVectorialValueChecker(epsilon1,
epsilon2));
This is a separate call from the optimize call, but it could be merged if
needed. However, I still consider being able to check convergence either in x
or in y is important and should be kept.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceExcepti
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904963#action_12904963
]
Dimitri Pourbaix commented on MATH-413:
---
> In light of all these problems, we should seriously re-examine whether the
> ConvergenceChecker approach is the right way to go.
Even without these problems, I am very sceptical about the usefulness of so
much flexibility. What users of numerical libraries have been used to is to
pass some kind of precision (absolute or relative) to a solver, minimizer ...
and to get the anser which matches the criterion.
Why would CM's users expect more? I do see why some CM developers would like
to give more, simply because it is fun to design such a general abstract
checker but do the users care? If we put the most flexible checker, the users
will have to fill it which might be a good reason to switch to a simpler (less
flexible) library.
As long as the documentation describes how the precision (argument) is used by
the class, the user knows what to expect.
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set
> (see e.g. "BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be
> a NPE. The alternative is to initialize a default checker that will never be
> used in case the user had intended to explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked
> "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it
> is never used.
> # There remains some duplicate code (such as the "multi-start loop" in the
> various "MultiStart..." implementations).
> # The "ConvergenceChecker" interface is very general (the "converged" method
> can take any number of "...PointValuePair"). However there remains a
> "semantic" problem: One cannot be sure that the list of points means the same
> thing for the caller of "converged" and within the implementation of the
> "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient
> evaluations to the function evaluation counter. In
> "LevenbergMarquartdOptimizer" for example, it would be unfair to do so.
> Currently I had to remove all tests referring to gradient and Jacobian
> evaluations.
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer",
> occurences of "OptimizationException" were replaced by the unchecked
> "ConvergenceException" but in some cases it might not be the most appropriate
> one.
> # "MultiStartUnivariateRealOptimizer": in the other classes
> ("MultiStartMultivariate...") similar to this one, the randomization is on
> the firts-g
[jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
[
https://issues.apache.org/jira/browse/MATH-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904592#action_12904592
]
Gilles commented on MATH-413:
-
IMO, the most urgent issue to be dealt with is the design decision to make the
convergence checking criteria independent from the optimization algorithm.
[This has a direct influence on the API and should be stabilized in v3.0.]
Points 1, 2, 6, 10 in the issue description indicate the problems with this
approach.
IMO, while modularization is often a good thing, the attempt produces several
drawbacks in this case:
* The user interface is more difficult than it could be: the user has to
specify a "complex" {{ConvergenceChecker}} object, instead of just {{double}}s
(relative and/or absolute threshold/tolerance/accuracy).
* At the CM level, there is no single implementation of {{ConvergenceChecker}}
that is applicable for every algorithm (see {{BrentOptimizer}} and
{{LevenbergMarquardtOptimizer}}).
* If we don't set a default, the user will get a NPE.
* If we set a default convergence checker, in an attempt to make usage easier,
it will still not be very useful because the user will have to set a new
checker as soon as he needs to change the algorithm accuracy.
* The user cannot rely on the description of convergence given by a
{{ConvergenceChecker}}'s implementation because if he chooses a checker that is
"incompatible" with the optimization algorithm, the behaviour is "undefined".
[I have overridden {{setConvergenceChecker}} to forbid the setting of anything
other than a {{BrentConvergenceChecker}} but this completely voids the intended
flexibility of having a separate convergence checking procedure.]
* If we want to enforce the strict separation between optimization and
convergence checking, we'll have to _modify_ standard algorithm so that they
fit into the mold (i.e. we must ensure that "convergence checking" and
"optimization" parts are indeed independent). This may not even be possible
without removing features of some algorithms (e.g. specific tolerance
definitions in "Levenberg-Marquardt"). And if it is done, then the resulting
algorithm will not be standard anymore!
In light of all these problems, we should seriously re-examine whether the
{{ConvergenceChecker}} approach is the right way to go.
In the end, it is very well possible the best that can be done is to pass
appropriate arguments to the constructor of each optimizer's implementation.
[With the consequence that if one wants to change the convergence checking
behaviour, one has to create a new instance. (Side note: this is consistent
with going towards instances' immutability.)]
> Miscellaneous issues concerning the "optimization" package
> --
>
> Key: MATH-413
> URL: https://issues.apache.org/jira/browse/MATH-413
> Project: Commons Math
> Issue Type: Bug
>Reporter: Gilles
> Fix For: 3.0
>
>
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|https://issues.apache.org/jira/browse/MATH-394]
> * [MATH-397|https://issues.apache.org/jira/browse/MATH-397]
> * [MATH-404|https://issues.apache.org/jira/browse/MATH-404]
> This issue collects the currently still unsatisfactory code (not necessarily
> sorted in order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used.
> "LevenbergMarquardtOptimizer" also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization
> algorithm creates problems (conceptual and practical):
> ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm
> passes "points" to the convergence checker, but the actual meaning of the
> points can very well be different in the caller (optimization algorithm) and
> the callee (convergence checker).
> ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend
> on the tolerances within the main algorithm. Since tolerances come with
> "ConvergenceChecker" and so can be changed at any time, it is awkward to
> adapt the values within the line search optimizer without exposing its
> internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync,
> although I did try to update them all.
> # Class "DirectSearchOptimizer" (in package "optimization.direct") inherits
> from class "AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base
> implementations (abstract class that contain the boiler-plate code) are in
> package "optimization.general" (e.g.
> "DifferentiableMultivariateVectorialOptimizer" and
> "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the converge
