Re: [Scilab-users] Optim & NelderMead use [closed]
Hi Paul,
You should be careful when using the square root function as it is not
differentiable at 0 (cf the attached file which illustrates this point
in your case). This will leads to issues and prevent optim from "really"
(in a sound way) converging towards the solution.
To address the issue, you can
- reformulate your objective function so that it is smooth near the
solution (you will still have an issue at 0 where g is not
differentiable though),
- use order 0 methods (still, some of them might fail on non-smooth
optimisation problems as the minimum might be very "narrow")
- dig into non-smooth optimisation (see e.g.
http://napsu.karmitsa.fi/nso/)
Regards,
Pierre
Le 16.01.2017 10:30, Carrico, Paul a écrit :
> Hi all
>
> After performing tests (and modifying the target function as it should have
> been done first), I can better understand how to use 'optim' and 'Neldermead'
> procedures.
>
> For my needs the mean flags are :
>
> - Step h in numderivative à usefull reading as "EE 221 Numerical
> Computing" Scott Hudson
>
> - The threshold epsg in optim (%eps is the default value - such high
> accuracy is not necessary for my application - furthermore using a value such
> as 1e-5 leads to err=1 that is correct for checking)
>
> - Ditto for Nelder-Mead and '-tolfunrelative' & '-tolxrelative'
>
> Now it works fine :-)
>
> Thanks all for the support
>
> Paul
>
> #
>
> mode(0)
>
> clear
>
> global count_use;
>
> count_use = 0;
>
> _// _
>
> function F=lineaire(X, A2, B2)
>
> F = A2*X+B2;
>
> endfunction
>
> _// _
>
> function G=racine(X, A1, B1)
>
> G = sqrt(A1*X) + B1;
>
> endfunction
>
> _// _
>
> function F=target(X, A1, B1, A2, B2)
>
> val_lin = lineaire(X,A2,B2);
>
> val_rac = racine(X,A1,B1);
>
> F = sqrt((val_lin - val_rac)^2);
>
> global count_use;
>
> count_use = count_use +1;
>
> endfunction
>
> _// Cost function: _
>
> function [F, G, IND]=cost(X, IND, A1, B1, A2, B2)
>
> F = target(X);
>
> _//g = numderivative(target, x.',order = 4); _
>
> G = numderivative(target, X.',1e-3, order = 4); _// 1E-3 => see EE 221
> "Numerical Computing" Scott Hudson_
>
> _// Study of the influence of h on the number of target function calculation
> & the fopt accuracy:_
>
> _// (epsg = %eps here)_
>
> _// h = 1.e-1 => number = 220 & fopt = 2.242026e-05_
>
> _// h = 1.e-2 => number = 195 & fopt = 2.267564e-07_
>
> _// h = 1.e-3 => number = 170 & fopt = 2.189495e-09 ++_
>
> _// h = 1.e-4 => number = 190 & fopt = 1.941203e-11_
>
> _// h = 1.e-5 => number = 215 & fopt = 2.131628e-13_
>
> _// h = 1.e-6 => number = 235 & fopt = 0._
>
> _// h = 1.e-7 => number = 255 & fopt = 7.105427e-15_
>
> _// h = 1.e-8 => number = 275 & fopt = 0._
>
> endfunction
>
> _// *_
>
> _// optimisation with optim_
>
> initial_parameters = [10]
>
> lower_bounds = [0];
>
> upper_bounds = [1000];
>
> nocf = 1000; _// number max of call of f_
>
> niter = 1000;_// number max of iterations_
>
> a1 = 30;
>
> b1 = 2.5;
>
> a2 = 1;
>
> b2 = 2;
>
> epsg = 1e-5;_// gradient norm threshold (%eps by defaut) --> lead to err
> = 1 !!!_
>
> _//epsg = %eps; // lead to Err = 13_
>
> epsf = 0; _//threshold controlling decreasing of f (epsf = 0 by defaut)_
>
> costf = list (cost, a1, b1, a2, b2);
>
> [fopt, xopt, gopt, work, iters, evals, err] =
> optim(costf,'b',lower_bounds,upper_bounds,initial_parameters,'qn','ar',nocf,niter,epsg,epsf);
>
> printf("Optimized value : %g\n",xopt);
>
> printf("min cost function value (should be as closed as possible to 0) ;
> %e\n",fopt);
>
> printf('Number of calculations = %d !!!\n',count_use);
>
> _// Curves definition_
>
> x = linspace(0,50,1000)';
>
> plot_raci = racine(x,a1,b1);
>
> plot_lin = lineaire(x,a2,b2);
>
> scf(1);
>
> drawlater();
>
> xgrid(3);
>
> f = gcf();
>
> _//f _
>
> f.figure_size = [1000, 1000];
>
> f.background = color(255,255,255);
>
> a = gca();
>
> _//a _
>
> a.font_size = 2;
>
> a.x_label.text = "X axis" ;
>
> a.x_location="bottom";
>
> a.x_label.font_angle=0;
>
> a.x_label.font_size = 4;
>
> a.y_label.text = "Y axis";
>
> a.y_location="left";
>
> a.y_label.font_angle=-90;
>
> a.Y_label.font_size = 4;
>
> a.title.text = "Title";
>
> a.title.font_size = 5;
>
> a.line_style = 1;
>
> _// Curves plot_
>
> plot(x,plot_lin);
>
> e1 = gce();
>
> p1 = e1.children;
>
> p1.thickness = 1;
>
> p1.line_style = 1;
>
> p1.foreground = 3;
>
> plot(x,plot_raci);
>
> e2 = gce();
>
> p2 = e2.children;
Re: [Scilab-users] Optim & NelderMead use [closed]
Hi Paul,
your cost function
*f*= sqrt((val_lin - val_rac)^2);
hasn't changed, since sqrt(x^2)=abs(x). What I meant before is replacing
*f*= abs(val_lin - val_rac);
by
*f*= (val_lin - val_rac)^2;
in order to make it differentiable. When using a non-differentiable cost
function together with numderivative, it seems logical that tweaking the
step size could artificially help convergence.
S.
Le 16/01/2017 à 10:30, Carrico, Paul a écrit :
Hi all
After performing tests (and modifying the target function as it
should have been done first), I can better understand how to use
‘optim’ and ‘Neldermead’ procedures.
For my needs the mean flags are :
-Step h in numderivative àusefull reading as “EE 221 Numerical
Computing" Scott Hudson
-The threshold epsg in optim (%eps is the default value – such high
accuracy is not necessary for my application – furthermore using a
value such as 1e-5 leads to err=1 that is correct for checking)
-Ditto for Nelder-Mead and ‘-tolfunrelative’ & ‘-tolxrelative’
Now it works fine :-)
Thanks all for the support
Paul
#
mode(0)
clear
globalcount_use;
count_use= 0;
/// /
function*f*=_lineaire_(*x*, *a2*, *b2*)
*f* = *a2***x*+*b2*;
endfunction
/// /
function*g*=_racine_(*x*, *a1*, *b1*)
*g* = sqrt(*a1***x*) + *b1*;
endfunction
/// /
function*f*=_target_(*x*, *a1*, *b1*, *a2*, *b2*)
val_lin= _lineaire_(*x*,*a2*,*b2*);
val_rac = _racine_(*x*,*a1*,*b1*);
*f*= sqrt((val_lin - val_rac)^2);
global count_use;
count_use = count_use +1;
endfunction
/// Cost function: /
function[*f*, *g*, *ind*]=_cost_(*x*, *ind*, *a1*, *b1*, *a2*, *b2*)
*f* = _target_(*x*);
///g = numderivative(target, x.',order = 4); /
*g* = _numderivative_(_target_, *x*.',1e-3, order = 4); /// 1E-3 =>
see EE 221 "Numerical Computing" Scott Hudson /
/// Study of the influence of h on the number of target function
calculation & the fopt accuracy:/
/// (epsg = %eps here)/
/// h = 1.e-1 => number = 220 & fopt = 2.242026e-05/
/// h = 1.e-2 => number = 195 & fopt = 2.267564e-07/
/// h = 1.e-3 => number = 170 & fopt = 2.189495e-09 ++/
/// h = 1.e-4 => number = 190 & fopt = 1.941203e-11/
/// h = 1.e-5 => number = 215 & fopt = 2.131628e-13/
/// h = 1.e-6 => number = 235 & fopt = 0./
/// h = 1.e-7 => number = 255 & fopt = 7.105427e-15/
/// h = 1.e-8 => number = 275 & fopt = 0./
endfunction
/// */
/// optimisation with optim/
initial_parameters= [10]
lower_bounds= [0];
upper_bounds= [1000];
nocf= 1000; /// number max of call of f/
niter= 1000; /// number max of iterations/
a1= 30;
b1= 2.5;
a2= 1;
b2= 2;
epsg= 1e-5; /// gradient norm threshold (%eps by defaut) --> lead to
err = 1 !!!/
///epsg = %eps; // lead to Err = 13/
epsf= 0; ///threshold controlling decreasing of f (epsf = 0 by defaut)/
costf= list (_cost_, a1, b1, a2, b2);
[fopt,xopt, gopt, work, iters, evals, err] =
optim(costf,'b',lower_bounds,upper_bounds,initial_parameters,'qn','ar',nocf,niter,epsg,epsf);
printf("Optimized value : %g\n",xopt);
printf("min cost function value (should be as closed as possible to 0)
; %e\n",fopt);
printf('Number of calculations = %d !!!\n',count_use);
/// Curves definition/
x= _linspace_(0,50,1000)';
plot_raci= _racine_(x,a1,b1);
plot_lin= _lineaire_(x,a2,b2);
_scf_(1);
drawlater();
xgrid(3);
f= _gcf_();
///f /
f.figure_size= [1000, 1000];
f.background= color(255,255,255);
a= _gca_();
///a /
a.font_size= 2;
a.x_label.text= "X axis" ;
a.x_location="bottom";
a.x_label.font_angle=0;
a.x_label.font_size= 4;
a.y_label.text= "Y axis";
a.y_location="left";
a.y_label.font_angle=-90;
a.Y_label.font_size= 4;
a.title.text= "Title";
a.title.font_size= 5;
a.line_style= 1;
/// Curves plot/
_plot_(x,plot_lin);
e1= _gce_();
p1= e1.children;
p1.thickness= 1;
p1.line_style= 1;
p1.foreground= 3;
_plot_(x,plot_raci);
e2= _gce_();
p2= e2.children;
p2.thickness= 1;
p2.line_style= 1;
p2.foreground= 2;
drawnow();
*/EXPORT CONTROL :
/**Cet email ne contient pas de données techniques
This email does not contain technical data*
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
___
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
