On 18 May 2010 12:09, Olaf Till <[email protected]> wrote: > On Tue, May 18, 2010 at 08:46:52AM +0100, Gary wrote: >> Thanks Olaf, see bellow for comments. >> >> On 17 May 2010 11:03, Olaf Till <[email protected]> wrote: >> > On Mon, May 17, 2010 at 08:43:20AM +0100, Gary wrote: >> >> I haven't rerun the script as it takes some time to complete, but I >> >> have attached the raw data from which I generated the spreadsheet. >> > >> > The last parameter is computed to be 0, which violates the lower bound >> > of 1e-30. Constraints (and bounds) are accounted for by leasqr with a >> > "projection algorithm" --- if a constraint is active and remains >> > active over the next step, the next parameters are computed to be at >> > the "border" of the constraint (e.g. exactly at the bound). This >> > computation was remarkably accurate in my own tests, but I think one >> > has to live with some inaccuracy. I'd guess that such inaccuracy is >> > the reason for the above violation. >> >> Is a step the same as an iteration? The 1st iteration of the leasqr >> algorithm ends around fit function call >> 103, > > Within a single iteration (if no non-linear constraints are given): > > - The gradient of the predicted values is determined. Since, AFAICS, > no function for an analytic gradient has been provided, this is done > with finite differences, which leads to as many calls of your fit > function as you have parameters, or twice as many if the "dp" > argument is not given or has positive elements. > > - A few search directions are tried, each time with one call to the > fit function. > > So with 25 parameters a bit more than 50 calls should be expected per > iteration, but not > 100 ... > >> During the debug run, one parameter that should be +ve and ~10^18 >> changes in the following way: >> >> CALL VALUE >> 1-102: +ve and ~ 10^-18 >> then -ve until >> 116: -1.2025e-19 >> 117: +1.2025e-19 >> 118: +1e-30 >> 119: -1.2025e-19 >> 120-: -ve >> >> Is this switching between the sign and being bound and then not bound >> within a singer iteration consistent with the inaccuracy of the >> methods for implementing bounds? > > I can't see what happens there without running the code. > >> > The lower bound should be just increased to a "reasonable" value where >> > small violations can be tolerated. I'm not sure at the moment how >> > parameter scale affects this problem, but would guess that one should >> > either increase the lower bound according to the scale of the >> > respective parameter, or scale the parameters (by applying a factor to >> > them in the model function) so that their order of magnitude gets >> > similar. >> >> Ok, so small in "small violations" means on an absolute and not >> relative scale? > > At least not relative to the bound. > >> For info, by using the absolute values of the >> parameters, the optimisation works well with parameters that are a >> factor of maybe 10^30 different. I will try scaling them to similar >> values and check to see if the bound violation still occurs. >> >> > Further notes: >> > >> > The initial parameters 14, 15, and 16 are outside the lower bound; >> > leasqr will initially set them to the lower bound with a warning (and >> > then start optimization). >> >> Yes, I checked this and the bounds are respected at the start of the run. >> >> > You have set niter = 2, which makes it pretty hopeless to achieve >> > convergence. Accordingly, kvg == 0 had been returned, which means that >> > no convergance had been achieved. >> >> This was purely to save time. As I said before, a full run takes a >> long time to complete. >> >> > Totally unrelated hint: It makes life easier if the response to a mail >> > is typed _below_ the text you respond to (avoid "top posting"). Don't >> > take me wrong for criticizing that :-) , it's better to keep to this >> > practice. >> >> I agree! I'm new to google mail and hadn't noticed that the default is >> to top post. >> >> > >> >> Perhaps a few additional comments in the function description text, >> explaining the limitations of the bounds would be useful to others who >> may encounter the same or similar problems. > > Yes, probably. > > dfdp does not cope well with the mentioned inaccuracy in bounds, I'll > try to fix that soon. It's probably also better to correct the > parameters to the bounds after each step, but only if no constraints > except bounds are specified. > > Olaf > > ------------------------------------------------------------------------------ > > _______________________________________________ > Octave-dev mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/octave-dev >
Re. the number of calls per iteration: I'm using the built-in finite differencing function to calculate the jacobian. If I specify iter = 1, leasqr stops after 102 calls to the function, but if I allow as many iterations as are needed to reach convergence (often 10 to 20 iterations are gone through), there are a little more than 50 calls per iteration. Is this the expected behaviour? Gary ------------------------------------------------------------------------------ _______________________________________________ Octave-dev mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/octave-dev
