Nick - a) The usual definition of 'number of significant digits' is -log10(relative precision). Thus a sigdig of 3 is a precision of 1 part in 1000, and a sigdig of 2 corresponds to 1% precision, not 10% as in your example. b) that being said, the sigdigs in the parameters reported by NONMEM need to be taken with a grain of salt - they probably represent best case, 'speed of light' type numbers where the real precision may be considerably worse. I do not know specifically how they are computed, but my guess is that it is based on the fact that in a converged problem, the relative gradient has been driven below some specified tolerance. One can then infer precision from the condition number of the Hessian of the overall objective function and the actual relative gradients. But NONMEM uses a quasi-Newton method - there is no Hessian available to the method, but only a stand-in accumulated curvature matrix (a 'pseudo Hessian') that is usually much better conditioned than the actual Hessian. The only thing that can really be concluded is that, at the moment the top level iteration is stopped and convergence declared based on the relative gradient, the next iteration , if it were done, would not change the parameter estimates by more than the reported sigdig value. This is quite a different conclusion than reported parameter estimates are with sigdigits of the 'true' values.
c) I know you have often argued that the failure of a covariance step has little or no evidential value for determining whether the minimization step was 'successful', and I generally agree with you. But the failure of the covariance step does mean that the Hessian could not be numerically estimated at all (failed the positive definiteness test). This does provide some additional evidence that one should be even more skeptical of the reported sigdig values of the parameter estimates. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nick Holford Sent: Tuesday, October 22, 2013 4:45 AM To: [email protected] Subject: Re: [NMusers] Change of NSIG or R matrix Xinting, First of all 'successful minimization' has nothing to do with a good model. NONMEM's internal decision to declare success or termination is often a pseudo-random choice. If you look at the sigdigs of the estimate you will typically find that the lowest value is 2.9 and many others are greater than 5. This gives you a clue to which parameters are well determined and which are less well known. It is a not a YES/NO decision. Second, NSIG determines the number of significant digits in the parameter estimates. If you choose a number less than 3 then it means you don't care if the answer is 10.1 or 10.9. They both have 2 sig digs but the estimates differ by nearly 10%. There is a large body of empirical literature that has relied on NSIG=3 (or more). I do not see any reason to ignore this in order to get a meaningless "minimization successful" message from a random number generator. I look forward to hearing from "many" to understand why they believe that "minimization successful" indicates that the model results are somehow better even though the parameter estimates have hardly any significant digits. Nick On 22/10/2013 9:17 p.m., Xinting Wang wrote: > Dear Nick, > > Thank you very much for your suggestion. Could you explain a little > bit about the statement regarding NSIG < 3? I seem to remember that > many suggested to use a smaller NSIG to get a successful minimization. > > Dear Leonid, > > I read about the recommendation of SIGL, NSIG and TOL, but I am not > quite familiar with the use of these options in subroutine ADVAN4. If > I set SIGL a fixed value, let's say 12, and NSIG 3, does this mean I > also have to identify a value for TOL in $subroutine? I appreciate > your help very much. > > Thank you both. > > Regards > > > > On 8 October 2013 21:59, Leonid Gibiansky <[email protected] > <mailto:[email protected]>> wrote: > > Yes, it should be fine to use S matrix if you cannot get default > to run, and use NSIG larger or smaller than default value of 3 > (although this is not guaranteed, usually NSIG does not change the > OF value or parameter estimates in any significant way). Note that > Nonmem manual recommends that SIGL >= 3*NSIG, TOL >= SIGL. > Separate SIGL can be set on COV step, and it is recommended that > SIGL >= 4*NSIG on COV step. In real life I've seen many examples > where larger NSIG and SIGL resulted in successful COV step, and > also many examples when default values were better (in getting COV > step). UNCONDITIONAL on COV step allows you to run COV even when > minimization ended with some error. > > Contrary to Nick's experience, I found that COV step is useful as > it reveals which of the model parameters are poorly estimated, and > that CI based on SE are usually quite good and are in a general > agreement with the bootstrap CI, but it may depend on the problem. > > Leonid > > > -------------------------------------- > Leonid Gibiansky, Ph.D. > President, QuantPharm LLC > web: www.quantpharm.com <http://www.quantpharm.com> > e-mail: LGibiansky at quantpharm.com <http://quantpharm.com> > tel: (301) 767 5566 <tel:%28301%29%20767%205566> > > > > > On 10/8/2013 3:57 AM, Xinting Wang wrote: > > Dear all, > > I have a naive question regarding the modeling building process in > NONMEM. With more and more covariates added in the model, I > often come > across an error message saying that "ERROR 134", or R MATRIX > SINGULAR. > > After searching from the internet, I learned that changing NSIG in > $ESTIMATION and MATRIX=S in $COV would be helpful for both > problems > respectively. And from my own experience, it dose help with > the modeling > building. > > However, my concern is, I used different NSIG and MATRIX in > the previous > steps. Is it proper to use different NSIGs and MATRICE in a > single model > building? If not, could you please explain this a little bit? > > Thank you in advance! > > Best Regards > -- > Xinting > Wang > > > > > -- > Xinting -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ +64(21)46 23 53 email: [email protected] http://holford.fmhs.auckland.ac.nz/ Holford NHG. Disease progression and neuroscience. Journal of Pharmacokinetics and Pharmacodynamics. 2013;40:369-76 http://link.springer.com/article/10.1007/s10928-013-9316-2 Holford N, Heo Y-A, Anderson B. A pharmacokinetic standard for babies and adults. J Pharm Sci. 2013: http://onlinelibrary.wiley.com/doi/10.1002/jps.23574/abstract Holford N. A time to event tutorial for pharmacometricians. CPT:PSP. 2013;2: http://www.nature.com/psp/journal/v2/n5/full/psp201318a.html Holford NHG. Clinical pharmacology = disease progression + drug action. British Journal of Clinical Pharmacology. 2013: http://onlinelibrary.wiley.com/doi/10.1111/bcp.12170/abstract NOTICE: The information contained in this electronic mail message is intended only for the personal and confidential use of the designated recipient(s) named above. This message may be an attorney-client communication, may be protected by the work product doctrine, and may be subject to a protective order. As such, this message is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by telephone and e-mail and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). Thank you. buSp9xeMeKEbrUze
