Dear Leonid
Thank you for this informative response. It'll definitely help me in finding my bearings. Cheers Andreas Von: Leonid Gibiansky [mailto:lgibian...@quantpharm.com] Gesendet: Dienstag, 7. September 2010 17:08 An: Steingötter Andreas Cc: nmusers; Rickmer Braren Betreff: Re: [NMusers] Negative eigenvalues, over-paramterization, finding most sensitive parameter Andreas, I doubt that one can give a general answer (without looking on the specific model) but here is my understanding of the situation: In over-parametrized models, there is one or more degenerate directions (in the parameter space) where changes of the parameters do not change the fit (i.e., where there is no data to estimate each parameter, only some combination). The model can be well-defined in the orthogonal directions. The simplest example is oral absorption: without IV data, F( bioavaialbility), CL and V are not definable. However, CL/F and V/F can be estimated. This leads to two different situations: if your critical parameter is in the "well-defined" space, then you may use it as a biomarker. If, on the other hand, this parameter is in the degenerate space, it cannot be used since its value is not stable. The burden of proof is of course on the presenter. One can support it by - small RSEs on the parameter of interest, if you can get them; - no correlation with other parameters (either in bootstrap samples, or in the history of SAEM iterations, or by investigation the variance-covariance matrix of the parameter estimates); - starting the model run with perturbed values of this parameter to show that the final estimate does not depend on the initial values; - etc. Alternative is to try to make the entire model stable by fixing some parameters at the biologically plausible values: if the model with fixed parameters is still flexible enough to describe the entire range of available data, one can use this model until some experimental results provide the data (and the need) to free and estimate those fixed parameters. Thanks Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 On 9/7/2010 2:42 AM, Steingötter Andreas wrote: Hello Nick, Leonid, Dieter As a beginner in NONMEM society, I am becoming very curious in your current discussion. In a related situation I may need some helpful comments and already excuse myself if this question has been answered many times before. THE SITUATION: We have tissue (let's say tumor tissue) that has some anatomical structure known by histology. So we know roughly how many blood vessels (to get an idea on blood flow/perfusion), how many vital tissue (to get an idea where the blood can distribute or perfuse into) and how many dead tissue (where only blood diffusion can take place) is present. We inject a substance (i.v. bolus) and macroscopically follow its kinetics through this tissue, i.e. have a concentration curve of this tissue. At the same time we can also measure the kinetics of other (neighboring) healthy tissues to generate additional concentration curves. All these curves exhibit bi- or multi-exponential behavior. First PROBLEM: We only observe on the macroscopic scale and therefore we have a mixture of tissue kinetics for each concentration curve. However, we are able to create a model that perfectly describes the concentration curves of all tissues as Dieter has done. This model is very likely to be over-parameterized. In a SECOND STEP we treat this tumor tissue and see some changes in tumor structure. But don't have clue how these changes in structure relate to changes in function, e.g. what rate constant, volume flow or distribution volume is most sensitive to such a change in blood vessels. For later purpose and to omit the need for histology the aim is to identify this sensitive parameter and use it as some kind of biomarker. NOW THE QUESTION: How to best proceed to (numerically) find this most sensitive parameter in the model? Do we start from the model that best describes the concentration curves and go backwards again. Do we pick a first potential parameter and reduce the model until this parameter is robust (shows no correlation) and do the same again for other possible candidates? Do we then end up with one model for each parameter of interest (which does not make sense to me)? To my understanding, for a given (rich) data set there can only be a compromise between model fit and robustness of parameter estimation and finally someone has to decide what that is. This compromise then needs to be tested and validated again and again by generating or including new data. BEGINNER's QUESTION: If we show that we have done the testing and tweaking with regard to what we (pretend to) know from physiology/biology/histology and are aware of (and describe) the uncertainty in parameter estimates for the selected, probably over parameterized model, would expert reviewers of your caliber still ask for more model simplification? Sorry for being so elaborate and many thanks for comments and critics of every description. Andreas Andreas Steingötter, PhD Division for Gastroenterology and Hepatology Department of Internal Medicine University Hospital Zurich Von: owner-nmus...@globomaxnm.com [mailto:owner-nmus...@globomaxnm.com] Im Auftrag von Nick Holford Gesendet: Dienstag, 7. September 2010 04:19 An: nmusers Betreff: Re: [NMusers] How serious are negative eigenvalues? Dieter, You ask: My question: can we trust this fit? The answer depends on why you are doing the modelling. If your goal is to describe the time course of concentrations then the overall ability of the model to describe what you saw depends on the totality of the model and its parameters. The model may be overparameterized but it may still do what you want it to do i.e. describe (and predict) the time course of concentrations in each compartment. If you are satisfied with the VPC showing that simulations from the model appropriately describe the observed concentrations then I think the answer to your question is yes. On the other hand if the goal is to estimate the size of one or more critical parameters then you will need to pay attention to how well these parameters are estimated. As Leonid has pointed out it seems that at least some of the model parameters are not well identified. This may be unimportant if the parameters you want to describe are robustly estimated. For example, if you had a simple PK model with samples mainly taken at steady state with few observations during absorption then you may get a good estimate of clearance but a rather poor estimate of KA. You cannot simply remove a parameter such as KA (you have to describe the sparse absorption somehow) but it will have little impact on the clearance estimate. Thus the model can be trusted for the purpose of estimating clearance but not absorption rate. Nick On 7/09/2010 12:11 a.m., Dieter Menne wrote: Dear Nmusers, we have very rich data from MRI concentration measurements, with 11 compartments and multiple compartments observed. The model is fit via SAEM (nburn=2000), and followed by an IMPMAP as in the described in the 7.1.2 manual. OMEGA is band with pair-wise block correlations in the following style: $OMEGA BLOCK(2) .02 ;CL 0.01 0.06 ; VC $OMEGA BLOCK(2) 5.4 ; QMVP 0.001 0.05 ;VMVP $OMEGA BLOCK(2) 0.06 ; QTVP 0.001 0.25 ;VTPV $EST PRINT=1 METHOD=SAEM INTERACTION NBURN=2000 NITER=200 CTYPE=2 NSIG=2 FILE=SAEM.EXT $EST METHOD=IMPMAP EONLY = 1 INTERACTION ISAMPLE=1000 NITER=5 FILE=IMP.EXT $COV PRINT=E UNCONDITIONAL Fits and CWRES diagnostics are perfect, and VPC checks are good. However, we have negative eigenvalues (the following example has been edited by removing digits) ETAPval = 0.2 0.2 0.3 0.04 0.8 0.95 0.003 0.1 0.6 0.4 0.9 0.1 0.5 0.4 0.2 0.8 0.3 0.3 0.4 0.01 0.8 ETAshr% = 13. 0.4 38 20 23 33 46 30 18 41 54 22 2. 26. 49. 12. 0.07 24. 18. 35. 2.5 EPSshr% = 7.5 8.1 Number of Negative Eigenvalues in Matrix= 7 Most negative value= -65339. Most positive value= 88796185.9 Forcing positive definiteness Root mean square deviation of matrix from original= 1.37E-003 My question: can we trust this fit? Dieter Menne Menne Biomed/University Hospital of Zürich -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53 email: n.holf...@auckland.ac.nz http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
