Robert Holmes wrote: > In my role as FRIAM's official Cassandra (I should get a T-shirt > printed), has anyone ever shown that these highly intensive > simulations give quantitatively better results than, say, something > written on Owen's laptop in NetLogo? Do we know that we get a better > assessment of (for example) the robustness of policies for stopping > epidemic spread or do we rely on the "more is better" argument? ("Of > course, the results are better - we have an NSF grant and 15 > supercomputers"). In a model, each added parameter has a statistical cost that needs to be justified statistically in terms of how more accurate the simulation prediction becomes. It happens that for agent models, there's also a strong practical incentive to reduce them because the sensitivity analysis gets so hard to do. For example, a sensitivity analysis can easily involve a huge space to explore even making many simplifying assumptions. If simulation only runs for 1 minute and there are 65536 plausible simulation configurations (e.g. 16 switches to throw, presumably from a space of many more implausible ones), that's still 45 days of runtime on one computer to fully explore the space and build a distribution of outcomes. It is not hard to find 16 plausible parameters to put into a realistic model.
Rare sequences of events can be important to disease propagation and their consequences can depend very much on population density, transportation behaviors of that population, and available health interventions. A modeler could describe a particular parameterization of a particular microcosm, but that's not probably what most planners are interested in, except to the extent some are just fishing around for a cartoon dramatization to show their boss. Unfortunately, to observe the rare event in a realistic setting, each instance of a simulation can be pretty substantial, because you're waiting for those rare events in a changing environment. Given a goal to correlate that changing environment with the rare events, it becomes all the harder because the frequently at which one can observe the co-occurring events is even more rare, thus getting a confidence interval on those combinations is hard. Facts about model components can be hard to come by, independent of intuition of dynamics about the whole thing behaves. You may not know exactly how a disease behaves in specific contexts, so you have to cover a range of possibility (e.g. one switch setting being the high and low range of what you think could happen best and worst case, assuming that it is linear and independent of other variables, which it may not be). Marcus ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org