Research Summary

Developing Methodology for Assessing Medium and Large Scale Environmental Models
E. David Ford, Rie Komuro, Marianne Turley and Joel Reynolds
We have developed a new method, and associated software, for assessment and inference of complex models. It uses multiple outputs from a model as goodness-of-fit measures that are assessed, simultaneously, against data or standards defined from a theory. The software, Poreto_Evolve, uses an evolutionary optimization algorithm to calculate the model's Pareto Optimal Set - the parameterizations that simulate unique non-dominated groups of output criteria effectively. Failure to achieve all criteria simultaneously is used as a diagnostic of deficiencies, with investigation of the Pareto Optimal Set providing guidance into deficiency sources. Our use of the Pareto concept as an analytical assessment tool is novel; the usual application is to define tolerance ranges in engineering design.

Currently we are applying this method to process-based models in biology and ecology, where complexity is normal. Scientists reviewing ecological modeling consider that inadequate general progress has been made in this field. We suggest this is due to lack of effective assessment, and that multiple criteria assessment provides an effective method. Our recent experience has also shown that this approach to multiple criteria optimization can provide an effective alternative to traditional methods of parameter inference for non-linear and high-order statistical models for complex data.

Our objective is to further develop the shareware, Pareto_Evolve, extend its range of application, and publicize its availability and applications through our web-site and discussion board. We propose a number of investigations and benchmarking exercises (additional to those already completed) to improve the search algorithm, the user-guidelines for applying the heuristic search software, and to further extend the methodology. We also propose a plan for integrating the approach into graduate education [and the local research community?] by creating a consulting center for ecological model assessment.