INTRODUCTORY REMARKS Lincoln E. Moses Administrator Energy Information Administration Department of Energy The Energy Information Administration (EIA) has a keen interest in the subject matter of this workshop and a strong reason to be one of its sponsors. These reasons come under several different rubrics. First, there are requirements in the law for EIA to validate its models and increase access to them by interested parties in the public. Second, we hope that from this conference we will gain many clues and indications as to how to improve the quality of some of our energy models. Third, specifically to advance the abilities to assess and validate models will respond to needs, both inside and outside of our organization. Fourth, we trust that the thought that appears in this meeting will advance not only the techniques of assessment and validation but will reach far toward deeper understanding of and ability to improve modeling itself. And, finally, we dare hope that a by-product of the larger technical understandings to be looked for will increase our ability to take a grip on one task which EIA regards as central to its work; that task is to "give useful indications of the uncertainty of each forecast." Thus, the Energy Information Administration is pleased to participate in the organization of the conference and in the publication of the results of the conference and looks forward to benefits from this round, and the possibility of further participation in future such rounds. WELCOME A. J. Goldman Chief Operations Research Division National Bureau of Standards Good morning. The unique capabilities of mathematical modeling, as an aid to decisions in vital public-sector areas like energy policy, are accompanied by some unique headaches. By the time one has dealt with the headaches of model design and implementation, one rarely--if I may mix my anatomical metaphors--has much stomach left for facing fully the remaining headaches of validation and assessment. These evaluative steps pose perplexing questions both conceptual and practical, questions that have forced their way to the forefront of our concerns here at the Center for Applied Mathematics of NBS. We are therefore most grateful: to you, for coming together to participate in this most enticing program; to Professor Gass, for his hard and thoughtful work in bringing it about; and to the Department of Energy's Energy Information Administration, for proposing and supporting it. I join you in looking forward to an intellectually exciting and significant experience. MODEL ASSESSMENT AND VALIDATION: ISSUES, STRUCTURE AND ENERGY INFORMATION ADMINISTRATION In an appendix to the recently released first volume of the Energy Information Administration's (EIA) Annual Report to Congress, [28], over sixty models are identified and briefly described which can be used to project and analyze energy production, consumption, prices and associated impacts. Ostensibly these models represent a substantial capability in support of the energy policy analysis process. However, generally common features of the models are that they are large, detailed, and in their operational form, resident on a computer. Precisely how the results of using such models are to be interpreted and communicated to the ultimate decisionmaker is in dispute. Serious questions, for example, can be raised as to the actual influence of the analysis systems developed so far. The need for developing better procedures in these areas is reflected in the highly successful program and attendance of this National Bureau of Standards Workshop on Validation and Assessment Issues of Energy Models. 2/ Questions concerning the usefulness of large computer models are not new, and evaluations of the problems at issue are available. The literature on model evaluation is growing, both in terms of the generic problem and for energy models in particular. Still, model assessments in practical terms have not been consistently attempted until now, except perhaps in the area of military operations research. 3/ Inspection of the topics considered at this Workshop reveals substantial differences in the opinions of the participants as to the status of model assessment activities. Consider that at various points during the program there are: reports on the outcome of model assessment projects; expressions of concern over who, generically, should presentations of rigorous procedures for achieving a questioning of whether or not assessment goals are a contention that there may not be meaningful model |