A WAY OF THINKING ABOUT MODEL ANALYSIS

Martin Greenberger

The Johns Hopkins University

Introduction

In the course of writing a review article on the assessment of energy policy models, I have been thinking a good deal about ways of classifying and describing the widening array of activities devoted to the study and investigation of models, as opposed to their development and use. With less frustration and arbitrariness than I might have expected in such an exercise, I finally arrived at two organizational schemata as explained in the review article (1). What I would like to do here is present the rationale for these schemata and show how it helps in reflecting on the routes being taken by model analysis and on where it is headed.

Please note that I am using the term "model analysis" and not "model assessment." For me, the first term includes the second, but does not have as specifically evaluative a connotation. The field of energy policy modeling is young and its growth has been lively, to say the least. Table 1 lists a sampling of the models and their uses, taken from an article written three years ago (2). Some of the most important models entering into energy policy today are not present there. With a field as green and lush as this one, there is a need for mowing and weeding. It is necessary to critique, and umpire, and evaluate. But it is also important to compare, understand, and explore. I employ the term "model analysis" to include all of these activities.

-

-

My two organizational schemata for model analysis are displayed in the form of a tree and a table. The tree is shown in Figure 1, the table a 4x3 matrix in Table 2. The tree has two main branches, corresponding to two principal modes for conducting model analysis, each with its own style and distinguishing set of objectives. The table focuses instead on who has taken the initiative for doing the analysis and under what circumstances. The tree and the table are two different perspectives for viewing the same array of activities. To avoid confusion, they are best dealt with separately. We shall discuss them one at a time.

Model

Table 1

Energy Policy Models, Methodologies, and Uses

Adams-Griffin

Baughman-Jaskow

Bechtel Supply

Brookhaven

Coal 1

Energy-economic effects of nuclear moratorium in
California.

FEA studies of industry requirements for energy
expansion..

ERDA evaluation of alternative energy technologies.
Congressional hearings on energy forecasts.

ERDA studies of economic impact of alternative
energy futures.

Department of Interior long-term energy forecasts.
Office of Energy Preparedness and Treasury
Department storage option studies.

Studies of nuclear alternatives (Ford-Mitre,
Committee on Nuclear and Alternative Energy
Systems).

FEA studies of oil embargo.

Impact of reduced energy consumption on the economy (Ford, EEI).

White House analysis of gas deregulation.

Alternative strategies for optimal economic growth.
ERDA studies of energy conservation.

Macroeconomic effects of a nuclear moratorium in
California.

EPRI industry studies.

Energy economic impact of alternative nuclear and fossil fuel strategies (Committee on Nuclear and Alternative Energy Systems).

Energy sector studies with emphasis on petrochemical industry.

National energy plan and FEA studies of oil and natural gas price decontrol.

Exploration of potential energy-economic growth.

Economic and environmental impacts of alternative energy futures.

Gulf Oil Co. and White House decisions on synthetic fuels.

American Gas Association natural gas studies.
Congressional hearings on Carter energy plan.

Accounting Charts of requirements and characteristics displaying numerical relationships.

Econometric: Mathematical (difference) equations solved simultaneously, with coefficients estimated statistically from historical data
Exogenous: Given or assumed, rather than calculated (endogenously) within the model.

Optimization Determination of "best" solutions by means of algebraic procedures.

Process representation Descnption of energy processes and markets in the form of a hierarchical network.

im dynamics. Mathematical (integral) equations solved recursively with coefficients estimated judgmentally from the modeler's experience and intuition

Two Kinds of Model Analysis: Their Overlap and Complementarity

The Tree

Let me begin with an admission. I tend to view model analysis from my own backyard. I was fortunate to have the opportunity while on leave from Johns Hopkins University during 1976 and 1977 to put into action some recommendations coming out of a study I had conducted with others the three years before. We had made our recommendations in the setting of an observation on the present sociology of the modeling field.

"Modelers mostly build and run their own models: that is
where the credits lie. Very few modelers run and analyze
the other fellow's model in any systematic way.... Modelers
are synthesizers and refiners more than analyzers, particularly
analyzers of other modeler's models. When possible at all,
such secondary analysis is too difficult and unrewarding an
activity to generate much interest. As a result, the inner
workings of a policy model are seldom understood by anyone
but the builders of the model (and not always by them).
This is a weak foundation for gaining the reliance and
trust of policymakers." (3, p.339)

We did not suggest that model builders be made to pay more attention to the models of others. We felt it was enough for them to do their own jobs well. What we did propose was

-

"development of a new breed of researcher/pragmatist the
model analyzer -- a highly professional and astute practioner
of the art and science of third-party model analysis. Such
analysis would be directed toward making sensitivity studies,
identifying critical points, probing questionable assumptions,
tracing policy conclusions, comprehending the effects of
simulated policy changes, and simplifying complex models

without distorting their key behavioral characteristics." (3, p. 339)

I took my leave at the Electric Power Research Institute where I served as manager of the Systems Program with a significant budget for sponsoring outside research and a charge to initiate a program of research germane and useful to the Institute and its electric utility membership.

In early 1976, it was already clear that policy models were attracting the attention of energy decisionmakers and regulators in Washington and state governments. There was a partly progressive, partly defensive readiness to accept the argument that projects designed to understand and objectively assess the use of these policy models would be in the interests of the electric utility industry. With the help of my staff and several key colleagues outside of the Institute, I was able to arrange for the establishment of a number of efforts directed at the analysis of energy policy models. Indirectly, these efforts were intended to promote an awareness of the need for model analysis and to stimulate the interest and development of people who would be qualified and able to carry on this kind of activity.