consequences may fall, inequities will very likely occur.

Another example of a seeming paradox concerns the "power" of CEA/CBA results. Some people argued that because many decisions are made in a political context, the results of any "objective" analysis would be heavily criticized and overwhelmed by other factors. Yet others argue that one of the factors in the potential misuse of CEA/CBA is its quantitative nature, allowing those involved in the decision process to "anchor" their arguments to what appear to be hard numbers. Are the results of CEA/CBA powerless? Or overly powerful?

The resolution of both these examples may lie in the distinction between the process or approach of CEA/CBA and the quantitative results of formal studies. As indicated by the two general findings of this assessment, many of the negative perceptions of CEA/CBA are based on the possible misuse or inappropriate use of formal study results. Viewed as a method of structuring the decision process, CEA/CBA need not hide or avoid questions of ethics or values, and it need not provide a deceptively quantitative answer to complex problems.

As an example of the difficulty of concentrating on quantifiable variables and how investigations of decision possibilities might be enhanced

by thinking in CEA/CBA terms, consider the cost effectiveness of CEA/CBA itself. OTA was frequently asked whether a CEA/CBA of CEA/ CBA might not be what is needed. And for a given decision situation that type of analysis might be very valuable. Approaching a CEA/ CBA of CEA/CBA in order to arrive at a quantitative, traditional bottom-line result, however, might lead analysts to list as a primary "cost" of CEA/CBA the resource costs involved in conducting and interpreting the studies. Thus, resource costs such as those identified in appendix C would be included, with dollars being the measure used. If, however, the analysts were less interested in a bottom-line figure for the CEA/CBA of CEA/CBA, they might consider the opportunity costs of analyses. That is, the more important aspect of the costs of CEA/ CBA may not be the dollars it takes to conduct it, but rather the alternate uses of those dollars and the alternate types of analysis and other activities that might occupy the attention of those concerned about more rational allocation of medical technologies (617). Would the funds and attention that could be devoted to CEA/ CBA be more productive if applied to efficacy and safety studies? To education or consciousness-raising of physicians? To more dissemination of existing knowledge of the costs and benefits of various technologies? To regulation of the use of technology? These are the questions that probably should be asked.

One of the key factors affecting the uses and usefulness of CEA/CBA has already been discussed in chapter 3: the technical, methodological feasibility of the technique. These methodological factors can be inherent aspects of CEA/ CBA, or they can be due to the state-of-the-art of CEA/CBA and thus more tractable.

The manner in which both types affect the usefulness of CEA/CBA, however, should be analyzed in the context or the environment of current or potential uses of CEA/CBA. In other words, the questions should be asked what is the decisionmaking context and how does it affect

the strengths or limitations of the methodology, and vice versa? For example, does the decision relate to a technology at an early stage in its lifecycle, such as bone marrow transplants? Or does it concern an established technology, such as appendectomy? Is the technology in question a diagnostic technology, such as the CT scanner, or a therapeutic one, such as renal dialysis?

The possibility of affecting the course of a technology's diffusion and use might be greater in early stages of its development, but the uncertainties about its health effects and its costs will generally be greater. Thus, it may be possible to

do a more valid or certain CEA/CBA later in the technology's lifecycle, but the information gained may be less valuable for public policy. The tradeoff required will vary depending on the specifics of the technology and the policy decision to be made. In addition, diagnostic technologies are often more difficult to study than other technologies because of the uncertainties involved in linking their use to health outcomes. Thus, studies of diagnostic technologies often tend toward the "net cost" end of the CEA/CBA spectrum, where the measures of outcome or benefit may be numbers of tests performed or levels of diagnostic accuracy.

In sum, the stage of development of the technology under study and the type of technology (or function of the technology) are two of the factors that will affect the specifics of analysis to be used, the uses to which analysis can be put, and the usefulness of resultant information. Other factors are the relative strength or importance of nonanalytical factors, such as politics or equity, in the decisions to be made; the ability of the sponsors of analysis to implement the results; the familiarity of sponsors and decisionmakers with formal analysis; the existence of adequate data relating to the technology, to the disease or other problem addressed by the technology, or on other possible effects of interventions based on analysis; the existence of economic incentives that match or run counter to the results of analysis, the types of decisions to be made (e.g., budget-based decisions or nonbudget-based decisions); and so on. Some of the factors that affect the use of CEA/CBA are listed in table 2.

One of the factors listed above is of particular importance: the quality and availability of data. Obviously, without data or estimates of data, there would be no CEA/CBAs. The quality of a CEA/CBA is directly related to the accuracy of the data used in it. For example, when good epidemiological data on the effects of a technology or the existence of disease are present, analysis will have a greater potential for being relevant and useful. A specific example of where epidemiological data have permitted analyses of high quality is in the area of smoking and its effects on health. Good data do not guarantee good analyses, however, because the quality of

Table 2.-Factors Affecting the Use of CEA/CBA Stage of Development of the Technologies Under Study. -Tradeoff required between availability/validity of data and ability to affect the future use of the technologies. Both the type of analysis and the usefulness of analysis will be affected.

Nature of Technologies Under Study and Function of Technologies Under Study.-In terms of function, diagnostic technologies, for example, often have indirect connections to health outcome and often lend themselves to the net cost type of CEA/CBA. In terms of the physical nature of technologies, surgery, for example, may involve additional uncertainties due to varying skills of surgeons and surgical settings. Both type and use of analysis will be af fected, but especially the type or specific methodological elements.

Social, Ethical, or Value Influences in the Decision Environment. Very similar, often overlapping with the above factor. Will affect both the type and uses of the analysis. The example of renal dialysis applies here. Abortion would serve as another example.

Quality of the Analysis.—Can be of at least four types: Analysis Subject to Inherent Methodological Limitations. -e.g., inability to adequately deal with equity concerns; influence of discount rate chosen on outcome of analysis.

Analysis Subject to State-of-the-Art Limitations.-e.g., difficulties in identifying and measuring many costs or effects.

Analysis Containing Errors of Omission or Commission. -These are errors not due to the state-of-the-art, e.g., failure to discount or perform sensitivity analysis when appropriate.

Analysis Subject to Data Limitations.-This factor will affect quality even though the other factors might have been adequately dealt with. Much cost and health outcome data are uncertain, difficult to retrieve, or simply nonexistent.

All four of these factors can affect the quality of analysis, which in turn affects the usefulness of the results.

Ability of Sponsors or Users of Analysis to Implement Results. The usefulness of analysis will naturally depend on the amount of control the user has over the particular technology or situation studied.

Experience/Familiarity of Users With the Type of Analysis Conducted. This factor will affect usefulness in two ways: it will be a direct influence on the acceptability of results, and it will affect the ability of the users to appropriately apply the results.

Existence of Economic Incentives in the Decision Environment. If the economic incentives relating to the use of the technology under study are in accord with the results, their acceptability will be great. If they run counter to the results, the usefulness will be limited, depending on the strength of the economic incentives. SOURCE: Office of Technology Assessment.

analysis is also affected by the other factors mentioned above. Similarly, the usefulness of analysis is dependent on those factors affecting quality as well as on a number of other factors (see table 2) relating to the decisionmaking and analytical contexts or environments.

There are many gaps in the data available for CEA/CBA, owing to such factors as methodological constraints, inadequate resources for data collection and interpretation, lack of communication between the users of data and those collecting it, and the sheer impossibility of collecting and analyzing all the data that could be used by someone, somewhere. The principal

POTENTIAL USERS OF CEA/CBA

Health care policies and other decisions are made at a variety of levels and in a variety of situations by an extremely broad range of individuals and groups. In theory, CEA/CBA results or approaches might be useful to any or all of these decisionmakers. Table 3 lists many of the decisionmakers-the list is not exhaustive but should provide an idea of how diverse and numerous the types of decisionmakers are. Three general classes of decisionmakers or potential users of CEA/CBA information are discussed in this assessment: individual medical

Table 3.-Partial List of Individuals and Groups Making or Influencing Resource Allocation

Federal agency charged with collecting and analyzing health data is the National Center for Health Statistics (NCHS). NCHS is currently involved in several developmental projects intended to clarify certain methodological issues related to the provision of data for CEA/CBA, especially in relation to cost-of-illness studies. (see app. B).

practitioners, nongovernmental institutions, and governmental/quasi-governmental institutions.

Individual Medical Practitioners

Despite the fairly small amount of empirical research on the subject, it seems safe to say that CEA/CBA has had little direct impact on individual physicians' behavior.3

Discussions with academic physicians indicate a consensus regarding the above point on CEA/CBA's lack of impact. Beyond that point, however, the consensus dissolves. There is disagreement, for example, concerning whether CEA/CBA has, and if so the extent to which, significantly affected physicians' consciousness of economic issues. Explanations for the lack of impact on practice are numerous, with emphasis on their relative importance varying dramatically from one observer to the next. And the consensus on current practice impact does not translate into agreement on the future role of CEA/CBA in influencing individual physician behavior: Opinion seems to be split roughly in half between those who believe that CEA/CBA will cause many physicians to alter their medical practices and those who anticipate continuation of the current absence of significant effect.

The principal explanations for CEA/CBA's lack of impact on physicians' behavior to date can be grouped under two headings:

1. The novelty of CEA/CBA in health care. Until very recently, the literature on health care

"The following discussion is taken from work done for OTA by Kenneth Warner of the University of Michigan (615).

CEA/CBA was sparse. As indicated in Background Paper #1, this has been particularly true in the medical literature. Relatively few physicians read the nonmedical health care literature; hence their exposure to the concepts and practice of CEA/CBA was minimal prior to the last few years. Needless to say, lack of exposure correlated highly with (and presumably caused in part) a lack of understanding of the technique and meaning of CEA/CBA.

The novelty of CEA/CBA in health care accounts for some of the quality problems in the published literature. While poor analytical quality certainly could be a barrier to application of the results of analysis, few observers cite it as a significant factor in the failure of physicians to apply findings to their practices.

In a similar vein, the uncertainties in analysis frequently prevent determination of an unequivocal conclusion in an analysis. Even when a firm "bottom line" is presented, nonquantified factors-for example, the distribution of costs and benefits-can make the conclusion far from definitive. Thus, one could argue that even highquality analyses frequently do not produce findings that can or should be translated directly into practice by individual physician decisionmakers. This seems an attractive explanation for physicians' nonresponse to analysis, particularly combined with whatever bewilderment they may feel as a result of their unfamiliarity with CEA/CBA. It is not, however, an explanation often noted in discussions on the subject. Most likely, this explanation presupposes that other, preliminary barriers to application of analysis have been surmounted; the evidence is to the contrary. Thus, one might anticipate that such inherent technical limitations of analysis will grow in importance as other barriers fall.

2. The irrelevance of much of CEA/CBA to medical practice decisionmaking. There are two basic sources of irrelevance, one substantive, one structural. In the substantive category, many CEA/CBAs have involved assessments of the desirability of social programs where social, and not individual, decisionmaking was at issue. Examples include the several studies of communicable disease control programs (e.g., measles, swine flu, etc.), community (or indus

try) based screening programs (e.g., hypertension), and fluoridation of municipal water supplies. The subject matter of such studies precludes a direct practice response by individual physicians.

While this too may serve as a useful partial explanation of the absence of behavioral response by individual practitioners, it cannot explain the total absence of such response, since much of the health care CEA/CBA literature is clearly relevant to individual practice decisionmaking. Nor is this an often-cited explanation. A more cogent argument concerns structural irrelevance: According to a strict economic interpretation, most physicians' interests in "costeffective care" deviate significantly from those of society. All physicians share an interest in understanding the efficacy and safety of medical technologies-technologies whose risks outweigh medical benefits are undesirable-but concerns with the economic side of cost effectiveness are either nonexistent or dependent on the physicians' economic environment. In general (619):

Cost data are psychologically remote. (The physician's) one-on-one relationship with the patient is not in the context of the cost to society.

The physician's economic circumstances, however, can produce in the physician an often subconscious reaction to costs. To a fee-forservice physician whose patients are well insured, the cost of a technology may be irrelevant, at least immediately. If the physician works within the context of prepayment, however, the professional concern with cost effectiveness begins to approach the social concern. In all cases, the patient's economic wherewithall often will be a major consideration: In an environment of prepayment or adequate insurance coverage, high costs of technologies do not translate into direct economic burdens on patients; hence the high costs are something of an abstraction to both the immediate patient and the physician.

This economic interpretation-emphasized by many knowledgeable observers-attributes the lack of effect of CEA/CBA on medical practice to its irrelevancy and even inconsistency

with medical norms, irrespective of the quality or quantity of the literature. Accordingly, unless the reimbursement system is changed, this argument suggests, the future will auger little change in the application of CEA/CBA to individual practice decisionmaking. According to this explanation, physicians' nonresponse to CEA/CBA is not necessarily a reflection of physicians' selfish monetary interests, or their indifference to economic considerations. Rather, nonresponse to CEA/CBA perhaps reflects physicians' fulfilling their roles as agents of their clients-patients. A physician's major responsibility may be to weigh all the costs and benefits to the patient and to his or her medical practice-i.e., the aggregate of all the patients of the physician.

This argument is not an entirely economic one, because the ethics of the doctor-patient relationship are involved. If a patient is not harmed economically by performance of a certain procedure, even though only a small medical benefit might be expected, what are the ethics of the individual physician's denying or recommending against the procedure in order to represent society's cost and benefit priorities? The differences between social and individual economic and ethical considerations constitute the only frequently advanced explanation for physicians' nonresponse to CEA/CBA that does not imply a brighter future for the ability of analysis to alter individuals' medical practice policies. Systemwide changes in the economic environment, such as growth in HMOS or major reimbursement reforms, might more closely aline the practice of medicine with the precepts of analysis. The strength of the explanation does not depend on lack of understanding of CEA/ CBA within the medical community; hence anticipated increases in familiarity with analysis need not promote the direct application of findings. Accordingly, barring external pressures, the economic incentives and ethical norms of medicine may very well continue to preclude widespread application by practitioners of the findings of health care CEA/CBAs, with the exception of the "easy" cases in which one procedure is demonstrated to be both more effective and less costly than an alternative.

Nongovernmental Institutions

A variety of nongovernmental institutions are potential consumers of CEA/CBAs. Insurers have a direct economic incentive to find and promote cost effectiveness in the provision of health care services; officials of major insurers, including Blue Cross/Blue Shield, have expressed their interest in the development of more and better CEA/CBAs to assist them with reimbursement decisions (see ch. 5). In an era of increasing restrictions on reimbursement, hospitals' interests in enhancing efficiency are obvious. HMOs also have a direct economic interest in cost-effective care: Greater efficiency translates into lower, more competitive membership rates and/or higher incomes for member physicians. Large business firms and unions have several reasons to be interested in CEA/ CBA: Greater efficiency in the provision of medical services to employees implies lower business costs or room for negotiation of other fringe benefits; health promotion and disease prevention among workers may increase productivity and reduce other costs of disability and morbidity; and so on. As major financers of the costs of illness, each of these organizations has not only an interest in promoting cost-effective care, but also has the market power to translate judgments concerning cost effectiveness into changes in health practice.

Another group of nongovernmental institutions, not directly involved in the financing of care, is evidencing interest in CEA/CBA: professional associations. Among such groups are the Association of American Medical Colleges, the American College of Physicians, the Resident Physicians section of the American Medical Association, and the cost-containment committees of several State medical societies. In part, the interest of such groups reflects concerns about the social implications of inefficient medical resource allocation; in part, interest may reflect a perception that if the health care community does not control cost inflation, Government regulators may attempt to do the job for it. Regardless of the motivation, the demonstrated interest suggests a receptivity to information that CEA/CBA in theory can pro