variation among HMOs with respect to whether and how these processes were formalized. An additional finding was that even though sophisticated forms of CEA/CBA were not perceived as relevant, the issues perceived to underlie CEA/CBA were considered an essential part of the decisionmaking process at all times.

More specifically, however, if the most potentially pertinent question in the HMO is "at what level of provision is 'X' cost-effective?" then the institutionalization of particular organizational procedures such as peer review of patterns of use of various resources, what constitutes inappropriate use, and who is qualified to designate their use in the HMO, as well as the monitoring of referral patterns, hospital admissions, and the like, are potentially very useful institutionalized mechanisms for assuring cost effectiveness in HMOs. Many HMOs already have instituted some of these formalities, and as a result have achieved dramatic cost savings (344).

Because of the great diversity of HMOs and variation in the competitive positions they hold in their respective communities, however, it is important that these procedures remain "individualized" (rather than standardized), i.e., each according to the HMO's need for analyses and/or review (295). Recently, GHAA has recommended to the council of the National Center for Health Care Technology that a study be undertaken with the purpose of developing criteria for the acquisition of technological resources in HMOs. The proposed study would indicate the probable impact of various acquisitions on the costs and utilization of these technologies and their relationship to per capita and aggregate costs, as well as the applicability of findings to other clinical organizational settings (300). The challenge of this proposal is formidable indeed. Still, the results might be useful if posed as a relative guide or as one model to follow.

Statutory Requirements for
CEA/CBA in HMOs

Until January 1, 1980, the regulations promulgated under the Health Planning and Resources Development Act of 1974 (Public Law 93-641) required that HMOs demonstrate the

cost effectiveness of their requests to State health planning agencies for CON approvals of capital expenditures. During 1979, an extended process of developing a detailed set of criteria to be used to ascertain the "cost effectiveness" of a given proposal raged on. The 1979 amendments (Public Law 96-79) to the health planning law, however, exempted HMOs from CON review by local HSAs and approval by State planning agencies. Neither Federal nor non-Federal HMOs are now covered under CON regulations. Because of the new health planning law, the controversial endeavor regarding cost-effectiveness criteria for HMO proposals was suspended. In January 1980, the relationship and responsibilities of HMOs to local HSAs and State health planning agencies were described as being in a state of limbo (159).

OHMO has a formalized system to assist potential or young HMOs in making cost-effective resource allocation decisions. HMOs are perceived as businesses by OHMO, and the main talent and skills required are cited as those of management. Therefore, under the authority of the technical assistance provisions of the Health Maintenance Organizations Act, OHMO has developed and published six manuals or review guides addressing the topics of: finance, quality assurance and health services delivery, management information systems, marketing, structural and contractual relationships, and management arrangement assessment (308).

To aid applicant HMOs in making cost-effective decisions, the review criteria provide cost ranges and reasonable upper cost limits at various levels of enrollment with which quoted bids may be compared. Similarly, in the quality assurance and health services delivery review guide, a table indicates the number of physicians required, as well as staffing pattern suggestions, at various levels of enrollment. (These standardized criteria for cost-effective performance are intended to serve as benchmarks only, and are not meant to indicate mandatory standards.) In summary, it seems that the analyses, decisionmaking practices, and criteria and standards formally institutionalized in the administrative procedures developed by OHMO are very similar in spirit to those less formalized in, but typical of, individual HMOs.

Most of the studies undertaken at OTA rely on the advice and assistance of an advisory panel of experts. The advisory panel for a particular assessment suggests source materials, subject areas, and perspectives to consider; assists in interpreting information and points of view that are assembled by OTA staff; and suggests possible conclusions based on the accumulation of information produced by the study. The panel members review staff and contract materials for accuracy and validity, discuss policy options of the study, and present arguments for and against the options and conclusions. They do not determine the report's final form, however, and are not responsible for its content, direction, or conclusions.

The advisory panel for the current assessment consisted of 12 experts with backgrounds in business, ethics, health policy, law, economics, statistics, and medicine. The panel was chaired by Dr. John R. Hogness, former President of the University of Washington and current President of the Association of Academic Health Centers. Two members of the OTA Health Program Advisory Committee, Dr. Stuart H. Altman and Dr. Frederick Mosteller, also served on the advisory panel.

The first panel meeting was held in Washington, D.C. (the site of all four panel meetings), on December 13, 1978. Panel members reviewed the study plan of the assessment, examined the need for specific case studies, and reviewed a number of suggestions for potential case study examinations. The case study approach, as part of the general study framework, is intended to illustrate the uses of CEA or CBA in health care, especially to evaluate medical technology, along with the impacts, and strengths, and weaknesses of these uses. The advisory panel was instrumental in helping the staff set goals for the assessment, establish boundaries for its focus, and define the basis for and role of CEA or CBA as a decisionassisting tool in the health care system. The panel was also helpful in identifying public and private sector uses and users of CEA/CBA techniques or information in decisionmaking.

To help select medical technologies for the final list of case studies, the following criteria were developed:

• examples of types of technology by function (preventive, diagnostic, therapeutic, and rehabilitative);

• examples of different stages of development and diffusion (not yet diffused, experimental or pilot, established in medical care);

• examples from different areas of medicine (such as general medical practice, pediatrics, obstetrics, radiology, and surgery);

• examples addressing medical problems that are important because of their high frequency or significant impacts;

examples of technologies with associated high costs;

• examples of technologies in widespread use; and • examples with sufficient evaluable literature. On the basis of these criteria and panel recommendations, OTA staff selected the case study topics. In addition, the Senate Finance Committee had specifically requested four case studies: psychotherapy, respiratory therapy, diagnostic X-ray, and length-ofstay in hospitals. Nineteen case studies (identified in app. F) were added to the study plan (Background Papers #2, #3, and #5).

The second advisory panel meeting was held on February 14, 1979. The panel reviewed the literature gathered by OTA staff, suggested additional references and sources of information, and evaluated the case study plan. In addition, it examined the staff's preliminary work on CEA and CBA methodology to identify strengths, weaknesses, and omissions. The panel was also helpful in commenting on the decision area papers (e.g., use of CEA in reimbursement coverage decisions) that were being prepared by the staff. Finally, it was asked to review a separate study being conducted as part of the overall assessment: The Management of Health Care Technology in Ten Countries (Background Paper #4).

Several subprojects were pursued: a survey of relevant State, Federal, local, and private and nonprofit agencies and organizations to determine the extent of use or support of CEA/CBA activities (app. B); and an extensive review of the health care literature to develop a comprehensive bibliography of health care CEAS, CBAS, and related publications.

The psychotherapy case study (Background Paper #3) used a separate, additional advisory panel that performed functions similar to the ones the overall advisory panel performed for the full assessment. The psychotherapy panel met on August 7, 1979, in Washington, D.C., to review the study plan, suggest improvements, and evaluate the background work performed. This panel met again on November 15, 1979, to review and critique the initial draft of the case study.

The third meeting of the advisory panel for the full assessment took place on October 16, 1979. In gener

al, the panel focused on reviewing the status of the various parts of the study. It was also asked to discuss the early drafts of the methodology report (Background Paper #1) and of the use of CEA in medicare reimbursement coverage.

During 1979, two additional subprojects were initiated: a survey of analysts who had performed CEAS and CBAS of health care technologies (app. C), and a paper examining the ethical implications of CEA and CBA techniques (app. D). The survey of analysts focused on the resource "costs" used to perform actual CEAS or CBAs. The analysts were also asked to estimate the expected or potential costs of an "ideal" study team doing CEA/CBA analyses of health care technology on a routine and continuing basis. The second subproject was a paper by the Hastings Institute on the ethical considerations of conducting and using efficiency-based analyses such as CEA and CBA in the health care system.

On January 18, 1980, the authors of the case studies assembled in Washington, D.C., to review the applicability of CEA/CBA to health care decisionmaking and to discuss the methodological or data problems they faced in trying to apply CEA/CBA to their case study areas. Case study authors also discussed the policy issues involved in the use of CEA/CBA in health care decisionmaking.

The final meeting of the advisory panel was held on March 28, 1980. At this meeting, the panel re

viewed drafts of the summary report, including the policy options for congressional consideration. Using the comments generated at this meeting, OTA staff revised the assessment report drafts.

The results of this assessment are being issued in six volumes (described in app. F). A two- or threetiered review process was used for each of the volumes and for each individual case study. The initial drafts were reviewed first by OTA staff and advisory panel members. In certain instances, outside reviewers were also asked for comments. After the authors completed their revisions based on the reviewers' suggestions and comments, the drafts were sent out for a second round of review by a much broader range of experts in a diversity of settings: Federal agencies, State or local offices, private and nonprofit organizations, academic institutions, practicing health professionals, consumer groups, and other selected individuals. Altogether, more than 400 individuals or organizations were asked to comment on drafts of case studies and other volumes of this assessment in the second round of review. The final report, the volume containing congressional options, was reviewed by more than 100. After appropriate revisions based on the comments received had been made, drafts were prepared for a final review by the assessment's advisory panel, by the Health Program Advisory Committee, and by other individual reviewers.