programmatic needs. It is not responsible for the administration of any health care delivery or reimbursement activities; it exists solely to conduct and sponsor health services research and to disseminate the results of that research to relevant Government agencies, the research community, and other interested parties.

The NCHSR statute's language is so broad that the Agency retains considerable leeway in its selection of specific research issues to pursue. To identify the areas of research it needs to conduct and support, NCHSR employs an informal consensus development process.

Priority Setting

The priority-setting process involves several steps. First, NCHSR canvasses policymakers, consumers, Government and non-Government experts, health care providers, professional associations, program administrators, and others, in an attempt to identify current and emerging issues that present the most immediate problems to the health care system. In the first cut at the list of suggested issues, a number of criteria are used, two of which are the relative importance of the issue and the perception that there is a good chance that the research will provide information that will contribute substantively to the policy process.

NCHSR staff, as well as outside professionals involved in various research areas, are involved in the culling process. The Director of NCHSR selects the top priority concerns from the major issues identified by this process. Before the list is adopted, it is submitted to an assorted group of NCHSR members and non-NCHSR experts for review. The priority issues that emerge from this process become the areas of health services research that NCHSR focuses on. The issue selection process is not a one-time occurrence, but rather an ongoing interaction between NCHSR and the health care community.

The list of priority areas does not lock out all other health care issues. NCHSR can, and does, consider "meritorious and potentially important" proposals for a range of nonpriority issues. In-house, or intramural, research can also pursue areas of interest that fall outside the priority areas.

Currently, NCHSR supports five general priority areas and a special studies category that is concerned with examining issues of emerging importance or of research methodology. The current priority areas are (438): 1) health care costs and cost containment, 2) health insurance, 3) planning and regulation, 4) technology and computer science applications, and 5) health

manpower

Evaluation of Research Proposals

NCHSR's health services research efforts are allocated among intramural research, extramural research grants and contracts, and center grants. Intramural research is subject to an informal review process within NCHSR. When the need arises, staff proposals for in-house research projects may also be reviewed by nonFederal experts. Projects are formulated according to special research needs or personnel capabilities at NCHSR. The in-house project proposal moves through the administrative hierarchy, receives comments and suggestions from the staff, and may be circulated to outside reviewers before final action is taken by NCHSR. Health services research centers receive funding apart from the extramural and intramural research budget. NCHSR's authorizing legislation called for the funding of at least six "existing and new centers for multidisciplinary health services research, evaluation, and demonstrations." There are eight such centers currently in operation (650).

NCHSR uses a dual review type of evaluation process to screen and select grant and contract applications it receives (385). This process is quite similiar to the peer review system described earlier for NIH. Grant and contract proposals that exceed $35,000 in direct costs are required to be reviewed for scientific and technical merit by study sections composed of non-Federal Government experts. Proposals that do not exceed $35,000 are evaluated by NCHSR staff, and where needed outside reviewers.

Explicit CEA/CBAs are not used in the priority-setting, project selection, or research evaluation processes. A few of the project criteria used in the selection process, however, make it likely that there will be significant emphasis on the

relationship between the application's budget (cost) and its potential outcomes (benefits). NCHSR receives far more applications than it has the funds to support, so in the project selection process, it does consider cost and effectiveness. The budget and outcomes criteria used to evaluate applications are considered with reference to the Agency's budget and goals.

CEA/CBA as Part of NCHSR's
Research Mission

NCHSR supports a number of cost-effectiveness and cost-benefit studies that cover a broad range of health services issues (439) and is the major supporter of CEAS and CBAs in the Federal health care research system at this time. Two of NCHSR's five research priority areashealth care costs and cost containment, and planning and regulation-specifically call for research using CEAs to examine the issues within these priority areas (438).

The CEAS and CBAS that NCHSR supports range from rigorous analyses of specific health care topics to broad studies of more complex health care issues. The results of these studies, like most of NCHSR's research products, are circulated via the Agency's formal publications dissemination process, conferences, seminars, journals, announcements, etc. NCHSR is in a position to monitor the eventual use of the research results by other agencies and decisionmakers. In most cases, though, the nature of the policymaking process makes it quite difficult to determine to what extent a given piece of information is used to reach a final policy decision. NCHSR is powerless to assure that the research results will actually be used in the decisionmaking process. The Agency's legislative mandate is to support health services research projects that answer, or at least address, the issues that the research community and policymakers feel are important. Decisions about whether and, if so, how to use the information that is generated are left up to agencies and decisionmakers outside NCHSR.

Health Care Financing Administration

HCFA, established in 1977 as a result of a major reorganization at the Department of Health,

Education, and Welfare (now DHHS), is a recent addition to the Federal Government's health services research community. This Agency is the organizational center that administers medicare/medicaid programs, the Professional Standards Review Organization (PSRO) program, and the research and statistics programs for these areas.

The legislation authorizing these programs allows HCFA to consider and pursue a range of health services research topics.17 HCFA is particularly responsible for sponsoring research that relates to its primary mission: the administration and evaluation of the medicare/medicaid area and the PSRO function. The mandate of the Office of Policy Planning and Research at HCFA, however, is broad enough to include a wider range of research topics. 18

HCFA currently has five major priority areas of grant support. Specifically, Agency support is focused on projects that meet one of the following criteria (284):

1. develops or demonstrates new financing mechanisms for health care services;

2. utilizes financing mechanisms to influence the effectiveness or delivery of health care services;

3. develops or demonstrates management or administrative procedures that will benefit HCFA programs;

4. develops knowledge or undertakes analyses of the basic nature and structure of health care costs and factors that affect their rate of increase; or

5. examines the economic and behavioral relationships between the financing of health care services and the total activities of the health care sector of the economy.

"The legislation that establishes HCFA's scope of responsibility is the Social Security Amendments of 1972 (Public Law 92-603). Sections 222 and 245 of this law provide for health services research and demonstration projects for a variety of reimbursement issues, performance incentives, etc. The National Health Planning and Resources Development Act of 1974 (Public Law 93-641) added provisions for work in ratesetting and development of quality data for providers.

18The Office of Policy Planning and Research of HCFA grew out of the Office of Research and Statistics (ORS), formerly of the Social Security Administration. ORS became a major source of support for health services research in the late 1930's.

Priority Setting

HCFA's priority research areas are chosen in an informal process much like the one used by NCHSR. Opinions, suggestions, and recommendations are solicited from a variety of people with a broad background of experience and training. Their recommendations are culled to form a list of research priorities. Although priority areas receive special attention in the selection process, new ideas and innovations are not automatically excluded from consideration if they fall outside these areas. Relevant examples of the issues on which HCFA is focusing are hospital costs, physician reimbursement, and the quality and effectiveness of various health care

areas.

Evaluation of Research Proposals

The peer review mechanism and the project evaluation process at HCFA are very similiar to the systems at NCHSR. The major difference between the two approaches is the formal involvement of HCFA staff in the initial technical review panels. Instead of being composed of all non-Federal experts like NCHSR's and NIH's panels, HCFA's review panels are composed of an equal mixture of HCFA staff, non-Federal experts, and non-HCFA Government experts (284). The criteria used to evaluate the merit of the research application are geared as much to HCFA's program needs as to technical and scientific merit. Final review and funding decisions are made by HCFA staff and administrators.

Use of CEA/CBA

HCFA is like the other agencies discussed above, in that it does not use explicit CEA/CBA to select research goals or grant proposals, or to evaluate project outcomes. Awareness of projects' "cost effectiveness" to the Agency's goals, however, is present.

HCFA focuses much of its substantive research on priority issues that try to determine the relative efficiency of various methods of delivering care. Many of its research solicitation areas are for work to arrive at CEA/CBA-like

evaluations of specific issues. Or, data are pursued that might assist HCFA in determining cost and effectiveness measures. Several of the projects recently completed, as well as a number of ongoing efforts, are directly focused on costeffectiveness issues (283).

The CEA/CBAs that HCFA is supporting are similar to those at NCHSR, in that they range from fairly rigorous attempts at examining the costs and effectiveness of a medical technology to being closer to effectiveness studies that include costs as an analytical afterthought, if at all. The information produced as a result of these analyses is combined with the other information and considerations that are part of HCFA's program responsibilities. At HCFA, as at the agencies previously described, the support, evaluation, and use of analyses or information in general is not usually a linear process. It is extremely difficult to point to a given piece of information, a cost-effectiveness study for instance, and determine where, how, or even if, the knowledge gained from a given research project was directly incorporated into a given policy decision. Nevertheless, since HCFA can tailor its use of CEA/CBA to suit its special program needs and might then be able to implement that information within its organizational boundaries, it could possibly serve as an instructive example of the support and use of CEA/CBA in the decisionmaking process. HCFA is in a position to evaluate the impact of its actions after programs or program changes have been in place for a period of time. NCHSR and NCHCT do not share this type of start-tofinish authority. NCHSR and NCHCT have more limited access and input to the decisionmaking process than do HCFA, NIH, and other health care agencies that maintain program responsibilities.

Although HCFA is a collection of fairly mature programs that have been reorganized into a new agency, the organization has not fully settled into its new niche. It will be interesting to note how, or if, HCFA uses any CEA/CBAS that are produced by HCFA's own research support system, by NCHSR, or via the new efforts of NCHCT.

Formal CEA/CBAs are not used or supported to a great extent in the health care R&D system. At the process (or administrative) level, Federal agencies rarely use explicit CEA/CBA as a decision-assisting tool. In only a few cases has CEA/CBA been instrumental in facilitating a given allocational or policy decision. (See "Survey of Agency Use of CEA/CBA," app. B.) The agencies do seem to make efforts to employ a "cost-effective" approach and attitude towards the decisions that are made regarding the setting of priorities, selection of research projects, and in the evaluation of research products. These attitudes and approaches are transferred into action primarily via external and internal peer review mechanisms. At the substantive level, several formal CEAS and CBAs have been or will be supported by NIH, NCHCT, NCHSR, and HCFA. The research done at this level may feed back to a degree into the administrative decisionmaking and evaluation processes at these agencies. A significant amount of the CEA/ CBA research will be used to add to the body of knowledge in the health care system.

Efficiency-based techniques such as CEA/ CBA have not gained a substantial foothold in the R&D decisionmaking process. The same can also be said with respect to the other formal methodologies that have received some use and recognition in the field of policy analysis (i.e., decision analysis, operations research, technology assessment, risk-benefit analysis, etc). The relatively infrequent reliance on these types of analyses stems from a number of factors related to the techniques and the nature of the health care system in general and the R&D process in specific. The complexity and uncertainty of the R&D endeavor and its eventual products does not lend itself well to the constraints of efficiency-based methodologies. This may be the most important reason for the lack of use of CEA/ CBA in R&D. The health care R&D process is extremely complex, essentially political, and quite often is passive. The Federal R&D process is authorized, funded, and supervised by representatives of the public and is under the scrutiny of the Nation as a whole. The R&D system must be responsible and responsive to the changing

needs and goals of the country; hence, it is a highly political process. Finally, the system tends to be passive. Many problems are often presented to the decisionmaker with the objectives preordained and the viable options available to attain the goals few in number. The R&D process is fueled by the imagination and initiative of those researchers outside the decisionmaking process. A research goal or national health policy objective can be established, but the system does not move without the initiative of those who must create and submit research ideas and plans to the funding agencies. To a significant degree, R&D-supporting agencies must wait for, and react to, the ideas, suggestions, efforts, and research findings of the thousands of health care researchers both within and outside Federal Government. It is extremely difficult for CEA/CBA to predict, evaluate, or include the importance of the dynamic aspects of R&D. As a result, CEA/CBA's credibility, usefulness, and input to the R&D decisionmaking process are limited.

CEA/CBA is most supported and used at the health services research end of the R&D spectrum and least supported and used at the basic and applied end. This situation follows logically from the inability of formal analytical techniques to adequately deal with the high level of uncertainty that is part of the technology R&D process. Predictions, opinions, and "guesstimates" are the tools of the trade in this area. One can include considerations of uncertain factors in CEA/CBA; sensitivity analysis can help to an extent. But the level of uncertainty remains high.

The methodological shortcomings of CEA/ CBA techniques are compounded by the attitudes of many of the potential users of CEA and CBA in the R&D system. The decisionmakers' perceptions regarding the usefulness and validity of CEA/CBA are such that many doubt that these techniques are either necessary or helpful in much of the R&D context. This view is particularly strong at the basic and applied research level. This situation should not be surprising. Until recently, cost containment and cost-effec

tiveness criteria were not heavily stressed in health care research. NIH has traditionally focused on the quality of the research supported and the safety and efficacy of the technologies developed as results of research efforts. Finding cures for health care problems, not saving money, has been the primary goal. At the health services end of the R&D spectrum, distribution, cost, and quality of care have received needed attention. Cost and effectiveness criteria have been part of the R&D efforts; but only parts of a much larger focus. Recently, decisionmakers have been asked to make them a larger part of their decisionmaking criteria. As a result, increasing numbers of decisionmakers are becoming aware of the uses (and possible abuses) of CEA and CBA techniques.

Another impediment to the use and support of CEA/CBA is the presence of other decisionassisting techniques that have been fairly successful in guiding and informing the decisionmaking process. Peer review panels, publication of results in reputable journals, advisory councils, conferences, and other mechanisms have all worked fairly well as "quality" controls and, to an extent, as cost-effectiveness filters. These processes are firmly in place, have performed reasonably well over the years, and have by and large produced commendable results. The pressure to maintain these existing support systems might tend to overwhelm any serious effort to incorporate CEA/CBA in the mainstream of the decisionmaking process.

The organizational and statutory frameworks are currently in place to allow the use of CEA or CBA in the decisionmaking efforts of the R&Dsupporting agencies. The limiting factors are the perceived need for and usefulness of the information that might result from CEA/CBA research.

If CEAS could be adapted to the need of biomedical R&D, the use of CEA-type studies at NIH might logically occur at the late transfer stages of a technology's movement from the lab to the clinic and at the consensus development meetings held by NIH and NCHCT. NIH may

also be able to incorporate some form of efficiency-based analysis in its center, contract, and intramural research efforts. NIH has more control over the formulation, direction, and evaluation of these types of research efforts than it does over extramural project grant research efforts. It is at these points in the decisionmaking and information-gathering processes that the CEA/CBA technique might help serve the needs of the NIH decisionmakers. NIH has in a very few instances used CEA techniques to evaluate the ongoing or completed programs and projects that are performed as adjuncts to the basic and applied research missions (e.g., health education programs, disease prevention advertising programs, information dissemination projects, screening programs, etc.). It might consider the usefulness of increasing its efforts along these lines. Once again: Doing CEA/CBA for any reason requires that the limits and usefulness of the analyses be kept in mind.

NCHCT could provide input into a range of decision points in the R&D process. Its legislative mandate authorizes the use of CEA/CBA to examine newly emerging and existing technologies. NCHCT cosponsors consensus development conferences at NIH and is charged with providing information to HCFA on reimbursement issues concerning medical technologies. It remains to be seen how NCHCT will use CEA/CBA in the decisionmaking process.

NCHSR and HCFA hold promise as supporters of CEA/CBA research. HCFA might also be a user of CEA/CBA information in its PSRO and medicare programs. These two agencies focus on technologies in later R&D stages and on the services end of the health care system. CEA/CBA has recently received increased attention in this area of research and one sees increasing use of this technique to examine a host of services research issues. 19

19Such use is discussed at greater length in Background Paper #1: Methodological Issues and Literature Review, prepared by OTA in conjunction with this assessment.