less information exists about patients' ratings and reports of technical aspects of care, they appear promising, especially for physicians' ambulatory care. Patients' assessments relate to both positive and negative aspects of care and can provide information about access. Like other outcome measures, however, patients' ratings may reflect factors other than quality, such as the preferences of the particular patients in a physician's practice.

Although not thoroughly validated in this report, certain situations suggest quite strongly that hospitals or physicians are providing care well below minimum acceptable levels of quality. The Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) refuses to accredit 1 to 2 percent of the hospitals that it surveys (524). Ninety percent of the hospitals surveyed by JCAHO are accredited with contingencies relating to deficiencies that the hospital is to correct within a certain period of time. Since JCAHO re

fuses accreditation only to hospitals with substantial failings, such refusal may be taken as an indication of a poor-quality hospital. Hospitals or offices in extreme disrepair, perhaps as an outgrowth of financial difficulties, also suggest poor quality. More specific to a particular condition, hospitals that have high birthweight-specific mortality rates probably offer lower quality care for newborns than hospitals with lower rates. Physicians who continue to perform outmoded procedures, such as those on the list developed by the National Blue Cross-Blue Shield Association, or physicians who perform complex surgery or other complex procedures without appropriate training and experience are likely to offer care of low quality.

In the course of evaluating specific quality indicators, this review has identified several deficiencies that pervade the field of quality assessment. Current techniques cannot adequately adjust for patient and environmental factors that may influence patient outcomes independently of the quality of care. This situation greatly impedes the use of outcome measures, such as hospital mortality rates, as indicators of quality. Nor has research validated possible quality indicators by linking structural measures of quality with appropriate process and desired patient outcomes, process measures of quality with subsequent patient outcomes, or desired patient outcomes with prior process. Although this report attempted to develop a framework for assessing quality from an individual consumer's perspective, a conceptual framework is still lacking for the most likely hazards of medical care, to indicate how medical care is likely to fail and how to test for each major failure.

Intertwined with the shortcomings of assessment techniques is the dearth of necessary data to assess the quality of care. Several of the indicators-hospital mortality rates, adverse events, malpractice compensation, evaluation of physicians' performance through a specific condition, volume of services, physician specialization, and patients' assessments-suffer from lack of uniform methods to code, collect, and report data, especially about specific diagnoses. No routinely collected data permit quality assessors to evaluate physicians' practices outside of hospitals. Addi

tional data (such as diagnostic information for Medicare ambulatory care) and methods (such as uniform reporting requirements) are needed to assess the quality of ambulatory care. Although more information is available on hospital than ambulatory care, the hospital discharge information required by Medicare contains little information on the patient's status on admission, before the person received care. Even with Medicare data sets, one cannot easily track the services that a patient has received from different providers on an inpatient and ambulatory basis. This deficiency makes it difficult to attribute specific patient outcomes to prior medical care, a problem that will intensify as care moves increasingly into ambulatory settings.

Although some information related to quality, most notably hospital mortality rates for Medicare patients, is becoming available to the public, other relevant information, such as the JCAHO contingencies that a hospital receives, is regularly compiled but not publicly available. Nor is information covering several years generally available on certain quality indicators, such as hospital mortality rates, adverse events, and volume of hospital procedures. Such longitudinal information would be less likely than information for a single year to reflect random influences and more likely to indicate relationships related to the quality of providers' care. Some current efforts are beginning to periodically generate information, such as the hospital mortality rates of Medicare beneficiaries, and systems could be established to regularly produce information on other indicators.

When considering making quality-of-care information more generally available, one must consider the likely effect on medical providers. The use of some indicators may create perverse incentives. In the absence of techniques that adequately adjust for patient differences, for example, evaluating the quality of hospitals by their mortality rates would entail an incentive for hospitals to transfer or avoid admitting severely ill patients. Similarly, using hospital-acquired infections or other adverse events as indicators of quality could undercut efforts to diagnose, document, and correct certain deficiencies. The same effect could arise from applying criteria to evaluate physicians' performance for a specific condition, such as hypertension. Evaluating hospitals or physicians by the volume of procedures that they perform might encourage them to relax their criteria for using these procedures and perhaps perform some unnecessarily. These are but a few examples of how a conflict might arise between a climate to encourage hospitals and physicians to examine and improve their care and efforts to make assessments of providers' quality more publicly available. In some cases, regulations have addressed a problem, such as hospitals' transfer of severely ill patients, but such regulations have not resolved the underlying conflict. This conflict is particularly troubling because most reviews of medical care, both public and private, rely on physicians and other medical professionals and will continue to do so.

POLICY OPTIONS

This report has identified some potentially useful indicators of the quality of care, but also several deficiencies associated with quality assessments to guide consumers' choice of hospitals and physicians. The remainder of this chapter examines approaches that Congress could take to remedy the problems noted above in five areas: to improve techniques available to assess the quality of care, to ensure that acceptable techniques are used to produce quality assessments, to im

prove the availability of data required for quality assessments, to disclose information to the public, and to disseminate information on quality to individuals and organizations. Policy options in each of these areas are summarized in table 1-5. These approaches represent policy options, not recommendations, for Congress. Although some of the options are related, others are mutually exclusive approaches to address a particular problem.

Table 1-5.-Summary of Policy Options for Congress To Address Problems With Quality-of-Care Indicators

To improve quality assessment techniques

Option 1: Mandate and fund research and demonstrations to improve quality assessment techniques.

To ensure the quality of quality assessments Option 2: Mandate the selection of indicators to assess quality for Medicare and Medicaid.

Option 3: With option 2, mandate the use of indicators to assess hospitals and physicians in Medicare and Medicaid. Option 4: With option 2, mandate briefings of State and local groups on selected indicators and construction methods.

To improve the availability of required data

Option 5: With options 1 and 2, require demonstrations to collect clinical data from hospitals and physicians to assess the quality of their care.

Option 6: With options 1 and 2, establish a task force to develop uniform requirements for reporting data.

To disclose information to the public

Option 7: Require Medicare and Medicaid hospitals to make

certain indicators public, including contingencies from JCAHO and results of HCFA's reviews.

Option 8: Permit PROS and HCFA to disclose information that identifies specific physicians.

To disseminate information to the public

Option 9: Establish an HHS office to disseminate quality information.

Option 10: Mandate and fund research and demonstrations on disseminating quality information.

SOURCE: Office of Technology Assessment, 1988.

Policy options must be considered in light of the fact that information on some of the indicators evaluated in this report is already being disseminated and used, namely information on hospital mortality rates, sanctions imposed by HHS on the recommendations of PROS, and physician specialization. As policymakers address problems of quality assessment, activities that will improve these indicators merit high priority, so that consumers and providers using current information are not misled. Moreover, efforts to identify and improve physicians and hospitals whose quality falls below acceptable levels deserve priority over efforts to distinguish among good-quality providers. Identifying poor-quality providers is not only more pressing for consumers and other providers, but also consistent with the obligation of the government to protect public health and safety and with the current state of quality assessment.

As the policy options illustrate, Congress could take three approaches, separately or together, to address these problem areas. One approach would be for Congress to create and maintain a legal cli

mate conducive to the flow of information needed to evaluate providers' quality and to inform consumers. This approach would entail removing any legal barriers to providers' participation in quality assessment and to public disclosure of information useful to consumers. As a second approach, Congress could use the leverage of the Medicare and Medicaid programs to encourage hospitals, physicians, and States to undertake desired actions, such as collecting data, constructing indicators of quality, and making information publicly available. As a third approach, Congress could mandate that the Federal Government directly undertake efforts to remedy deficiencies regarding quality assessments for consumers.

Although whether a particular governmental activity is considered appropriate may depend on one's philosophy of government, consensus, if not unanimity, supports a government role in the flow of information. Scholars have often cited information to exemplify a good that is in everyone's interest to have but in no one's interest to finance individually. Like the responsibility for promoting public health and preserving national security, the responsibility for ensuring adequate vital public information may fall to government. This situation need not imply that the government itself undertake the desired activities. Some private sector organizations, notably the Joint Commission and the Institute of Medicine, already have considerable expertise and work underway. The Federal Government could stimulate private sector and State initiatives, promote the coordination of public and private activities, and cooperate in public-private enterprises. The discussion of the policy options below considers how Congress could encourage or use such non-Federal organizations.

Two relevant issues then arise for public policy: whether public information about hospital and physician quality has sufficient importance to justify governmental action and which approaches or options are likely to prove most effective in bringing about the desired results. As described earlier, individuals and organizations from many quarters support increased publicly available information on the quality of medical care for several reasons: so that consumers and providers can identify poor-quality physicians and

hospitals, so that people can learn over time how to choose and interact with providers, and so that consumers through their choices over time can influence providers to improve their quality of care. The relative merits of different strategies to accomplish these ends are discussed under each option.

To Improve Quality Assessment
Techniques

Although considerable work has been done to develop techniques to assess the quality of medical care, in general indicators require much refinement. The evaluation of the indicators in this report has brought to light several critical areas in which quality assessment techniques remain wanting. The inadequacy of techniques for taking into account factors other than quality that affect patients' outcomes impedes the public's interpretation of outcome measures, such as hospital mortality rates. Although the vast majority of medical care takes place in ambulatory settings, methods to assess physicians' ambulatory care are still in their infancy. Even more basic to quality assessment, the ability of structural and outcome indicators to measure the quality of care has not been validated by linking the results to the medical care process. Nor is there general agreement on the criteria and standards by which the medical care process should be judged. Identifying poor-quality providers is the immediate need, but techniques are also needed to distinguish levels of goodquality providers.

Option 1: Mandate and earmark funds for the Department of Health and Human Services, the Veterans Administration, and the Department of Defense to strengthen research and demonstrations to improve techniques for assessing the quality of medical care.

The Federal Government has a special interest in supporting quality assessment research. In addition to its role in developing basic research techniques, the Federal Government accounts for 30 percent of the Nation's medical expenditures, primarily through the Medicare program for elderly and disabled people and the Medicaid program for certain poor people, but also through the Veterans Administration for veterans and the Depart

ment of Defense for military personnel and their families.

Despite Federal and private funding of research on quality assessment (see app. E), serious gaps remain, and efforts do not flow from a systematic, long-term agenda. Few projects are attempting to validate outcome measures against the medical care that patients received or to examine the validity of structural measures of quality. Several projects are working on techniques to adjust outcome measures for relevant patient characteristics, but few of these plan to incorporate clinical information on a patient's status when the patient first sought medical care, information that is vital to assessing the quality of care that was subsequently provided. A continuing need to provide the basis for quality assessments of providers' performance is research on the clinical efficacy of common medical procedures. Currently funded projects do not appear to be laying the groundwork needed to assess the quality of medical care in ambulatory settings, an activity that the Omnibus Budget Reconciliation Act of 1986 (Public Law 99-509) stipulates that PROs are to undertake beginning no sooner than January 1989.

Yet another type of research needed to further the field of quality assessment is research on the criteria and standards for evaluating physicians'

and hospitals' performance. Drawing on the literature and expert opinion, some researchers have formulated criteria and standards to assess providers' performance for certain conditions. Generally accepted review criteria for many conditions are lacking, however, and quality assessors, including those in PROs, usually rely on their own criteria. For the most part, existing criteria and standards have not been tested. The generic screens that PROS apply to Medicare inpatient cases, for example, have not been validated. Nor has the process that PROS and the HHS Office of the Inspector General use to determine sanctions been evaluated. How to modify criteria over time to incorporate changes in medical knowledge and practice poses an additional challenge. Without some mechanism to take technological change into account, evaluating the quality of care through criteria and standards runs the risk of inhibiting medical advances.

Under the option described here, Congress not only would require the Federal agencies engaged in health services research and health care delivery to give high priority to research and demonstrations designed to improve quality assessment techniques, but also would earmark funds for this purpose. Federal agencies in turn could identify their research priorities and fund researchers to pursue them. Congress could rely on a decentralized research strategy, with each agency continuing to work independently. Alternatively, instead of continuing fragmented efforts in this field, Congress could establish a specific locus of responsibility for quality assessment research, in either an existing or newly created office.

That the Federal Government finances or provides medical care on a large scale gives it both the economic interest and the mechanisms to refine quality assessment techniques. The Government has considerable opportunity to amass data required for developing quality assessment techniques and to test alternative assessment methods across population subgroups, geographical regions, and medical care settings. Much could be learned by examining population-based data from Medicare. From these data, for example, researchers could derive statistics on the average and range of mortality rates for certain conditions. Those statistics could then be used to inform consumers

of the risks of specific treatments and to serve as a benchmark for developing standards to evaluate providers. In addition to amassing useful data, the Government also has the ability to bring together experts from medical specialty societies and other parties at interest to develop criteria and standards for assessment.

This option raises the issue of what is to be gained from targeting funds or creating a new locus for quality assessment at this time. With the assistance of expert groups, the Joint Commission is developing and testing measures of clinical performance and patient risk (see app. D). The Omnibus Budget Reconciliation Act of 1986 (Public Law 99-509) mandated certain studies related to quality assessment, including one now underway at the Institute of Medicine to examine criteria and standards for assessment. Congress may prefer to await the results of these studies and others underway at HCFA and the National Center for Health Services Research and Health Care Technology Assessment (see app. E) before mandating that additional research on specific topics be undertaken. Alternatively, gaps in current research, such as work on survey instruments for patients' assessments of their care, could be identified and corresponding projects could be undertaken to avoid further delay.

To Ensure the Quality of
Quality Assessments

Efforts to assess the quality of medical providers and to make the results public have mushroomed in recent years. In the Federal arena, the most notable effort was HCFA's release in 1986 and 1987 of the mortality rates experienced by Medicare beneficiaries in hospitals across the country. During those same years, the PRO for California disclosed publicly Medicare mortality rates in all California hospitals. Individual States, notably Massachusetts, New York, and Pennsylvania, are beginning to assess the quality of hospitals and physicians and plan to make the information public in the near future.

Private activities are also increasing. Individual hospitals, organizations of hospitals, large clinics, and HMOs are engaged in assessing the quality of their own care. These private organi