this chapter examines intensively one criterion that is frequently found in occurrence screensnamely, nosocomial infections. (Another common element in almost all screens-hospital deaths, or some subset of deaths-is analyzed as a potential indicator of quality in ch. 4 of this report.) A shortcoming of the use of nosocomial infection rates as a quality indicator is that a single indicator may effectively identify quality problems in a specific type of patient or clinical service but not address problems in other areas of medical care; very poor-quality care may go unregistered. A major strength of existing hospital screening systems may well be the use of multiple criteria to identify problems. On the other hand, multiple variables complicate analysis, even under ideal research conditions. Where relevant research related to occurrence screens has been done, this chapter notes it. The remainder of the chapter is organized as follows. First, occurrence screens that might be considered standardized because they have been developed at the State or national level are described. Then, the reliability, validity, and feasibility of using either nosocomial infections or "standard" occurrence screens as indicators of the quality of care are examined. Finally, conclusions are stated, and the policy implications of using adverse events as indicators of the quality of care in hospitals are explored. STATE- AND NATIONAL-LEVEL OCCURRENCE SCREENS In the vast majority of cases, hospitals design and implement their own screening systems for adverse events. Under development or already in place, however, are a number of national and State-level activities that use the same general methods and approach. In the private sector, for example, the Maryland Hospital Association has undertaken a project to find a limited number of data elements (clinical indicators) that could be commonly defined and would permit meaningful comparisons among hospitals for the purpose of assessing quality. Nine indicators were tested in pilot Maryland hospitals beginning in 1985, and today, following deletions, additions, and revisions of various indicators, the study is being conducted in more than 40 voluntarily participating hospitals. The indicators being studied include nosocomial infections, surgical wound infections, autopsy rates, newborn deaths, perioperative deaths, cesarean sections, hospital readmissions, unplanned admissions following ambulatory surgery, intensive care unit readmissions, and unscheduled returns to the operating room (607). The State of Pennsylvania's Health Care Cost Containment Council collects data on two elements that are usually considered occurrences, nosocomial infections and hospital readmissions (484). Because the data are collected on every hospital patient discharged, adverse events can be linked to specific physicians and services. Moreover, for every hospitalized patient, Pennsylvania hospitals are required to submit to the State Council an indicator of the severity of illness (MedisGroups methodology) along with other more standard discharge abstract information. The Pennsylvania reporting system is currently being implemented, and published statistics that include patient severity of illness adjustments are not expected before 1990. Other States have demonstrated interest in similar reporting systems. Colorado, for example, has issued regulations effective January 1989 that require reporting patient severity of illness levels as part of required hospital discharge abstracting systems (140). JCAHO expects to expand its accreditation activities to include the use of clinical indicators to screen hospital cases for quality problems (324). Three JCAHO task forces, working on obstetrical, anesthesia-related, and hospitalwide clinical indicators, have identified structure, process, and outcome clinical criteria that are currently being tested as screens for quality problems in pilot hospitals. The hospitalwide indicators being evaluated are shown in table 5-2. Also shown in that table are the most important patient risk factors or covariates that might also influence outcomes. JCAHO is continuing to develop indicators for a variety of clinical areas, but use of the clinical Table 5-2.-JCAHO Hospitalwide Clinical Indicators Being Evaluated as Screens for Hospital Quality Problems HOSPITALWIDE CLINICAL INDICATORS BEING EVALUATED 1. Unplanned readmissions to a hospital shortly after inpatient surgery 2. Unplanned admissions to a hospital shortly after outpatient surgery or specified procedures 3. Development or worsening of pressure ulcers (decubiti) 4. Development of wound infections after clean or clean-contaminated surgical procedures 5. Development of pneumonia in patients treated in special care units 6. Development of infections related to the use of intravascular devices in special care units 7. Proper timing of antibiotic prophylaxis for specified surgical procedures 8. Appropriate use of blood culture sensitivities in the treatment of bacterial sepsis 9. Development of complications associated with suboptimal methods of administration and monitoring of specified medications 10. Commission of important medication errors resulting in death or major morbidity 11. Mortality of patients with specified medical conditions either during hospitalization or within 30 days of admission if death occurs at another institution to which the patient was transferred 12. Mortality of patients after specified surgical procedures either during hospitalization or within 30 days of admission if death occurs at another institution to which the patient was transferred 13. Mortality among patients treated in the hospital for injuries sustained immediately prior to treatment when death occurs within 30 days of injury or during a hospitalization that was precipitated by the occurrence of the injury Supplemental information collected Patient risk factors (covariates) that might influence outcomes Age Height and weight Braden Risk Scale on admission to hospital and special care units Glasgow Coma Score on admission to hospital and special care units Trauma Score of patients prior to treatment for injuries Diagnoses on admission to hospital, immediately prior to operation or specified procedure, and on admission to special care units (6 digit ICD-9-CM) Types of surgical or other specified procedures, if any (4 digit ICD-9-CM) Nature of surgical or other specified procedures, if any (scheduled, urgent, or unscheduled) Type and site of intravascular devices used in special care unit Selected chronic medications on admission to hospital Selected laboratory values on admission to hospital, immediately prior to operation or specified procedure, and on admission to special care units Temperature, pulse, respiration, and systolic and diastolic blood pressure on admission to hospital, immediately prior to operation or specified procedure, and on admission to special care units Other information For patient admitted after outpatient procedure: stated reason for admission Patient with endotracheal tube or tracheostomy in special care unit Use of nasogastric tube in special care unit SOURCE: Joint Commission on the Accreditation of Healthcare Organizations, National Invitational Forum on Clinical Indicators (Chicago, IL: November 1987). indicators as part of the accreditation process is not expected to be fully implemented until 1990 at the earliest. The U.S. Department of Defense screens about 10 percent of all discharges from its 167 hospitals using exhaustive process and outcome clinical criteria that were developed by consensus panels of experts (447). This screening takes place under the Department of Defense Civilian External Peer Review Program. All cases involving 1 of 34 specific diagnoses or 14 problems are sampled, and the patients' medical records specially abstracted by medical record technicians. The abstracted information is computerized, and the screen of clinical criteria then applied. Cases failing the computer screen (about 10 to 20 percent fail) are reviewed by physicians. The adverse event screening program that has had the most far-reaching impact to date is HCFA's "generic quality screen," which is used to screen hospitalized Medicare patients for quality problems (see table 5-3). Since July 1986, utilization and quality control peer review organizations (PROs) have been required to apply the generic screens to every case they review (about one-fourth of all Medicare discharges). Nurse reviewers examine patients' medical records, and if a screen is failed, the medical record is referred *1. Adequacy of discharge planning Table 5-3.-HCFA's Generic Quality Screensa No documented plan for appropriate followup care or discharge planning as necessary, with consideration of physical, emotional, and mental status/needs at the time of discharge. 2. Medical stability of the patient at discharge a. Blood pressure on day before or day of discharge systolic-less than 85 or greater than 180 diastolic-less than 50 or greater than 110 b. Temperature on day before or day of discharge greater than 101° F oral (rectal 102° F) c. Pulse less than 50 (or 45 if the patient is on a beta blocker), or greater than 120 within 24 hours of discharge d. Abnormal results of diagnostic services which are not addressed or explained in the medical record e. Intravenous fluids or drugs on the day of discharge (excludes KVOS, antibiotics, chemotherapy, or total parenteral nutrition) f. Purulent or bloody drainage of postoperative wound within 24 hours prior to discharge 3. Deaths a. During or following elective surgery b. Following return to intensive care unit, coronary care or special care unit within 24 hours of being transferred out c. Other unexpected death *4. Nosocomial infections a. Temperature increase of more than 2° F more than 72 hours from admission b. Indication of an infection following an invasive procedure (e.g., suctioning, catheter insertion, tube feeding, surgery) 5. Unscheduled return to surgery within same admission for same condition as previous surgery or to correct operative problem (exclude "staged" procedures) 6. Trauma suffered in the hospital a. Unplanned removal or repair of a normal organ (i.e., removal or repair not addressed in operative consent) *b. Fall with injury or untoward effect (including but not limited to fracture, dislocation, concussion, laceration, etc.) c. Life-threatening complications of anesthesia d. Life-threatening transfusion error or reaction e. Hospital acquired decubitus ulcer f. Care resulting in serious or life-threatening complications, not related to admitting signs and symptoms, including but not limited to the neurological, endocrine, cardiovascular, renal or respiratory body systems (e.g., resulting in dialysis, unplanned transfer to special care unit, lengthened hospital stay) g. Major adverse drug reaction or medication error with serious potential for harm or resulting in special measures to correct (e.g., intubation, cardiopulmonary resuscitation, gastric lavage) including but not limited to the following: i. Incorrect antibiotic ordered by the physician (e.g., inconsistent with diagnostic studies or the patient's history of drug allergy) ii. No diagnostic studies to confirm which drug is correct to administer iii. Serum drug levels not performed as needed iv. Diagnostic studies or other measures for side effects not performed as needed (e.g., BUN, creatinine, intake and output) aFor entries marked with an asterisk in this table, the PRO reviewer is to record the failure of the screen, but need not refer to physician reviewer. SOURCE: U.S. Department of Health and Human Services, Health Care Financing Administration, Health Standards and Quality Bureau, 1986-1988 PRO Scope of Work, Baltimore, MD, Nov. 4, 1985. to a physician advisor for further review. Only the physician advisor can declare a case a "quality problem." On the basis of this information, the PROS build provider profiles for their own internal use; they also take corrective actions RELIABILITY OF THE INDICATOR As noted earlier in this chapter, OTA chose an adverse outcome used in almost all existing occurrence and generic screens for indepth review ranging from education to intensified review, and ultimately to sanctions (see ch. 6). Appendix D provides a full description of the PROS' review procedures and responsibilities. namely, nosocomial infections. One reason for selecting nosocomial, or hospital-acquired, infections is that such infections are quite prevalent. Six percent of all U.S. hospitalizations are complicated by nosocomial infections, amounting to more than four million nosocomial infections per year (269). Of course, infections may also be acquired in the community prior to admission to the hospital. Nosocomial infections are defined as infections that are not known to be present or incubating at the time of admission. The most common nosocomial infections are urinary tract infections (42 percent), followed by surgical wound infections (24 percent), pneumonia (10 percent), and infections of the bloodstream (bacteremia) (5 percent). These four types of infections account for about 80 percent of all nosocomial infections. Almost three-quarters of nosocomial infections occur among patients undergoing surgery (273). The most difficult obstacle to the reliable measurement of nosocomial infections is the lack of standardized case finding. Reliable measurement of infections requires that trained surveillance personnel search actively for cases using standardized clinical definitions of infections (269). No system of routine data collection is completely sensitive in identifying nosocomial infections, and the surveillance techniques that are used in case finding in various hospitals differ fundamentally (616). The likelihood that nosocomial infections will be clearly recorded in a patient's medical record and/or coded on a hospital discharge abstract varies widely by hospital, but relying on written diagnoses is generally an inaccurate method of determining infection rates (232). One study in a university hospital found, for example, that 43 percent of nosocomial infections were not coded in the hospital discharge abstract (409). A study sponsored by the Centers for Disease Control (CDC) showed that reliable measurement of, and changes in, nosocomial infection rates at various sites are possible in a large-scale data collection effort that relies on medical record review (275). The Study on the Efficacy of Nosocomial Infection Control (SENIC) Project evaluated the efficacy of the infection surveillance and control programs established between 1970 and 1975-76 in a representative sample of U.S. hospitals. Incidence rates of nosocomial infections in four sites (urinary tract, surgical wound, lower respiratory tract, and bloodstream) were determined from a random sample of medical records in each of the 2 years in 338 hospitals stratified by size, teaching status, and infection control activity. To meas ure nosocomial infection rates reliably, CDC devised a standardized method of making diagnoses via retrospective review of patients' medical records, and it validated the method's accuracy through a series of pilot studies. Nonphysician CDC reviewers, who underwent careful training and infield monitoring, abstracted relevant data, recorded them on standardized forms, and applied a set of standardized algorithms to arrive at the infection diagnoses. The retrospective chart review method used in the SENIC Project (by nonphysicians following a standardized procedure) compared favorably (average sensitivity of 0.74) with the "gold standard" method of physician-epidemiologists supervising intensive prospective data collection teams (230,275). As measured against this standard, physician self-reporting forms were least sensitive (0.14 to 0.34) in finding cases of nosocomial infection, and clinical surveillance for evidence of fever, antibiotic use, or both were only moderately sensitive (0.47 to 0.59) (230). Because the recognition of infections depends in part on physicians' propensity for ordering the cultures and chest X-rays that confirm the presence of infection, the SENIC Project also analyzed the use of these diagnostic tests in the sample hospitals (270). Generally, the researchers found an increase over time in the use of diagnostic tests, and the increased use of these tests was associated with increased recognition of infectious diseases. More importantly, despite clinical agreement on the efficacy of these diagnostic tests, hospitals differed significantly in diagnostic medical practices. Hospitals with high rates of culturing, working up fevers, and obtaining chest Xrays showed higher observed rates of nosocomial infections. This finding presents an additional measurement problem that cannot be resolved through better or standardized data collection efforts (270). If nosocomial infection rates were used as indicators of quality in cross-hospital comparisons, those hospitals that were effectively identifying nosocomial infections through appropriate testing could be penalized. Because no diagnostic testing is necessary to confirm the presence of surgical wound infections, a possible solution would be to compare infection rates only for this subset of nosocomial infections (270,305). HCFA's Generic Quality Screens As shown in table 5-3, HCFA's generic quality screens apply two criteria related to nosocomial infections (item 4): "a) temperature increase of more than 2 degrees more than 72 hours from admission; and b) indication of an infection following an invasive procedure." Depending on how individual PROS interpret and use the nosocomial infection screens, results could vary greatly. Nurse reviewers searching for "indications of an infection," for example, could either rigorously review all laboratory records, progress notes, and nursing notes or simply look for documentation of antibiotic use or specific laboratory test results. According to an initial report from HCFA on the use of the generic quality screen by PROS, more of the discharges reviewed failed the nosocomial infection screen (5 percent) than any other (except medical stability at discharge), but fewer than 15 percent of these cases upon further review by a physician advisor actually had a significant medical problem (653). The physician advisor must decide which of the discharges that have failed the nosocomial infection screen constitute actual "quality problems." There are no guidelines on how clinically to ascertain a quality problem; the judgment is primarily subjective. Thus, at present, the same case that is considered a problem in one PRO (or by one physician advisor) might be discounted by another PRO. In some PROS, for example, the physician advisors were not counting nosocomial infections as quality problems if the infections were treated appropriately (487). Recently revised guidelines on the application of the generic quality screens clarify that nosocomial infections should be counted regardless of therapy (652). Nonetheless, there is obviously a severe reliability problem with HCFA's generic quality screen that results from the subjective nature of the physician advisor's audit. Only summary data on the generic quality screens (neither hospitals nor physicians are identified) are forwarded by the PROS to HCFA. Data reported to HCFA for the first year during which the generic screens were used showed wide variation in the incidence of screen failures and of confirmed quality problems across PROS (660). In several PROS, fewer than 5 percent of cases failed any screen; in other PROS, more than 40 percent failed. In cases of screen failures, the percentage of confirmed quality problems ranged from zero to 100 percent. To ameliorate substantial reliability problems, the so-called SuperPRO, an independent contractor, is charged to re-review a sample of each PRO's cases to validate the determinations of nurse reviewers and physician advisors. In its first review of the application of the generic quality screens in 45 PROS, the SuperPRO found 8.9 percent of sample cases with quality problems v. only 3.8 percent reported by the PROS (654). In response to critiques, HCFA has revised the generic quality screens for the PROS' third round of contracts, which will probably begin in early 1989 (652). The revised generic quality screens have several changes (see app. D). In the future, for example, nurse reviewers will flag a case as a nosocomial infection only if two or more indications listed in new HCFA guidelines are present in a patient's chart.3 In addition, all PROs have Indicators of a nosocomial infection include: temperature elevation of 101 degrees Fahrenheit or greater; elevated white blood count and/or left shift; isolation of organism from body fluids or specimens; appropriate radiographic imaging abnormalities; purulent drainage; heat, redness, focal tenderness and/or pain; pyuria, dysuria; and productive cough (652). |