prehensive level of care, for example, is that the "emergency service offers reasonable care in determining whether an emergency exists, renders lifesaving first aid, and makes appropriate referral to the nearest facilities that are capable of providing needed services." There must be some mechanism for providing physician coverage at all times in Level IV emergency facilities, but the mechanism is to be defined by the medical staff of the hospital. That the standard does not require immediate availability is reflected in JCAHO standards for Level III and higher emergency facilities. Level III facilities, for example, are required to have at least one physician available to the emergency care area within approximately 30 minutes. 12 The impact of having a trained and experienced physician available in an emergency department at all times has not been evaluated, so the relative validity of these standards cannot be judged.

In addition, it is noteworthy that only one set of standards or guidelines for emergency services-JCAHO's-requires that a hospital have a provision for providing emergency care 24 hours a day, 7 days a week (325).

12Level I and II hospitals are required to have at least one physician experienced in emergency care on duty in the emergency care area at all times. In addition, in Level I hospitals, there must be inhospital physician coverage by members of the medical staff or by senior-level residents for at least medical, surgical, orthopedic, obstetric/gynecological, pediatric, and anesthesiology services.

Trauma Center Designations.-A review by the Centers for Disease Control of mortality data for 1984 shows that unintentional injuries were the leading cause of "years of potential life lost" before the age of 65 (440). A large proportion of efforts to decrease the number of deaths caused by injury have focused on injury prevention, but considerable attention has also been directed to the designation and implementation of emergency medical service systems (e.g., 454, 455; the Federal Emergency Medical Services Systems Act of 1973 [revised in 1975, repealed in 198113]). In an organized emergency system, some hospitals are designated as regional trauma centers, to which severely multiply injured individuals are brought for treatment.

Intuitively, one expects that treatment and outcome in trauma centers will be better than elsewhere because of the immediate availability of rapid transportation, highly trained field personnel and emergency physicians, modern diagnostic tools, and experienced trauma surgeons (543). The only current national guidelines for trauma centers have been devised by the American College of Surgeons' Committee on Trauma (26). The American College of Surgeons' guidelines incorporate resources for both prehospital and hospital care. For hospitals, the guidelines specify the desired characteristics for three levels of trauma care. The two highest levels (Levels I and II) have similar requirements for patient care; the highest level (Level I) has additional requirements for education and research in trauma. Level III trauma center hospitals serve communities that do not have all the resources usually associated with Level I or II institutions; Level III facilities must have a "maximum commitment to trauma care commensurate with resources." Thus, for example, a Level III hospital might have a surgeon and other personnel on call rather than in-house. Nonetheless, a Level III facility would be called a trauma center by the American College of Surgeons.

According to a recent survey by the American College of Surgeons, approximately 177 hospitals

13 The Federal Government devolved much of its leadership responsibilities to States by folding the Emergency Medical Services Systems Act program into the Preventive Health and Health Services block grant.

have Level I trauma centers, 138 of which are designated as Level I by some external body; the remainder are self-designations by hospitals themselves. About 157 hospitals have Level II trauma centers, 124 of which are so designated by some external body (127). Table 9-5 indicates that only 19 States designate trauma centers using either the guidelines of the American College of Surgeons or a modified version of those guidelines.

The availability of various surgical, as opposed to medical, personnel is a major requirement for meeting the American College of Surgeons' guidelines, although there are numerous other require

RELIABILITY OF THE INDICATOR

Accreditation schemes for hospitals overall and for particular services are, it is clear, highly variable. To a consumer interested in neonatal intensive care, certification by the State of Ohio for a particular level of neonatal intensive care would convey much more information than the fact that a hospital with a neonatal intensive care unit had received JCAHO accreditation. Similarly, to a person interested in cancer care, a hospital's membership in the Association of Community Cancer Centers or designation as a Comprehensive Cancer Center by the National Cancer Institute would not convey the same type of approval as would approval by the Cancer Commission of the American College of Surgeons. Overall, HCFA's certification process is not as rigorous as JCAHO's accreditation process.

Some States have developed specific requirements for hospitals to offer specific services, but

ments as well (26). For example, the American College of Surgeons recommends that a trauma team be organized and directed by a surgeon. The surgeon-directed trauma team is to evaluate the patient initially, and a surgeon is to be responsible for the patient's overall care. A physician with special competence in care of the critically injured is to be a designated "member" of the trauma team, and is to continuously staff the overall emergency department, but not be the head of the trauma team. Although the need for surgeons to deliver most trauma care is generally acknowledged, there is some controversy about who should design and manage the overall service (44).

the types of services under these regulations and the specific requirements differ across States. In California, for example, emergency services are considered a supplemental service and appear as such on hospital licenses and published information for consumers (113,345); New York is about to change a similar regulation to make emergency services a basic requirement (472).

At the level of the individual State standard, there is considerable variation, because States develop their standards through statute and regulation, and statutes vary across States. The reliability of the surveyors and the survey process may vary as well. Hospitals surveyed by JCAHO, for example, have complained that judgments regarding their compliance with the same standard may vary considerably between survey periods. In part, the variation is due to periodic revision by JCAHO of its standards, a necessity.

VALIDITY OF THE INDICATOR

Accreditation for scope of hospital services is not a single entity, and individual standards themselves may vary in the extent to which they have been validated. Optimally, perhaps, standards and guidelines for scope of services would be based on medical practice with systematically demonstrated efficacy. The problem, however, is

that much of medical practice is not based on evidence from scientific studies (628). Decisions about the "best" staff, equipment, and organization for a particular service or a particular problem are often the result of clinical judgment. Thus, most standards have been developed through expert consensus.

@Excludes trauma center designations by counties and cities, of which there are few. Specialized centers include such facilities as spinal cord injury centers and burn centers.

CSome States, for example, use survey teams from or suggested by the American College of Surgeons.

SOURCE: H. Champion (Chief Director, Critical Surgical Care Services, Medstar, Washington Hospital Center), and H. Tetes (Attorney at Law, Bricker & Eckler), "State Laws and Regulations," Washington, DC, 1988.

Expert consensus may be an appropriate basis for establishing standards and guidelines for hospitals overall and for particular conditions, services, and departments. For some services, however, groups of experts disagree with one another, either on the need to establish standards or on the content of the standards themselves. The consumer is then left with the puzzling question of which group of standards or guidelines is more valid.

The validity of particular standards and guidelines could be demonstrated with studies of relationships between standards or guidelines and good process and outcomes, determined post hoc. Some standards and guidelines, such as those for neonatal intensive care and trauma centers, have been subjected to some such study, but most have not. JCAHO's hospital accreditation standards for hospitals have been subjected to very little study, and HCFA's hospital certification standards subjected to none. The studies that have been conducted have had methodological problems. For the most part, they have relied on retrospective analysis and outcomes as criteria and have not been conducted by independent observers.

One significant problem, applicable to all standards and guidelines, is that the standards or guidelines may change over time, sometimes significantly (388), a situation that makes the results of studies conducted at one point in time not applicable to subsequent standards. Frequent changes in standards may, of course, be necessary to reflect changes in technology and medical practice.

Validity of Overall Hospital
Accreditation

There has been little attempt to validate overall JCAHO accreditation as an indicator of the quality of care. An important factor limiting studies seeking to validate JCAHO accreditation is that accreditation is refused or withdrawn for so few hospitals that the mere fact of accreditation may not be very sensitive to variations in quality. The few studies of the validity of JCAHO

14At the time Hyman collected his data, JCAHO was using the terminology "recommendations" rather than "contingencies."

accreditation as an indicator of the quality of care have yielded inconclusive or noncomparable results.

Hyman obtained the results of JCAHO surveys for New York City hospitals (312). Unexpectedly, Hyman found that publicly supported hospitals had better JCAHO contingency scores14 than voluntary not-for-profit hospitals on 9 of 11 functions. Friedman analyzed the relationships between numbers of JCAHO contingencies and HCFA's 1984 hospital mortality data (237). The result was a very low, statistically insignificant correlation, but this result is not surprising given the problems with HCFA's measure of hospital mortality (see ch. 4). One internal JCAHO study found a high level of agreement among JCAHO senior clinical and administrative staff as to the significance of several categories of standards for ensuring quality patient outcomes, but actual outcomes or process criteria were not used as validation standards (572).

Because JCAHO accreditation means that hospitals will be certified by HCFA, HCFA is required by law to validate JCAHO's results (Subsection 1864(c) of the Social Security Act). Every year, HCFA requests that State surveyors survey a small sample of JCAHO-accredited hospitals, stratified to be representative of hospitals nationally. HCFA also asks State surveyors to investigate patient complaints that seem to have substance. The State surveyors perform JCAHO validation surveys for HCFA using the Medicare conditions of participation. If a State surveyor finds that a hospital has significant deficiencies that could affect the health and safety of patients, the hospital is placed under State surveillance until the deficiencies are corrected. The hospital is no longer deemed to meet the Medicare conditions of participation, and the State monitors the correction of any deficiency.

HCFA conducted the last published JCAHO validation survey in fiscal year 1983, and transmitted it to Congress in 1986 (639). In general, JCAHO hospitals were found to be in compliance with HCFA's requirements. Any conclusion that JCAHO standards are valid because of their compliance with HCFA's requirements, however, depends on the validity of HCFA's survey process,

and that process has not been validated. In addition, the discrepancy rates that HCFA found between HCFA's deficiencies and JCAHO's contingencies would mean that 276 hospitals in any single year, and as many as 750 hospitals overallis would be out of compliance on some condition of participation.

One future source of information for developing and validating JCAHO (and HCFA) standards is JCAHO's Agenda for Change project (see app. D). This project is attempting to develop more valid and condition-specific standards, including clinical process and outcome indicators. A potential JCAHO clinical indicator for obstetrics, for example, is birthweight-specific hospital mortality rates; hospitals designating themselves as high level neonatal intensive care units may have to meet a minimum birthweight-specific mortality rate. This project is being pilot-tested now with a small sample (324). In addition, JCAHO is progressing with plans to revamp its structural indicators so that they reflect the characteristics of effective health care organizations.

Validity of Standards and Guidelines for Specific Services

Many of the available studies of the validity of trauma center designations as indicators of the quality of care are methodologically flawed. Those that rely in whole or in part on autopsy

15HCFA's 1983 validation surveys found that up to 15 percent of hospitals surveyed were not in compliance with HCFA standards, although they had been in compliance with JCAHO's standards. If the 15 percent noncompliance rate is multiplied by the total number of JCAHO-surveyed hospitals (5,000), the number of hospitals not in compliance with HCFA standards would be 750.

studies, for example, are biased in that not all deaths result in autopsies. Some studies use different sources of information to determine causes of death. In one study of the San Diego County Regional Trauma System, for example, the causes of deaths in trauma centers were taken from a trauma registry, but the causes of death in comparison hospitals were taken from autopsies (564).

Perhaps more important, most studies of trauma center designations tend to be uncontrolled; that is, they merely compare patient outcomes before and after implementation of a trauma system. Studies that merely compare outcomes before and after implementation of a trauma system do not take into account factors other than medical care that may be responsible for reducing death rates from trauma (543). These factors may include simultaneous changes, such as reductions in speed limits and enhanced enforcement of drunk driving laws. In studies of standards and guidelines for neonatal intensive care, most of the research has been done only on Level III neonatal intensive care units (194), and the validation standards have been outcome measures, primarily mortality. Plans are underway to conduct studies of neonatal intensive care units using process criteria for validation.

Standards for emergency services have not been subject to the same amount of study that trauma center designations have, perhaps because the scope of services in emergency rooms is so broad. A knowledgeable observer concluded that there is no dependable knowledge about interhospital differences in emergency department performance or about the sources and correlates of such differences; there is also no dependable knowledge about the factors and conditions that facilitate or hinder emergency department effectiveness (245, 246).

If validated, compliance with external standards for scope of hospital services is potentially an extremely valuable and easily accessible indicator of the quality of care for consumers. Currently, JCAHO and the American College of Surgeons both provide hospitals with a certificate to

post. JCAHO's certificate addresses overall hospital accreditation, not individual services. Detailed reports on the results of JCAHO surveys of hospitals would be more informative; but these results are for the most part not easily obtained.