Page images
PDF
EPUB

amine the volume-outcome relationship. These studies pertain to both hospital and physician volume. Although most research relates to hospital volume, a growing body of literature focuses on the relative importance of physician volume in contrast or in addition to hospital volume.

This chapter examines the reliability, validity, and feasibility of using the volume-outcome rela

RELIABILITY OF THE INDICATOR

Information on the volume of procedures and diagnoses and on inhospital mortality is routinely available from two sources: hospital discharge abstracts and insurance claims. There are several problems with the reliability of hospital discharge abstract data. Errors can occur at different points during the data collection process: in recording the patient's diagnosis or procedure onto the medical chart, in the translation of the chart onto discharge abstract forms, or in the transformation of discharge abstract forms into large-scale computerized data systems. Several studies of the accuracy of hospital abstracting suggest a high error rate (450,532). Moreover, inaccuracy in the data may be the result of random errors, such as misapplication of coding rules or the selection of vague diagnosis codes, or may be the result of purposeful misspecification of a patient's principal diagnosis in order to achieve an optimal diagnosisrelated group (DRG) for Medicare payment purposes. Recently, a reabstracting study noted that incorrect DRGs were originally assigned 20.8 percent of the time in 1984-85 and that 61.7 percent of these errors benefited the hospital (304).

Insurance claims data—especially non-Medicare insurance claims data-usually include less information about diagnoses than do routinely collected hospital discharge abstract data. Moreover, coding problems in the case of claims data may be worse than those in the case of hospital discharge abstracts. The problem is especially acute for diagnoses; procedures are generally well coded (131).

The pertinent question here, however, is not whether coding errors occur, but how such errors affect volume-outcome studies. The miscoding of

tionship as an indicator of the quality of medical care, and explores the issues of causality as well as other relevant conceptual and methodological issues. How volume data might be used by consumers in choosing hospitals and physicians is discussed, and further necessary research is outlined.

a diagnosis or procedure may cause undercounts or overcounts of the number of patients in certain categories. Many of the diagnoses and procedures that have been studied in the volumeoutcome literature are so important to a patient's hospitalization and the categories are so broad, however, that miscounts of patients are probably not an important concern. Total hip replacement, for example, would be unlikely to be overlooked. Moreover, in many studies, volume is specified as a series of categories (e.g., high, medium, and low), so a small amount of random undercounting or overcounting is not crucial. Miscoding of patients' simultaneously existing illnesses (comorbidities) may be a problem in case-mix adjustments to reflect patient differences. The problems in adjusting for patient differences in the analysis of volume are similar to those present in the analysis of hospital-specific mortality outcomes (see ch. 4).

Volume-outcome studies are generally crosssectional, and changes in the accuracy of data over time are less important than systematic differences across hospitals. When analyses are focused on individual hospitals, the reliability of data is an important concern, because misclassification could result in the mislabeling of a hospital as a good- or poor-quality provider. When the investigation concerns the identification and exploration of the hypothesized relationship between volume and outcome, the reliability of data is less of a key concern. Suppose there are random errors across hospitals in the coding of diagnoses. Such errors will affect the precision with which relationships are estimated, but if the errors are uncorrelated with volume, the volumeoutcome effect will not be altered.

VALIDITY OF THE INDICATOR

Table 8-1 presents a summary listing of the 15 procedures and diagnoses investigated in the 26 studies used for the analysis in this chapter. The studies are grouped in the left hand column by research team and by the publication date of the first article by the team (e.g., all three studies by Kelly and her colleagues are shown together). To check which authors studied a particular diagnosis or procedure, read down the column for a given procedure or diagnosis.

Of the 15 procedures and diagnoses investigated in the 26 studies, 13 are surgical procedures. Only 2 are medical diagnoses: acute myocardial infarction ("heart attack") and newborn diseases. The study of surgical procedures is easier than the study of medical diagnoses for several reasons. First, surgical procedures are generally well identified and coded both on hospital discharge abstracts and insurance claims. The occurrence of an operation is rarely in dispute, even though the choice of procedure or necessity for it may be questioned by various physicians.3 The determination of some diagnoses, on the other hand, is often quite difficult; comparably trained clinicians may disagree on an individual patient's diagnosis.

Second, although severity of illness may vary with both surgically treated patients and medically treated patients, it is less likely to be a major source of bias in volume-outcome studies of patients treated surgically. Surgery is usually used to increase longevity or to correct a problem that interferes with the quality of a person's life but is not immediately life-threatening. Thus, a surgically treated patient is often in reasonably good health on admission to the hospital, and shortterm mortality is more likely to reflect the effects of treatment than to reflect the patient's initial health status. In medical admissions, on the other hand, there is greater variation in the complexity of cases, and a patient's health status on admission may be a more important determinant of short-term outcomes than the quality of care ren

'In some cases, there may be miscoding of which procedure occurred, for example, revision of total hip versus total hip replacement.

dered is. Thus, the paucity of good measures of patients' severity of illness probably has a greater impact on studies involving medical admissions than studies involving surgical admissions.

Measures of Volume and Outcome

Volume is measured in several ways in the 26 studies reviewed by OTA:

• categorical variables (e.g., low-and highvolume groups, or a four-or five-category classification),

• a continuous variable (e.g., a count of number of patients, which allows for a linear relation),

• volume and volume squared (which allows for either linear or "U"-shaped curves), or • log of volume (which allows for a stronger effect at low volumes and progressively weaker effects at higher volumes).

Most of the studies measure volume for a single year, although some studies use other periods. One study uses a hybrid: the proportion of patients in a hospital (a continuous measure) treated by surgeons with low volumes (a dichotomous variable) (307).

Four measures of patient outcomes are used in the 26 studies:

• inhospital mortality,

• mortality within a fixed period of time, • complications or health status measures, and • long hospital stays as a proxy for complications.

The use of mortality as an outcome measure of quality has some limitations (see ch. 4). For some procedures and diagnoses, mortality is so rare an event that it is difficult to determine whether an occasional death indicates a pattern of poor quality or a chance occurrence. Important biases may also be introduced because discharge policies controlled by hospitals can affect inpatient mortality rates. In hospitals that transfer patients with severe complications to other, more appropriate facilities, such as regional tertiary hospitals, there are likely to be lower mor

Abdominal

aortic aneurysm

Acute myocardial

infarction

Appendectomy

Bilary tract surgery

catheterization

Cardiac

Table 8-1.-Studies Reviewed by OTA on the Relationship Between Volume and Outcome for Specific Diagnoses and Procedures

Coronary artery bypass graft

surgery

Femur fracture

[blocks in formation]
[blocks in formation]

12. Shortell and LoGerfo, 1981 (571)

13. Hertzer, et al., 1984 (295)
14. Flood, et al., 19849 (217).
15. Rosenblatt, et al., 1985 (538)
16. Riley and Lubitz, 1985 (520)
17. Kempczinski, et al., 1986 (349).
18. Sloan, et al., 1986h (582).
19. Kelly and Hellinger, 1986 (347)
20. Kelly and Hellinger, 1987 (348)

21. Kelly, forthcoming' (346)
22. Roos, et al., 1986 (531)

23. Roos, et al., 1987 (533)

24. Wennberg, et al., 1987 (697)
25. Showstack, et al., 1987 (573)
26. Fowles, et al., 1987 (227)
Abbreviations: HV hospital volume; PV

[subsumed][subsumed][subsumed][subsumed][subsumed][ocr errors][subsumed]
[blocks in formation]

9Flood, et al. (1984), also studied amputation of lower limb, nonsurgical gallbladder diagnosis, and nonsurgical ulcer diagnosis.

Sloan, et al. (1986), also studied morbid obesity surgery, mastectomy, nephrectomy, and spinal fusion.

Kelly (forthcoming) also studied atherosclerosis, cranial injury, diabetes, and hypertension.

SOURCE: Office of Technology Assessment, 1988.

[blocks in formation]
[subsumed][subsumed][ocr errors]
[graphic]

Hernia

Hysterectomy

Intestinal
operation

Newborn diseases

Prostatectomy

Stomach

operation

Total hip

replacement

Vascular surgery

tality rates. In hospitals with longer average stays, there is a greater chance of observing a death. Suppose, for example, that one hospital typically keeps patients for 10 days after a certain surgical procedure, while another hospital works to get patients on their feet and discharges them after a week. If a certain fraction of patients from each hospital experiences a fatal heart attack on the 8th to 10th days after surgery, these deaths will be counted in the inhospital mortality rate of only the first hospital. Because of these biases, some researchers calculate mortality rates with respect to a fixed window, such as 30 days, after admission (643).

Complications and other measures of patients' health status are less objectively measured than mortality. In some instances, a clearly identified procedure, such as a reoperation, indicates a poor outcome. Other measures, such as surgical wound infections, are less reliably coded across hospitals (see ch. 5).

One final measure of quality is even further removed from a direct measure of outcome. Luft and his colleagues use the proportion of patients that stay a very long time in the hospital as a proxy for complication rates (306,307,573). They argue that if one chooses a length of stay exceeded by only 10 percent of all patients, then a hospital with far more than 10 percent of its patients staying that long or longer may be experiencing poor outcomes. Although this argument is plausible, it has not been validated by determining whether those patients with very long hospital stays truly have complications, or stay longer, for example, because nursing home beds are scarce.

Differences in Patient Characteristics

A major problem in analysis of the volumeoutcome relationship is the potentially confounding effect of differences in patient characteristics. Every patient is different, and individual factors strongly influence outcomes. Even if these patient differences are random, the estimation of a volume-outcome relation will be made more difficult because of the "noise" due to these random effects. This point is illustrated in figure 8-2 which plots the inpatient mortality rates for patients undergoing coronary artery bypass graft (CABG)

surgery in 78 California hospitals in 1983 (574). Although there generally appear to be lower rates of poor outcomes at higher volumes (a negative linear relationship), there is substantial variation among hospitals at given volume levels-variation due in part to patient-related factors.

The crucial question is whether more or less severely ill patients are consistently admitted to high-volume hospitals. If they are, an observed association between outcome and volume could be due entirely to patient mix. The true answer to this question would be found by random assignment of large numbers of patients to institutions with varying volume levels. Random assignment, with sufficiently large numbers of patients, would reduce to insignificance the likelihood that patient-related factors account for the observed differences in outcomes. Unfortunately, since such an experiment would be enormously expensive and impossible because of ethical considerations, one is left with attempts to control for patients' differences by various statistical means.

There are two general approaches to dealing with differences in patient mix across hospitals. The first is to specify the procedure or diagnosis for study as carefully and narrowly as possible. The intent of this approach is to set patient selection criteria that result in a homogeneous group of patients. For example, patients undergoing CABG surgery who also have heart valve surgery have mortality rates about three times as high as those of patients undergoing CABG surgery only (573). Since some hospitals may specialize in uncomplicated CABG surgery while others have a large share of patients also requiring valve surgery, results may be biased unless one focuses on patients with CABG surgery only.

The second approach, which can be combined with the first, is to include variables in the analysis that may capture risk differences among the patients included in the study. In theory, each of these additional variables could be used to further stratify the study population of patients, but this approach is limited by an ever shrinking sample size. In many studies, therefore, patient selection criteria are combined with statistical controls. The patient's age, race, and sex are classic variables used in analyses. Transfer from another hos

Ratio of actual to expected mortality rates

Figure 8-2.-Ratio of Actual to Expected Mortality Rates by Volume of Patients Undergoing Coronary Artery Bypass Graft Surgery in California, 1983

6.0

5.5

5.0

4.5

4.0

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0

0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 9501000 Volume of patients by hospital

[blocks in formation]

SOURCE: JA Showstack, K.E. Rosenfeld, D.W. Garnick, et al, Institute for Health Policy Studies, University of California, unpublished data, San Francisco, 1987.

pital is often a powerful indicator of a patient at higher risk of a poor outcome (393). Counts of the number of secondary diagnoses or procedures or the presence of specific diagnoses or procedures also are used (394,573,582). In some instances, diagnostic information is combined to form a disease "stage" indicative of the severity of the principal diagnosis (346,347,348).

The problem of differences among patients has been highlighted in the literature on using mortality data to evaluate hospital performance (78, 189). To some extent, the problem is more severe if the focus is on studying individual hospitals rather than on studying the hypothesized relationship between volume and mortality. If a specific hospital is identified as having a significantly above average mortality rate, the hospital administration is likely to claim that unmeasured differences in patient mix account for the observed results. Upon careful examination of the medical

records, one may find that some patients entering the hospital with severe problems do account for an elevated mortality rate (see ch. 4). Precisely what clinical characteristics, if any, are similarly correlated with volume is not clear.

Research Findings

Statistical methods used in the volume-outcome studies listed in table 8-1 range from simple comparisons of high- and low-volume groups to fairly sophisticated causal models. Regression models were commonly used because they can include a large number of patient and/or hospital variables as explanatory factors. In some cases, logistic models were used to account explicitly for the 0,1 nature of patient mortality. Three papers used simultaneous equation models to estimate both the influence of volume on outcomes and the influence of outcomes on volume (307,393,397).

« PreviousContinue »