Rutgers Center of Alcohol Studies, 1993. pp. 357-377. Bliss, F.H.; Moos, R.H.; and Bromet, E.J. Monitoring change in community-oriented treatment programs. J Community Psychol 4:315-326, 1976.

Brown, H.P., and Peterson, J.H. Assessing spirituality in addiction treatment and follow-up: Development of the Brown-Peterson Recovery Process Inventory (B-PRPI). Alcoholism Treat Q 8:21-50, 1991.

Burling, T.A.; Reilly, P.M.; Moltzen, J.O.; and Ziff,

D.C. Self-efficacy and relapse among inpatient drug and alcohol abusers: A predictor of outcome. J Stud Alcohol 50:354-360, 1989. Cannon, D.S., and Baker, T.B. Emetic and electric shock alcohol aversion therapy: Assessment of conditioning. J Consult Clin Psychol 49:2033, 1981.

Carroll, S. Spirituality and purpose in life in alcoholism recovery. J Stud Alcohol 54:297-301, 1993.

Chaney, E.F.; O'Leary, M.R.; and Marlatt, G.A.

Skill training with alcoholics. J Consult Clin
Psychol 46:1092-1104, 1978.

Chevalley, C., and Urquhart, J. Medication event monitoring to project impact of patients' dosing errors on clinical trial results. Abstracts of the 8th Conference, International Society of Clinical Biostatistics, Gothenburg, Sweden, 1987.

Connors, G.J. "Measurement of AA Involvement in Alcoholism Treatment Research." Presentation at the Annual Meeting of the Association for the Advancement of Behavioral Therapy, San Diego, November 1994. Connors, G.J.; Tarbox, A.R.; and Faillace, L.A. Changes in alcohol expectancies and drinking behavior among treated problem drinkers. J Stud Alcohol 54:676-683, 1993. DiClemente, C.C., and Hughes, S.O. Stages of change profiles in outpatient alcoholism treatment. J Subst Abuse 2:217-235, 1990. DiClemente, C.C.; Carbonari, J.P.; Montgomery, R.P.G.; and Hughes, S.O. The Alcohol Abstinence Self-Efficacy Scale. J Stud Alcohol 55:141-148, 1994.

DiClemente, C.C.; Carroll, K.M.; Connors, G.J.,

and Kadden, R.M. Process assessment in treatment matching research. J Stud Alcohol [Suppl] 12:156-162:1994.

Elkins, R.L. Covert sensitization treatment of alcoholism: Contributions of successful conditioning to subsequent abstinence maintenance. Addict Behav 5:67-89, 1980. Ellsworth, R., and Maroney, R. Characteristics of psychiatric programs and their effects on patients' adjustment. J Consult Clin Psychol 39:436-44, 1972.

Etheridge, R.M.; Craddock, S.G.; Dunteman, G.H.; and Hubbard, R.L. Treatment services in two national studies of community-based drug abuse treatment programs. J Subst Abuse 7:926, 1995.

Fawcett, J.; Clark, D.C.; Aagesen, C.A.; Pisani, V.D.; Tilkin, J.M.; Sellers, D.; McGuire, M.; and Gibbons, R.D. Double-blind, placebo-controlled trial of lithium carbonate therapy for alcoholism. Arch Gen Psychiatry 44:248-256, 1987. Finney, J.W. Enhancing substance abuse treatment evaluations: Examining mediators and moderators of treatment effects. J Subst Abuse 7:135-150, 1995.

Finney, J.W., and Monahan, S.C. The cost-effectiveness of treatment for alcoholism: A second approximation. J Stud Alcohol, in press. Finney, J.W., and Moos, R.H. Environmental assessment and evaluation research: Examples from mental health and substance abuse programs. Evaluation Program Planning 7:151-167, 1984.

Fischer, J. The relationship between alcoholic patients' milieu perception and measures of their drinking during a brief follow-up period. Int J Addict 14:1151-1156, 1979. Fuller, R.K.; Branchey, L.; Brightwell, D.R.; Derman, R.M.; Emrick, C.D.; Iber, F.L.; James, K.; Lacoursiere, R.B.; Lee, K.K.; Lowenstam, I.; Maany, I.; Neiderhiser, D.; Nocks, J.J.; and Shaw, S. Disulfiram treatment of alcoholism: A Veterans Administration cooperative study. JAMA 256(11):1449-1455, 1986. Gerstein, D.R. Policy-relevant research on drug treatment. In: Cartwright, W.S., and Kaple, J.M., eds. Economic Costs, Cost-Effectiveness, Financing, and Community-Based Drug Treatment. National Institute on Drug Abuse, Research Monograph No. 113. Rockville, MD: NIDA, 1991.

Gilbert, F.S. The development of a "Steps Questionnaire." J Stud Alcohol 52:353-360, 1991. Glaser, F.B. Anybody got a match? Treatment research and the matching hypothesis. In: Edwards, G., and Grant, M., eds. Alcoholism Treatment in Transition. Baltimore: University Park Press, 1980. pp. 178-196.

Gorski, T.T. Questionnaire of Twelve Step Completion. Independence, MO: Herald House/Independence Press, 1990.

Harris, R.; Linn, M.; and Pratt, T. A comparison of dropouts and disciplinary discharges from a therapeutic community. Int J Addict 15:749756, 1980.

Hawkins, J.D.; Catalano, R.F.; and Wells, E.A. Mea

suring effects of a skills training intervention for drug abusers. J Consult Clin Psychol 54:661-664, 1986.

Heimberg, R.G.; Mueller, G.P.; Holt, C.S.; Hope,

D.A.; and Libowitz, M.R. Assessment of anxiety in social interaction and being observed by others: The Social Interaction Anxiety Scale and the Social Phobia Scale. Behav Ther 23:53-73, 1992.

Herrera, J.M., and Lawson, W.B. Effects of consultation on the ward atmosphere in a state psychiatric hospital. Psychol Rep 60:423-428, 1987.

Holder, H.; Longabaugh, R.; Miller, W.R.; and Rubonis, A.V. The cost effectiveness of treatment for alcoholism: A first approximation. J Stud Alcohol 52:517-540, 1991.

Horvath, A.O., and Luborsky, L. The role of the therapeutic alliance in psychotherapy. J Consult Clin Psychol 61:561-573, 1993. Hubbard, R.L.; Marsden, M.E.; Cavanaugh, J.V.; and Ginzburg, H.M. Drug Abuse Treatment: A National Study of Effectiveness. Chapel Hill, NC: University of North Carolina Press, 1989. Humphreys, K.; Noke, J.N.; and Moos, R.H. "The Diversity of Recovering Substance Abuse Staff Members' Professional Roles and Beliefs About Addiction." Paper presented at the Meeting of the Kettil Bruun Society of Social and Epidemiological Research on Alcohol, September 1994.

Institute of Medicine. Prevention and Treatment of Alcohol Problems: Research Opportunities. Washington, DC: National Academy Press, 1989. Ito, J.R.; Donovan, D.M.; and Hall, J.J. Relapse prevention in alcohol aftercare: Effects on drinking outcome, change process, and aftercare attendance. Br J Addict 83:171-181, 1988. Ito, J.R., and Donovan, D.M. Predicting drinking outcome: Demography, chronicity, coping, and aftercare. Addict Behav 15:553-559, 1990. Johnsen, J.; Stowell, A.; Bache Wiig, J.E.; Stensrud,

T.; Ripel, A.; and Morland, J. A double-blind placebo controlled study of healthy volunteers given a subcutaneous disulfiram implantation. Br J Addict 82:607-613, 1987. Jones, S.L., and Lanyon, R.I. Relationship

between adaptive skills and outcome of alcoholism treatment. J Stud Alcohol 42:521-525, 1981.

Jones, S.L.; Kanfer, R.; and Lanyon, R.I. Skill training with alcoholics: A clinical extension. Addict Behav 7:285-290, 1982.

Kadden, R.M.; Getter, H.; Cooney, N.L.; and Litt, M.D. Matching alcoholics to coping skills or interactional therapies: Posttreatment results. J Consult Clin Psychol 57:698-704, 1989. Kadden, R.M.; Litt, M.D.; Cooney, N.L.; and Busher, D.A. Relationship between role-play measures of coping skills and alcoholism treatment outcome. Addict Behav 17:425-437, 1992.

Litman, G.K.; Eiser, J.R.; and Rawson, N.S.B. Towards a typology of relapse: A preliminary report. Drug Alcohol Depend 2:157-162:1977. Litman, G.K.; Eiser, J.R.; Rawson, N.S.B.; and Oppenheim, A.N. Differences in relapse precipitants and coping behaviour between alcohol relapsers and survivors. Behav Res Ther 17:89-94, 1979.

Litman, G.K., and Stapleton, J. An instrument for measuring coping behaviors in hospitalized alcoholism: Implications for relapse prevention treatment. Br J Addict 78:269-276, 1983. Litman, G.K.; Stapleton, J.; Oppenheim, A.N.; Peleg, M.; and Jackson, P. The relationship between coping behaviours, their effectiveness and alcoholism relapse and survival. Br J Addict 79:283-291, 1984. Mackay, W.P.; Donovan, D.M.; and Marlatt, G.A. Cognitive and behavioral approaches to alcohol abuse. In: Frances, R., and Miller, S., eds. Clinical Textbook of Addictive Disorders. New York: Guilford, 1991. pp. 452-481.

Marlatt, G.A. Craving for alcohol, loss of control, and relapse: A cognitive behavioral analysis. In: Nathan, P.E.; Marlatt, G.A.; and Loberg, T., eds. Alcoholism: New Directions in Behavioral Research and Treatment. New York: Plenum, 1978.

Mayer, J.E., and Koeningsmark, C.P.S. Self-efficacy, relapse and the possibility of posttreatment denial as a stage in alcoholism. Alcohol Treat Q 8(4):1-17, 1992. McCaughrin, W.C., and Price, R.H. Effective out

patient drug misuse treatment organizations: Program features and selection effects. Int J Addict 27:1335-1358, 1992. McClintock, C. Evaluators as applied theorists. Evaluation Practice 11:1-12:1990.

McKay, J.R.; Alterman, A.I.; McLellan, A.T.; and Snider, E.C. Treatment goals, continuity of care, and outcome in a day hospital substance abuse rehabilitation program. Am J Psychiatry 151:254-259, 1994.

McKay, J.R.; Maisto, S.A.; and O'Farrell, T.J. Endof-treatment self-efficacy, aftercare, and drinking outcomes of alcoholic men. Alcoholism Clin Exp Res 17:1078-1083, 1993. McLellan, A.T.; Alterman, A.I.; Cacciola, J.; Metzger, D.; and O'Brien, C.P. A new measure of substance abuse treatment: Initial studies of the Treatment Services Review. J Nerv Ment Dis 180:101-110, 1992.

McLellan, A.T.; Arndt, I.O.; Metzger, D.S.; Woody, G.E.; and O'Brien, C.P. The effects of psychosocial services in substance abuse treatment. JAMA 269:1953-1959, 1993. McLellan, A.T.; Luborsky, L.; Cacciola, J.; Griffith, J.; Evans, F.; Barr, H.; and O'Brien, C.P. New data from the Addiction Severity Index: Reli

ability and validity in three centers. J Nerv Ment Dis 173:412-423, 1985. McLellan, A.T.; Grissom, G.R.; Brill, P.; Metzger,

D.S.; and O'Brien, C.P. Substance abuse treatment in the private setting: Are some programs more effective than others? J Subst Abuse Treat 10:243-254, 1993. McLellan, A.T.; Woody, G.E.; Luborsky, L.; and Goehl, L. Is the counselor an "active ingredient" in substance abuse rehabilitation? An examination of treatment success among four counselors. J Nerv Ment Dis 176:423-430, 1988. Miller, P.M. Behavioral Treatment of Alcoholism. New York: Pergamon, 1976.

Miller, W.R.; Taylor, C.A.; and West, J.C. Focused versus broad-spectrum behavior therapy for problem drinkers. J Consult Clin Psychol 48:590-601, 1980.

Moffett, L.A. Assessing the social system of a ther

apeutic community: Interpersonal orientations, social climate, and norms. Int J Ther Communities 5:110-119, 1984.

Moffett, L.A., and Flagg, C. Real and ideal social climate perceptions of residents, staff, and visitors in a therapeutic community for substance-dependent patients. Ther Communities 14:103-118, 1993.

Monti, P.M.; Abrams, D.B.; Binkoff, J.A.; Zwick, W.R.; Liepman, M.R.; Nirenberg, T.D.; and Rohsenow, D.J. Communication skills training, communication skills training with family and cognitive behavioral mood manageiment training for alcoholics. J Stud Alcohol 51:263-270, 1990.

Monti, P.M.; Rohsenow, D.J.; Abrams, D.B.; Zwick, W.R.; Binkoff, J.A.; Munroe, S.M.; Fingeret, A.L.; Nirenberg, T.D.; Liepman, M.R.; Pedraza, M.; Kadden, R.M.; and Cooney, N.L. Development of a behavior analytically derived alcohol-specific role-play assessment instrument. J Stud Alcohol 54:710-721, 1993.

Moos, R.H. Evaluating Treatment Environments: A Social Ecological Approach. New York: Wiley, 1974.

Moos, R.H. Group Environment Scale Manual. 2nd ed. Palo Alto, CA: Consulting Psychologists Press, 1986a.

Moos, R.H. Work Environment Scale Manual. 2nd ed. Palo Alto, CA: Consulting Psychologists Press, 1986b.

Moos, R.H. Community-Oriented Programs Envi

ronment Scale Manual. 2nd ed. Palo Alto, CA: Consulting Psychologists Press, 1988a. Moos, R.H. Assessing the program environment: Implications for program evaluation and design. In: Conrad, K.J., and Roberts-Gray, C., eds. Evaluating Program Environments. San Francisco: Jossey-Bass, 1988b.

Moos, R.H. Ward Atmosphere Scale Manual. 2nd ed.

Palo Alto, CA: Consulting Psychologists Press, 1989.

Moos, R.H., and Finney, J.W. The treatment setting in alcoholism program evaluations. Ann Behav Med 8:33-39, 1986.

Moos, R.H., and Finney, J.W. Alcoholism program evaluations: The treatment domain. J Drugs Soc 2:31-51, 1987/88.

Moos, R.H.; Finney, J.W.; and Cronkite, R.C. Alcoholism Treatment: Context, Process, and Outcome. New York: Oxford University Press, 1990. Moos, R.H.; Finney, J.; and Maude-Griffin, P. The social climate of self-help and mutual support groups: Assessing group implementation, process, and outcome. In: McCrady, B.S., and Miller, W.R., eds. Research on Alcoholics Anonymous: Opportunities and Alternatives. New Brunswick, NJ: Rutgers Center on Alcohol Studies, 1993. pp. 251-274.

Moos, R.H., and Lemke, S. Group Residences for Older Adults. New York: Oxford University Press, 1994.

Moos, R.H.; Pettit, E.; and Gruber, V. Characteristics and outcomes of three models of community residential care for substance abuse patients. J Subst Abuse 7:99-116, 1995. Morgenstern, J.; Frey, R.M.; McCrady, B.S.; Labouvie, E.; and Neighbors, C. Examining mediators of change in traditional chemical dependency treatment. J Stud Alcohol, in press. Moyers, T.B., and Miller, W.R. Therapists' conceptualizations of alcoholism: Measurement and implications for treatment decisions. Psychol Addict Behav 7:238-245, 1993.

Najavits, L.M., and Weiss, R.D. Variations in therapist effectiveness in the treatment of patients with substance use disorders: An empirical review. Addiction 89:679-688, 1994. Nirenberg, T.D., and Maisto, S.A. The relationship between assessment and alcohol treatment. Int J Addict 25:1275-1285, 1990.

Office of Applied Studies. National Drug and Alcoholism Treatment Unit Survey (NDATUS): 1991 Main Findings Report. Rockville, MD: Substance Abuse and Mental Health Services Administration, 1991.

Office of Applied Studies. Drug Services Research Survey. Final Report: Phase I. Rev. Rockville, MD: Substance Abuse and Mental Health Services Administration, 1992a.

Office of Applied Studies. Drug Services Research Survey. Final Report: Phase II. Rockville, MD: Substance Abuse and Mental Health Services Administration, 1992b.

Peterson, K.A.; Swindle, R.W.; Paradise, M.A.; and Moos, R.H. Substance Abuse Treatment Programming in the VA: Staffing, Patients, Policies, and Services. Palo Alto, CA: Program Evaluation and Resource Center, 1993.

Peterson, K.A.; Swindle, R.W.; Phibbs, C.S.; Recine, B.; and Moos, R.H. Determinants of readmission following inpatient substance abuse treatment: A national study of VA programs. Med Care 32:535-550, 1994a.

Peterson, K.A.; Swindle, R.W.; Paradise, M.A.; and Moos, R.H. Substance Abuse Treatment Programming in the VA: Program Structure and Treatment Processes. Palo Alto, CA: Program Evaluation and Resource Center, 1994b. Price, R.H., and D'Aunno, T.A. The organization and impact of outpatient drug abuse treatment services. In: Watson, R.R., ed. Drug and Alcohol Reviews. Vol. 3. Treatment of Drug and Alcohol Abuse. Clifton, NJ: Humana Press, 1992.

Price, R.H., and Moos, R.H. Toward a taxonomy of inpatient treatment environments. J Abnorm Psychol 84:181-188, 1975.

Prochaska, J.O.; DiClemente, C.C.; and Norcross, J.C. In search of how people change: Applications to addictive behaviors. Am Psychol 47:1102-1114, 1992.

Project MATCH Research Group. Project MATCH: Rationale and methods for a multisite clinical trial matching alcoholic patients to treatment. Alcoholism Clin Exp Res 17:1130-1145, 1993.

Rosenberg, H. Relapsed versus non-relapsed alco

hol abusers: Coping skills, life events, and social support. Addict Behav 8:183-186, 1983. Rounsaville, B.J.; O'Malley, S.; Foley, S.; and Weissman, M.M. Role of manual-guided training in the conduct and efficacy of interpersonal psychotherapy for depression. J Consult Clin Psychol 56:681-687, 1988.

Sanchez-Craig, M.; Spivak, K.; and Davila, R. Superior outcome of females over males after brief treatment for the reduction of heavy drinking: Replication and report of therapist effects. Drug Alcohol Depend 86:867-876, 1991. Simpson, D.D.; Dansereau, D.F.; and Joe, G.W.

The DATAR project: Cognitive and behavioral enhancements to community-based treatments. In: Fletcher, B.W.; Inciardi, J.A.; Tims, F.M.; Delaney, P.; and Horton, A.M., eds. The Effectiveness of Innovative Strategies in the Treatment of Drug Abuse. Westport, CT: Greenwood Press, in press.

Smith, D.E., and McCrady, B.S. Cognitive impairment among alcoholics: Impact on drink refusal skill acquisition and treatment outcome. Addict Behav 16:265-274, 1991. Snow, M.G.; Prochaska, J.O.; and Rossi, J. Processes of change in Alcoholics Anonymous: Maintenance factors in long-term sobriety. J Stud Alcohol 55:362-371, 1994.

Steiner, H.; Haldipur, C.; and Stack, L. The acute

admission ward as a therapeutic community. Am J Psychiatry 139:897-901, 1982. Swindle, R.W.; Peterson, K.A.; Paradise, M.J.; and Moos, R.H. Measuring substance abuse program treatment orientations: The Drug and Alcohol Program Treatment Inventory. J Subst Abuse 7:61-78, 1995.

Timko, C. The Quality of Psychiatric Treatment
Programs. Palo Alto, CA: Center for Health

Care Evaluation, Department of Veterans
Affairs Medical Center, 1994.

Timko, C. Policies and services in residential substance abuse programs: Comparisons with psychiatric programs. J Subst Abuse 7:43-59, 1995.

Tonigan, J.S.; Miller, W.R.; and Montgomery, H.A. "The Twelve-Step Participation Questionnaire." Unpublished instrument, University of New Mexico, Center on Alcoholism, Substance Abuse, and Addictions (CASAA), 1990. Twentyman, C.T.; Greenwald, D.P.; Greenwald, M.A.; Kloss, J.D.; Kovaleski, M.E.; and ZibungHoffman, P. An assessment of social skill deficits in alcoholics. Behav Assess 4:314-326, 1982.

Valle, S.K. Interpersonal functioning of alcoholism counselors and treatment outcome. J Stud Alcohol 42:783-790, 1981.

Waltz, J.; Addis, M.E.; Koerner, K.; and Jacobson, N.S. Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. J Consult Clin Psychol 61:620-630, 1993.

Wells, E.A.; Catalano, R.F.; Plotnick, R.; Hawkins, J.D.; and Brattesani, K.A. General versus drugspecific coping skills and posttreatment drug use among adults. Psychol Addict Behav 3:8-21, 1989.

Wells, E.A.; Peterson, P.L.; Gainey, R.R.; Hawkins, J.D.; and Catalano, R.F. Outpatient treatment for cocaine abuse: A controlled comparison of relapse prevention and twelve-step approaches. Am J Drug Alcohol Abuse 20:1-17, 1994.

Yahr, H.T. A national comparison of public and private-sector alcoholism treatment delivery system characteristics. J Stud Alcohol 49:233239, 1988.

Yeaton, W.H. The development and assessment of valid measures of service delivery to enhance inference in outcome-based research: Measuring attendance at self-help group meetings. J Consult Clin Psychol 62:686694, 1994.

Young, R.M.; Oei, T.P.S.; and Crook, G. Development of a drinking self-efficacy questionnaire. J Psychopathology Behav Assess 13:1-15, 1991.

This is not only an exciting time to be conducting alcoholism outcome evaluation but also a critical one. In many States, such evaluation is now legislatively mandated, with future appropriations tied to the demonstration of treatment program effectiveness. In parallel, the proliferation of statistical software for personal computers has dramatically increased the type and complexity of techniques available to the evaluator.

Although some concern has been raised about how the democratization of the tools of evaluation may precipitate their inappropriate use (e.g., Pedhazer 1982), indirect evidence suggests that increased accessibility has had a positive effect in the field of alcoholism evaluation. Miller et al. (1995a) found, for example, that the methodologic quality of research outcome studies has improved significantly in the past 20 years, much of this due to selection of assessment instruments with known psychometric properties and the appropriate use of multivariate techniques. Likewise, Alcoholics Anonymous researchers have become more sophisticated in their approaches to measurement of AA processes (e.g., Gilbert 1991) and in techniques to evaluate the predictors and outcomes of AA affiliation (Emrick et al. 1993; Tonigan et al., in press).

Chapter Overview

The purpose of this chapter is to sensitize the reader to a variety of fundamental issues

that arise in the conduct of outcome evaluation in alcoholism treatment. Discussion of the relative merits of specific self-report measures (Sobell and Sobell, this volume) and biological markers (Anton et al., this volume) of alcohol consumption are reviewed in detail elsewhere in the Guide and are not reiterated. The chapter begins with a general discussion of the importance of using assessment instruments with strong psychometric properties in outcome evaluation.

The second section reviews the unitary and multidimensional definitions of alcoholism and describes how these models identify relevant measures of change. Study findings are critiqued, and recommendations are made. This is followed by an overview of the purposes of outcome evaluation in the field of alcoholism treatment, highlighting some of the separate issues involved in individual and group-based evaluation. The next section briefly reviews the importance of collecting representative measures of pretreatment drinking, frequency of followup, and assessment order effects. The final section outlines some practical considerations in alcohol outcome evaluation, including interviewer role and training, instrument consistency, and data entry.

The Value of Reliable Measures

Much has been written on how, and under what conditions, reliability and valid