Page images
PDF
EPUB

and such a study is otherwise called for, the user should conduct the study of fairness.

C. Technical standards for content validity studies (1) Appropriateness of content validity studies. Users choosing to validate a selection procedure by a content validity strategy should determine whether it is appropriate to conduct such a study in the particular employment context. A selection procedure can be supported by a content validity strategy to the extent that it is a representative sample of the content of the job. Selection procedures which purport to measure knowledges, skills, or abilities may in certain circumstances be justified by content validity, although they may not be representative samples, if the knowledge, skill, or ability measured by the selection procedure can be operationally defined as provided in section 14C(4) below, and if that knowledge, skill, or ability is a necessary prerequisite to successful job performance.

A selection procedure based upon inferences about mental processes cannot be supported solely or primarily on the basis of content validity. Thus, a content strategy is not appropriate for demonstrating the validity of selection procedures which purport to measure traits or constructs, such as intelligence, aptitude, personality, commonsense, judgment, leadership, and spatial ability. Content validity is also not an appropriate strategy when the selection procedure involves knowledges, skills, or abilities which an employee will be expected to learn on the job.

(2) Job analysis for content validity. There should be a job analysis which includes an analysis of the important work behavior(s) required for successful performance and their relative importance and, if the behavior results in work product(s), an analysis of the work product(s). Any job analysis should focus on the work behavior(s) and the tasks associated with them. If work behavior(s) are not observable, the job analysis should identify and analyze those aspects of the behavior(s) that can be observed and the observed work products. The work behavior(s) selected for measurement should be critical work behavior(s) and/or important work behavior(s) constituting most of the job.

(3) Development of selection procedures. A selection procedure designed to measure the work behavior may be developed specifically from the job and job analysis in question, or may have been previously developed by the user, or by other users or by a test publisher.

(4) Standards for demonstrating content validity. To demonstrate the content validity of a selection procedure, a user should show that the behavior(s) demonstrated in the selection procedure are a representative sample of the behavior(s) of the job in question or that the selection procedure provides a representative sample of the work product of the job. In the case of a selection procedure

measuring a knowledge, skill, or ability, the knowledge, skill, or ability being measured should be operationally defined. In the case of a selection procedure measuring a knowledge, the knowledge being measured should be operationally defined as that body of learned information which is used in and is a necessary prerequisite for observable aspects of work behavior of the job. In the case of skills or abilities, the skill or ability being measured should be operationally defined in terms of observable aspects of work behavior of the job. For any selection procedure measuring a knowledge, skill, or ability the user should show that (a) the selection procedure measures and is a representative sample of that knowledge, skill, or ability; and (b) that knowledge, skill, or ability is used in and is a necessary prerequisite to performance of critical or important work behavior(s). In addition, to be content valid, a selection procedure measuring a skill or ability should either closely approximate an observable work behavior, or its product should closely approximate an observable work product. If a test purports to sample a work behavior or to provide a sample of a work product, the manner and setting of the selection procedure and its level and complexity should closely approximate the work situation. The closer the content and the context of the selection procedure are to work samples or work behaviors, the stronger is the basis for showing content validity. As the content of the selection procedure less resembles a work behavior, or the setting and manner of the administration of the selection procedure less resemble the work situation, or the result less resembles a work product, the less likely the selection procedure is to be content valid, and the greater the need for other evidence of validity.

(5) Reliability. The reliability of selection procedures justified on the basis of content validity should be a matter of concern to the user. Whenever it is feasible, appropriate statistical estimates should be made of the reliability of the selection procedure.

(6) Prior training or experience. A requirement for or evaluation of specific prior training or experience based on content validity, including a specification of level or amount of training or experience, should be justified on the basis of the relationship between the content of the training or experience and the content of the job for which the training or experience is to be required or evaluated. The critical consideration is the resemblance between the specific behaviors, products, knowledges, skills, or abilities in the experience or training and the specific behaviors, products, knowledges, skills, or abilities required on the job, whether or not there is close resemblance between the experience or training as a whole and the job as a whole.

(7) Content validity of training success. Where a measure of success in a training program is used as a selection procedure and the content of a training program is justified on the basis of content validity, the use should be justified on the relationship between the content of the training program and the content of the job.

(8) Operational use. A selection procedure which is supported on the basis of content validity may be used for a job if it represents a critical work behavior (1.e., a behavior which is necessary for performance of the job) or work behaviors which constitute most of the important parts of the job.

(9) Ranking based on content validity studies. If a user can show, by a job analysis or otherwise, that a higher score on a content valid selection procedure is likely to result in better job performance, the results may be used to rank persons who score above minimum levels. Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance.

D. Technical standards for construct validity studies—(1) Appropriateness of construct validity studies. Construct validity is a more complex strategy than either criterion-related or content validity. Construct validation is a relatively new and developing procedure in the employment field, and there is at present a lack of substantial literature extending the concept to employment practices. The user should be aware that the effort to obtain sufficient empirical support for construct validity is both an extensive and arduous effort involving a series of research studies, which include criterion related validity studies and which may include content validity studies. Users choosing to justify use of a selection procedure by this strategy should therefore take particular care to assure that the validity study meets the standards set forth below.

(2) Job analysis for construct validity studies. There should be a job analysis. This job analysis should show the work behavior(s) required for successful performance of the job, or the groups of jobs being studied, the critical or important work behavior(s) in the job or group of jobs being studied, and an identification of the construct(s) believed to underlie successful performance of these critical or important work behaviors in the job or jobs in question. Each construct should be named and defined, so as to distinguish it from other constructs. If a group of jobs is being studied the jobs should have in common one or more critical or important work behaviors at a comparable level of complexity.

(3) Relationship to the job. A selection procedure should then be identified or developed which measures the construct identified in

accord with paragraph (2) above. The user should show by empirical evidence that the selection procedure is validly related to the construct and that the construct is validly related to the performance of critical or important work behavior(s). The relationship between the construct as measured by the selection procedure and the related work behavior(s) should be supported by empirical evidence from one or more criterion-related studies involving the job or jobs in question which satisfy the provisions of section 14B above.

(4) Use of construct validity study without new criterion-related evidence (a) Standards for use. Until such time as professional literature provides more guidance on the use of construct validity in employment situations, the Federal agencies will accept a claim of construct validity without a criterion-related study which satisfies section 14B above only when the selection procedure has been used elsewhere in a situation in which a criterion-related study has been conducted and the use of a criterion-related validity study in this context meets the standards for transportability of criterion-related validity studies as set forth above in section 7. However, if a study pertains to a number of jobs having common critical or important work behaviors at a comparable level of complexity, and the evidence satisfies paragraphs 14B (2) and (3) above for those jobs with criterion-related validity evidence for those jobs, the selection procedure may be used for all the jobs to which the study pertains. If construct validity is to be generalized to other jobs or groups of jobs not in the group studied, the Federal enforcement agencies will expect at a minimum additional empirical research evidence meeting the standards of paragraphs section 14B (2) and (3) above for the additional jobs or groups of jobs.

(b) Determination of common work behaviors. In determining whether two or more jobs have one or more work behavior(s) in common, the user should compare the observed work behavior(s) in each of the jobs and should compare the observed work product(s) in each of the jobs. If neither the observed work behavior(s) in each of the jobs nor the observed work product(s) in each of the jobs are the same, the Federal enforcement agencies will presume that the work behavior(s) in each job are different. If the work behaviors are not observable, then evidence of similarity of work products and any other relevant research evidence will be considered in determining whether the work behavior(s) in the two jobs are the same.

DOCUMENTATION OF IMPACT AND VALIDITY EVIDENCE

SEC. 15. Documentation of impact and validity evidence- A. Required information. Users of selection procedures other than those

users complying with section 15A(1) below should maintain and have available for each job information on adverse impact of the selection process for that job and, where it is determined a selection process has an adverse impact, evidence of validity as set forth below.

(1) Simplified recordkeeping for users with less than 100 employees. In order to minimize recordkeeping burdens on employers who employ one hundred (100) or fewer employees, and other users not required to file EEO-1, et seq., reports, such users may satisfy the requirements of this section 15 if they maintain and have available records showing, for each year:

(a) The number of persons hired, promoted, and terminated for each job, by sex, and where appropriate by race and national origin;

(b) The number of applicants for hire and promotion by sex and where appropriate by race and national origin; and

(c) The selection procedures utilized (either standardized or not standardized).

These records should be maintained for each race or national origin group (see section 4 above) constituting more than two percent (2%) of the labor force in the relevant labor area. However, it is not necessary to maintain records by race and/or national origin (see section 4 above) if one race or national origin group in the relevant labor area constitutes more than ninetyeight percent (98%) of the labor force in the area. If the user has reason to believe that a selection procedure has an adverse impact, the user should maintain any available evidence of validity for that procedure (see sections 7A and 8).

(2) Information on impact-(a) Collection of information on impact. Users of selection procedures other than those complying with section 15A(1) above should maintain and have available for each job records or other information showing whether the total selection process for that job has an adverse impact on any of the groups for which records are called for by sections 4B above. Adverse impact determinations should be made at least annually for each such group which constitutes at least 2 percent of the labor force in the relevant labor area or 2 percent of the applicable workforce. Where a total selection process for a job has an adverse impact, the user should maintain and have available records or other information showing which components have an adverse impact. Where the total selection process for a job does not have an adverse impact, information need not be maintained for individual components except in circumstances set forth in subsection 15A(2)(b) below. If the determination of adverse impact is made using a procedure other than the "four-fifths rule," as defined in the first sentence of section 4D above, a justification, consistent

with section 4D above, for the procedure used to determine adverse impact should be available.

(b) When adverse impact has been eliminated in the total selection process. Whenever the total selection process for a particular job has had an adverse impact, as defined in section 4 above, in any year, but no longer has an adverse impact, the user should maintain and have available the information on individual components of the selection process required in the preceding paragraph for the period in which there was adverse impact. In addition, the user should continue to collect such information for at least two (2) years after the adverse impact has been eliminated.

(c) When data insufficient to determine impact. Where there has been an insufficient number of selections to determine whether there is an adverse impact of the total selection process for a particular job, the user should continue to collect, maintain and have available the information on individual components of the selection process required in section 15(A)(2)(a) above until the information is sufficient to determine that the overall selection process does not have an adverse impact as defined in section 4 above, or until the job has changed substantially.

(3) Documentation of validity evidence—(a) Types of evidence. Where a total selection process has an adverse impact (see section 4 above) the user should maintain and have available for each component of that process which has an adverse impact, one or more of the following types of documentation evidence:

(1) Documentation evidence showing criterion-related validity of the selection procedure (see section 15B, below).

(11) Documentation evidence showing content validity of the selection procedure (see section 15C, below).

(111) Documentation evidence showing construct validity of the selection procedure (see section 15D, below).

(iv) Documentation evidence from other studies showing validity of the selection procedure in the user's facility (see section 15E, below).

(v) Documentation evidence showing why a validity study cannot or need not be performed and why continued use of the procedure is consistent with Federal law.

(b) Form of report. This evidence should be compiled in a reasonably complete and organized manner to permit direct evaluation of the validity of the selection procedure. Previously written employer or consultant reports of validity, or reports describing validity studies completed before the issuance of these guidelines are acceptable if they are complete in regard to the documentation requirements contained in this section, or if they satisfied requirements of guidelines which were in effect when the validity study

179-106 0-98--3

was completed. If they are not complete, the required additional documentation should be appended. If necessary information is not available the report of the validity study may still be used as documentation, but its adequacy will be evaluated in terms of compliance with the requirements of these guidelines.

(c) Completeness. In the event that evidence of validity is reviewed by an enforcement agency, the validation reports completed after the effective date of these guidelines are expected to contain the information set forth below. Evidence denoted by use of the word "(Essential)" is considered critical. If information denoted essential is not included, the report will be considered incomplete unless the user affirmatively demonstrates either its unavailability due to circumstances beyond the user's control or special circumstances of the user's study which make the information irrelevant. Evidence not so denoted is desirable but its absence will not be a basis for considering a report incomplete. The user should maintain and have available the information called for under the heading "Source Data" in sections 15B(11) and 15D(11). While it is a necessary part of the study, it need not be submitted with the report. All statistical results should be organized and presented in tabular or graphic form to the extent feasible.

B. Criterion-related validity studies. Reports of criterion-related validity for a selection procedure should include the following information:

(1) User(s), location(s), and date(s) of study. Dates and location(s) of the job analysis or review of job information, the date(s) and location(s) of the administration of the selection procedures and collection of criterion data, and the time between collection of data on selection procedures and criterion measures should be provided (Essential). If the study was conducted at several locations, the address of each location, including city and State, should be shown.

(2) Problem and setting. An explicit definition of the purpose(s) of the study and the circumstances in which the study was conducted should be provided. A description of existing selection procedures and cutoff scores, if any, should be provided.

(3) Job analysis or review of job information. A description of the procedure used to analyze the job or group of jobs, or to review the job information should be provided (Essential). Where a review of job information results in criteria which may be used without a full job analysis (see section 14B(3)), the basis for the selection of these criteria should be reported (Essential). Where a job analysis is required a complete description of the work behavior(s) or work outcome(s), and measures of their criticality or importance should be provided (Essential). The report should describe the basis on which the

behavior(s) or outcome(s) were determined to be critical or important, such as the proportion of time spent on the respective behaviors, their level of difficulty, their frequency of performance, the consequences of error, or other appropriate factors (Essential). Where two or more jobs are grouped for a validity study, the information called for in this subsection should be provided for each of the jobs, and the justification for the grouping (see section 14B(1)) should be provided (Essential).

(4) Job titles and codes. It is desirable to provide the user's job title(s) for the job(s) in question and the corresponding job title(s) and code(s) from U.S. Employment Service's Dictionary of Occupational Titles.

(5) Criterion measures. The bases for the selection of the criterion measures should be provided, together with references to the evidence considered in making the selection of criterion measures (essential). A full description of all criteria on which data were collected and means by which they were observed, recorded, evaluated, and quantified, should be provided (essential). If rating techniques are used as criterion measures, the appraisal form(s) and instructions to the rater(s) should be included as part of the validation evidence, or should be explicitly described and available (essential). All steps taken to insure that criterion measures are free from factors which would unfairly alter the scores of members of any group should be described (essential).

(6) Sample description. A description of how the research sample was identified and selected should be included (essential). The race, sex, and ethnic composition of the sample, including those groups set forth in section 4A above, should be described (essential). This description should include the size of each subgroup (essential). A description of how the research sample compares with the relevant labor market or work force, the method by which the relevant labor market or work force was defined, and a discussion of the likely effects on validity of differences between the sample and the relevant labor market or work force, are also desirable. Descriptions of educational levels, length of service, and age are also desirable.

(7) Description of selection procedures. Any measure, combination of measures, or procedure studied should be completely and explicitly described or attached (essential). If commercially available selection procedures are studied, they should be described by title, form, and publisher (essential). Reports of reliability estimates and how they were established are desirable.

(8) Techniques and results. Methods used in analyzing data should be described (essential). Measures of central tendency (e.g., means) and measures of dispersion (e.g.,

standard deviations and ranges) for all selection procedures and all criteria should be reported for each race, sex, and ethnic group which constitutes a significant factor in the relevant labor market (essential). The magnitude and direction of all relationships between selection procedures and criterion measures investigated should be reported for each relevant race, sex, and ethnic group and for the total group (essential). Where groups are too small to obtain reliable evidence of the magnitude of the relationship, need not be reported separately. Statements regarding the statistical significance of results should be made (essential). Any statistical adjustments, such as for less then perfect reliability or for restriction of score range in the selection procedure or criterion should be described and explained; and uncorrected correlation coefficients should also be shown (essential). Where the statistical technique categorizes continuous data, such as biserial correlation and the phi coefficient, the categories and the bases on which they were determined should be described and explained (essential). Studies of test fairness should be included where called for by the requirements of section 14B(8) (essential). These studies should include the rationale by which a selection procedure was determined to be fair to the group(s) in question. Where test fairness or unfairness has been demonstrated on the basis of other studies, a bibliography of the relevant studies should be included (essential). If the bibliography includes unpublished studies, copies of these studies, or adequate abstracts or summaries, should be attached (essential). Where revisions have been made in a selection procedure to assure compatability between successful job performance and the probability of being selected, the studies underlying such revisions should be included (essential). All statistical results should be organized and presented by relevant race, sex, and ethnic group (essential).

(9) Alternative procedures investigated. The selection procedures investigated and available evidence of their impact should be identified (essential). The scope, method, and findings of the investigation, and the conclusions reached in light of the findings, should be fully described (essential).

(10) Uses and applications. The methods considered for use of the selection procedure (e.g., as a screening device with a cutoff score, for grouping or ranking, or combined with other procedures in a battery) and available evidence of their impact should be described (essential). This description should include the rationale for choosing the method for operational use, and the evidence of the validity and utility of the procedure as it is to be used (essential). The purpose for which the procedure is to be used (e.g., hiring, transfer, promotion) should be described (essential). If weights are assigned to dif

ferent parts of the selection procedure, these weights and the validity of the weighted composite should be reported (essential). If the selection procedure is used with a cutoff score, the user should describe the way in which normal expectations of proficiency within the work force were determined and the way in which the cutoff score was determined (essential).

(11) Source data. Each user should maintain records showing all pertinent information about individual sample members and raters where they are used, in studies involving the validation of selection procedures. These records should be made available upon request of a compliance agency. In the case of individual sample members these data should include scores on the selection procedure(s), scores on criterion measures, age, sex, race, or ethnic group status, and experience on the specific job on which the validation study was conducted, and may also include such things as education, training, and prior job experience, but should not include names and social security numbers. Records should be maintained which show the ratings given to each sample member by each rater.

(12) Contact person. The name, mailing address, and telephone number of the person who may be contacted for further information about the validity study should be provided (essential).

(13) Accuracy and completeness. The report should describe the steps taken to assure the accuracy and completeness of the collection, analysis, and report of data and results.

C. Content validity studies. Reports of content validity for a selection procedure should include the following information:

(1) User(s), location(s) and date(s) of study. Dates and location(s) of the job analysis should be shown (essential).

(2) Problem and setting. An explicit definition of the purpose(s) of the study and the circumstances in which the study was conducted should be provided. A description of existing selection procedures and cutoff scores, if any, should be provided.

(3) Job analysis-Content of the job. A description of the method used to analyze the job should be provided (essential). The work behavior(s), the associated tasks, and, if the behavior results in a work product, the work products should be completely described (essential). Measures of criticality and/or importance of the work behavior(s) and the method of determining these measures should be provided (essential). Where the job analysis also identified the knowledges, skills, and abilities used in work behavior(s), an operational definition for each knowledge in terms of a body of learned information and for each skill and ability in terms of observable behaviors and outcomes, and the relationship between each knowledge, skill, or ability and each work behavior, as well as

« PreviousContinue »