have to be of such a nature that activities and experiments would be performed by the students themselves. It would further have to have an evaluation design so that we could determine the results of the program. It would promote effective solutions to be carried out in socially acceptable ways. This problem, again, was addressed to the Title III Office. Again, they found the quality of the application of sufficient merit to fund this program. You have before you a replica of the finished product.

Teachers in our own school system were able to develop a program of such high quality that dissemination to all seventeen thousand districts will be possible through a major publishing company. Because the program was developed at the local level, cost of dissemination has been held to a modest price. Knowing the limited resources of school systems, economy was a major factor; thus no component within a kit is consumed in use. Each kit is prepared for one week's instruction. Therefore, the initial cost of the kit is prorated, not over thirty children, but over thirty children times the thirty-six week school year. Therefore, over 1,000 students may be exposed to each unit in a given y ear. The kit material will also last for years; the point being that this is a most economical package and one which already, in pilot form, has been proven most effective. It will be used in various districts throughout the nation during the spring of this year and by thousands of youngsters in many districts during 1973–74.

A third problem confronted us and confronts many districts. Children were entering school with certain socialization problems. Hyperactivity and distractability were diminishing the effects of instruction. Attention span was short. So many stimuli bombard students today in the community, in the home, and in their total environment that a direct response through a program specifically geared to meet these challenges was demanded. This program, which is now in its second year, begins in kindergarten and directs the student through specific visual activities to improve attention and concentration, aptitudes which can be translated to any academic discipline. The materials for this program are very attractive as you see them displayed before you. Again, all of the developmental expertise existed within the school system. Costs for dissemination are kept at a minimum through production efforts which are executed by the school system or under its direct control.

All three of these efforts were deemed necessary and of high priority. None of them could have been responded to successfully without the assistance of the Title III Program. The financial resources were simply not available at the local level. Assistance was needed in program design, evaluation, and dissemination vehicles in order to see to it that these products were shared with other districts. Title III has permitted us to respond to three specific critical needs. The response has been the development of total programs. The response has also been to provide a delivery system with total evaluation and a dissemination vehicle which now permits us to provide these materials at absolute minimal cost to any one or more of the 17,000 districts in the United States.

An evaluation design was a mandatory part of each of these projects and thus quality control was maintained throughout the project period through field testing and statistical evaluation. Since each of these programs was developed at the local level, control was maintained in terms of student interest and teaching facility. Each program was developed by classroom teachers and other school personnel. The programs each had a three year time span and therefore efficiency had to be maintained in order to complete the program by the termination date. This has been done.

In closing, may I merely state again that the resources provided by Title III have allowed Union to respond to three very real needs which not only existed in our township, but which are common to many or most districts. These programs could not have been developed without Title III funds and Title III staff assistance.

STATEMENT OF DR. EVELYN OGDEN, DIRECTOR OF PROGRAM MANAGEMENT (TITLE III COORDINATOR) Based on DATA ABSTRACTED FROM AN INDEPENDENT EVALUATION OF THE TITLE III PROGRAM IN THE STATE OF NEW JERSEY

DATA CONCERNING THE EFFECTIVENESS OF TITLE III: THE PRODUCT AND THE PROCESS

Title III is a program addressed directly to the problems of education. It is not a basic research program, but rather utilizes the results of research in developing operational solutions to identified problems. Planned solutions are field tested, evaluated and analyzed.

Title III is also a demonstration program. It presumes that if solutions that work are found they will be adopted locally, as well as by other districts with similar problems.

HOW SUCCESSFUL HAS TITLE III BEEN? WHY HAS IT SUCCEEDED OR FAILED? First it is necessary to explain what we mean by success and failure. Projects are funded based on proposed solutions aimed at attaining specific objectives. These objectives must include projections directly related to student learning. A successful project is one which accomplishes its objectives. An unsuccessful project is one which does not attain its objectives.

WHAT ARE THE INDICATORS OF SUCCESS OR FAILURE BASED ON THE NEW JERSEY EXPERIENCE?

Every project is evaluated based on its objectives, each year, by trained, independent evaluation teams. These evaluation reports indicate the following: 63% have evidence that they are meeting all their objectives; 29% have evidence of meeting most objectives; and 8% are meeting few or more of their objectives. Percentages do not adequately reflect the human impact of the meaning of success in these projects. Let me give you a few examples of what they mean educationally for children:

In an inner city disadvantaged school, children whose tested mean IQ when they entered school was 80, now have a tested mean IQ of 100.5. Their academic profile now does not differ significantly from that of the advantaged suburban control group or from national norms. These achievements in learning have been maintained for almost three years.

Special education children, in another district, once isolated from their peers, have been returned to regular classes for part or all of the day. As a result special education students learn more and feel better about themselves. Regular classroom teachers, special education teachers and parents support the success of the program. Finally, the program once developed resulted in a decrease of in the cost of educating these children. This project extended to other districts throughout the state could result in the savings of millions of education dollars. Most projects are multi-dimensional, and address more than the improvement of the basic skills. For example, a humanities program has been able to document that creativity in children participating has been enhanced.

The stories of all the 74 active Title III projects are documented in the State's Office of Program Development.

Another important indicator of success is the adoption of the new practices by the district after Title III funding ceases. Under Title III, project funding is usually based on a three-year projection. An analysis of the 53 projects which have completed a full cycle since 1970 yields the following data:

The critical year for adoption by the project district appears to be the first year after federal funding ceases. All projects adopted at that time are being continued at the local level. Have all projects succeeded? The answer is no. Nine projects have been terminated prior to the completion of their planned development period, for failure to meet project objectives. Some projects, in the past, have been unable to document either the success or failure of their ideas.

WHAT LOCAL FACTORS AFFECT PROJECT SUCCESS OF FAILURE?

The complimentary but differentiated roles played by Federal, State and Local agencies account, in large part, for the success of development in New Jersey. Development of an idea into a solution which significantly improves education is a complex process. It is naive to assume that the mere dispersion of money to individuals with ideas will result, with any consistency, in successful programs. Individuals do not exist or implement programs in a vacuum. Solutions, school districts and communities are complex. On the other hand, individuals, at the local district level have not had a history of systematically applying known research findings to the solution of their problems, nor a history of bringing about substantive change. This trend is in the process of being reversed as a result of involvement in Title III processes. The local educator is closest to the problems of education and should more adequately identify the need for change. Creativity in problem solving is plentiful at the local district level. Field testing new ideas in a local school setting also improves the chances for replication in other districts. We have evolved a process over the past few years which relies on federal, state ,and local cooperation aimed at maximizing success in Title III. What then is the Title III process that works?

This process is comprehensive in that it covers all phases of development and diffusion. It begins with the technical assistance provided by the State to people wanting to apply for Title III funding and ends with evaluation of programs adopted from Title III districts by non-Title III districts.

The process implemented by the State Development team, is an application of the systems approach to education change. It requires that problems be identified in measurable terms and that program objectives be stated in parallel terms. It requires that research related to the problem be analyzed, that alternative solutions be considered. It requires that the solution be presented in detail. including consideration of the impact of the solution on all people concerned. It requires a detailed evaluation design. It requires job expectation sheets, budget details, and contract specifications. Finally, it requires that the activities be presented visually in sequential order (through PERT), so that projects can be effectively monitored and carried out.

The process includes the development of a close working relationship between the project staff locally and the State Department of Education. A great deal of technical assistance is given to local educators as they develop and implement their plans. This partnership improves chances of local success as well as focusing on the broader objective of diffusion of solutions.

Monitoring of programs is continuous with each project visited by a project expeditor approximately five times a year. State Department staff do not conduct the on-site evaluations of Title III projects, but they arrange for the evaluations to be conducted by independent teams.

The U.S. Office plays a significant role in the process by setting standards for implementing the legislation and holding states accountable for their processes through a State Plan. In addition, the U.S. Office of Education has established a standard set of criteria for judging the success of projects. Finally, the U.S. Office of Education has the role of disseminating successful practices across state lines.

WHAT TOTAL FACTORS AFFECT PROJECT SUCCESS OR FAILURE?

Our experience has shown that a major factor in determining the success or failure of a project is the quality of the systematic comprehensive plan upon which it is based. The process outlined above aims at assuring the quality of that preliminary planning.

Our experience has also shown that systematic continuous contact with projects by the State Department during implementaion and high standards of accountability are essential if success is to be maximized. The development cost of the project is only moderately related to success. Sucessful projects in N.J. range from a low of $7500 to a high of $300,000. However, the cost of maintaining and adopting the new practices is directly related to the adoption rate.

The size, location or population served does not show a significant correlation. In N.J., small rural districts, inner city districts, as well as suburban districts have all produced nationally validated projects.

DATA CONCERNING THE ADOPTION AND DISSEMINATION OF TITLE III:

THE PRODUCT AND THE PROCESS

Developing successful educational practices in the funded district is important. However, an even broader goal of Title III is the dissemination of successful practices.

HOW SUCCESSFUL HAS TITLE III BEEN IN SPREADING SUCCESSFUL PRACTICES?

We can document that practices developed under Title III have been adopted by 185 other districts in the State. In addition, practices developed in New Jersey under Title III are currently being used in districts in 30 other States, 250 districts in New Jersey have adopted practices developed under Title III in other States. These figures represent only those adoptions we have documented. It is our opinion that many more adoptions have taken place than we have been able to document.

What are the factors related to the spread of new practices in education?

Systematic dissemination of educational practices is relatively new. In the early years of ESEA Title III the prevailing assumption among educators seemed to be, "if a program works, other people will adopt it." Projects were encouraged to disseminate material almost as soon as they were funded. This was also true of programs developed by foundations and other federal and State agencies. The result was a huge quantity of descriptive materials and generalized claims of effectiveness. Education does not need more practices; it needs practices that work.

Before a systematic process for diffusion of successful ideas could be implemented, procedures had to be developed for documenting success or failure. Therefore, early emphasis in Title III was on evaluation.

When we learned how to evaluate program effectiveness, we discovered an error in a basic assumption. We had assumed programs which work would be adopted by others. What we found, in New Jersey anyway, was that unsuccessful projects were just as likely to be adopted. We also found that the cost of a practice, the materials available, the personnel needed to implement a procedure and many other variables were related to adoption rates of educational practices.

Two years ago New Jersey instituted a validation procedure. In order to be validated, a project had to have confirmed evidence that it made a significant difference in student learning. It had to have materials available that another district would need to adopt the program. Finally, it had to have reasonable documented adoption costs. Validated projects were endorsed by the New Jersey Commissioner of Education as "Programs That Work". Only projects meeting validation standards were funded for dissemination.

Other States and the U.S. Office of Education were also interested in validating projects as a basis for dissemination. We now have a National Validation procedure. This procedure establishes precise criteria for judging projects in terms of effectiveness, marketability, cost-effectiveness and innovativeness. In the future, potential consumers will know precisely what is meant when a project is a validated success.

We have learned that dissemination of valid programs does not happen automatically. If Title III has had a major flaw, it has been its failure until recently to apply a systems approach to dissemination.

In recent years several successful approaches to dissemination have been found. For example: The "producer-consumer" model is being used to disseminate nine Nationally Validated New Jersey projects. Under this model, producer districts receive funds for dissemination based on specific dissemination objectives. The staff of the producer district provide technical assistance, training and materials to other districts wanting to adopt the program.

Still other procedures are being tested for systematically disseminating projects that are successful.

Improved national dissemination of educational successes is needed. National Validation will yield hundreds of validated projects. States cannot assume the responsibility for systematically disseminating programs across State lines. Failure to effectively disseminate Nationally has in the past and will perpetuate in the future the dilemma and costlines of "reinvention of the wheel" in State after State.

SUMMARY

Title III is a program which can document its effectiveness. Title III has evolved a sophisticated effective process for systematically developing the new products and processes needed in education. Dissemination processes are approaching this level of development. It is on the threshold of reaching its full potential, with the investments of time, effort and money up to this date beginning to bring in a return to the National "educational" treasury.

In closing, I would like to make some recommendations to the committee concerning continuation of ESEA Title III legislation:

1. That the intent of the ESEA Title III be preserved and protected in any new legislation, namely, to improve education through development and dissemination of programs based on research which address common problems in education.

2. That the Federal, State and Local cooperative relationship be retained.

3. That legislation provide for additional funding for dissemination of Nationally Validated projects. Such funding might be in the form of a formula based on the developmental costs of the project.

NOTE. Data utilized in this report was abstracted from an independent evaluation of Title III in New Jersey conducted by Education and Public Affairs, Washington, D.C. STATEMENT BY ROBERT W. WARD, STATE EDUCATION DIRECTOR, Office of Program DEVELOPMENT, NEW JERSEY STATE DEPARTMENT OF EDUCATION

In conclusion, we have presented our case for development: When you, the Congress, decide, we hope that you agree with us that Title III is an imperative. We hope that you agree that education—a multi-billion dollar industry-must have development capability.

We hope that you agree that the New Jersey story of Title III has hard data which says that when you provide capability, money, and a systematic approach to change that includes in the process all the people and institutions to be affected by the change, you get positive results and that you have the evidence.

We hope that you agree that Title III has produced valid products that offer solutions that can be trusted. For the first time, education may have its own Pure Food and Drug Act.

We hope that you agree that Title III has demonstrated you can influence how the big money-the operation money-is spent.

For example: In our State, the development capability has been applied beyond Title III. Development assistance has been provided for: Bilingual Education; Title I programs (Dale Avenue Title III Project); and Environmental programs. The result has been that monies appropriated in these categories are better spent. As you know, people are saying the millions for compensatory education have been wasted, that compensatory education does not work, and yet you know that thousands of children need programs that compensate for their disadvantaged state. It is, therefore, clear that there is nothing wrong with the compensatory education concept. What is wrong is that we need more programs that workprograms that have demonstrated a measurable result. Yes, programs that do what they say they will do. Programs that Title III can produce.

Finally, we hope you agree that with the limited money available to education, we just cannot continue to spend money on untested, poorly designed curriculum. We must have a Bell Labs and a Western Electric component in education if we are to meet our commitment of insuring that each individual will be provided the opportunity to achieve his full educational potential.

Yes, ladies and gentlemen, the Title III community of the nation stands ready to provide the evidence to show that planning and development is an educational imperative and that the Congress will continue the work it began in 1965, when in its wisdom, it passed the Elementary and Secondary School Act.

The question is not should we continue Title III. The question should be how many more development dollars are needed to insure the best use of the operational funds presently available. How do we get better results from the billions we are spending already? We think that we have made a case for development. We hope you agree.

TITLE III PROJECTS AND MATERIALS DISSEMINATED TO SCHOOL DISTRICTS IN NEW

JERSEY

"Central Iowa Math Project for Low Achievers"

Des Moines, Iowa

250 Programs in use.