Are SBIR Programs Meeting Their Goals? on agreements, about 39 percent reported that the products or services resulting from the SBIR project were being sold commercially. In comparison, for 18 percent of these Phase II completions that had not received follow-on funding commitments, firms reported that they were selling the results of their SBIR project commercially. In its 1987 annual report to the Congress on SBIR programs,* SBA provided information on efforts by firms that had received seven or more Phase I SBIR awards to commercialize their SBIR projects. SBA made these observations in response to concerns that firms with large numbers of SBIR awards were not taking adequate steps to ensure the commercialization of the resulting projects. SBA determined that no particular problems existed with the management and commercialization of multiple awards. In SBA's opinion, companies with multiple awards were “just as committed, or more so, to the successful performance and commercialization of SBIR projects...." Firms that responded to our questionnaire concerning their SBIR projects SBA and agencies with SBIR programs seek to accomplish the program goal of fostering and encouraging participation by minority and disadvantaged small businesses through outreach efforts to inform them about SBIR programs. SBA defines a minority and disadvantaged small business concern as one that is at least 51 percent owned by one or more minority and disadvantaged individuals or, in the case of any publicly owned business, at least 51 percent of the voting stock of which is owned by one or more minority and disadvantaged individuals and *Fourth Year Results Under the Small Business Innovation Development Act of 1982, SBA (Washington, D.C.: June 1987), p. 11. Are SBIR Programs Meeting Their Goals? whose management and daily business operations are controlled by one or more of such individuals. A minority and disadvantaged individual is defined as a member of any According to SBA data, the percentage of money awarded minority and disadvantaged small businesses was lower in fiscal years 1986 and 1987 than in previous years. However, SBA officials believe that firms have little incentive to report their minority status correctly and that the data on minority firm participation in SBIR may contain some inaccuracies. The amount of SBIR money awarded to minority and disadvantaged firms SBA officials believe, however, that the minority award amounts Are SBIR Programs Meeting Their Goals? the SBIR proposal process, SBA officials believe a firm has little incentive to report its status accurately. SBA and agencies with SBIR programs have undertaken outreach efforts to encourage participation by minority and disadvantaged firms, often as part of general outreach efforts to inform small businesses about SBIR. These efforts have taken several forms: national conferences; regional seminars; and mailings to state agencies, historically minority universities and colleges, and individual firms. For example, in April 1987 DOD, NASA, and DOE held a joint 2-day workshop on the SBIR program that was sponsored by Virginia state government. In addition, a session for minority and disadvantaged firms was held in October 1987, as part of a conference in Atlanta attended by all SBIR agencies. The SBIR program has attracted some minority and disadvantaged firms that have not previously participated in federal contracting activities. About 26 percent of the projects by minority and disadvantaged firms identified in our questionnaire sample were performed by firms that had not had a contract or grant from the federal government prior to receiving their first SBIR award. SBA sponsored a study during 1985 to identify minority and disadvantaged firms capable of and interested in participating in the SBIR program. The study was completed in 1986 and the over 300 firms identified were entered in the SBIR mail list system and sent publications on the program. The study found that the number of firms that are primarily R&D-Oriented is small compared to the total number of minority and disadvantaged firms. The study also found that many minority and disadvantaged individuals who have the technical training and capability for participation in the program are employed in large corporations or in the government and are not interested in applying for the program. Quality of SBIR Research Projects Measuring Research Overall, 29 percent of the SBIR projects were judged to be of higher qual- In reauthorizing SBIR programs in 1986, the Congress asked us to report on how the quality of SBIR research projects compares with other research supported by each agency. To measure research quality, we sent questionnaires to project officers responsible for overseeing and monitoring SBIR and other research projects at the five agencies responsible for 96 percent of SBIR funds. We asked them to compare the quality of specific SBIR research projects with other research that they manage. We identified techniques that had been developed to assess research Chapter 2 discussed some ways in which agencies try to ensure the quality of their SBIR research projects. Agency project selection procedures, for example, seek to identify and fund SBIR proposals of high scientific and technical merit. In addition, agencies make some use of follow-on funding agreements as a way to identify proposals of high potential for commercial development. 1Citation analysis measures the number of times a scientific article is referred to in subsequent research articles and is intended to show how useful the research has been to other scientists. See Research Funding As an Investment: Can We Measure the Returns? Office of Technology Assessment (Washington, D.C.: April 1986). Quality of SBIR Research Projects Overall Assessment of We decided, on the basis of our own experience and the views of science policy experts we consulted, that the most feasible additional approach to measuring research quality was to enlist the judgments of technically knowledgeable persons who were familiar with the SBIR project but were not actually participating in the research. In addition to SBIR projects, agency project officers are normally responsible for other research activities. Therefore, we asked agency project officers to compare SBIR research with other research for which they were also responsible. SBIR research is a relatively small part of the responsibilities of most project officers. Almost 80 percent of the project officers responding to our questionnaire said that SBIR proposals and projects required no more than 10 percent of their time. Their remaining time was devoted to nonSBIR R&D proposals and projects and to other activities. To measure research quality, we asked project officers to compare spe- Overall, about half of the SBIR projects were judged to be of about the same quality as other research under the project officer's responsibility. As table 3.1 shows, 50 percent of the SBIR projects were rated as having about the same overall quality as other research, while 29 percent were regarded as somewhat or much better and 19 percent were regarded as somewhat or much worse. A similar rating pattern is found for most of the specific factors regarding research quality. For all but one of the factors, more projects were rated better than were rated worse than other projects. The one exception was the quality of scientific and technical facilities and resources, for which 14 percent of |