Page images
PDF
EPUB

Are SBIR Programs Meeting Their Goals?

[merged small][merged small][ocr errors]

on agreements, about 39 percent reported that the products or services resulting from the SBIR project were being sold commercially. In comparison, for 18 percent of these Phase II completions that had not received follow-on funding commitments, firms reported that they were selling the results of their SBIR project commercially.

In its 1987 annual report to the Congress on SBIR programs,* SBA provided information on efforts by firms that had received seven or more Phase I SBIR awards to commercialize their SBIR projects. SBA made these observations in response to concerns that firms with large numbers of SBIR awards were not taking adequate steps to ensure the commercialization of the resulting projects. SBA determined that no particular problems existed with the management and commercialization of multiple awards. In SBA's opinion, companies with multiple awards were “just as committed, or more so, to the successful performance and commercialization of SBIR projects...."

Firms that responded to our questionnaire concerning their SBIR projects
indicated that the number of SBIR awards received makes little differ-
ence in the rate of commercialization. We examined the data reported by
firms that had received 11 or more Phase I awards. Of the projects that
had completed Phase II, 25 percent had been performed by firms with
11 or more Phase I awards. For both groups of firms, about 25 percent
of the completed projects resulted in products or services that were
being sold commercially.

SBA and agencies with SBIR programs seek to accomplish the program goal of fostering and encouraging participation by minority and disadvantaged small businesses through outreach efforts to inform them about SBIR programs.

SBA defines a minority and disadvantaged small business concern as one

that is at least 51 percent owned by one or more minority and disadvantaged individuals or, in the case of any publicly owned business, at least 51 percent of the voting stock of which is owned by one or more minority and disadvantaged individuals and

*Fourth Year Results Under the Small Business Innovation Development Act of 1982, SBA (Washington, D.C.: June 1987), p. 11.

Are SBIR Programs Meeting Their Goals?

[merged small][ocr errors]

whose management and daily business operations are controlled by one or more of such individuals.

A minority and disadvantaged individual is defined as a member of any
of the following groups: Black Americans, Hispanic Americans, Native
Americans, Asian-Pacific Americans, and Subcontinent Asian
Americans.

According to SBA data, the percentage of money awarded minority and disadvantaged small businesses was lower in fiscal years 1986 and 1987 than in previous years. However, SBA officials believe that firms have little incentive to report their minority status correctly and that the data on minority firm participation in SBIR may contain some inaccuracies.

The amount of SBIR money awarded to minority and disadvantaged firms
increased each year from fiscal years 1984 through 1987. (See table
2.7.) When compared with total money awarded to small business,
minority and disadvantaged firms received about 12 percent in 1984
and 1985 and about 8.5 percent in 1986 and 1987. The percentage of
Phase I SBIR awards received by minority and disadvantaged firms
remained about the same for fiscal years 1985 to 1987, but the percent-
age of Phase II awards received by these firms was lower in fiscal years
1986 and 1987 than it was in 1985.

[blocks in formation]

SBA officials believe, however, that the minority award amounts
reported may not be accurate. Firms report minority and disadvantaged
status voluntarily on their proposals, and SBA has identified cases in
which individual firms have been inconsistent, identifying themselves as
minority and disadvantaged on some proposals but not on others.
Because minority and disadvantaged firms do not receive preference in

Are SBIR Programs Meeting Their Goals?

the SBIR proposal process, SBA officials believe a firm has little incentive to report its status accurately.

SBA and agencies with SBIR programs have undertaken outreach efforts to encourage participation by minority and disadvantaged firms, often as part of general outreach efforts to inform small businesses about SBIR. These efforts have taken several forms: national conferences; regional seminars; and mailings to state agencies, historically minority universities and colleges, and individual firms. For example, in April 1987 DOD, NASA, and DOE held a joint 2-day workshop on the SBIR program that was sponsored by Virginia state government. In addition, a session for minority and disadvantaged firms was held in October 1987, as part of a conference in Atlanta attended by all SBIR agencies.

The SBIR program has attracted some minority and disadvantaged firms that have not previously participated in federal contracting activities. About 26 percent of the projects by minority and disadvantaged firms identified in our questionnaire sample were performed by firms that had not had a contract or grant from the federal government prior to receiving their first SBIR award.

SBA sponsored a study during 1985 to identify minority and disadvantaged firms capable of and interested in participating in the SBIR program. The study was completed in 1986 and the over 300 firms identified were entered in the SBIR mail list system and sent publications on the program. The study found that the number of firms that are primarily R&D-Oriented is small compared to the total number of minority and disadvantaged firms. The study also found that many minority and disadvantaged individuals who have the technical training and capability for participation in the program are employed in large corporations or in the government and are not interested in applying for the program.

Quality of SBIR Research Projects

Measuring Research
Quality

Overall, 29 percent of the SBIR projects were judged to be of higher qual-
ity than other agency research, and 50 percent were judged as of similar
quality. However, project officers judged SBIR projects differently on
some factors important to research quality, and officers differed among
agencies in how SBIR projects were rated. For example, project officers at
all agencies rated SBIR projects higher than other agency research con-
cerning the likelihood that the project will lead to inventing and com-
mercializing new products, processes, or services. Agency project
officers differed on other factors, however, such as the likelihood that
the project will lead to new scientific and technical discoveries. Many of
the important differences among agencies paralleled the differing
emphasis on SBIR program objectives that was described in chapter 2.

In reauthorizing SBIR programs in 1986, the Congress asked us to report on how the quality of SBIR research projects compares with other research supported by each agency. To measure research quality, we sent questionnaires to project officers responsible for overseeing and monitoring SBIR and other research projects at the five agencies responsible for 96 percent of SBIR funds. We asked them to compare the quality of specific SBIR research projects with other research that they manage.

We identified techniques that had been developed to assess research
quality but determined that they were not appropriate to our needs.
According to the Office of Technology Assessment, the only quantitative
measure of research quality is by analyzing research publications
through techniques such as citation analysis.1 Because SBIR projects
involve applied research and do not usually produce scientific articles,
this way of measuring research quality was not appropriate to our
needs.

Chapter 2 discussed some ways in which agencies try to ensure the quality of their SBIR research projects. Agency project selection procedures, for example, seek to identify and fund SBIR proposals of high scientific and technical merit. In addition, agencies make some use of follow-on funding agreements as a way to identify proposals of high potential for commercial development.

1Citation analysis measures the number of times a scientific article is referred to in subsequent research articles and is intended to show how useful the research has been to other scientists. See Research Funding As an Investment: Can We Measure the Returns? Office of Technology Assessment (Washington, D.C.: April 1986).

Quality of SBIR Research Projects

Overall Assessment of
Research Quality

We decided, on the basis of our own experience and the views of science policy experts we consulted, that the most feasible additional approach to measuring research quality was to enlist the judgments of technically knowledgeable persons who were familiar with the SBIR project but were not actually participating in the research. In addition to SBIR projects, agency project officers are normally responsible for other research activities. Therefore, we asked agency project officers to compare SBIR research with other research for which they were also responsible.

SBIR research is a relatively small part of the responsibilities of most project officers. Almost 80 percent of the project officers responding to our questionnaire said that SBIR proposals and projects required no more than 10 percent of their time. Their remaining time was devoted to nonSBIR R&D proposals and projects and to other activities.

To measure research quality, we asked project officers to compare spe-
cific SBIR projects with other research projects that they were responsi-
ble for, according to nine factors that we had identified as potentially
relevant to research quality (by consulting science policy experts,
reviewing published material, and pretesting questionnaires), and to
assess overall project quality. These factors, which are listed in table
3.1, included, among others, the likelihood that the project would lead to
new scientific/technical discoveries or to inventing and commercializing
new products, processes, and services. In order to focus on projects that
had been going on long enough to produce results, we sent question-
naires to 530 project officers concerning the 739 projects begun during
1983 and 1984 that had been later selected for Phase II awards. Appen-
dixes III, IV, and V contain additional information on our questionnaires
and the project officers' responses.

Overall, about half of the SBIR projects were judged to be of about the same quality as other research under the project officer's responsibility. As table 3.1 shows, 50 percent of the SBIR projects were rated as having about the same overall quality as other research, while 29 percent were regarded as somewhat or much better and 19 percent were regarded as somewhat or much worse. A similar rating pattern is found for most of the specific factors regarding research quality.

For all but one of the factors, more projects were rated better than were rated worse than other projects. The one exception was the quality of scientific and technical facilities and resources, for which 14 percent of

« PreviousContinue »