Are SBIR Programs Meeting Their Goals?

Using SBIR Programs to Meet Federal R&D Needs

Agencies with large R&D programs have different needs because of different mission responsibilities and different ways of managing and overseeing research. These differences are reflected in the solicitation of SBIR proposals, the ranking and selecting of such proposals for funding, and the management of the SBIR projects. Despite these differences, agency and department heads generally indicated that their SBIR programs were helping to meet R&D needs. About three quarters of the project officers also responded that SBIR programs probably or definitely helped meet agency R&D needs. In addition, project officers said that through SBIR programs, agencies support many projects that they would not otherwise sponsor. In their opinion, about half of the projects probably or definitely would not have been funded if the agency did not have an SBIR program.

Differences in Agency
R&D Needs

DOD and NASA conduct a high proportion of applied research and devel-
opment to meet specific defense, aeronautic, and space technology needs
in addition to some basic research. Much of their applied research and
development is performed through contracts with private industry,
under the supervision of agency managers. On the other hand, NSF and
HHS fund a much higher proportion of basic research through grants to
universities than do DOD and NASA. Such basic research is performed
with little supervision by NSF or HHS officials. DOE, like DOD and NASA,
supports applied research but, like NSF and HHS, also supports basic
research, particularly in the field of high energy and nuclear physics.

NASA and DOD conduct their SBIR programs primarily to meet specific objectives as an integral part of agency R&D programs. In contrast, SBIR projects at NSF and HHS differ from other research at these agencies in that they have an applied research focus and emphasize private sector commercialization. NSF and HHS solicit proposals within broad technological areas and emphasize the selection of proposals with high potential for private sector commercialization. As a result, SBIR programs at these agencies are less coordinated with other agency research, which tends to be fundamental in nature and does not emphasize commercialization. At DOE, SBIR projects in some areas, such as magnetic fusion and basic energy research, are geared toward specific agency R&D objectives, while those in other areas, such as energy conservation and fossil fuel, focus on private sector commercialization.

Are SBIR Programs Meeting Their Goals?

Agencies Differ in
Management of SBIR
Programs

Table 2.4: Responses Concerning
Frequency of Monitoring SBIR Projects

The difference in how agencies seek to meet R&D needs is reflected in how they solicit, select, and manage SBIR proposals. For instance, DOD'S annual SBIR solicitation identifies specific tasks in hundreds of different technical areas, such as the design of body armor, self-sealing truck radiators, and underground chemical storage technology. In contrast, NSF's annual solicitation simply lists about 20 general scientific areas, such as materials research and advanced scientific computing, with a few examples of potential projects from each, and encourages any proposals that fall under these general headings. The National Institutes of Health, which manage almost all HHS research, have a policy of considering any proposal in the health area, whether or not it is responsive to a research area specified in its solicitation.

In addition to differing in the solicitation of proposals, agencies also differ in how they rank SBIR proposals for funding. DOD and NASA follow a decentralized approach in which research managers throughout the agency rank proposals for funding. NSF and HHS use a more centralized approach that relies upon experts from outside the agency to rank projects. At DOE, SBIR proposal reviews are carried out by experts from both inside and outside the agency.

Agencies also differ in their management of SBIR projects. As table 2.4 shows, project officers at DOD and NASA are much more likely to stay in close touch with SBIR awardees over the course of the research project than those in NSF and HHS. DOD and NASA SBIR program managers told us that their agency project officers normally stay in close contact with SBIR and other research contractors to monitor mission-related applied research. In contrast, NSF and HHS project officers normally have less contact with grant recipients because there is no direct agency oversight of research, according to SBIR managers at these agencies.

Are SBIR Programs Meeting Their Goals?

Attitudes of Agency
Officials

Table 2.5: Responses on Whether the
SBIR Programs Help Meet Agency R&D
Needs

In their written responses to us concerning SBIR R&D, the heads of 11
agencies and departments provided information on how SBIR programs
helped meet their agency R&D needs. Our questionnaire to project
officers also asked whether SBIR programs helped meet agency R&D needs
and what contribution individual SBIR projects had made in meeting R&D
needs.

The 11 agency and department heads generally replied that their SBIR programs were helping to meet R&D needs. (See ch. 4.) Their responses differed, however, in the specific contributions reported for SBIR programs. DOD and NASA, for example, emphasized how SBIR projects helped fulfill R&D mission needs. On the other hand, NSF stated that its SBIR program complemented its basic research programs by providing a linking mechanism to the marketplace. Like NSF, HHS said that the primary purpose of its SBIR program was to increase the commercialization of the results of federally funded R&D.

Many project officers monitoring SBIR projects also believed that SBIR programs helped meet agency R&D needs. Of the respondents to our questionnaire, 41 percent reported that the SBIR program definitely helped meet agency R&D needs, and another 37 percent thought that the program probably did so. Only 10 percent thought that SBIR programs probably or definitely made no contribution to agency research needs. However, as table 2.5 shows, agency project officers differed in their attitudes.

At NASA and DOD, where SBIR projects are solicited, selected, and managed
to meet specific R&D objectives, a high percentage of project officers
believe that the SBIR program definitely or probably helps meet agency
R&D needs. On the other hand, at NSF and HHS, where SBIR projects are not

Are SBIR Programs Meeting Their Goals?

Table 2.6: Responses Concerning the
Extent That Individual SBIR Projects
Have Contributed to the R&D Goals of
the Agency

closely related to agency programs to support basic research in universi-
ties, fewer project officers believe that SBIR programs are meeting
agency R&D needs.

When asked about specific projects, officers responded that 23 percent
of the projects had made a great or very great contribution to agency
R&D goals, while another 65 percent had made at least some contribu-
tion. As table 2.6 shows, project officers at DOD and NASA were more
likely than those at other agencies to judge their projects as making a
large contribution to agency R&D goals.

Comments provided by project officers on their questionnaires indicate that DOD and NASA SBIR projects contributed to R&D goals by meeting specific R&D objectives. For example, an Air Force monitor said that one SBIR project had contributed by significantly advancing bearing technology for turbine engines. A NASA project officer said that a project to develop a new cooling procedure had made a moderate contribution by helping develop new ways to shield superconducting magnets. Because NSF does not direct SBIR projects toward specific research objectives, project officer comments identified general, rather than specific, benefits to the agency. One project officer, for example, said that research on a new chemical process made some contribution to meeting agency research goals. He noted that the SBIR mission did not exactly coincide with NSF's basic science orientation but that the SBIR effort to apply science was healthy for the agency.

A larger proportion of project officers at NASA and DOD than at the other three agencies identified the SBIR program as a moderately or very important element of their agency's overall research program-69 percent at NASA and 65 percent at DOD. At DOE, 40 percent believed SBIR was a moderately or very important research program element; at HHS, 32 percent; and at NSF, 28 percent.

Are SBIR Programs Meeting Their Goals?

SBIR Programs Fund
Projects That Agencies
Might Not Support
Otherwise

In their comments on questionnaire responses, SBIR project officers indi-
cated several ways in which their SBIR programs contributed to research
objectives. For example, one NASA project officer noted that the program
attracted talent "hidden" in small businesses to R&D areas important to
his division, while another said that the SBIR program was an excellent
vehicle for starting up projects not in the mainline of agency R&D
which might become part of the mainline R&D if successful. Similarly, a
DOD project officer commented that the SBIR program provided an easy
method to forge relationships with innovative small businesses and
allowed a method of judging the state of the art.

Through SBIR programs, agencies support many projects that they would not otherwise sponsor. SBIR project officers reported that 52 percent of their projects probably or definitely would not have been funded by the agency if the SBIR program did not exist and were uncertain about an additional 30 percent. In their opinion, only about 17 percent of SBIR projects were likely to have been funded without an SBIR program.

There are some differences, however, among agencies over whether
projects would have been funded if the SBIR program did not exist. At
DOD, project officers thought that the agency would definitely or proba-
bly have funded 23 percent of the projects, while at the next highest
agencies (NSF and HHS), the percentage was 16 percent. For the five agen-
cies, the percentage of projects that probably or definitely would not
have obtained funding without the SBIR program ranged from 47 percent
to 62 percent.

At NSF and HHS, SBIR projects have an applied research focus and emphasize private sector commercialization so they are different from most other agency research. Thus, it is reasonable to expect that many SBIR projects would not have obtained non-SBIR funding at these agencies. However, at DOD and NASA where SBIR projects are more similar to other agency research activities, the large percentage of projects (49 percent at DOD and 59 percent at NASA) that would not have received funding outside the SBIR program is more unexpected.

According to questionnaire responses, DOD and NASA are using SBIR projects to undertake high-risk research—research in areas where results are less easy to achieve. In these two agencies, about half of the Phase II SBIR projects were rated by project officers as having higher levels of risk than non-SBIR projects that they managed. Only 13 percent of the projects in these agencies were assessed as having lower levels of