Page images
PDF
EPUB

Quality of SBIR Research Projects

the projects were judged to be better than other research, while 27 percent were judged to be worse. Responses concerning the likelihood that the project will lead to inventing and commercializing new products, processes, or services were more positive than for other factors. For this factor, most projects (53 percent) were regarded as better than other research, while 29 percent were judged about the same. About 12 percent were judged worse than other research.

Table 3.1: Questionnaire Responses Concerning SBIR Project Quality in Comparison With Non-SBIR Research
Percent

[blocks in formation]

Differences Among
Agencies Regarding
SBIR Project Quality

Although most SBIR projects were judged to be about the same overall quality as other research, the pattern of responses differed among the agencies covered by our questionnaires. In general, these differences in agency response paralleled the differences in emphasis on SBIR goals that were described in chapter 2. At DOD and NASA, agencies that emphasize the SBIR goal of meeting federal R&D needs, project officers rated SBIR projects high on almost all factors in comparison with other research. In contrast, HHS and NSF project officers rated SBIR projects very high concerning the likelihood of private sector commercialization, a goal that

Quality of SBIR Research Projects

these agencies emphasize in their SBIR programs, but lower on some other factors-as table 3.2 indicates.

To compare agency responses in table 3.2, we assigned numerical values to the questionnaire responses, as follows:

[blocks in formation]

Responses of "unable to judge" or "not applicable" were not included in this analysis. For each factor, we added up the numerical value of the agency responses and divided by the number of responses to obtain an average agency response for each factor. As table 3.2 shows, the average scores in many cases were generally slightly above or very close to O, indicating that many projects were regarded as of much the same quality as non-SBIR research.

[blocks in formation]

Note: Individual questionnaire responses were assigned numerical values to develop an overall agency evaluation, as follows:

[blocks in formation]

At one extreme, NASA project officers rated SBIR projects higher than other research on all factors. DOD's responses are close to, but not quite as positive as, those from NASA. DOD project officers rated SBIR projects better than other research on all but one factor: scientific/technical facilities and resources.

At the other extreme, NSF project officers rated SBIR projects as lower in research quality than other projects overall and lower on six of the nine specific factors. HHS project officers were negative in their overall comparison of SBIR research quality to other agency research and very close to neutral on six of the remaining nine factors. NSF and HHS project officers were, however, very positive concerning the likelihood that SBIR projects would lead to invention and commercialization.

Quality of SBIR Research Projects

Table 3.3: Share of Project Officer's Non-
SBIR Research Time Devoted to Basic
Research

DOE's responses fell between the extremes established by the other agencies, in the assessment of overall research quality, and in several of the specific factors. The assessment of SBIR projects performed by DOE'S Office of Program Analysis and dated August 1988 shows a real, although small, difference between the overall average ratings of SBIR and non-SBIR projects, with the non-SBIR projects having a higher rating.2

In comparing responses among agencies, it should be noted that project officers differ among agencies in the amount of non-SBIR basic research that they oversee, as table 3.3 shows. This table indicates that more project officers at NSF and HHS than at other agencies reported devoting all, or almost all, of their time to overseeing basic research when they were not working with SBIR projects.

[blocks in formation]

As table 3.4 shows, project officers who spent all, or almost all, of their non-SBIR R&D time on basic research differed from other project officers in their responses concerning research quality.

2The DOE assessment was based on evaluations provided by 17 independent scientific and technical panels that reviewed samples of SBIR and non-SBIR projects.

[blocks in formation]

For all but one of the factors in table 3.4, project officers who spent all, or almost all, of their Non-SBIR R&D time on basic research were less likely than other project officers to regard their SBIR projects as better than other research for which they were responsible. For example, 20 percent of the project officers who spent all, or almost all, of their nonSBIR time on basic research said that the SBIR project was of better overall quality than other research, compared with 35 percent of other project officers. However, the project officers who spent all, or almost all, of their non-SBIR project time on basic research were about as likely as the others to assess their SBIR project as more likely than other research to lead to inventing and commercializing new products, processes, services.

or

« PreviousContinue »