Questionnaire Methodology

In preparing this report, we used three survey instruments, as follows:

a survey of small businesses that had received SBIR awards,

a questionnaire to project officers responsible for monitoring SBIR projects at DOD, DOE, HHS, NASA, and NSF containing general questions on their agencies' SBIR program, and

a questionnaire to the same project officers concerning specific SBIR projects.

For this report, we sent the survey contained in appendix II to small
businesses using the same sample of SBIR projects that was used in our
previous report, Federal Research: Small Business Innovation Research
Participants Give Program High Marks.' The sample of projects we used
was drawn from lists of projects conducted during fiscal years 1983
through 1985 by the 12 federal agencies that sponsored SBIR projects
during this period. Questionnaires were sent to all firms having projects
except for projects funded by DOD, DOE, HHS, NASA, and NSF. For those
agencies, we selected a representative sample as shown in table V.1. In
addition, we sent questionnaires concerning all Phase II projects desig-
nated as complete by the responsible agency at the time of our survey
for the previous report. We assigned appropriate weights during the
data analysis to account for the agency of the project and whether or
not Phase II was complete. Table V.1 shows the sample size for each
agency and the weighted number of projects for each agency in our anal-
ysis. (A copy of the survey is in app. II.)

The sample was designed to have sampling errors of no more than 5 percent at the 95-percent confidence level (sampling errors for subsets of the sample could be higher). (App. II shows sampling errors in parentheses for selected key variables.)

1(GAO/RCED-87-161BR, July 27, 1987).

Questionnaire Methodology

Questionnaire Procedures

Survey Results

Table V.1: Sampling Plan

We developed the questionnaire after discussions with agency officials and consultants. We conducted pretests with eight companies in the Washington, D.C., and Boston areas that participated in SBIR projects. During each session, an individual respondent filled out the questionnaire in the presence of two GAO observers. After the pretests, we revised the questionnaire as necessary to increase clarity and ease of response.

We mailed questionnaires to the principal investigator of each project in the sample. Because we based our sample on projects rather than companies, 212 companies received 2 or more questionnaires. A total of 954 companies received our questionnaire.

We sent follow-up letters to nonrespondents, including a second copy of the questionnaire, and also sent a final reminder to nonrespondents to encourage them to return their questionnaires.

We received 1,113 completed questionnaires out of 1,406 that were mailed, yielding a response rate of 79 percent. These responses were weighted to account for our stratified sampling of agency projects. Appendix II shows the questionnaire and the frequency of responses to individual questions.

Questionnaire Methodology

General Questions to
Project Officers

Working with agency officials at DOD, DOE, HHS, NASA, and NSF, we identified and sent questionnaires to 530 officers who had been responsible for monitoring and/or assessing the 739 SBIR projects started at these agencies during fiscal years 1983 and 1984 that resulted in Phase II awards. These five agencies are responsible for 96 percent of all SBIR funds.

Questionnaire Procedures

Survey Results

Questionnaire
Concerning Specific
SBIR Projects

We developed questions concerning the SBIR program after discussions with agency officials and consultants. We conducted pretests with SBIR project officers at DOD, DOE, HHS, and NSF. During each session an individual project officer filled out the questionnaire in the presence of two GAO observers. After pretesting, the questionnaire was revised as necessary to increase clarity and ease of response.

We sent follow-up letters to nonrespondents, including a second copy of the questionnaire. Later, we made a final follow-up to the remaining nonrespondents by telephone.

We received 495 completed questionnaires from the 530 project officers that we had identified, yielding a response rate of 93.4 percent. Appendix III shows the questionnaire and the frequency of responses to individual questions.

Each project officer who received a questionnaire with general questions about the SBIR program also received one or more questionnaires about specific SBIR projects that were started during fiscal years 1983 and 1984 that resulted in Phase II awards, a total of 739 projects. We followed up nonresponses to this questionnaire in conjunction with the questionnaire concerning general questions about the SBIR program. We received questionnaires concerning 691 projects, a response rate of 93.5 percent.

The questionnaire concerning specific SBIR projects was developed and pretested in conjunction with the general questions concerning the SBIR program. Appendix IV shows the questionnaire and the frequency of responses to individual questions.

Letter From the Department of Agriculture Concerning the SBIR Program

I am pleased to respond to your request for an evaluation by the U.S. Department of
Agriculture (USDA) of the effectiveness of the Small Business Innovation Research (SBIR)
program within the USDA. This evaluation is based in part on extensive consultations by
the SBIR Office with grantees, scientists who have served on both SBIR and USDA
Competitive Research Grants Office (CRGO) panels, and various USDA officials. It is also
based upon information documented by the SBIR Coordinator, Dr. Charles F. Cleland, who
has made nearly 30 site visits to Phase I and Phase II grantees since he joined USDA's
SBIR program in May of 1987.

In our opinion the SBIR program is proving to be a sound investment of Federal R&D funds for the following reasons:

(1) The research community that applies to the SBIR program is completely
different from that which applies to the Competitive Research Grants program,
which is USDA's primary extramural research grant program. In FY '87, the
Competitive Research Grants program received a total of 1653 grant proposals
with only eight coming from private profit organizations. A total of 363 grants
were awarded with just two going to private profit organizations (both were
awarded to Weyerhaeuser Company). The SBIR program in FY '87 received 178
Phase I applications and 24 Phase II applications, and made 23 Phase I awards
and 12 Phase II awards. Thus, for science and technology-based small business
firms, the SBIR program represents their best opportunity for access to USDA
R&D funds.

(2) The quality of successful SBIR proposals compares favorably to the quality of
successful proposals submitted to the Competitive Research Grants program.
Scientists who have served as panelists in both programs indicate that while the
nature of the research is clearly different, the scientific and technical merit is
very high in both cases. Competitive Research Grants projects are usually for
a two to three year period and are focused on basic research, while SBIR Phase
I grants are for only 6 months and have a more applied focus. Consequently,
there are limitations on what can realistically be proposed in a Phase I grant,
but this does not detract from the scientific merit of the proposals.