« PreviousContinue »
12. unquoted strings containing quoted-string-characters
type mismatch on function reference (using string as an argument)
DEF with string variable for parameter
DEF with multiple parameters
misplaced or missing END-statement
17. null entries in various lists (INPUT, DATA, READ, e.g.)
When developing programs to test for possible enhancements, also tried to assist the user in confirming what the actual processor behavior is, so that it may be checked against the documentation. For example, the program that tests whether the implementation accepts "<" and ">" for comparison of strings also displays the implicit character collating sequence if the comparisons are accepted. When the implementation accepts an error program be sure to check that the documentation does in fact describe the actual interpretation of the error
by the test program. If the error program is rejected, the processor's error message should be a reasonably accurate description of the erroneous construction.
There is a summary of the requirements for error handling in the form of pseudo-code in section 3.2.2 (Figure 1).
220.127.116.11 Informative Tests
Informative tests are very much like implementation must accept and process them, since they are syntactically standard. The difference is that the standard only recommends, rather than requires, certain aspects of their behavior. The pass/fail message (described below) and other program output indicates when a test is informative and not mandatory. All the informative tests have to do with the quality (as opposed to the existence) of various mathematical facilities. Specifically, the accuracy of the numeric operations and approximated functions and the randomness of the RND function are the subjects of informative tests. Some of the standard tests also have individual sections which are informative, and again the pass/fail message is the key to which sections informative and which mandatory. If numeric accuracy is important for your purposes, either as an implementor or a user, you should analyze closely the results of the informative tests.
There are three kinds of documentation in the test serving three complementary purposes:
The user's manual (this document). The purpose of this manual is to provide a global description of the test system and how it relates to the standard and to conformance. At a more detailed level, there is also a description of each functional group of programs and the particular things you should watch for when running that group.
Program output. As far as possible, the programs attempt to explain themselves and how they must be interpreted to determine conformance. Nonetheless, they make sense only in the context of some background knowledge of the BASIC standard and conformance (more detail below on output format).
Remarks in the source code. Using the REM statement, the programs attempt to clarify their own internal logic, should you care to examine it. Many of the programs are algorithmically trivial enough that remarks are superfluous, but otherwise remarks are there to guide your understanding of how the programs are intended to work.
There is a format for program output consistent throughout the test sequence. program first prints its identifying. sequence number and title. The next line lists the sections of the ANSI standard to which this test applies. After this program header, there is general information, if any, pertaining to the whole program. Following all this program-level output there is a series of one or more sections, numbered sequentially within the program number. Each section tests one aspect of the general feature being exercised by the program. Every section header displays the section number and title and any information pertinent to that section. Then the message, "BEGIN TEST." appears, after which the program attempts execution of the feature under test. At this point, the test may print information to help the user understand how execution is proceeding.
Then comes the important part: a message, surrounded by asterisks, announcing "*** TEST PASSED ***" or "*** TEST FAILED If the test cannot diagnose its own behavior, it will print a conditional pass/fail message, prefacing the standard message with a description of what must or must not have happened for the test to pass. Be careful to understand and apply these conditions correctly. It is a good idea to read the ANSI standard with special attention in conjunction with this sort of test, so that you can better understand the point of the particular section.
There is no pass/fail message for the error tests, since there is, of course, no standard semantics prescribed for a non-standard construction. As mentioned above, error programs usually generate messages to help you diagnose the behavior of the processor when it does accept such a program.
After the pass/fail message will come a line containing "END TEST. 11 which signals that the section is finished. If there is another section, the section header will appear next. If not, there will be a message announcing the end of the program. Note that each section passes or fails independently; all sections not just the last, must print "*** TEST PASSED ***" for the program as a whole to pass. Figure 2 contains a schematic outline of standard program output.
Format of Test Program Output
PROGRAM FILE nn: descriptive program title.
ANSI STANDARD XX.x, yy.y
message if a feature is used before being tested, cf. section 4.1 and general remarks about the purpose of the program
SECTION nn. 1: descriptive section title.
interpretive message for error or exception tests
message to assist analysis of processor behavior for error program
SECTION nn.2: descriptive section title.
SECTION nn.m: descriptive section title.
END PROGRAM nn
5 FUNCTIONAL GROUPS OF TEST PROGRAMS
This section contains information specific to each of the groups and sub-groups of programs within the test sequence. Groups are arranged hierarchically, as reflected in the numbering system. The sub-section numbers within this section correspond to the group numbering in the table of section 6.1, e.g., section 18.104.22.168 of the manual describes functional group 12.1.2.
It is the purpose of this section to help you understand the overall objectives and context of the tests by providing information supplementary to that already in the tests. This section will generally not simply repeat information contained in the tests themselves, except for emphasis. Where the tests require considerable user interpretation, this documentation will give you the needed background information. Where the tests are self-checking, this documentation will be correspondingly brief. We suggest that you first read the comments in this section to get the general idea of what the tests are trying to do, read the relevant sections of the ANSI standard to learn the precise finally run the programs themselves, comparing their output to the sample output in Volume 2. The messages written by the test programs are intended to tell you in detail just what behavior is necessary to pass, but these messages are not the vehicle for explaining how that criterion is derived from the standard. Program output should be reasonably intelligible itself, but it is better understood in the broader context of the standard and its conformance rules.
5.1 Simple PRINTing Of String Constants
This group consists of one program which tests that the implementation is capable of the most primitive type of PRINTing, that of string constants and also the null PRINT. Note that it
is entirely up to you to determine whether the test passes or fails by assuring that the program output is consistent with the expected output. The program's own messages describe what is expected. You may also refer to the sample output in Volume 2 to see what the output should look like.
5.2 END And STOP
This group tests the means of bringing BASIC programs to normal termination. These capabilities are tested early, since all the programs use them. Both END and STOP cause execution to stop when encountered, but STOP may appear anywhere in the program any number of times. There must be exactly one END statement in a program, and it must be the last line in the source code. Thus, END serves both as a syntactic marker for the end of the program, and is also executable.