detailed examination of instructions (to find out which store references by the object program involve a replaced instruction location) and more frequent interrupts.

"The situation savours of 'trick' programming. It is apparent that the monitoring process will be simplified if there is some guarantee that these oddities are absent from object programs." (Wetherfield, 1966, pp. 162–163).

7.16 "MAID (Monroe Automatic Internal Diagnosis) is a program that tells a machine how to measure its circuitry and test performance on sample problems - computer hypochondria.' (Whiteman, 1966, p. 67).

"Think of the thousands of dollars saved by tightening up that one most-used program loop. Instrumentation can identify which loop is the most used." (Sutherland, 1965, pp. 12-13).

7.18 "On-Line Instrumentation will bring us better understanding of the interplay of the programs and data within the computer. Simple devices and programs to keep track, on-line, of what the computer does will bring us better understanding of what our information reprocessing systems are actually doing." (Sutherland, 1965, p. 9).

7.19 "The process of building a pilot system configuration and then evaluating it, modifying it, and improving it is very costly both in time and money. Another approach is possible. Before he builds the system, the designer should be able to test his concepts on a simulation model of a document retrieval system. One such model for simulating information storage and retrieval systems was designed by Blunt and his co-workers at HRB-Singer, Inc., under a contract with the Office of Naval Research. In this model, the input parameters for the simulation reflects the configuration of the system, the work schedule of the system, the work schedule of the personnel, equipment availability, the likelihood and effect of errors in processing and the location and availability of the system user. Simulation output provides a study of system response time (both delay time and processing time), equipment and personnel work and idle time and the location and size of the data queues. The systems designer can thus vary the inputs, use the model to simulate the interactions among personnel, equipment, and data at each step of the information processing cycle, and then determine the effect on the system response time." (Borko, 1967, p. 55).

7.20 "Simulation is a tool for investigation and, like any tool, is limited to its inherent potential. Moreover, the realization of this potential is dependent upon economics, the craftsmanship of the

designer and the ingenuity of the user. Digital simulation can expedite the analysis of a complex system under various stimuli if the aggregate can be divided into elements whose performance can be suitably described. If the smallest elements into which we can divide a system are themselves unpredictable (even in a probabilistic sense) digital simulation is not feasible. (Conway, et al., 1959, p. 94). This feasibility test uncovers an important limitation in today's simulation technology with respect to information systems. In many respects some of the more important man-information-system interactions cannot now be described in a formal manner; hence, cannot be characterized for digital simulation. For example, one can calculate the speed and costs of processing an inquiry, but cannot predict if the output will satisfy the user or estimate its impact on his operations.

"This limitation, therefore, (1) restricts simulation applications to examining the more mechanical aspects of data processing, or (2) forces the design engineer to adopt some simplifying assumptions concerning the effects of man's influence on the system. An example of the first point is a data flow simulation examining the rate of data processing without regard to the quality of the types and mixes of equipment and personnel. This capability for examining the resultant effects in varying parameters of the system enable the design engineer to explore more alternatives in less time and at less cost than ever before; e.g., he can develop costcapability curves for different possible system configurations under both present and anticipated processing needs. Neglecting this aspect of systems analysis has sometimes led to the implementation of a system saturated by later requirements and confronted by an unnecessary high cost for modification or replacement." (Blunt et al., 1967, pp. 75-76).

"To use simulation techniques in evaluating different computer systems, one must be able to specify formally the expected job mix and constraints under which the simulated system must operate, e.g., operating time per week. Equally important, one must carefully select a set of characteristics on which the competing systems will be judged. For different installations the most im portant characteristics may well be different. Each system under consideration is modelled, simulation runs are executed, and the results are compared on the selected characteristics.

"Unfortunately, the ideal case seldom occurs. Often the available information about the computer system's expected job mix is very limited. Furthermore, it is a well-known fact that an installation's job mix itself may be strongly influenced both qualitatively and quantitatively by the proposed changes in the system. For example, many of the difficulties with early time-sharing systems can be attributed to the changes in user practices caused by the introduction of the system. When statistics on job mix are available, they are often expressed in averages. Yet, it may be most important to simulate

a system's performance under extreme conditions. Finally, it is often difficult to show that a simulation is valid that is, that it actually does simulate the system in question." (Huesmann and Goldberg, 1967, p. 150).

7.21 "The field of information retrieval has been marked by a paucity of mathematical models, and the basis of present operational computer retrieval systems is essentially heuristic in design." (Baker, 1965, p. 150).

"The semantic and linguistic aspects of information retrieval systems also lend themselves poorly to the rigidity of models and model techniques, for which claims often lack empirical support." (Blunt, 1965, p. 105).

7.22 "There are structures which can easily be defined but which present-day mathematics cannot handle because of the limitations of presentday theory." (Hayes, 1963, p. 284).

"Markov models cannot, in general, be used to represent processes where other than random queuing is used." (Scherr, 1965, p. 32).

"Clearly, we need some mathematical models permitting the derivation of methods which will accomplish the desired results and for which criteria of effectiveness can be determined. Such models do not appear often in the literature." (Bryant, 1964, p. 504).

"First, it will be necessary to construct mathematical models of systems in which content, structure, communication, and decision variables all appear. For example, several cost variables are usually included in a typical operations research model. These are either taken as uncontrollable or as controllable only by manipulating such other variables as quantity purchased or produced, time of purchase or production, number and type of facilities, and allocation of jobs to these facilities. These costs, however, are always dependent on human performance, but the relevant variables dealing with personnel, structure, and communication seldom appear in such models. To a large extent this is due to the lack of operational definitions of many of these variables and, consequently, to the absence of suitable measures in terms of which they can be characterized." (Ackoff, 1961, p. 38).

"Mathematical analysis of complex systems is very often impossible; experimentation with actual or pilot systems is costly and time consuming, and the relevant variables are not always subject to control. . . .

"Simulation problems are characterized by being mathematically intractable and having resisted solution by analytic methods. The problems usually involve many variables, many parameters, functions which are not well-behaved mathematically, and random variables. Thus, simulation is a technique of last resort." (Teichroew and Lubin, 1966, p. 724).

"The complex systems generally encountered

in the real world do not lend themselves to neat mathematical formulations and in most cases the operations analyst is forced to reduce the problem to simpler terms to make it tractable." (Clapp, 1967, p. 5).

"Admittedly the degree to which identifiable factors can be measured-compared to the influence of unidentifiable factors - does help determine whether or not an approach can be scientific. It acts as a limit on the area where scientfic methods can be applied. Precision in model building is relating to the difficulty of the problem and the state of human knowledge concerning specific techniques and their application." (Kozmetsky and Kircher, 1956, p. 137).

7.23 "There is no guarantee that a model such as latent class analysis, factor analysis, or anything else borrowed from another field will meet the needs of its new context; however this should not dissuade one from investigating such plausible models." (Baker, 1965, p. 150).

"Models must be used but must never be believed. As T. C. Chamberlain said, 'science is the holding of multiple working hypotheses'." (Tukey and Wilk, 1966, p. 697).

7.24 "System simulation or modeling was subsequently proposed as a substitute for deriving test. problems and is still generally accepted as such even though its use introduced the new difficulty of determining and building meaningful models." (Davis, 1965, p. 82).

"The biggest problem in simulation modeling, as in all model building, is to retain all 'essential' detail and remove the nonessential features." (Scherr, 1965, p. 109).

“The fundamental problem in simulation of digital networks is that of economically constructing a mathematical model capable of faithfully replicating the real network's behavior in regard to simulation objectives." (Larsen and Mano, 1965, p. 308).

7.25 "At this time, there exists no specialpurpose simulation programming language specifically for use with models of digital computer systems. The general-purposes languages, such as SIMSCRIPT, GPSS, etc., all have faults which render them unsuitable for this type of work." (Scherr, 1965, p. 43).

"The invention of an an adequate simulation language promises to be the crowbar needed to bring the programming of operating systems to the level of sophistication of algebraic or commercial programming languages." (Perlis, 1965, p. 192). "The technique of input simulation . . . can be very expensive. The programs necessary to create the simulated inputs are far from trivial and may well constitute a second system larger than the operational system." (Steel, 1965, p. 232).

7.26 "Those programs which require the simulated computer system and job mix to be specified in algebraic or assembly languages have proved useful; but as general computer systems simulation tools, they require too much difficult recording to be

completely satisfactory. One way to improve upon this situation has been to use languages specifically designed to simulate systems. Teichroew and Lubin in a recent review have listed more than twenty languages, among them GPSS, SIMSCRIPT, SOL, and CSL. These simulation languages allow the modeller to specify the computer configuration and job mix in a more convenient manner." (Huesmann, and Goldberg, 1967, p. 152).

7.27 "One of the more exotic applications of digital computers is to simulate a digital computer on another entirely different type of computer. Using a simulation program, application programs developed for the first computer, the source computer, may be executed on a second computer, the object computer.

"Simulation obviously provides many advantages in situations where a computer is replaced by a different computer, for which the applications have not yet been programmed. Simulation techniques enable an installation to continue solving problems using existing programs after the new computer has been installed and the old one removed . . . "Another situation in which simulation is advantageous is during the development of a new computer. Once specifications for the new computer have been established, programming of applications for the computer can proceed in parallel with hardware developments. The use of a simulator in this situation enables the users to debug their applications before the hardware is actually available." (Trimble, 1965, p. 18).

"One of the most successful applications of the recent microprogramming technology is in the simulation of computers on computers.

"The microprogram control and the set of microprogram routines are in effect a simulation program that simulates the programmer's instruction set on a computer whose instruction set is the set of elementary operations. It may be equally possible to simulate computers with other programmer instruction sets in terms of the same set of elementary operations. This, slightly oversimplified perhaps, is the idea of hardware assisted simulation that is now usually called emulation." (Rosen, 1968, p. 1444).

"As a result of simulation's ability to deal with many details, it is a good tool for studying extensive and complicated computer systems. With simulation, one may assess the interaction of several subsystems, the performances of which are modified by internal feedback loops among the subsystems. For instance, in a teleprocessing system where programs are being read from drum storage and held temporarily in main storage, the number of messages in the processing unit depends upon the drum response time, which depends upon the drum access rate, which, in turn, depends upon the number of messages in main storage. In this case, only a system-wide simulation that includes communication lines, processing unit, and I/O subsystems will determine the impact of varying program priorities

on main-storage usage. Studies of this nature can become very time consuming unless parameter selections and variations are carefully limited. It is no small problem to determine which are the major variations that affect the system. In this aspect, simulation is not as convenient as algorithmic methods with which many variations can be tabulated quickly and cheaply." (Seaman, 1966, p. 177).

7.28 "The [SIMSCRIPT] notation is an augmented version of FORTRAN, which is acceptable; but this organization does not take advantage of the modularity of digital systems.

"SIMSCRIPT is an event-based language. That is, the simulation is described, event by event, with small programs, one per event. Each event program (or sub-program) must specify the times for the events following it. Conditional scheduling of an event is extremely difficult." (Scherr, 1965, p. 43).

7.29 Ewards points out that “. . . the preparation of so-called scenarios, or sequences of events to occur as inputs to the simulation, is a major problem, perhaps the most important one, in the design of simulations, especially simulations of information-processing systems." (Edwards, 1965,

p. 152).

7.30 "Parallel processes can be rendered sequential, for simulation purpose; difficulties then arise when the processes influence each other, leading perhaps to incompatibilities barring a simultaneous development. Difficulties of this type cannot be avoided, as a matter of principle, and the system is thus not deterministic; the only way out would be to restore determinism through recourse to appropriate priority rules. This approach is justified only if it reflects priorities actually inherent in the system." (Caracciolo di Forino, 1965, p. 18).

7.31 "As a programming language, apart from simulation, SIMULA has extensive list processing facilities and introduces an extended co-routine concept in a high-level language." (Dahl and Nygaard, 1966, p. 671).

7.32 "The LOMUSS model of the Lockheed UNIVAC on-line, time-sharing, remote terminal system simulated two critical periods . . . and provided information upon which the design of the 1108 system configuration was based. An effort is continuing which will monitor the level and characteristics of the workload, equipment utilization, turnaround time, etc., for further model validation." (Hutchinson and Maguire, 1965, pp. 166-167).

"A digital computer is being used to simulate the logic, determine parts values, compute subunit loading, write wiring lists, design logic boards, print checkout charts and maintenance charts. Simulating the logic and computing the loading of subunits gives assurance that a computer design will function properly before the fabrication starts. After the logic equations are simulated, it is a matter of hours until all fabrication information and checkout information is available. Examples are given of the use of these

techniques on the design and fabrication of a large scale military computer." (Malbrain, 1959, p. 4-1). "The new EDP Machine Logic and Control Simulator, LOCS, is designed to facilitate the simulation of data processing systems and logic on the IBM 7090 Data Processing System . . . The inputs of LOCS consist of a description of the machine to be simulated, coded in LOCS language, and a set of test programs coded in either the procedure language of the test problems (e.g., FORTRAN) or in the instruction language of the simulated machine . . . The outputs of LOCS consist of the performance statistics, computation results, and diagnostic data which are relevant to both the test programs and the design of the simulated machine." (Zucker, 1965, pp. 403-404).

66

7.33 . . . A system of software and hardware which simulates, with a few exceptions, a multiple set of IBM System/360 computing systems (hardware and software) that are simultaneously available to many users." (Lindquist et al., 1966, p. 1775).

7.34 "Another simulation program designed to simulate multiprocessor systems is being developed by R. Goldstein at Lawrence Radiation Laboratory. Written in SIMTRAN, this program is specifically designed to simulate the OCTOPUS computer system at LRL which includes an IBM 7030 STRETCH, two IBM 7094's, two CDC 6600's, one CDC 3600, two PDP 6's, an IBM 1401, and various I/O devices. A parameterized input table specifies the general multiprocessing configuration, the data transmission rates, memory sizes and buffer sizes. Any other hardware variations and operating system characteristics are introduced with SIMTRAN routines. As output the program produces figures on actual memory utilization, graphs of memory access time, graphs of overhead, graphs of response time, and graphs of several other relevant variables." (Huesmann and Goldberg, 1967, p. 153).

7.35 "One of the more exotic applications of digital computers is to simulate a digital computer on another entirely different type of computer. Using a simulation program, application programs developed for the first computer, the source computer, may be executed on a second computer, the object computer." (Trimble, 1965, p. 18).

7.36 "The flexibility of a digital computer enables one to try out complicated picture processing schemes with a relatively small amount of effort. To facilitate this simulation, a digital picture scanner and cathode-ray tube display was constructed. Pictures were scanned with this system, the signal was recorded on a computer magnetic tape, and this tape was used as input to a program that simulated a picture-transmitting system." (Huang and Tretiak, 1965, pp. 45-46).

"Computer simulation of a letter generator using the above grammar is a comparatively straightforward programming task. Such a program, making

use of COMPAX, has been written for CDC 3600 system at the Tata Institute of Fundamental Research, Bombay." (Narasimhan, 1966, p. 171).

At IBM, there has been developed a computer program for image-forming systems simulation (IMSIM/1), so that the photo-optical design engineer can study performance before such systems are actually built (Paris, 1966).

7.37 "The first model developed matches CTSS . . . Next, a simple, first-come, first-served round-robin scheduling procedure will be substituted. Then, a model which incorporates multiprogramming techniques with the CTSS hardware configuration will be developed. Finally, a simple continuous-time Markov model will be used to represent both single-processor and multiple-processor time-shared systems.

"The primary result obtained is that it is possible to successfully model users of interactive computer systems and systems themselves with a good degree of accuracy using relatively simple models." (Scherr, 1965, p. 31).

"The final model to be simulated will represent a system in which swapping and processor operation are overlapped. While a program is being run by the processor, the program which was running previously is dumped and the next program to run is loaded. Since loading and dumping cannot occur simultaneously, there must be room in the core memory for at least two complete user programs — the program being dumped or loaded. Should two programs intended to run in sequence not fit together in the core memory, the processor must be stopped to complete the swapping." (Scherr, 1965, pp. 40-41).

Other examples of experiments in computer simulation of multiple access and time-shared systems include the following: "The development of the simulation program now provides a first-comefirst-serve queue unloading strategy. Continuing effort, however, will provide for optional strategies, e.g., selecting the data unit with the shortest servic ing time, consideration of what flow will minimize idle time at the central processor etc." (Blunt, 1965, p. 15).

"Project MAC is using a display to show the dynamic activity of jobs within its scheduling algorithm. Watching this display, one can see jobs moving to higher and lower priority queues as time passes." (Sutherland, 1965, p. 13).

7.38 "A great advance also is the wide application of digital computers for the simulation of recognizing and adaptive systems. Digital simulation extraordinarily facilitates and accelerates conducting experiments in this realm by permitting extremely effective experimental investigations without expenditure of materials and with little time spent." (Kovalevsky, 1965, p. 42).