« PreviousContinue »
entering the system, with special problems likely to be involved, for example, in the dating of reports. (Croxton, 1955). As Davis (1967) also points out, the timeliness of information contained in the system depends not only on the time of its input but also upon the date or time it was recorded or reported and the date the information itself was originally acquired, including the special case of the "elastic ruler" (Birch, 1966).2.19 Another typical problem is that of transliterations and transcriptions between items or messages recorded in many different languages.2.20
A crucial area of R & D concern is that of the accuracy, integrity, and reliability of information in the system, although these questions are all too often neglected in system design and use 2.21 Again, Davis emphasizes the importance of information content controls. These may be achieved, on input, either by error-detecting checks on quantitative data or by “correctness control through “common sense' or logical checks.” (Davis, 1967, p. 10.) 2.22 Thus, the use of reliability indicators and automatic inference capabilities may provide significant advantages in improved information handling systems in the future.2.23
One of the obvious difficulties in controlling accuracy and reliability of the information content of items in the system is that of correction and updating cycles.2.24 More commonly, however, errors affecting the accuracy and reliability of information are those of human errors in observation, recording, or transcription and those of transmission or equipment failure during communication and input. The incidence of such errors is in fact inevitable and poses a continuing challenge to the system designers which becomes increasingly severe as the systems themselves become more complex.2.25
It is to be noted, of course, that a major area of R & D concern in the communication sciences is that of information theoretic approaches to error detection, correction, and control. In terms of generalized information processing systems, however, we shall assume that advanced techniques of message encoding and decoding are available to the extent required, just as we assume adequate production quality controls in the manufacture and acceptance testing of, say, magnetic cores. Thus our concern here is with regard to the control, detection, and (where feasible), correction, of errors in information content of items in an information processing system or network, regardless of whatever protective encoding measures have been employed.
It should be recognized first of all that any formulation of an information-carrying message or record is an act of reportage, whether it is performed by man or by machine. Such reportage may itself be in error (the gunshots apparently observed during riot conditions may have been backfiring from a truck, the dial indicator of a recording instrument may be out of calibration, and the like). The recording of the observation may be in error: misreading of, say, the dial indicator, transposition of digits in
copying a numerical data display, and accidental or inadvertent misspellings of names are obvious examples.
With respect to errors introduced by transmission, examples of R & D requirements and progress were cited in the first report in this series (“Information Acquisition, Sensing, and Input", Section 3.4). Two further examples to be noted here include the discussion by Hickey (1966) of techniques designed to handle burst-type errors 2.26 and a report by Menkhaus (1967) on recent developments at the Bell Telephone Laboratories.2.27 For checking recording and/or transmission errors, a variety of error detection devices (such as input interlocks,2.28 parity information,2.29 check digits,2.30 hash totals,2.31 for: mat controls and message lengths 2.32) have been widely used. 2.33
Problems introduced by alphanumeric digit transpositions or simple misspellings can often be attacked and solved by computer routines, provided that there is some sort of master authority list, or file, or the equivalent of this in terms of prior conditional matching.2.34 For example, Alberga (1967) discusses the comparative efficiency of various methods of detecting errors in character strings.
The use of contextual information for error detection and possible correction in the case of automatic character recognition processes has been noted in a previous report in this series, that on information acquisition, sensing, and input. This is, of course, a special case of misspelling.2.35 Some of the pertinent literature references include Edwards and Chambers (1964), Thomas and Kassler (1967) and Vossler and Branston (1964). The latter investigators, in particular, suggest the use of lookup dictionaries specialized as to subject field and analysis of part-of-speech transitions.2.36
Context analysis is important, first, because for the human such capabilities enable him to predict (and therefore skim over or filter out) message redundancies and to decide, in the presence of uncertainties between alternative message readings, the most probably correct message contents when noise, errors, or omissions occur in the actual transmission of the message.?
2.37 Context analysis also provides means for automatic error detection and error correction in the input of text at the character level, the word level, and the level of the document itself such as the detection of changes in terminology or the emergence of new content in a given subject field. For example, “various levels of context can be suggested, ranging from that of the characters surrounding the one in question to the more nebulous concept of the subject class of the document being read.” (Thomas and Kassler, 1963, p. 5). In automatic character recognition, in particular, consideration has been given to letter digrams, trigrams, and syllable analysis approaches 2.38 as well as to dictionary lookups.
Special problems, less amenable to contextual considerations, arise in the case of large files
containing many names (whether of persons or of drugs, for example) which are liable to misspellings or variant spellings or which are homonymous.2.39 Information control requirements in such cases may involve the use of phonetic indexing techniques 2.40
error detection and correction mechanisms.2.41
Automatic inference and consistency checks may be applied to error detection and error correction as well as to identification and authentication procedures. Waldo and DeBacker (1958) give an early example as applied to chemical structure data.2.42 A man-machine interactive example has been described by North (1968).2.43 For the future, however, it can be predicted that: “Ways must be found for the machine to freely accept and use incomplete, qualitative information, to mix that information with internally-derived information, and to accept modifications as easily as the original information is accepted.” (Jensen, 1967, p. l-1).
Finally, we note that, in its broadest sense, the term "control” obviously implies the ability to predict whether a given machine procedure will or will not have a solution and whether or not a given computer program, once started running, will ever come to a halt. The field of information control may thus include the theories of automata, computability, and recursive functions, and questions of the equivalence of Turing machines to other formal models of computable processes.
2.1.3. Other System Design Requirements
ments, their locations and the probable workloads (both as to types and also as to throughputs required), are the necessary analyses of the resources presently potentially available. Resources analysis typically involves considerations of manpower availabilities, technological possibilities, and alternative procedural potentialities.
The question may well be raised with respect to an obvious spectrum of R & D requirements. Certainly there will be continuing areas of R & D concern with respect to advanced hardware technologies in processor and storage system design, and in materials and techniques that are related to these requirements. Next there are problems of "software" - that is, of programming techniques to take full advantage of parallel processing capabilities, associative memory accessing and organization, multiprogrammed and multiple-access system control.
Certain requirements are obviously overriding because they permeate the total system design and because they interact with many or all of the sub-systems involved. These include the problems of comparative pay-offs between various possible assemblies of hardware and software, the questions of programming languages
languages and of
and of suitable hierarchies of such languages, and the problems of man-machine interaction especially in the case of time-shared or multiple access systems.
Similarly, the requirements for handling a variety of input and output sensing modalities and for processing more than one 1-0 channel in an effectively simultaneous operation clearly indicate needs for continuing research and development efforts in the design and use of parallel processing techniques, multi-processor networks, time-shared multiple access scheduling, and multi-programming.
Hierarchies of languages are implied, ranging from those in which the remote console user speaks to the machine system in a relatively natural language (constrained to a greater or lesser degree) to those required for the highly sophisticated executive control, scheduling, file protection, accounting, monitoring, and instrumentation programs. For the future, increasing consideration needs to be given not only to hierarchies of languages for using systems, but to hierarchies of systems as well.2.51
There are, of course, concurrent hardware research, development, and effective usage requirements in all or most of these areas. Improvements in microform storage efficiency, lower per bit information-representation costs, communication channel utilization economies, improved quality of facsimile reproduction and transmission of items selected or retrieved, are obvious examples of directly foreseeable future demands. Some of the above considerations will be discussed in later sections of this report. Here we are concerned in particular with resources analysis in terms of system modularity, configuration and reconfiguration, and with provisions for safeguarding the information to be handled in the system.
Other system design considerations with respect to requirements analysis include questions of centralization or decentralization of functions and facilities, including compromises such as clusters; 2.44 questions of batch-processing as against time-sharing or mixtures of these modes,2.45 and questions of formatting, normalization,2.46 and standardization.2.47
A final area of requirements analysis involves the questions of system design change and modification 2.48 and of system measurement.2.49 In particular, information on types of system usage by various clients provides the basis for periodic re-design of system procedures and for appropriate reorganization of files. Such feedback information may also provide the client with system statistics that enable him to tailor his interest-profile or search strategy considerations to both the available collection characteristics and to his own selection requirements. As Williams suggests, 2.50 this kind of facility is particularly valuable in systems where the client himself may establish and modify the categories of items in the files that are most likely to be of interest to him.
2.2. Resources Analysis
Collateral with comprehensive analyses of potential system clienteles, their needs and require
2.2.1. System Modularity, Configuration, and
Reconfiguration Today, in increasingly complex information processing systems, there are typically requirements for considerable modularity and replication of system components in order to assure reliable, dependable, and continuous operation.2.52 The possibilities for the use of parallel processing techniques are receiving increased R & D attention. Such techniques may be used to carry out data transfers simultaneously with respect 2.53 to the processing operations, to provide analyses necessary to convert sequential processing programs into parallel-path programs,2.54 or to make allocations of system resources more efficiently because constraints on the sequence in which processing operations are executed can be relaxed.2.55
In terms of system configuration and reconfiguration, there is a continuing question of the extent of desirable replication of input-output units and other components or sub-assemblies. This may be particularly important for multiple-access and multiple-use systems.2.56 A particularly important system configuration feature desired as a resource for largescale information processing systems is that of open-endedness.2.57
System reconfigurations, often necessary changing task orders are received, are particularly important in the area of shifting the system facilities for system self-checking and repair.2.58 Thus Amdahl notes that “the process of eliminating and introducing components when changing tasks is reconfiguration. The time required to reconfigure upon occurrence of a malfunction may be a critical system parameter," (Amdahl, 1965, p. 39) and Dennis and Glaser emphasize that "the ability of a system to adapt to new hardware, improved procedures and new functions without interfering with normal system operation is mandatory." (Dennis and Glaser, 1965, p. 5.)
and the emergence of computer-communication networks obviously imply some degree of both modularity and replication of components, providing thereby some measure of safeguarding and recovery protection.2.62 An extensive bibliographic survey of proposed techniques for improving system reliability by providing various processes for introducing redundancy is provided by Short (1968).2.63
Protective redundancy of system components is, as we have seen, a major safeguarding provision in design for high system reliability and availability.2.64 In terms of continuing R & D concerns, however, we note the desirability of minimizing the costs of replication 2.65 and the possibilities for development of formal models that will facilitate the choice of appropriate trade-offs between risks and costs.
Finally, there are the questions of resources analysis with respect to the safeguarding of the information in the system or network that is, the provisions for recovery, backup, rollback, and restart or repeat of messages, records, and files.2.67 The importance of adequate recovery techniques in the event of either system failure or destruction or loss of stored data, can hardly be overestimated.2.68
The lessons of the Pentagon computer installation fire, in the early days of automatic data processing operations, still indicate today that, in many situations, separate-site replication of the master files (not only of data but also often of programs) is mandatory 2.69 Otherwise, the system designer
. must determine whether or not the essential contents of the machine-usable master files can be recreated from preserved source data.2.70 If the file contents can be recreated, then the designer must decide in what form and on what storage media the backup source records are to be preserved.2.71
In terms of system planning and resource analysis for information processing network design, we note the following questions:
Can the network continue to provide at least minimal essential services in the case of one or more accidental or deliberate breaks in the links?
What are the minimal essential services to be maintained at fail-safe levels? To what extent will special priorities and priority re-scheduling be required?
Must dynamic re-routing of information flow be applied, or will store-and-forward with delayed re-routing techniques suffice?
There are known techniques for evaluating optimum or near-optimum paths through complex paths in the sense of efficiency (economic, workload balancing, and throughput or timeliness considerations). Can these techniques be reapplied to the fail-safe or fail-softly requirements must
methods and algorithms be developed?
What are the fallback mechanisms at all levels and nodes of the system for: (a) specific failures
2.2.2. Safeguarding and Recovery Considerations
A first and obvious provision for "fail-safe" (or, more realistically, "fail-softly'') 2.59 operation of an
“ information processing system network is that of adequate information controls (for example, as discussed above) on the part of all member systems and components in the network.2.60 This requirement reflects, of course, the familiar ADP aphorism of ‘garbage in, garbage out'. Again, the total system must be adequately protected from inadvertent misuse, abuse, or damage on the part of its least experienced user or its least reliable component. Users must be protected from unauthorized access and exploitation by other users, and they also must be protected from the system itself, not only in the sense of equitable management, scheduling, and costing but also in the sense that system failures and malfunctions should not cause intolerable delays or irretrievable losses.2.61
Tie-ins to widespread communication networks
at a particular node, (b) breaks of one or more specific link(s), (c) massive failures, such as the New York area power blackout?
In general, with respect to areas of R & D concern affecting safeguarding and recovery provisions, we may conclude with Davis that “Rarely, if ever, are measurements made of the ability of the system to respond when partially destroyed or malfunctioning, of the length of time required
for changing the system response to internal change in direction or to external stimuli, of the length of time necessary for a newcomer to be inserted into his assigned role in the system, of the redundancy, backup, or alternatives available at times of partial or total system destruction, and so forth. Clearly, there will be no adequately constructed system until such measures of effectiveness are understood and incorporated into system design." (Davis, 1964,
3. Problems of System Networking
“a. Load sharing among major computer cen
ters “b. Data pick-up from remote test sites (or from
airborne tests). In some cases real-time processing and retransmission of results to the
test site would be desirable. "c. Providing access for Plant A to a computer
center at Location B. Plant A might have a medium-scale, small-scale, or no computer of
its own. “d. Data pick-up from dispersed plants and offices
for processing and incorporation in overall reports. The dispersed points might be in the same locality as the processing center, or possibly as much as several thousand miles
away." (Perlman, 1961, p. 209.) Three special areas of system network planning may be noted in particular. These are the areas of network management and control, of distribution requirements, and of information flow requirements.
Steadily mounting evidence of the nearly inevitable development of information-processing. system networks, computer-communication utilities, and multiply-shared, machine-based, data banks illuminates a major and increasingly critical area of R & D concern. In this area, the problems of “organized complexity” 3.1 are likely to be at least an order of magnitude more intractable than they are today in multiprogrammed systems, much less in those systems requiring extensive manmachine interaction.
It is probable, in each of these three fields of development, that there has been and will continue to be for some time to come: (1) inadequate requirements and resources fact-finding and analysis, 3.2 (2) inadequate tools for system design, 3.3 and (3) the utter lack of appropriate means for evaluation in advance of extensive (and expensive) alternatives of system design and implementation.3.4 Certainly the problems of system networking will involve those of priority scheduling and dynamic allocation and reallocation in aggravated form.3.5 Moreover, the extensive prior experience in, for example, message-switching systems, is likely to be of relatively little benefit in the interactive system network.3.6
In particular, the practical problems of planning for true network systems in the areas of documentation and library services have scarcely begun to be attacked.3.7 Nevertheless, the development of computer-communications networks has begun to emerge as the result of some or all of the following factors: (1) Requirements for data acquisition and collec
tion from a number of remote locations.3.8 (2) Demands for services and facilities not readily
available in the potential user's immediate
locality. (3) Recognized needs to share data, programs and
subroutines, work loads, and system resources.3.9 In addition, various users may share the specialized facilities offered by one or more
of the other members of the network. 3.10 Similar requirements were considered by various major members of the aerospace industry as early as 1961, as follows:
3.1. Network Management and Control
Requirements Effective provisions for network management and control derive directly from the basic objectives and mission of the network to be established. First, there are the questions with respect to the potential users of the system such as the following: 1. What are the objectives of the system itself? Is
it to be a public system, free and accessible to all? 3.11 Is it to serve a spectrum of clientele interests, privileges, priorities, and different levels of need-to-know? Is it subject, in the provision of its services, to constraints of national security, constitutional rights (assurance of protection of the individual citizen's right to the security, among other things, of his “papers” from unreasonable searches and seizures), laws and regulations involving penalties for violation such as “Secrecy of Com
munications,” and copyright inhibitions? 2. What are the charging and pricing policies, if
any, to be assessed against different types of service, different types of clients, and
“6. What are the budgetary constraints and
financially allowable rate of development?” (Davis, 1968, p. 4-5).
The factors of geographical coverage, location and facilities of participants, and membership point to some of the distribution requirements, to be considered next.
3.2. Distribution Requirements
different priorities of service to the different
members of the clientele? 3.12 3. What different protections may be built into
the system for different contributors with varying degrees of requirements for restric
tions upon access to or use of their data? 3.13 4. What are the priority, precedence, and inter
rupt provisions required in terms of the
clientele? 3.14 Next are the questions, in terms of the potential client-market, of the location, accessibility, cost, volume of traffic, and scheduling allocations for some determinate number of remote terminals, user stations, and communication links.
Then there are the questions of the performance and technological characteristics required with respect to these terminals, stations, and links.3.15 Are the central system and the communication network both capable of handling, effectively simultaneously, the number of individual stations or links required? Does the communication system itself impose limitations on bandwidths available, data transmission rates, number of channels operable effectively in parallel? Are alternate transmission modes available in the event of channel usurpation or nonavailability for other reasons? Is effectively on-line responsiveness of the communication system linkages required and if so to what extent?
More generally, the following design and planning questions should be studied in depth if there is to be effective management and control: "1. What is the scope of the network?
a. Its geographical coverage
e. Required rate of development
a. Privacy requirements
c. System management programs “3. What are the network management and
a. Standardization 3.16
f. Cost of services
a. FCC regulations
f. Federal agency jurisdiction . . .
A major area of concern with respect to distribution requirements in information processing network planning is that of the question of the type and extent of centralization or decentralization of the various system functions. There is first the possibility of a single master, supervisory, and control processing center linked to many geographically dispersed satellite centers (which carry out varying degrees of preprocessing and postprocessing of the information handled by the central system) and terminals. Secondly, several interconnected but independent processors may interchange control and supervisory functions as workload and other considerations demand.3 Still another possibility is regional centralization such has been recommended for a national documentation network, for example.3.19
Different compromises in network and system design to meet distribution requirements are also obviously possible.3.20 However, a variety of special problems may arise with respect to distribution requirements when some of the network functions are decentralized.3.21
Then there is the question of whether or not the network is to be physically distributed that is, "the term 'distributed network' is best used to delineate those communications networks based on connecting each station to all adjacent stations, rather than to just a few switching points, as in a centralized network." (Baran, 1964, p. 5). This distribution requirement consideration is closely related to information flow analysis and planning, especially with respect to assurance of continuing productive operation when certain parts of the network are inoperative.3.22 It should be noted, moreover, that “solving the data base management problem has been beyond the state of the art.” (Dennis, 1968, p. 373).
3.3. Information Flow Requirements
In general, it may be concluded that "to determine the correct configuration, certain basic factors must be investigated. These factors generally relate to the information flow requirements and include the following: 1. The kind of information to be transmitted
through the communications network and the types of messages.