Thus, while Simpson's and Lagrange's work had shown the arithmetic mean to be increasingly 'good' as n→ ∞, Bernoulli's and Laplace's work implied that the arithmetic mean was 'best' only in the limiting case of infinitely poor precision.

As noted above, Gauss discovered the great algebraic and arithmetical advantages of the technique of Least Sum of Squared Residuals in 1795. In 1797 he attempted to justify this procedure via the calculus of probabilities, concluding that determination of "most probable values" of unknown quantities is impossible unless the law of error is known explicitly. "When this is not the case, nothing remains but to assume such a function as an hypothesis. It seemed to him most natural to proceed first the other way around and to look for that function on which the whole theory should be based if for the simplest case there is to result the rule generally accepted as good, namely, that the arithmetic mean of several values obtained for the same unknown through observations of equal reliability is to be considered as the most probable value" (14, p. 98). By June 1798 (13, p. 113) he had completed his now famous 'proof' of the Method of Least Squares, in which he (a) adopted as a postulate the Principle of the Arithmetic Mean, (b) utilized the concept that repetition of a measurement process generates a probability distribution of errors, and (c) applied Bayes's method of inverse probability-without reference to Thomas Bayes (1702-1761). Starting from these premises he showed that if the arithmetic mean of n independent measurements of a single magnitude is to be the most probable value of this magnitude a posteriori, then the errors X1 = Y1-T of the individual measurements Y, must be distributed in accordance with the law of error

values of the essential parameters have uniform a priori distributions, then the most probable values of the unknown implied by a given set of observational data are given identically by the application of the technique of Least Sum of Squared Residuals. He did not publish these results, however, until 1809, in Book II, Section 3, of his Theory of the Motion of Heavenly Bodies Moving about the Sun in Sections (21).

Gauss was well aware that this derivation of his now famous law of error and consequent justification of the technique of Least Sum of Squared Residuals was merely an extension of the Principles of the Arithmetic Mean and stood or fell with this Principle. Thus, he remarked that the principle that "the most probable system of values of the unknown quantities [is that for which] the sum of the squares of the differences between the observed and computed values of the functions [observed] is a minimum . . . must, everywhere be considered an axiom with the same propriety as the arithmetical mean of several observed values of the same quantity is adopted as the most probable value" (21, art. 179). But his analysis of the Method of Least Squares remains notable because he recognized that "the constant h can be considered as a measure of the precision [praecisionis] of the observations" and then went on to give (1) the formula for the precision of a linear function independent observations of equal or unequal precisions, and (2) the rule for weighting results of unequal precision so as to obtain the combined result of maximum attainable precision. These are everlasting accomplishments of his first 'proof'.

Laplace greatly strengthened Gauss's first 'proof' almost immediately after its publication, by his discovery (22 pp. 383389) that, under certain very general conditions (not considered in full generality by Laplace) the distributions of linear functions, and hence of the arithmetic means, of n independent errors can be approximated (when properly scaled) by

Gauss's law of error (5), with the error of the approximation tending to zero as n→ ∞. From this it follows directly that the Method of Least Squares as developed by Gauss leads to 'most probable values' (under "very general conditions") when the number of independent observations involved is large. The Method of Least Squares was, therefore, regarded firmly established, not merely on grounds of algebraic and arithmetical convenience, but also via the calculus of probabilities at least when the number of independent observations is large!

as

IV. Minimum Errors of Estimation

and Gauss's Second 'Proof'

As noted above, Laplace suggested in 1774 (20) that the 'best mean' to take in practical astronomy is that function of the observations which has an equal probability of over- and under-estimating the true value, showed that this is equivalent to adopting the principle of Least Mean Absolute Error of Estimation, and gave an algorithm for finding this particular function of three observations in a one-parameter case. By this algorithm his 'best mean' is given by the abscissa T (y1, Y2, Y3) that divides the area under the curve f(y1-7)f(y27) f(y3-7), considered as a function of 7, into two equal halves, f(x) being the law of error involved. In 1778 (23), Laplace extended this agreement to the case of n independent observations and termed this procedure "the most advantageous method" of estimation. This

approach was invented anew and fully explored by E. J. G. Pitman in 1939 (24). Unfortunately, it usually leads to intractable equations for the "most advantageous" estimates, except for very special choices of the law of error. Thus, in 1811 (25), Laplace found that, among all laws of

servations is the "most advantageous" estimator of T.

By adopting instead the principle of Least Mean Squared Error of Estimation and the requirement that the resulting "best mean" should yield the true values of the quantities concerned if it should happen that all of the observations were entirely free from error, Gauss showed in 1821-23 (26, 27) that, when the resulting 'best values' are linear functions of the observations, then they are identically the same as those given by the technique of Least Sum of Squared Residuals (which provides the practical modus operandi for obtaining them), and that in this important case the Least Mean Squared Error property is completely independent of the law of error involved. This fact, which mathematical statisticians today express by say. ing that the Method of Least Squares yields minimum variance linear unbiased estimators of the unknown magnitudes concerned under "very general conditions", is considered by many mathematical statisticians today to be the real theoretical basis of the Method of Least Squares. Henri Poincaré (1854-1912) remarked in 1893-94 (28, p. 168), "This approach justifies the [Method of Least Squares] independently of the law of errors is, thus, a refutation of Gauss's [earlier] reasoning [and] it is rather strange that this refutation is due to Gauss himself". And it is equally surprising that this bestlinear-unbiased-estimator property of Least Squares seems to be unknown to many

users of the Method of Least Squares today.

V. Concluding Remarks

The robust survival of the Method of Least Squares as a valuable tool of applied science no doubt stems in part from the algebraic and arithmetical advantages of Least Sum of Squared Residuals and in part from the fact this procedure also yields estimates of Least Mean Squared Error in the important case when the end results are linear functions of the basic observations. This one-to-one correspond

ence between minimizing some function of the residuals and minimizing the same function of Errors of Estimation appears to be a unique property of Least Squares. And although the Method of Least Squares does not lead to the best available estimates of unknown parameters when the law of error is other than the Gaussian, if the number of independent observations available is much larger than the number of parameters to be determined the Method of Least Squares can be usually counted on to yield nearly-best estimates.

References

(1) Russell, Bertrand. Human Knowledge: Its Scope and Limits. Simon and Schuster, New York, 1948, p. 344.

(2) Whittaker, E. T. and Robinson, G. The Calculus of Observations. Blackie & Son, Ltd., London, 1924. 2nd edition, 1932; 3rd edition, 1940.

(3) Legendre, Adrien Marie. Nouvelles méthodes pour la détermination des orbites des comètes. Paris, 1805. Appendix, 'Sur laMéthode des moindres quarrés', pp. 72-80. (English translation of pp. 72-75, by Henry A. Ruger and Helen M. Walker, in David Eugene Smith, A Source Book in Mathematics, McGraw-Hill Co., Inc., New York, 1929, pp. 576-579. Reprinted in 2 vols., Dover Publications, Inc., New York, 1959.)

(4) Cotes, Roger. Aestimatio errorum in mixta mathesi, per variationes partium trianguli plani et spherici, Opera Miscellania (appended to his Harmonia Mensurarum, Cantabrigiae, 1722), pp. 1-22.

(5) Euler, Leonhard. Pièce qui a remporté le prix de l'Academie royale des sciences en 1748, sur les inégalités du mouvement de Saturne et de Jupiter. Paris, 1749.

(6) Mayer, Johann Tobias. Abhandlung über die Umwälzung des Mondes um seine Axe, Kosmographische Nachrichten und Sammlungen, Vol. I (1748), pp. 52-183 (published 1750).

(7) Boscovich, Roger, Joseph. De Litteraria Expeditione per Pontificiam ditionem, et Synopsis amplioris Operis, ac habentur plura ejus cx exemplaria etiam sensorum impressa, Bononiensi Scientiarum et Artum Instituto Atque Academia Commentarii, Tomus IV, pp. 353-96, 1757.

(8) Stay, Benedict. Philosophiae Recentioris, a Benedicto Stay in Romano Archigynasis Publico Eloquentare Professore, versibus traditae, Libri X, cum adnotationibus et Supplementis P. Rogerii Josephi Boscovich S.J., Tomus II, Romae, 1760.

(9) Eisenhart, Churchill. Boscovich and the Combination of Observations. Chapter 7 in R. J.

Boscovich, F.R.S.: Studies of His Life and Work, edited by Lancelot Law Whyte, Allen and Unwin, Ltd., London, 1961.

(10) Simon, Pierre, Marquis de Laplace. Mémoire sur la Figure de la Terre. Mémoires de l'Académie royale des Sciences de Paris, pour Pannée 1783, pp. 17-46, Paris, 1786. Reprinted in Oeuvres de Laplace, National Edition, Vol. 11, Gauthier-Villars, Paris, 1895.

(11) Laplace. Sur les degrés mesurés des méridiens, et sur les longueurs observées sur pendule, Histoire de l'Académie royale des inscriptions et belles lettres, avec les Mémoires de littérature tirées des registres de cette académie, Année 1789, 18-43 of the Memoires. Paris, 1792. (12) Laplace. Traité de Mécanique Céleste, Vol. II. Paris, 1799. Reprinted as Vol. II of Oeuvres de Laplace, Paris, 1843; National Edition, Gauthier-Villars, Paris, 1878; English translation, with notes and commentary, by Nathaniel Bowditch, Boston, 1832.

(13) Dunnington, Guy Waldo. Carl Friedrich Gauss: Titan of Science. Hafner Publishing Co., New York, 1955.

(14) Gauss, Carl Friedrich. Summary of his paper "Theoria combinationis observationum erroribus minimis obnoxiae, pars prior." Göttingische gelehrte Anzeigen, February 26, 1821; Gauss Werke, vol. IV, p. 96-100.

(15) Simpson, Thomas. A letter to the Right Honourable George Earl of Macclesfield, President of the Royal Society, on the advantage of taking the mean of a number of observations, in practical astronomy, Phil. Trans. Roy. Soc. London, Vol. 49, Part I, pp. 82-93, 1755.

(16) Demoivre, Abraham. The Doctrine of Chances: or a Method of Calculating the Probability of Events in Play. London, 1718 (2nd ed., 1738; 3rd ed., 1756).

(17) Simpson. An attempt to show the advantage arising by taking the mean of a number of observations in practical astronomy. In his Miscellaneous Tracts on Some Curious and Very Interesting Subjects in Mechanics, PhysicalAstronomy, and Speculative Mathematics, pp. 64-75. J. Nourse, London, 1757.

(18) Joseph-Louis, Comte de Lagrange. Mémoire sur l'utilité de la méthode de prendre le milieu entre les résultats de plusieurs observations; dans lequel on examine les avantages de cette méthode par le calcul des probabi.ités; et où l'on résoud différents problèmes relatifs à cette matière. Miscellanea Taurinensia, Vol. 5 (1770-1773), pp. 167-232 of Mathematics portion. Reprinted in Oeuvres de Lagrange, Vol. 2, pp. 173-234. Gauthier-Villars, Paris, 1868.

(19) Bernoulli, Daniel. Dijudicatio maxime probabilis plurium observationum discrepantium atque verisimillima inductio inde formanda. Acta Academiae Scientiorum Petropolitanae, Vol. I (1777), Pt. I, pp. 3-23 of the Memoirs. St.

Petersburg, 1778; English translation by C. G. Allen, Biometrika, Vol. 48, Pts. 1 and 2 (June 1961), pp. 3-13.

(20) Laplace. Problème III: Déterminer le milieu que l'on doit prendre entre trois observations données d'un même phénomène. Pp. 634644 of his "Mémoire sur la probabilité des causes par les événements", Mémoires de Mathématique et de Physique, Vol. 6, pp. 621657, Paris, 1774.

(21) Gauss. Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium. Frid. Perthes et I. H. Besser, Hamburg, 1809; Reprinted in Carl Friedrich Gauss Werke, Band VII. Gotha, 1871. English translation by Charles Henry Davis, Boston, 1857; reprinted by Dover Publications, Inc., New York, 1963.

(22) Laplace. Mémoire sur les approximations des formu.es qui sont fonctions de trèsgrands nombres, et sur leur application aux probabilités, Mémoires de la classe des Sciences Mathématiques et Physiques, de l'Institut de France Année 1809, pp. 353-415; Supplement pp. 559565. Paris, 1810.

(23) Laplace. Mémoire sur les Probabilités, Histoire de l'Académie royale des Sciences de

Paris, Annee 1778, pp. 227-332. Paris, 1781.

(24) Pitman, E. J. G. The estimation of the location and scale parameters of a continuous population of any given form, Biometrika, Vol. XXX, Parts 3 and 4 (Jan. 1939), pp. 391-421.

(25) Laplace. Mémoire sur les intégrales définies et leur application aux probabilités, et spécialement à la recherche du milieu qu'il faut choisir entre les résultats des observations, Mémoires de la Classe des Sciences Mathéma tiques et Physiques de l'Institut Impérial de France, Année 1810, pp. 279-347. Paris, 1811.

(26) Gauss. Theoria combinationis observationum erroribus minimis obnoxiae. Pars prior. [presented 15 Feb. 1821.] Commentationes societas regiae scientiarum Göttingensis recentiores, Vol. V, pp. 33-62, 1823. Reprinted in Carl Friedrich Gauss Werke, Vol. IV, pp. 1-26. Göttingen, 1873.