Least Square Estimation and Orthogonality

YAP Von Bing

Department of Statistics and Data Science
National University of Singapore
Singapore

Published in The Mathematician Educator, 2023, Vol 4, No. 1, pp 55-65.

Abstract

Two measurement problems give rise to intractable systems of linear equations. By making a heuristic assumption on the measurement errors, least square estimates emerge as approximate solutions. An elegant generalisation to more complicated problems is afforded by the theory of Euclidean space, particularly orthogonal projection. This is an attempt to make the connection explicit. Orthogonality still plays a crucial role in the current, statistical view on least square estimation, which lies in the heart of the ubiquitous technique of linear regression.

Download pdf full text