Least Squares

<< LinearAlgebra`MatrixManipulation`
<<LinearAlgebra`Orthogonalization`

Part 1.  An example: Data on cancer deaths

Answer the questions in Part 1 here.

Part 2.  Modeling the cancer death data

Enter the vectors 1, x, and y.

one  =  {1,  1,  1,  1,  1,  1,  1,  1,  1}
x  =  {2.5, 2.6, 3.4, 1.3, 1.6, 3.8, 11.6, 6.4, 8.3}
y  =  {147, 130, 130, 114, 138, 162, 208, 178, 210}

Construct a matrix U whose columns are an orthonormal basis for Span(1, x).

Z = 
        Transpose[
          GramSchmidt[{one, x},  
            Normalized -> False]];
Z // MatrixForm
z1 = Transpose[Z][[1]];
z2 = Transpose[Z][[2]];
Norm[x_] := Sqrt[Dot[x, x]];
u1 = z1/Norm[z1];
u2 = z2/Norm[z2];
U = Transpose[{u1, u2}]; U // MatrixForm

Find the projection of y on the span of the vectors 1 and x.

Part 3.  The normal equations

Construct the matrix X.

X  =  Transpose[{one,  x}];   X // MatrixForm

Find the coefficient matrix and constant vector for the normal equations.

Solve the normal equations for coefficients b and m for the least squares line y = m x + b.

Confirm that your coefficients give the same solution as the projection of y on Col(X).

Part 4.  Summary