Section
4:
Using Matrix Operations
111
However,
the
normal equations
are
very sensitive
to
rounding
errors. (Orthogonal factorization, discussed
on
page
113,
is
relatively insensitive
to
rounding errors.)
The
weighted least-squares problem
is a
generalization
of the
ordinary least-squares problem.
In it you
seek
to
minimize
n
HWr|||=
X>?r?
where
W is a
diagonal
n X n
matrix with positive diagonal
elements
wlt
w2,
...,
wn.
Then
||Wr|||
=
(y
-
Xb)rWrW(y
-
Xb)
and any
solution
b
also satisfies
the
weighted normal equations
These
are the
normal equations with
X and y
replaced
by WX and
Wy.
Consequentially,
these
equations
are
sensitive
to
rounding
errors also.
The
linearly constrained least-squares problem involves finding
b
such
that
it
minimizes
MIHIy-
subject
to the
constraints
Cb
= d
/ *
Z^A
V=i
=
dt
for i
—
1,
2,...,
ml .
This
is
equivalent
to
finding
a
solution
b to the
augmented normal
equations
XrX
CT
C
0
where
1, a
vector
of
Lagrange multipliers,
is
part
of the
solution
but
isn't
used further. Again,
the
augmented equations
are
very
sensitive
to
rounding errors. Note also
that
weights
can
also
be
included
by
replacing
X and y
with
WX and Wy.