Define the concept of steepest descent and show how to compute the optimal stepsize
|
Choose
such that
is minimized i.e.
Setting the above expression equal to zero gives the optimal
:
Note that since
is symmetric
Formulate the steepest descent (or gradient method) method and write a pseudocode which implements it.
|
Note that
. Then the minimal
is given by
Given
For
Since
is symmetric, positive definite,
where
is upper triangular (Cholesky Factorization).
Then
Hence,
is symmetric:
since
symmetric
is positive definite:
since
positive definite
Given
For