-
let us consider a problem of minimizing a objective function
with two variables x and y given by
f(x,y) = x - y + 2x2 + 2xy + y2 and start
with initial values X1 = [ 0   0]
grad f = [ 1 + 4x + 2y    -1 + 2x +2y]
grad f(X1) = [  1  -1]
S1 = - grad f(X1) =
[ -1   1]
-
to find X2 we need to find the optimal step
length l1. To find l1 we need minimize
f(X1 + l1S1)
= f(-l1,l1) = 0 we get
l1 = 1 and
X2 = X1 +
l1S1 = [  -1   1]
since grad f(X2) = [ -1   -1] is not
equal to zero we proceed to the next iteration
-
step 2:
S2 = [  1   1]
l2 = 0.2
X3 = X2 + l2
S2 = [  -0.8   1.2]
-
step 3:
S3 = [  1   1]
l3 = 0.2
X4 = X3 + l3
S3 = [  -1.0   1.4]
the process is continued till grad f becomes zero for a
particular value of Xi