BFGS Algorithm Overview
The BFGS method is an iterative procedure that approximates the
inverse Hessian matrix H_k at each iteration. The update of the
current solution \mathbf{x}_k involves computing a search
direction and step length. The general steps of the BFGS algorithm are
as follows:
- Initialize \mathbf{x}_0 and choose an initial guess for the inverse Hessian approximation H_0 , typically H_0 = I (the identity matrix).
- For each iteration k , do the following:
- Compute the gradient at the current point: \nabla f(\mathbf{x}_k) .
- Compute the search direction:
\mathbf{p}_k = -H_k \nabla f(\mathbf{x}_k)
- Perform a line search to find an appropriate step size \alpha_k , which minimizes f(\mathbf{x}_k + \alpha_k \mathbf{p}_k) .
- Update the current point:
\mathbf{x}_{k+1} = \mathbf{x}_k + \alpha_k \mathbf{p}_k
- Compute the new gradient \nabla f(\mathbf{x}_{k+1}) .