Purple::GradientDescent Class Reference

Inheritance diagram for Purple::GradientDescent:

List of all members.

Public Types
	GoldenSection
	BrentMethod
enum	OptimalStepSizeMethod { GoldenSection, BrentMethod }
Public Member Functions
	GradientDescent (ObjectiveFunction *)
	GradientDescent (void)
virtual	~GradientDescent (void)
Vector< double >	getInitialArgument (void)
OptimalStepSizeMethod	getOptimalStepSizeMethod (void)
double	getFirstStepSize (void)
double	getOptimalStepSizeTolerance (void)
double	getGradientNormGoal (void)
int	getMaximumNumberOfIterations (void)
int	getShowPeriod (void)
double	getWarningStepSize (void)
void	setInitialArgument (Vector< double >)
void	setOptimalStepSizeMethod (OptimalStepSizeMethod)
void	setFirstStepSize (double)
void	setOptimalStepSizeTolerance (double)
void	setGradientNormGoal (double)
void	setMaximumNumberOfIterations (int)
void	setShowPeriod (int)
void	setWarningStepSize (double)
Vector< double >	getMinimalArgument (void)
void	print (void)
void	load (char *)
void	save (char *)
void	saveOptimizationHistory (char *)
Private Member Functions
double	getGoldenSectionOptimalStepSize (double, double, Vector< double >, Vector< double >)
double	getBrentMethodOptimalStepSize (double, double, Vector< double >, Vector< double >)
double	getMinimum (Vector< double >)
double	getMaximum (Vector< double >)
Private Attributes
Vector< double >	initialArgument
double	gradientNormGoal
int	maximumNumberOfIterations
int	showPeriod
double	firstStepSize
double	optimalStepSizeTolerance
double	warningStepSize
Vector< double >	evaluationHistory
Vector< double >	gradientNormHistory
OptimalStepSizeMethod	optimalStepSizeMethod

---

Detailed Description

This concrete class represents the gradient descent optimization algorithm for an objective function.

See also:

ObjectiveFunction.

OptimizationAlgorithm.

Definition at line 29 of file GradientDescent.h.

---

Member Enumeration Documentation

enum Purple::GradientDescent::OptimalStepSizeMethod

Available optimization operators for obtaining the optimal step size.

Enumerator:

GoldenSection
BrentMethod

Definition at line 38 of file GradientDescent.h.

---

Constructor & Destructor Documentation

Purple::GradientDescent::GradientDescent

(

ObjectiveFunction *

newObjectiveFunction

)

General constructor. It creates a gradient descent object associated to an objective function object. It also initializes the class members to their default values:

Initial argument: Random point whithin the objective function domain.

Optimization operators:

• Optimal step size method = Brent method;

Optimization parameters:

• First step size: 1.0e-3.
• Optimal step size tolerance: 1.0e-3.

Stopping criteria:

• Evaluation goal: -1.0e69.
• Gradient norm goal: 0.0.
• Maximum time: 1.0e6.
• Maximum number of iterations: 1000.

User stuff:

• Warning step size: 1000.
• Show period: 25.

Parameters:

newObjectiveFunction,:

Pointer to an objective function object.

See also:

ObjectiveFunction.

OptimizationAlgorithm.

Definition at line 62 of file GradientDescent.cpp.

Purple::GradientDescent::GradientDescent

(

void

)

Default constructor. It creates a gradient descent optimization algorithm object not associated to any objective function object. It also initializes the class members to their default values:

Optimization operators:

• Optimal step size method = Brent method;

Optimization parameters:

• First step size: 1.0e-3.
• Optimal step size tolerance: 1.0e-3.

Stopping criteria:

• Evaluation goal: -1.0e69.
• Gradient norm goal: 0.0.
• Maximum time: 1.0e6.
• Maximum number of iterations: 1000.

User stuff:

• Warning step size: 1000.
• Show period: 25.

See also:

OptimizationAlgorithm.

Definition at line 138 of file GradientDescent.cpp.

Purple::GradientDescent::~GradientDescent

(

void

)

[virtual]

Destructor.

Definition at line 167 of file GradientDescent.cpp.

---

Member Function Documentation

double Purple::GradientDescent::getGoldenSectionOptimalStepSize	(	double	initialStepSize,
		double	evaluation,
		Vector< double >	argument,
		Vector< double >	searchDirection
	)			[private]

This method returns the optimal step size by searching in a given direction to locate the minimum of the objective function in that direction. It uses the golden section method.

Parameters:

	initialStepSize,:	Initial step size in line search.
	evaluation,:	Objective function evaluation value.
	argument,:	Objective function argument Vector.
	searchDirection,:	Search direction Vector.

See also:

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 508 of file GradientDescent.cpp.

double Purple::GradientDescent::getBrentMethodOptimalStepSize	(	double	initialStepSize,
		double	evaluation,
		Vector< double >	argument,
		Vector< double >	searchDirection
	)			[private]

This method returns the optimal step size by searching in a given direction to locate the minimum of the objective function in that direction. It uses the Brent's method.

Parameters:

	initialStepSize,:	Initial step size in line search.
	evaluation,:	Objective function evaluation value.
	argument,:	Objective function argument Vector.
	searchDirection,:	Search direction Vector.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 728 of file GradientDescent.cpp.

double Purple::GradientDescent::getMinimum

(

Vector< double >

v

)

[private]

This method returns the minimum element in a Vector of double precision numbers.

Parameters:

v,:

Vector of double precision numbers.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>)

Definition at line 1780 of file GradientDescent.cpp.

double Purple::GradientDescent::getMaximum

(

Vector< double >

v

)

[private]

This method returns the maximum element in a Vector of double precision numbers.

Parameters:

v,:

Vector of double precision numbers.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>)

Definition at line 1806 of file GradientDescent.cpp.

Vector< double > Purple::GradientDescent::getInitialArgument

(

void

)

This method returns the initial objective function argument to be used by the gradient descent method for optimization.

Definition at line 194 of file GradientDescent.cpp.

GradientDescent::OptimalStepSizeMethod Purple::GradientDescent::getOptimalStepSizeMethod

(

void

)

This method returns the search direction method used for optimization.

See also:

getFletcherReevesSearchDirection(Vector<double>, Vector<double>, Vector<double>).

getPolakRibiereSearchDirection(Vector<double>, Vector<double>, Vector<double>).

Definition at line 183 of file GradientDescent.cpp.

double Purple::GradientDescent::getFirstStepSize

(

void

)

This method returns the initial step size for line search in the first iteration of the gradient descent.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 232 of file GradientDescent.cpp.

double Purple::GradientDescent::getOptimalStepSizeTolerance

(

void

)

This method returns the tolerance value in line search.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 245 of file GradientDescent.cpp.

double Purple::GradientDescent::getGradientNormGoal

(

void

)

This method returns the goal value for the norm of the objective function gradient. This is used as a stopping criterium when optimizing a function.

Definition at line 206 of file GradientDescent.cpp.

int Purple::GradientDescent::getMaximumNumberOfIterations

(

void

)

This method returns the maximum number of iterations to be performed by the gradient descent method during the optimization process. This is used as a stopping criterium when optimizing an objective function.

Definition at line 218 of file GradientDescent.cpp.

int Purple::GradientDescent::getShowPeriod

(

void

)

This method returns the number of iterations between the optimization showing progress.

Definition at line 270 of file GradientDescent.cpp.

double Purple::GradientDescent::getWarningStepSize

(

void

)

This method returns the step size value at wich a warning message is written to the screen during line search.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 259 of file GradientDescent.cpp.

void Purple::GradientDescent::setInitialArgument

(

Vector< double >

newInitialArgument

)

This method sets a new initial objective function argument to be used by the gradient descent method for optimization.

Parameters:

newInitialArgument,:

Initial argument Vector.

Definition at line 300 of file GradientDescent.cpp.

void Purple::GradientDescent::setOptimalStepSizeMethod

(

GradientDescent::OptimalStepSizeMethod

newOptimalStepSizeMethod

)

This method sets a new optimal step size method to be used for optimization with the gradient descent method.

Parameters:

newOptimalStepSizeMethod,:

Optimal step size method.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 287 of file GradientDescent.cpp.

void Purple::GradientDescent::setFirstStepSize

(

double

newFirstStepSize

)

This method sets a new initial step size for line search in the first iteration of the gradient descent.

Parameters:

newFirstStepSize,:

First step size.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 414 of file GradientDescent.cpp.

void Purple::GradientDescent::setOptimalStepSizeTolerance

(

double

newOptimalStepSizeTolerance

)

This method sets a new tolerance value to be used in line minimization.

Parameters:

newOptimalStepSizeTolerance,:

Line search tolerance.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 443 of file GradientDescent.cpp.

void Purple::GradientDescent::setGradientNormGoal

(

double

newGradientNormGoal

)

This method sets a new the goal value for the norm of the objective function gradient. This is used as a stopping criterium when optimizing an objective function.

Parameters:

newGradientNormGoal,:

Goal value for the norm of the objective function gradient.

Definition at line 331 of file GradientDescent.cpp.

void Purple::GradientDescent::setMaximumNumberOfIterations

(

int

newMaximumNumberOfIterations

)

This method sets a new maximum number of iterations in the optimization process.

Parameters:

newMaximumNumberOfIterations,:

Maximum number of iterations.

Definition at line 358 of file GradientDescent.cpp.

void Purple::GradientDescent::setShowPeriod

(

int

newShowPeriod

)

This method sets a new number of iterations between the optimization showing progress.

Parameters:

newShowPeriod,:

Show period.

Definition at line 385 of file GradientDescent.cpp.

void Purple::GradientDescent::setWarningStepSize

(

double

newWarningStepSize

)

This method sets a new step size value at wich a warning message is written to the screen during line minimization.

Parameters:

newWarningStepSize,:

Warning step size value.

See also:

getGoldenSectionOptimalStepSize(double, double, Vector<double>, Vector<double>).

getBrentMethodOptimalStepSize(double, double, Vector<double>, Vector<double>).

Definition at line 473 of file GradientDescent.cpp.

Vector< double > Purple::GradientDescent::getMinimalArgument

(

void

)

[virtual]

This method optimizes an objective function according to the gradient descent algorithm. It returns the minimal argument of the objective function. Optimization occurs according to the optimization operators and the optimization parameters.

Implements Purple::OptimizationAlgorithm.

Definition at line 1031 of file GradientDescent.cpp.

void Purple::GradientDescent::print

(

void

)

This method prints to the screen the optimization operators and the optimization parameters concerning the gradient descent object:

Initial argument.

Optimization operators:

• Optimal step size method.

Optimization parameters:

• First step size.
• Optimal step size tolerance.

Stopping criteria:

• Evaluation goal.
• Gradient norm goal.
• Maximum time.
• Maximum number of iterations.

User stuff:

• Warning step size.
• Show period.

Definition at line 1349 of file GradientDescent.cpp.

void Purple::GradientDescent::load

(

char *

filename

)

This method loads a gradient descent object from a data file. Please mind about the file format, wich is specified in the User's Guide.

Initial argument.

Optimization operators:

• Optimal step size method.

Optimization parameters:

• First step size.
• Optimal step size tolerance.

Stopping criteria:

• Evaluation goal.
• Gradient norm goal.
• Maximum time.
• Maximum number of iterations.

User stuff:

• Warning step size.
• Show period.

Parameters:

filename,:

Filename.

See also:

save(char*).

Definition at line 1592 of file GradientDescent.cpp.

void Purple::GradientDescent::save

(

char *

filename

)

This method saves the gradient descent object to a data file.

Initial argument.

Optimization operators:

• Optimal step size method.

Optimization parameters:

• First step size.
• Optimal step size tolerance.

Stopping criteria:

• Evaluation goal.
• Gradient norm goal.
• Maximum time.
• Maximum number of iterations.

User stuff:

• Warning step size.
• Show period.

Parameters:

filename,:

Filename.

See also:

load(char*).

Definition at line 1462 of file GradientDescent.cpp.

void Purple::GradientDescent::saveOptimizationHistory

(

char *

filename

)

This method saves the optimization history to a data file:

1. Iteration.
2. Objective function evaluation.
3. Objective function gradient norm.

Parameters:

filename,:

Filename.

Definition at line 1722 of file GradientDescent.cpp.

---

Member Data Documentation

Vector<double> Purple::GradientDescent::initialArgument [private]

Initial argument.

Definition at line 45 of file GradientDescent.h.

double Purple::GradientDescent::gradientNormGoal [private]

Objective function gradient norm goal. It is used as a stopping criterion.

Definition at line 50 of file GradientDescent.h.

int Purple::GradientDescent::maximumNumberOfIterations [private]

Maximum number of iterations. It is used as a stopping criterion.

Definition at line 55 of file GradientDescent.h.

int Purple::GradientDescent::showPeriod [private]

Number of iterations between the optimization showing progress.

Definition at line 59 of file GradientDescent.h.

double Purple::GradientDescent::firstStepSize [private]

Inititial step size in line search for first iteration of gradient descent.

Definition at line 63 of file GradientDescent.h.

double Purple::GradientDescent::optimalStepSizeTolerance [private]

Tolerance for the optimal step size.

Definition at line 67 of file GradientDescent.h.

double Purple::GradientDescent::warningStepSize [private]

Step size value at wich a warning message is written to the screen.

Definition at line 71 of file GradientDescent.h.

Vector<double> Purple::GradientDescent::evaluationHistory [private]

Evaluation of objective function optimization history.

Definition at line 75 of file GradientDescent.h.

Vector<double> Purple::GradientDescent::gradientNormHistory [private]

Gradient norm of objective function optimization history.

Definition at line 79 of file GradientDescent.h.

OptimalStepSizeMethod Purple::GradientDescent::optimalStepSizeMethod [private]

Definition at line 90 of file GradientDescent.h.

---