Minimisation function.
Arguments
~~~~~~~~~
The arguments, which should all be strings, specify the minimiser as well as its options. A
minimum of one argument is required. As this calls the minfx function 'generic_minimise'
the full list of allowed arguments is shown below in the reproduced 'generic_minimise'
docstring. Ignore all sections except those labelled as minimisation algorithms and
minimisation options. Also do not select the Method of Multipliers constraint algorithm as
this is used in combination with the given minimisation algorithm if the keyword argument
'constraints' is set to 1. The grid search algorithm should also not be selected as this is
accessed using the 'grid' function instead. The first argument passed will be set to the
minimisation algorithm while all other arguments will be set to the minimisation options.
Keyword arguments differ from normal arguments having the form 'keyword = value'. All
arguments must precede keyword arguments in python. For more information see the examples
section below or the python tutorial.
Keyword Arguments
~~~~~~~~~~~~~~~~~
func_tol: The function tolerance. This is used to terminate minimisation once the function
value between iterations is less than the tolerance. The default value is 1e-25.
grad_tol: The gradient tolerance. Minimisation is terminated if the current gradient value
is less than the tolerance. The default value is None.
max_iterations: The maximum number of iterations. The default value is 1e7.
constraints: A boolean flag specifying whether the parameters should be constrained. The
default is to turn constraints on (constraints=True).
scaling: The diagonal scaling boolean flag. The default that scaling is on (scaling=True).
verbosity: The amount of information to print to screen. Zero corresponds to minimal
output while higher values increase the amount of output. The default value is 1.
Diagonal scaling
~~~~~~~~~~~~~~~~
Diagonal scaling is the transformation of parameter values such that each value has a
similar order of magnitude. Certain minimisation techniques, for example the trust region
methods, perform extremely poorly with badly scaled problems. In addition, methods which
are insensitive to scaling such as Newton minimisation may still benefit due to the
minimisation of round off errors.
In Model-free analysis for example, if S2 = 0.5, te = 200 ps, and Rex = 15 1/s at 600 MHz,
the unscaled parameter vector would be [0.5, 2.0e-10, 1.055e-18]. Rex is divided by
(2 * pi * 600,000,000)**2 to make it field strength independent. The scaling vector for
this model may be something like [1.0, 1e-9, 1/(2 * pi * 6e8)**2]. By dividing the unscaled
parameter vector by the scaling vector the scaled parameter vector is [0.5, 0.2, 15.0]. To
revert to the original unscaled parameter vector, the scaled parameter vector and scaling
vector are multiplied.
Examples
~~~~~~~~
To apply Newton minimisation together with the GMW81 Hessian modification algorithm, the
More and Thuente line search algorithm, a function tolerance of 1e-25, no gradient
tolerance, a maximum of 10,000,000 iterations, constraints turned on to limit parameter
values, and have normal printout, type any combination of:
relax> minimise('newton')
relax> minimise('Newton')
relax> minimise('newton', 'gmw')
relax> minimise('newton', 'mt')
relax> minimise('newton', 'gmw', 'mt')
relax> minimise('newton', 'mt', 'gmw')
relax> minimise('newton', func_tol=1e-25)
relax> minimise('newton', func_tol=1e-25, grad_tol=None)
relax> minimise('newton', max_iter=1e7)
relax> minimise('newton', constraints=True, max_iter=1e7)
relax> minimise('newton', verbosity=1)
To use constrained Simplex minimisation with a maximum of 5000 iterations, type:
relax> minimise('simplex', constraints=True, max_iter=5000)
Note
~~~~
All the text which follows is a reproduction of the docstring of the generic_minimise
function from the minfx python package. Only take note of the minimisation algorithms and
minimisation options sections, the other sections are not relevant for this function. The
Grid search and Method of Multipliers algorithms CANNOT be selected as minimisation
algorithms for this function.
The section entitled Keyword Arguments is also completely inaccessible therefore please
ignore that text.
Generic minimisation function.
This is a generic function which can be used to access all minimisers using the same set of function arguments. These are the function tolerance value for convergence tests, the maximum number of iterations, a flag specifying which data structures should be returned, and a flag specifying the amount of detail to print to screen.
Minimisation output
===================
The following values of the 'full_output' flag will return, in tuple form, the following data:
- 0: 'xk',
- 1: '(xk, fk, k, f_count, g_count, h_count, warning)',
where the data names correspond to:
- 'xk': The array of minimised parameter values,
- 'fk': The minimised function value,
- 'k': The number of iterations,
- 'f_count': The number of function calls,
- 'g_count': The number of gradient calls,
- 'h_count': The number of Hessian calls,
- 'warning': The warning string.
Minimisation algorithms
=======================
A minimisation function is selected if the minimisation algorithm argument, which should be a string, matches a certain pattern. Because the python regular expression 'match' statement is used, various strings can be supplied to select the same minimisation algorithm. Below is a list of the minimisation algorithms available together with the corresponding patterns.
This is a short description of python regular expression, for more information, see the regular expression syntax section of the Python Library Reference. Some of the regular expression syntax used in this function is:
- '[]': A sequence or set of characters to match to a single character. For example, '[Nn]ewton' will match both 'Newton' and 'newton'.
- '^': Match the start of the string.
- '$': Match the end of the string. For example, '^[Ll][Mm]$' will match 'lm' and 'LM' but will not match if characters are placed either before or after these strings.
To select a minimisation algorithm, set the argument to a string which matches the given pattern.
Unconstrained line search methods::
___________________________________________________________________________________________
| | |
| Minimisation algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Back-and-forth coordinate descent | '^[Cc][Dd]$' or '^[Cc]oordinate[ _-][Dd]escent$' |
| | |
| Steepest descent | '^[Ss][Dd]$' or '^[Ss]teepest[ _-][Dd]escent$' |
| | |
| Quasi-Newton BFGS | '^[Bb][Ff][Gg][Ss]$' |
| | |
| Newton | '^[Nn]ewton$' |
| | |
| Newton-CG | '^[Nn]ewton[ _-][Cc][Gg]$' or '^[Nn][Cc][Gg]$' |
|___________________________________|_____________________________________________________|
Unconstrained trust-region methods::
___________________________________________________________________________________________
| | |
| Minimisation algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Cauchy point | '^[Cc]auchy' |
| | |
| Dogleg | '^[Dd]ogleg' |
| | |
| CG-Steihaug | '^[Cc][Gg][-_ ][Ss]teihaug' or '^[Ss]teihaug' |
| | |
| Exact trust region | '^[Ee]xact' |
|___________________________________|_____________________________________________________|
Unconstrained conjugate gradient methods::
___________________________________________________________________________________________
| | |
| Minimisation algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Fletcher-Reeves | '^[Ff][Rr]$' or '^[Ff]letcher[-_ ][Rr]eeves$' |
| | |
| Polak-Ribiere | '^[Pp][Rr]$' or '^[Pp]olak[-_ ][Rr]ibiere$' |
| | |
| Polak-Ribiere + | '^[Pp][Rr]\+$' or '^[Pp]olak[-_ ][Rr]ibiere\+$' |
| | |
| Hestenes-Stiefel | '^[Hh][Ss]$' or '^[Hh]estenes[-_ ][Ss]tiefel$' |
|___________________________________|_____________________________________________________|
Miscellaneous unconstrained methods::
___________________________________________________________________________________________
| | |
| Minimisation algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Simplex | '^[Ss]implex$' |
| | |
| Levenberg-Marquardt | '^[Ll][Mm]$' or '^[Ll]evenburg-[Mm]arquardt$' |
|___________________________________|_____________________________________________________|
Constrained methods::
___________________________________________________________________________________________
| | |
| Minimisation algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Method of Multipliers | '^[Mm][Oo][Mm]$' or '[Mm]ethod of [Mm]ultipliers$' |
| | |
| Logarithmic barrier function | 'Log barrier' |
|___________________________________|_____________________________________________________|
Global minimisation methods::
___________________________________________________________________________________________
| | |
| Minimisation algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Simulated Annealing | '^[Ss][Aa]$' or '^[Ss]imulated [Aa]nnealing$' |
|___________________________________|_____________________________________________________|
Minimisation options
====================
The minimisation options can be given in any order.
Line search algorithms. These are used in the line search methods and the conjugate gradient methods. The default is the Backtracking line search. The algorithms are::
___________________________________________________________________________________________
| | |
| Line search algorithm | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Backtracking line search | '^[Bb]ack' |
| | |
| Nocedal and Wright interpolation | '^[Nn][Ww][Ii]' or |
| based line search | '^[Nn]ocedal[ _][Ww]right[ _][Ii]nt' |
| | |
| Nocedal and Wright line search | '^[Nn][Ww][Ww]' or |
| for the Wolfe conditions | '^[Nn]ocedal[ _][Ww]right[ _][Ww]olfe' |
| | |
| More and Thuente line search | '^[Mm][Tt]' or '^[Mm]ore[ _][Tt]huente$' |
| | |
| No line search | '^[Nn]o [Ll]ine [Ss]earch$' |
|___________________________________|_____________________________________________________|
Hessian modifications. These are used in the Newton, Dogleg, and Exact trust region algorithms::
___________________________________________________________________________________________
| | |
| Hessian modification | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Unmodified Hessian | '^[Nn]o [Hh]essian [Mm]od' |
| | |
| Eigenvalue modification | '^[Ee]igen' |
| | |
| Cholesky with added multiple of | '^[Cc]hol' |
| the identity | |
| | |
| The Gill, Murray, and Wright | '^[Gg][Mm][Ww]$' |
| modified Cholesky algorithm | |
| | |
| The Schnabel and Eskow 1999 | '^[Ss][Ee]99' |
| algorithm | |
|___________________________________|_____________________________________________________|
Hessian type, these are used in a few of the trust region methods including the Dogleg and Exact trust region algorithms. In these cases, when the Hessian type is set to Newton, a Hessian modification can also be supplied as above. The default Hessian type is Newton, and the default Hessian modification when Newton is selected is the GMW algorithm::
___________________________________________________________________________________________
| | |
| Hessian type | Patterns |
|___________________________________|_____________________________________________________|
| | |
| Quasi-Newton BFGS | '^[Bb][Ff][Gg][Ss]$' |
| | |
| Newton | '^[Nn]ewton$' |
|___________________________________|_____________________________________________________|
For Newton minimisation, the default line search algorithm is the More and Thuente line search, while the default Hessian modification is the GMW algorithm.
@keyword func: The function which returns the value.
@type func: func
@keyword dfunc: The function which returns the gradient.
@type dfunc: func
@keyword d2func: The function which returns the Hessian.
@type d2func: func
@keyword args: The tuple of arguments to supply to the functions func, dfunc, and d2func.
@type args: tuple
@keyword x0: The vector of initial parameter value estimates (as an array).
@type x0: numpy rank-1 array
@keyword min_algor: A string specifying which minimisation technique to use.
@type min_algor: str
@keyword min_options: A tuple to pass to the minimisation function as the min_options keyword.
@type min_options: tuple
@keyword func_tol: The function tolerance value. Once the function value between iterations decreases below this value, minimisation is terminated.
@type func_tol: float
@keyword grad_tol: The gradient tolerance value.
@type grad_tol: float
@keyword maxiter: The maximum number of iterations.
@type maxiter: int
@keyword A: Linear constraint matrix m*n (A.x >= b).
@type A: numpy rank-2 array
@keyword b: Linear constraint scalar vector (A.x >= b).
@type b: numpy rank-1 array
@keyword l: Lower bound constraint vector (l <= x <= u).
@type l: numpy rank-1 array
@keyword u: Upper bound constraint vector (l <= x <= u).
@type u: numpy rank-1 array
@keyword c: User supplied constraint function.
@type c: func
@keyword dc: User supplied constraint gradient function.
@type dc: func
@keyword d2c: User supplied constraint Hessian function.
@type d2c: func
@keyword print_flag: A flag specifying how much information should be printed to standard output during minimisation. 0 means no output, 1 means minimal output, and values above 1 increase the amount of output printed.
@type print_flag: int
@keyword print_prefix: The text to add out to the front of all print outs.
@type print_prefix: str
@keyword full_output: A flag specifying which data structures should be returned.
@type full_output: bool
|
