Package minimise :: Module generic
[hide private]
[frames] | no frames]

Module generic

source code

Functions [hide private]
 
generic_minimise(func=None, dfunc=None, d2func=None, args=(), x0=None, min_algor=None, min_options=None, func_tol=1e-25, grad_tol=None, maxiter=1000000.0, A=None, b=None, l=None, u=None, c=None, dc=None, d2c=None, full_output=0, print_flag=0, print_prefix='')
Generic minimisation function.
source code

Imports: match, grid, coordinate_descent, steepest_descent, bfgs, newton, ncg, cauchy_point, dogleg, steihaug, exact_trust_region, fletcher_reeves, polak_ribiere, polak_ribiere_plus, hestenes_stiefel, simplex, levenberg_marquardt, method_of_multipliers


Function Details [hide private]

generic_minimise(func=None, dfunc=None, d2func=None, args=(), x0=None, min_algor=None, min_options=None, func_tol=1e-25, grad_tol=None, maxiter=1000000.0, A=None, b=None, l=None, u=None, c=None, dc=None, d2c=None, full_output=0, print_flag=0, print_prefix='')

source code 
Generic minimisation function.

This is a generic function which can be used to access all minimisers using the same set of
function arguments.  These are the function tolerance value for convergence tests, the maximum
number of iterations, a flag specifying which data structures should be returned, and a flag
specifying the amount of detail to print to screen.


Keyword Arguments
~~~~~~~~~~~~~~~~~

func:  The function which returns the value.

dfunc:  The function which returns the gradient.

d2func:  The function which returns the Hessian.

args:  The tuple of arguments to supply to the functions func, dfunc, and d2func.

x0:  The vector of initial parameter value estimates (as an array).

min_algor:  A string specifying which minimisation technique to use.

min_options:  A tuple to pass to the minimisation function as the min_options keyword.

func_tol:  The function tolerance value.  Once the function value between iterations decreases
below this value, minimisation is terminated.

grad_tol:  The gradient tolerance value.

maxiter:  The maximum number of iterations.

A:  Linear constraint matrix m*n (A.x >= b).

b:  Linear constraint scalar vector (A.x >= b).

l:  Lower bound constraint vector (l <= x <= u).

u:  Upper bound constraint vector (l <= x <= u).

c:  User supplied constraint function.

dc:  User supplied constraint gradient function.

d2c:  User supplied constraint Hessian function.

full_output:  A flag specifying which data structures should be returned.

print_flag:  A flag specifying how much information should be printed to standard output during
minimisation.  0 means no output, 1 means minimal output, and values above 1 increase the amount
of output printed.


Minimisation output
~~~~~~~~~~~~~~~~~~~

The following values of the 'full_output' flag will return, in tuple form, the following data

    0:  'xk',
    1:  '(xk, fk, k, f_count, g_count, h_count, warning)',

where the data names correspond to

    'xk':      The array of minimised parameter values,
    'fk':      The minimised function value,
    'k':       The number of iterations,
    'f_count': The number of function calls,
    'g_count': The number of gradient calls,
    'h_count': The number of Hessian calls,
    'warning': The warning string.


Minimisation algorithms
~~~~~~~~~~~~~~~~~~~~~~~

A minimisation function is selected if the minimisation algorithm argument, which should be a
string, matches a certain pattern.  Because the python regular expression 'match' statement is
used, various strings can be supplied to select the same minimisation algorithm.  Below is a
list of the minimisation algorithms available together with the corresponding patterns.

This is a short description of python regular expression, for more information, see the
regular expression syntax section of the Python Library Reference.  Some of the regular
expression syntax used in this function is:

    '[]':  A sequence or set of characters to match to a single character.  For example,
    '[Nn]ewton' will match both 'Newton' and 'newton'.

    '^':  Match the start of the string.

    '$':  Match the end of the string.  For example, '^[Ll][Mm]$' will match 'lm' and 'LM' but
    will not match if characters are placed either before or after these strings.

To select a minimisation algorithm, set the argument to a string which matches the given
pattern.


Parameter initialisation methods:
___________________________________________________________________________________________
|                                   |                                                     |
| Minimisation algorithm            | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Grid search                       | '^[Gg]rid'                                          |
|___________________________________|_____________________________________________________|


Unconstrained line search methods:
___________________________________________________________________________________________
|                                   |                                                     |
| Minimisation algorithm            | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Back-and-forth coordinate descent | '^[Cc][Dd]$' or '^[Cc]oordinate[ _-][Dd]escent$'    |
|                                   |                                                     |
| Steepest descent                  | '^[Ss][Dd]$' or '^[Ss]teepest[ _-][Dd]escent$'      |
|                                   |                                                     |
| Quasi-Newton BFGS                 | '^[Bb][Ff][Gg][Ss]$'                                |
|                                   |                                                     |
| Newton                            | '^[Nn]ewton$'                                       |
|                                   |                                                     |
| Newton-CG                         | '^[Nn]ewton[ _-][Cc][Gg]$' or '^[Nn][Cc][Gg]$'      |
|___________________________________|_____________________________________________________|


Unconstrained trust-region methods:
___________________________________________________________________________________________
|                                   |                                                     |
| Minimisation algorithm            | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Cauchy point                      | '^[Cc]auchy'                                        |
|                                   |                                                     |
| Dogleg                            | '^[Dd]ogleg'                                        |
|                                   |                                                     |
| CG-Steihaug                       | '^[Cc][Gg][-_ ][Ss]teihaug' or '^[Ss]teihaug'       |
|                                   |                                                     |
| Exact trust region                | '^[Ee]xact'                                         |
|___________________________________|_____________________________________________________|


Unconstrained conjugate gradient methods:
___________________________________________________________________________________________
|                                   |                                                     |
| Minimisation algorithm            | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Fletcher-Reeves                   | '^[Ff][Rr]$' or '^[Ff]letcher[-_ ][Rr]eeves$'       |
|                                   |                                                     |
| Polak-Ribiere                     | '^[Pp][Rr]$' or '^[Pp]olak[-_ ][Rr]ibiere$'         |
|                                   |                                                     |
| Polak-Ribiere +                   | '^[Pp][Rr]\+$' or '^[Pp]olak[-_ ][Rr]ibiere\+$'     |
|                                   |                                                     |
| Hestenes-Stiefel                  | '^[Hh][Ss]$' or '^[Hh]estenes[-_ ][Ss]tiefel$'      |
|___________________________________|_____________________________________________________|


Miscellaneous unconstrained methods:
___________________________________________________________________________________________
|                                   |                                                     |
| Minimisation algorithm            | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Simplex                           | '^[Ss]implex$'                                      |
|                                   |                                                     |
| Levenberg-Marquardt               | '^[Ll][Mm]$' or '^[Ll]evenburg-[Mm]arquardt$'       |
|___________________________________|_____________________________________________________|


Constrained methods:
___________________________________________________________________________________________
|                                   |                                                     |
| Minimisation algorithm            | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Method of Multipliers             | '^[Mm][Oo][Mm]$' or '[Mm]ethod of [Mm]ultipliers$'  |
|___________________________________|_____________________________________________________|



Minimisation options
~~~~~~~~~~~~~~~~~~~~

The minimisation options can be given in any order.


Line search algorithms.  These are used in the line search methods and the conjugate gradient
methods.  The default is the Backtracking line search.
___________________________________________________________________________________________
|                                   |                                                     |
| Line search algorithm             | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Backtracking line search          | '^[Bb]ack'                                          |
|                                   |                                                     |
| Nocedal and Wright interpolation  | '^[Nn][Ww][Ii]' or                                  |
| based line search                 | '^[Nn]ocedal[ _][Ww]right[ _][Ii]nt'                |
|                                   |                                                     |
| Nocedal and Wright line search    | '^[Nn][Ww][Ww]' or                                  |
| for the Wolfe conditions          | '^[Nn]ocedal[ _][Ww]right[ _][Ww]olfe'              |
|                                   |                                                     |
| More and Thuente line search      | '^[Mm][Tt]' or '^[Mm]ore[ _][Tt]huente$'            |
|                                   |                                                     |
| No line search                    | '^[Nn]one$'                                         |
|___________________________________|_____________________________________________________|



Hessian modifications.  These are used in the Newton, Dogleg, and Exact trust region algorithms.
___________________________________________________________________________________________
|                                   |                                                     |
| Hessian modification              | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Unmodified Hessian                | '[Nn]one'                                           |
|                                   |                                                     |
| Eigenvalue modification           | '^[Ee]igen'                                         |
|                                   |                                                     |
| Cholesky with added multiple of   | '^[Cc]hol'                                          |
| the identity                      |                                                     |
|                                   |                                                     |
| The Gill, Murray, and Wright      | '^[Gg][Mm][Ww]$'                                    |
| modified Cholesky algorithm       |                                                     |
|                                   |                                                     |
| The Schnabel and Eskow 1999       | '^[Ss][Ee]99'                                       |
| algorithm                         |                                                     |
|___________________________________|_____________________________________________________|



Hessian type, these are used in a few of the trust region methods including the Dogleg and Exact
trust region algorithms.  In these cases, when the Hessian type is set to Newton, a Hessian
modification can also be supplied as above.  The default Hessian type is Newton, and the default
Hessian modification when Newton is selected is the GMW algorithm.
___________________________________________________________________________________________
|                                   |                                                     |
| Hessian type                      | Patterns                                            |
|___________________________________|_____________________________________________________|
|                                   |                                                     |
| Quasi-Newton BFGS                 | '^[Bb][Ff][Gg][Ss]$'                                |
|                                   |                                                     |
| Newton                            | '^[Nn]ewton$'                                       |
|___________________________________|_____________________________________________________|


For Newton minimisation, the default line search algorithm is the More and Thuente line search,
while the default Hessian modification is the GMW algorithm.