1、 Optimization
a) Local Optimizationi. minimize(fun, x0[, args, method, jac, hess, ...]) Minimization of scalar function of one or more variables.
ii. minimize_scalar(fun[, bracket, bounds, ...]) Minimization of scalar function of one variable.
iii. OptimizeResult Represents the optimization result.
iv. OptimizeWarning
b) Equation (Local) Minimizers
i. leastsq(func, x0[, args, Dfun, full_output, ...]) Minimize the sum of squares of a set of equations.
ii. least_squares(fun, x0[, jac, bounds, ...]) Solve a nonlinear least-squares problem with bounds on the variables.
iii. nnls(A, b) Solve argmin_x || Ax - b ||_2 for x>=0.
iv. lsq_linear(A, b[, bounds, method, tol, ...]) Solve a linear least-squares problem with bounds on the variables.
c) Global Optimization
i. basinhopping(func, x0[, niter, T, stepsize, ...]) Find the global minimum of a function using the basin-hopping algorithm
ii. brute(func, ranges[, args, Ns, full_output, ...]) Minimize a function over a given range by brute force.
iii. differential_evolution(func, bounds[, args, ...]) Finds the global minimum of a multivariate function.
d) Rosenbrock function
i. rosen(x) The Rosenbrock function.
ii. rosen_der(x) The derivative (i.e.
iii. rosen_hess(x) The Hessian matrix of the Rosenbrock function.
iv. rosen_hess_prod(x, p) Product of the Hessian matrix of the Rosenbrock function with a vector
2、 Fitting
curve_fit(f, xdata, ydata[, p0, sigma, ...]) Use non-linear least squares to fit a function, f, to data.
3、 Root finding
1）Scalar functions
a) brentq(f, a, b[, args, xtol, rtol, maxiter, ...]) Find a root of a function in a bracketing interval using Brent’s method.
b) brenth(f, a, b[, args, xtol, rtol, maxiter, ...]) Find root of f in [a,b].
c) ridder(f, a, b[, args, xtol, rtol, maxiter, ...]) Find a root of a function in an interval.
d) bisect(f, a, b[, args, xtol, rtol, maxiter, ...]) Find root of a function within an interval.
e) newton(func, x0[, fprime, args, tol, ...]) Find a zero using the Newton-Raphson or secant method.
2）Multidimensional
root(fun, x0[, args, method, jac, tol, ...]) Find a root of a vector function.
fsolve(func, x0[, args, fprime, ...]) Find the roots of a function.
broyden1(F, xin[, iter, alpha, ...]) Find a root of a function, using Broyden’s first Jacobian approximation.
broyden2(F, xin[, iter, alpha, ...]) Find a root of a function, using Broyden’s second Jacobian approximation.
4、 Linear Programming
a) linprog(c[, A_ub, b_ub, A_eq, b_eq, bounds, ...]) Minimize a linear objective function subject to linear equality and inequality constraints.b) linprog_verbose_callback(xk, **kwargs) A sample callback function demonstrating the linprog callback interface.
5、 Utilities
a) approx_fprime(xk, f, epsilon, *args) Finite-difference approximation of the gradient of a scalar function.
b) bracket(func[, xa, xb, args, grow_limit, ...]) Bracket the minimum of the function.
c) check_grad(func, grad, x0, *args, **kwargs) Check the correctness of a gradient function by comparing it against a (forward) finite-line_search(f, myfprime, xk, pk[, gfk, ...]) Find alpha that satisfies strong Wolfe conditions.
d) show_options([solver, method, disp]) Show documentation for additional options of optimization solvers.
e) LbfgsInvHessProduct(sk, yk) Linear operator for the L-BFGS approximate inverse Hessian.
6、 Nonlinear solvers
Large-scale nonlinear solvers:
newton_krylov(F, xin[, iter, rdiff, method, ...]) Find a root of a function, using Krylov approximation for inverse Jacobian.
anderson(F, xin[, iter, alpha, w0, M, ...]) Find a root of a function, using (extended) Anderson mixing.
General nonlinear solvers:
broyden1(F, xin[, iter, alpha, ...]) Find a root of a function, using Broyden’s first Jacobian
approximation.
broyden2(F, xin[, iter, alpha, ...]) Find a root of a function, using Broyden’s second Jacobian
approximation.
Simple iterations:
excitingmixing(F, xin[, iter, alpha, ...]) Find a root of a function, using a tuned diagonal Jacobian approximation.
linearmixing(F, xin[, iter, alpha, verbose, ...]) Find a root of a function, using a scalar Jacobian approximation.
diagbroyden(F, xin[, iter, alpha, verbose, ...]) Find a root of a function, using diagonal Broyden Jacobian approximation