-
M-W definition: the mathematical procedures (as finding the
maximum of a function) to make something as effective as possible
-
solution should satisfy accompanying equality or inequality constraints
-
non-probabilistic optimization
-
solution in a functional form that predicts data points
-
minimum mean square error measure typically used
-
example -- regression analysis, splines, SVMs etc.
-
probabilistic optimization
-
some information theoretic measure is optimized to estimate
the underlying probability distribution
-
typically used criteria -- likelihood, entropy, posterior
probability