Optimization and Model Selection

Overview

You will learn about optimization methods and Bayesian model selection

Goals

• Learn about Optimization and Stochastic Gradient Descent

• Learn about Bayesian Model Selection

Lecture Materials

• Slides

• Optimization

• Bayesian Model Selection

Homework Assignment

• None

Supplemental Readings

• Convex Functions

• Jensen’s Inequality

• Finite Difference Equations

• Automatic Differentiation

• Rosenbrock function

• Nelder-Mead method

• Conjugate Gradient Method

• Newton’s CG method

• Powell’s method

• BFGS method

• Stochastic Gradient Descent

• Adam Optimizer

• Softmax Function

• Jeffreys proposed scale to interpret a Bayes Factor