
Explanation of the concept of Learning Rate (step size) in Machine Learning Algorithms. Started with discussion on stationary points on multivariate and higher order functions and then discussed Stochastic Gradient Descent, Momentum and various Adaptive Learning Rate algorithms.