Optimization is about finding the best model. With greater model complexity it becomes increasingly important to avoid overfitting: finding a model that is best for one specific dataset but does not generalize well to others.
Slides for optimization video (PDF)
Slides for overfitting video (PDF)
Notebook for optimization (partially complete)
Notebook for regularization (partially complete)
Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".
For attribution, please cite this work as
Loftus (2021, Feb. 16). Neurath's Speedboat: Optimization and overfitting. Retrieved from http://joshualoftus.com/ml4ds/05-optimization/
BibTeX citation
@misc{loftus2021optimization,
author = {Loftus, Joshua},
title = {Neurath's Speedboat: Optimization and overfitting},
url = {http://joshualoftus.com/ml4ds/05-optimization/},
year = {2021}
}