Hyperparameter optimization (HPO) is crucial to getting the best possible performance from your machine learning models. BOHB (Bayesian Optimization and Hyperband) is a recently developed algorithm that combines the best parts of two popular approaches to the HPO problem. It allows highly flexible declaration of the hyperparameter configuration space, parallel search of computing resources, and large numbers of hyperparameters. Best yet, there's a fantastic open source implementation in the Python package hpbandster.

Bayesian optimization methods create a model of the function that maps hyperparameter configurations to the model performance. They use this model to choose new hyperparameter configurations to test and refine the resulting model. These methods focus on configuration selection.

Hyperband, on the other hand, is a bandit strategy that focuses on configuration evaluation. It uses an adaptive multi-resolution approach to get quick and dirty estimates of many more hyperparameter configurations than is possible in the Bayesian optimization framework. It uses this increased evaluation speed to determine what seems promising and what should be excluded as it randomly searches through the configuration space. This results in a fast, flexible and parallelizable algorithm.

www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to exchange ideas and learn from each other. The global PyData network promotes discussion about best practices, new approaches and emerging technologies for data management, processing, analysis and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

PyData conferences strive to be accessible and community-driven, with presentations at beginner to advanced levels. PyData tutorials and lectures provide participants with the latest project features and advanced use cases. 00:00 Welcome!
00:10 Help us add timestamps or captions to this video! See description for details.

Want to help add timestamps to our YouTube videos to increase discoverability? Read more here: https://github.com/numfocus/YouTubeVideoTimestamps

Please take the opportunity to connect and share this video with your friends and family if you find it helpful.