| Channel | Publish Date | Thumbnail & View Count | Download Video |
|---|---|---|---|
 | Publish Date not found |  0 Views |
https://www.meetup.com/Deep-Learning-NYC/
Modern learning models are characterized by large hyperparameter spaces. To adequately explore these large spaces, we need to evaluate a large number of configurations, typically orders of magnitude more configurations than available parallel workers. Given the growing cost of model training, we would ideally like to perform this search in approximately the same time it takes to train a single model. We address this challenge by introducing ASHA, a simple and robust hyperparameter tuning algorithm with solid theoretical grounding that exploits parallelism and aggressive early-stopping. Our extensive empirical results show that ASHA outperforms state-of-the-art hyperparameter tuning methods; scales linearly with the number of workers in distributed environments; converges to a high-quality configuration in half the time required by Vizier (Google's internal hyperparameter tuning service) in an experiment with 500 workers; and competes favorably with specialized neural architecture search methods on standard benchmarks.
Bio of Ameet: Ameet Talwalkar is an Assistant Professor in the Department of Machine Learning at Carnegie Mellon University and Co-Founder and Chief Scientist at Determined AI. His primary interests are in the area of statistical machine learning, including problems at the intersection of systems and learning. His current work is motivated by the goal of democratizing machine learning, with a focus on topics related to the scalability, automation, and interpretability of learning algorithms and systems. He led the initial development of the MLlib project in Apache Spark, co-authored the thesis book “Foundations of Machine Learning” (2012, MIT Press), and created an award-winning edX MOOC on distributed machine learning.
Please feel free to share this video with your friends and family if you found it useful.
