Crandi

04 Lr Finder Ipynb Colab

Leo Migdal
-
04 lr finder ipynb colab

There was an error while loading. Please reload this page. There was an error while loading. Please reload this page. This how-to guide demonstrates how we can leverage the FastaiLRFinder handler to find an optimal learning rate to train our model on. We will compare the results produced with and without using the handler for better understanding.

In this example, we will be using a ResNet18 model on the MNIST dataset. The base code is the same as used in the Getting Started Guide. We will first train the model with a fixed learning rate (lr) of 1e-06 and inspect our results. Let’s save the initial state of the model and the optimizer to restore them later for comparison. Let’s see how we can achieve better results by using the FastaiLRFinder handler. But first, let’s restore the initial state of the model and optimizer so we can re-train them from scratch.

When attached to the trainer, this handler follows the same procedure used by fastai. The model is trained for num_iter iterations while the learning rate is increased from start_lr (defaults to initial value specified by the optimizer, here 1e-06) to the upper bound called end_lr. This increase can be linear (step_mode="linear") or exponential (step_mode="exp"). The default step_mode is exponential which is recommended for larger learning rate ranges while linear provides good results for small ranges. Colab is a hosted Jupyter Notebook service that requires no setup to use and provides free access to computing resources, including GPUs and TPUs. Colab is especially well suited to machine learning, data science, and education.

Check out our catalog of sample notebooks illustrating the power and flexiblity of Colab. Read about product updates, feature additions, bug fixes and other release details. Check out these resources to learn more about Colab and its ever-expanding ecosystem. We’re working to develop artificial intelligence responsibly in order to benefit people and society.

People Also Search

There Was An Error While Loading. Please Reload This Page.

There was an error while loading. Please reload this page. There was an error while loading. Please reload this page. This how-to guide demonstrates how we can leverage the FastaiLRFinder handler to find an optimal learning rate to train our model on. We will compare the results produced with and without using the handler for better understanding.

In This Example, We Will Be Using A ResNet18 Model

In this example, we will be using a ResNet18 model on the MNIST dataset. The base code is the same as used in the Getting Started Guide. We will first train the model with a fixed learning rate (lr) of 1e-06 and inspect our results. Let’s save the initial state of the model and the optimizer to restore them later for comparison. Let’s see how we can achieve better results by using the FastaiLRFind...

When Attached To The Trainer, This Handler Follows The Same

When attached to the trainer, this handler follows the same procedure used by fastai. The model is trained for num_iter iterations while the learning rate is increased from start_lr (defaults to initial value specified by the optimizer, here 1e-06) to the upper bound called end_lr. This increase can be linear (step_mode="linear") or exponential (step_mode="exp"). The default step_mode is exponenti...

Check Out Our Catalog Of Sample Notebooks Illustrating The Power

Check out our catalog of sample notebooks illustrating the power and flexiblity of Colab. Read about product updates, feature additions, bug fixes and other release details. Check out these resources to learn more about Colab and its ever-expanding ecosystem. We’re working to develop artificial intelligence responsibly in order to benefit people and society.