Torch Optim Pytorch 1 11 0 Documentation

Created On: Jun 13, 2025 | Last Updated On: Aug 24, 2025 torch.optim is a package implementing various optimization algorithms. Most commonly used methods are already supported, and the interface is general enough, so that more sophisticated ones can also be easily integrated in the future. To use torch.optim you have to construct an optimizer object that will hold the current state and will update the parameters based on the computed gradients. To construct an Optimizer you have to give it an iterable containing the parameters (all should be Parameter s) or named parameters (tuples of (str, Parameter)) to optimize. Then, you can specify optimizer-specific options such as the learning rate, weight decay, etc.

Click here to download the full example code A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance. This implementation uses the nn package from PyTorch to build the network. Rather than manually updating the weights of the model as we have been doing, we use the optim package to define an Optimizer that will update the weights for us. The optim package defines many optimization algorithms that are commonly used for deep learning, including SGD+momentum, RMSProp, Adam, etc. Total running time of the script: ( 0 minutes 0.000 seconds)

© Copyright 2019-2024, PyKEEN Project Team. PyTorch is an optimized tensor library for deep learning using GPUs and CPUs. Features described in this documentation are classified by release status: Stable (API-Stable): These features will be maintained long-term and there should generally be no major performance limitations or gaps in documentation. We also expect to maintain backwards compatibility (although breaking changes can happen and notice will be given one release ahead of time). Unstable (API-Unstable): Encompasses all features that are under active development where APIs may change based on user feedback, requisite performance improvements or because coverage across operators is not yet complete.

The APIs and performance characteristics of these features may change. Click here to download the full example code Learn the Basics || Quickstart || Tensors || Datasets & DataLoaders || Transforms || Build Model || Autograd || Optimization || Save & Load Model This section runs through the API for common tasks in machine learning. Refer to the links in each section to dive deeper. PyTorch has two primitives to work with data: torch.utils.data.DataLoader and torch.utils.data.Dataset.

Dataset stores the samples and their corresponding labels, and DataLoader wraps an iterable around the Dataset. PyTorch offers domain-specific libraries such as TorchText, TorchVision, and TorchAudio, all of which include datasets. For this tutorial, we will be using a TorchVision dataset. PyTorch is an optimized tensor library for deep learning using GPUs and CPUs. Features described in this documentation are classified by release status: Stable: These features will be maintained long-term and there should generally be no major performance limitations or gaps in documentation.

We also expect to maintain backwards compatibility (although breaking changes can happen and notice will be given one release ahead of time). Beta: These features are tagged as Beta because the API may change based on user feedback, because the performance needs to improve, or because coverage across operators is not yet complete. For Beta features, we are committing to seeing the feature through to the Stable classification. We are not, however, committing to backwards compatibility. Prototype: These features are typically not available as part of binary distributions like PyPI or Conda, except sometimes behind run-time flags, and are at an early stage for feedback and testing. torch-optimizer – collection of optimizers for PyTorch.

https://www4.comp.polyu.edu.hk/~cslzhang/paper/CVPR18_PID.pdf https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization

People Also Search

• torch.optim — PyTorch 1.11.0 documentation
• torch.optim — PyTorch 2.9 documentation
• PyTorch: optim — PyTorch Tutorials 1.11.0+cu102 documentation
• torch.optim.lr_scheduler — pykeen 1.11.0 documentation
• PyTorch documentation — PyTorch 2.9 documentation
• Distributed Optimizers — PyTorch 1.11.0 documentation
• Quickstart — PyTorch Tutorials 1.11.0+cu102 documentation
• torch.optim.Optimizer.step — PyTorch 1.11.0 documentation
• PyTorch documentation — PyTorch 1.11.0 documentation
• Welcome to pytorch-optimizer's documentation!