PyTorch
{{#invoke:Infobox|infobox}} PyTorch is an open-source machine learning library for Python, based on Torch,[1][2][3] used for applications such as natural language processing.[4] It is primarily developed by Facebook's artificial-intelligence research group,[5][6][7] and Uber's "Pyro" Probabilistic programming language software is built on it.[8]
PyTorch provides two high-level features:[9]
- Tensor computation (like NumPy) with strong GPU acceleration
- Deep neural networks built on a tape-based autodiff system
History
Facebook operates both PyTorch and Caffe2, but incompatibility made it difficult to transform a PyTorch-defined model into Caffe2 or vice versa. The Open Neural Network Exchange (ONNX) project was created by Facebook and Microsoft in September 2017 for converting models between frameworks. Caffe2 was merged into PyTorch at the end of March 2018.[10]
PyTorch Tensors
In terms of programming, Tensors can simply be considered multidimensional arrays. Tensors in PyTorch are similar to NumPy arrays, with the addition being that Tensors can also be used on a GPU that supports CUDA. PyTorch supports various types of Tensors.[11]
Modules
Autograd Module
PyTorch uses a technique called automatic differentiation. A recorder records what operations have performed, and then it replays it backward to compute the gradients. This technique is especially powerful when building neural networks in order to save time on one epoch by calculating differentiation of the parameters at the forward pass itself.
Optim Module
torch.optim is a module that implements various optimization algorithms used for building neural networks. Most of the commonly used methods are already supported, so there is no need to build them from scratch.
nn Module
PyTorch autograd makes it easy to define computational graphs and take gradients, but raw autograd can be a bit too low-level for defining complex neural networks. This is where the nn module can help.
See also
References
| references-column-width
| references-column-count references-column-count-{{#if:1|{{{1}}}}} }}
| {{#if:
| references-column-width }} }}" style="{{#if:
| {{#iferror: {{#ifexpr: 1 > 1 }}
| Template:Column-width
| Template:Column-count }}
| {{#if:
| Template:Column-width }} }} list-style-type: {{#switch:
| upper-alpha
| upper-roman
| lower-alpha
| lower-greek
| lower-roman = {{{group}}}
| #default = decimal}};">