TopolearnEvery input, weight, activation and gradient in PyTorch is a tensor, so everything else builds on it.
A tensor (torch.Tensor) is a typed, n-dimensional array. If you know NumPy's ndarray, you already know most of it: tensors have a shape, a dtype and support element-wise math, slicing and broadcasting. Two things are new: a tensor has a device (cpu, cuda, mps) and it can record operations for automatic differentiation.
Create one from Python data with torch.tensor(data). PyTorch infers the dtype: Python ints become torch.int64, floats become torch.float32 (not float64 as in NumPy), and bools become torch.bool.
Useful attributes and methods:
t.shape(ort.size()) returns atorch.Size, which behaves like a tuple.t.ndimis the number of dimensions,t.numel()the number of elements.t.dtypeandt.devicetell you how and where the data is stored.- Indexing a single element still gives a 0-dimensional tensor; call
.item()to get a plain Python number, and.tolist()for nested lists.
Tensors are the currency of the library: models take tensors in and return tensors out, and the training loop is a sequence of tensor operations. Getting comfortable printing shapes and dtypes is the most useful debugging habit you can build.
Example
import torch
t = torch.tensor([[1, 2, 3], [4, 5, 6]])
print(t.shape) # torch.Size([2, 3])
print(t.dtype) # torch.int64
print(t.device) # cpu
print(t.ndim, t.numel()) # 2 6
f = torch.tensor([1.0, 2.0])
print(f.dtype) # torch.float32
print(t[1, 2]) # tensor(6)
print(t[1, 2].item()) # 6
print(t.tolist()) # [[1, 2, 3], [4, 5, 6]]Remember
- A tensor = n-d array + dtype + device (+ optional gradient tracking)
- Python floats become float32 by default, unlike NumPy's float64
.item()turns a one-element tensor into a Python number
Watch out for
- Comparing a 0-d tensor to a number and expecting a Python bool everywhere; use
.item()when you need a plain value.