TopolearnMost beginner runtime errors are dtype or device mismatches, and precision choices drive speed and memory.
Every tensor has a dtype and a device. Common dtypes:
torch.float32: the default for weights and activations.torch.float16andtorch.bfloat16: 2-byte floats used for fast mixed-precision training.torch.int64(torch.long): indices and class labels;CrossEntropyLossexpects labels in this type.torch.bool: masks.
Convert with t.to(torch.float32) or shortcuts like t.float(), t.long(). Mixed-dtype arithmetic follows type promotion: int64 + 0.5 gives float32, and true division / on integers always returns a float.
The device says where the data lives. t.to("cuda") copies a tensor to the first NVIDIA GPU, "mps" targets Apple silicon, and "cpu" brings it back. .to() returns a new tensor; it does not modify the original (modules are the exception, they move in place).
Operations require all operands on the same device. Adding a CPU tensor to a CUDA tensor raises RuntimeError: Expected all tensors to be on the same device. The usual pattern is to pick a device once at the top of a script and create or move everything there:
device = "cuda" if torch.cuda.is_available() else "cpu"Moving data between host and GPU is relatively slow, so avoid round trips inside loops, such as calling .item() or .cpu() on every step.
Example
import torch
x = torch.arange(4) # int64
y = x.to(torch.float32) / 2
print(y) # tensor([0.0000, 0.5000, 1.0000, 1.5000])
print((x / 2).dtype) # torch.float32 (true division)
print((x + 0.5).dtype) # torch.float32 (type promotion)
h = y.to(torch.bfloat16)
print(h.dtype, h.element_size()) # torch.bfloat16 2
device = "cuda" if torch.cuda.is_available() else "cpu"
z = y.to(device) # copy (y itself is unchanged)
print(z.device.type) # cuda or cpu
print(y.device) # cpuRemember
- float32 for weights, int64 for labels and indices, bool for masks
.to(device)returns a copy; operands must share one device- Integer
/gives floats; mixed dtypes are promoted
Watch out for
- Passing float labels to
CrossEntropyLossfor class indices, or int64 inputs to aLinearlayer. - Calling
.item()every iteration, which forces a GPU sync.