TopolearnReading binary files, sensors and C data, or keeping heterogeneous records compact, calls for structured dtypes.
A structured dtype describes a record made of named fields, each with its own type, like a C struct:
dt = np.dtype([("id", "i4"), ("name", "U8"), ("temp", "f8")])An array of that dtype stores records back to back. a["temp"] returns a view of one field across all records (shape stays the same, dtype becomes float64), and a[0] returns one record. You can assign to fields (a["temp"] += 1), sort by them with np.sort(a, order="temp"), and filter with masks built from fields.
The layout is explicit: dt.itemsize and dt.fields show size and byte offsets. Pass align=True to pad fields like a C compiler would. This makes structured arrays the right tool when bytes come from somewhere with a fixed format: np.frombuffer(raw_bytes, dtype=dt) or np.fromfile(path, dtype=dt) parses a binary file of records with no loop at all. Prefix types with < or > to fix byte order, e.g. ">u2" for big-endian network data.
Fields can be sub-arrays (("pos", "f4", (3,))) or nested structures.
When should you not use them? For analysis of tabular data, pandas is far more convenient: structured arrays have no group-by, joins or missing-value handling, and arithmetic across records is awkward because ufuncs mostly don't operate on structured dtypes directly. Note also that a field view over interleaved records is strided, so operations on one field don't get the cache efficiency of a separate contiguous column. np.rec.array offers attribute access (r.temp) but is mostly legacy.
Example
import numpy as np
dt = np.dtype([("id", "i4"), ("name", "U8"), ("temp", "f8")])
a = np.array([(1, "oslo", 3.5), (2, "lima", 19.0), (3, "cairo", 28.2)], dtype=dt)
print(a["name"]) # ['oslo' 'lima' 'cairo']
print(a[1]) # (2, 'lima', 19.0)
a["temp"] += 1.0 # field view, in place
print(a[a["temp"] > 10]["id"]) # [2 3]
print(np.sort(a, order="temp")["name"]) # coldest first
print(dt.itemsize, dt.fields["temp"][1]) # 44 36 (bytes, offset)
# parse packed big-endian binary records: uint16 id, float32 value
rec = np.dtype([("id", ">u2"), ("value", ">f4")])
raw = bytes.fromhex("0001 40490fdb 0002 402df854".replace(" ", ""))
r = np.frombuffer(raw, dtype=rec)
print(r["id"], r["value"].round(4)) # [1 2] [3.1416 2.7183]Remember
- A structured dtype lists named fields with their own types and offsets
a['field']is a view; sort withorder=and filter with field masksfrombuffer/fromfilewith a structured dtype parses binary records without loops- For tabular analysis prefer pandas
Watch out for
- Using structured arrays as a DataFrame substitute and fighting missing group-by, join and NA support.