TopolearnArrays are Arrow's basic building block; every table, file and exchange between libraries is made of them.
An Arrow array is an immutable, typed, one-dimensional column of values stored contiguously in memory, with a separate bitmap that marks which slots are null. Every value in it has the same data type.
pa.array([1, 2, None]) infers int64 and records one null. You can force a type with type=pa.int32() or pa.float32(). Common types are pa.int8() through pa.int64(), unsigned pa.uint32(), pa.float64(), pa.bool_(), pa.string() (UTF-8), pa.large_string(), pa.binary(), pa.date32(), pa.timestamp("ms", tz="UTC") and pa.decimal128(10, 2).
Nulls are first-class in every type, so an integer column with a gap stays integer. That is a key difference from classic NumPy, which has no null concept.
Useful attributes: arr.type, len(arr), arr.null_count, arr.nbytes. arr[0] returns an Arrow scalar (call .as_py() for the Python value), and arr.to_pylist() converts the whole thing. arr.slice(1, 2) is zero-copy.
Arrays never grow or change. To combine data you build new arrays, or use a ChunkedArray: a logical column made of several arrays of the same type, created with pa.chunked_array([a, b]). Table columns are ChunkedArrays, which is how data read in pieces stays copy-free.
Example
import pyarrow as pa
a = pa.array([1, 2, None, 4])
print(a.type, len(a), a.null_count) # int64 4 1
b = pa.array([1.5, 2.25], type=pa.float32())
print(b.type) # float
print(a[1], a[1].as_py()) # 2 2
ts = pa.array([0, 86_400_000], type=pa.timestamp("ms", tz="UTC"))
print(ts.to_pylist()[1]) # 1970-01-02 00:00:00+00:00
chunked = pa.chunked_array([a, pa.array([5, 6])])
print(chunked.num_chunks, len(chunked)) # 2 6
print(a.slice(1, 2).to_pylist()) # [2, None] (zero-copy view)Remember
- Arrays are immutable, typed and carry a null bitmap
- Pass
type=to control the type instead of relying on inference - ChunkedArray = one logical column made of several arrays
Watch out for
- Trying to append to or assign into an Arrow array: they are immutable; build a new one.