TopolearnCombining batches, adding features as columns and chunking work are routine array plumbing.
`np.concatenate(arrays, axis=0)` joins a sequence of arrays along an existing axis. All other dimensions must match. For 2-D data, axis=0 adds rows and axis=1 adds columns.
`np.stack(arrays, axis=0)` joins along a new axis. All inputs must have the same shape; stacking ten (28, 28) images gives (10, 28, 28). That's the difference to remember: concatenate keeps ndim, stack adds one.
Convenience wrappers:
np.vstack: stack vertically (rows); 1-D inputs are treated as single rowsnp.hstack: stack horizontally; for 1-D inputs it simply joins them end to endnp.column_stack: turn 1-D arrays into columns of a 2-D array, handy for building feature matricesnp.block([[A, B], [C, D]]): assemble a matrix from blocks
Splitting goes the other way. np.split(a, 3) cuts into 3 equal parts (raising if they can't be equal), np.split(a, [2, 5]) cuts at indices 2 and 5, and np.array_split(a, 3) allows unequal parts, which is what you want for chunking work across workers. np.hsplit and np.vsplit are axis-specific forms. Pieces from split are views.
Every stacking function allocates a new array and copies all inputs. Calling np.concatenate repeatedly inside a loop is quadratic: each iteration copies everything accumulated so far. Collect the pieces in a Python list and concatenate once at the end, or preallocate the output and fill it with slices.
Example
import numpy as np
a = np.array([[1, 2], [3, 4]])
b = np.array([[5, 6]])
print(np.concatenate([a, b], axis=0)) # 3x2: b added as a row
print(np.concatenate([a, a], axis=1).shape) # (2, 4)
imgs = [np.zeros((28, 28)) for _ in range(10)]
print(np.stack(imgs).shape) # (10, 28, 28) new axis
h = np.array([1.70, 1.82]); w = np.array([65, 80])
X = np.column_stack([h, w])
print(X) # [[ 1.7 65. ] [ 1.82 80. ]]
x = np.arange(10)
print(np.split(x, [3, 7])) # [0 1 2], [3 4 5 6], [7 8 9]
print([len(p) for p in np.array_split(x, 4)]) # [3, 3, 2, 2]
chunks = []
for i in range(3):
chunks.append(np.full(2, i))
print(np.concatenate(chunks)) # [0 0 1 1 2 2] join onceRemember
concatenatejoins along an existing axis;stackcreates a new onecolumn_stackbuilds a 2-D table from 1-D columnsarray_splitallows uneven chunks;splitrequires even ones- Join once at the end; concatenating in a loop is quadratic
Watch out for
- Concatenating 1-D arrays with axis=1, which fails because 1-D arrays have only axis 0; use
column_stackorstack(..., axis=1).