TopolearnComplex transformations, like joining datasets or building lookup tables, need variables, folds and helpers.
Variables: expr as $x | body evaluates body with $x bound to each output of expr. It is how you keep an outer value while descending: .team as $t | .members[] | {team: $t, name}. Destructuring works too: . as {name: $n, tags: [$first]} | ....
reduce folds a stream into one value: reduce .[] as $item (init; update). Inside update, . is the accumulator.
- Sum:
reduce .[] as $x (0; . + $x). - Build a lookup object:
reduce .[] as $u ({}; .[$u.id | tostring] = $u.name).
foreach is similar but outputs each intermediate state, useful for running totals.
Functions are defined with def name: body; or with parameters def name(f): body;. Parameters are filters, not values: def twice(f): f | f;. Prefix a parameter with $ to receive a value instead: def add_tax($rate): . * (1 + $rate);. Functions can be recursive and can be kept in a file loaded with -L dir and import "name" as n; or include.
A powerful join pattern combines these: load one file into a lookup object with --slurpfile or INDEX(.[]; .id), then enrich the other: INDEX(users[]; .id) as $u | orders[] | .user = $u[.user_id | tostring].name.
Example
echo '[3,5,7]' | jq 'reduce .[] as $x (0; . + $x)' # 15
echo '[1,2,3]' | jq -c '[foreach .[] as $x (0; . + $x)]' # [1,3,6] running total
echo '{"team":"core","members":[{"name":"Ann"},{"name":"Bob"}]}' |
jq -c '.team as $t | .members[] | {team: $t, name}'
# {"team":"core","name":"Ann"} {"team":"core","name":"Bob"}
echo '[100,250]' | jq -c 'def add_tax($r): . * (1 + $r); map(add_tax(0.25))' # [125,312.5]
echo '[1,2]' | jq -c 'def twice(f): f | f; map(twice(. * 10))' # [100,200]
cat > d.json <<'EOF'
{"users":[{"id":1,"name":"Ann"},{"id":2,"name":"Bob"}],"orders":[{"oid":7,"user_id":2}]}
EOF
jq -c 'INDEX(.users[]; .id) as $u | .orders[] | .user = $u[.user_id | tostring].name' d.json
# {"oid":7,"user_id":2,"user":"Bob"}Remember
- expr as $x | body binds variables
- reduce .[] as $x (init; update) folds a stream
- def f(g): ...; takes filters; def f($v): takes values
Watch out for
- Expecting a function parameter without $ to be a value; it is a filter evaluated against the current input.