Pipes, commas and building JSON

jq · Essentials · 10 min

Reshaping API output into exactly the fields you need is jq's everyday job.

A jq program is built from small filters combined with operators.

  • The pipe a | b feeds every output of a into b. .users[] | .email is the same as .users[].email, but pipes let you add any filter in between.
  • The comma a, b runs both filters on the same input and outputs the results of both: .name, .age.

Object construction creates new JSON:

  • {name: .user.name, id: .user.id} builds an object with the given keys.
  • Shorthand: {name, id} means {name: .name, id: .id}.
  • Keys can be computed with parentheses: {(.key): .value}.
  • If a value produces several outputs, you get several objects, which can surprise you.

Array construction [ ... ] collects all outputs of the inner expression into one array: [.items[].price].

Putting it together, .users[] | {name, email: .contact.email} turns a verbose API response into a compact list of objects, one per user. Add -c for compact one-line output, handy for further processing or grep.

Parentheses group as you would expect, and | binds more loosely than ,, so .a, .b | f means (.a, .b) | f.

Example

cat > r.json <<'EOF'
{"users":[{"name":"Ann","id":1,"contact":{"email":"ann@x.io"}},
          {"name":"Bob","id":2,"contact":{"email":"bob@x.io"}}]}
EOF
jq -c '.users[] | {name, email: .contact.email}' r.json
# {"name":"Ann","email":"ann@x.io"}
# {"name":"Bob","email":"bob@x.io"}
jq -c '[.users[].id]' r.json                      # [1,2]
jq -r '.users[0] | .name, .id' r.json             # Ann  1
jq -c '.users | map({(.name): .id}) | add' r.json # {"Ann":1,"Bob":2}

Remember

  • a | b pipes each output of a into b
  • a, b emits the outputs of both
  • {name, id: .x} builds objects; [ ... ] collects outputs

Watch out for

  • Writing {name: .tags[]} and getting one object per tag; a multi-output value multiplies objects.

Check yourself

  1. What does `jq -c '{a: (1,2)}' <<< 'null'` output?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: jqlang.org/manual/