Strings, regex and output formats

jq · Practical · 10 min

Turning JSON into text other tools consume, such as CSV, log lines or shell arguments, is a classic jq job.

String interpolation puts values into text: "\(.name) is \(.age)". Any expression works inside \( ), and non-strings are converted to JSON text.

Common string functions:

  • split(", ") and join("-") convert between strings and arrays; ascii_downcase, ascii_upcase, ltrimstr("v"), rtrimstr(".json"), startswith, endswith.
  • tostring and tonumber convert; length counts characters.
  • Regexes (PCRE-like, via Oniguruma): test("^err"; "i") returns a boolean, capture("(?<user>\\w+)@(?<host>.+)") returns an object of named groups, sub(re; repl) and gsub replace, scan(re) emits all matches. Backslashes must be doubled inside jq strings.

Format strings turn data into text safely:

  • @csv and @tsv turn an array into a CSV or TSV row, with correct quoting. Use with -r.
  • @sh quotes values for use as shell arguments, @uri percent-encodes, @base64 / @base64d encode and decode, @html and @json escape.
  • They also prefix a string: @uri "https://x.io/q?s=\(.term)" encodes only the interpolated parts.

A typical report: jq -r '.[] | [.id, .name, .email] | @csv' produces a CSV file you can open in a spreadsheet.

Example

echo '[{"id":1,"name":"Ann Lee","email":"ann@x.io"},{"id":2,"name":"Bob, Jr","email":"BOB@X.IO"}]' > u.json
jq -r '.[] | "\(.id): \(.name)"' u.json                 # 1: Ann Lee   2: Bob, Jr
jq -r '.[] | [.id, .name, .email] | @csv' u.json         # 1,"Ann Lee","ann@x.io"
                                                         # 2,"Bob, Jr","BOB@X.IO"
jq -c '.[] | .email | ascii_downcase | capture("(?<user>[^@]+)@(?<host>.+)")' u.json
# {"user":"ann","host":"x.io"}  {"user":"bob","host":"x.io"}
jq -c 'map(.name | split(" ") | .[0])' u.json            # ["Ann","Bob,"]
jq -r '.[] | select(.email | test("x\\.io$"; "i")) | .id' u.json   # 1  2
jq -r '@uri "https://s.io/?q=\(.[0].name)"' u.json       # https://s.io/?q=Ann%20Lee

Remember

  • "\(expr)" interpolates values into strings
  • test, capture, sub, gsub, split, join for text work
  • @csv/@tsv on arrays, @sh, @uri, @base64 for safe output

Watch out for

  • Using @csv on an object or a string; it needs an array, so build one with [ ... ] first.

Check yourself

  1. Which produces a correctly quoted CSV row from fields id and name?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: jqlang.org/manual/

Builds on

Unlocks

Nothing builds on this yet.