TopolearnBeyond one-liners awk is a small, fast language for reports and data reshaping that runs everywhere.
Longer programs go in a file and run with awk -f report.awk data. Pass shell values in with -v name=value instead of splicing them into the quoted program.
Useful built-ins:
printfwith%s %d %.2f %-10sfor aligned columns;sprintfreturns the string instead.- String functions:
length(s),substr(s, start, len)(1-based),index(s, t),split(s, arr, sep)which returns the count,tolower,toupper. sub(re, repl [, target])andgsub(...)replace in place and return the number of replacements;&inreplis the match.match(s, re)setsRSTARTandRLENGTH.
User functions: function name(a, b, tmp) { ... }. By convention the extra parameters after spaces are local variables, because awk has no other way to declare them. Arrays are passed by reference, scalars by value.
With several input files, FNR is the line number within the current file while NR keeps counting. So NR == FNR is true only while reading the first file. This is the idiom for joins and lookups: load the first file into an array, then use it on the second.
gawk adds extras (strftime, gensub, --csv, sorted array traversal), but code using them won't run on mawk or BSD awk.
Example
printf 'u1 Ana\nu2 Bo\n' > users.txt
printf 'u2 15.5\nu1 7\nu2 4.5\n' > orders.txt
# join: load users (first file), then total orders per user name
awk 'NR == FNR { name[$1] = $2; next }
{ total[name[$1]] += $2 }
END { for (u in total) printf "%-5s %6.2f\n", u, total[u] }' users.txt orders.txt | sort
# Ana 7.00
# Bo 20.00
awk -v limit=10 '$2 > limit { print $1 }' orders.txt # u2
awk 'function shout(s) { return toupper(s) "!" }
{ n = split($0, parts, "-"); print n, shout(parts[1]) }' <<< 'ab-cd-ef'
# 3 AB!Remember
- Pass shell values with -v, longer programs with -f file.awk
- NR == FNR { ...; next } loads the first file for a join
- Extra function parameters are the way to declare locals
Watch out for
- Splicing a shell variable into the awk program text; quotes in the value break it. Use -v.