awk patterns, BEGIN and END

grep, sed & awk · Practical · 12 min

Patterns let one awk command replace a grep plus a cut, and END blocks turn a filter into a report.

Each awk rule is pattern { action }. The pattern decides which records the action runs on:

  • a comparison: $3 > 100, $1 == "alice", NF > 0;
  • a regex against the whole line: /ERROR/, or against a field: $2 ~ /^api/, $2 !~ /test/;
  • combinations with &&, ||, !;
  • a range /start/,/stop/, like in sed.

If the action is omitted, the default action prints the line, so awk '$3 > 100' behaves like a field-aware grep. If the pattern is omitted, the action runs for every line.

Two special patterns run outside the input loop: BEGIN { ... } runs once before any input is read (set FS, OFS, print a header) and END { ... } runs once after the last line (print totals). In END, NR holds the total number of records.

Variables need no declaration: they start as an empty string, which is also 0 in numeric context. That is why { sum += $2 } END { print sum } just works.

Comparisons are numeric when both sides look like numbers and string-based otherwise. Be careful with fields like 007 versus "7" written as a string constant: $1 == "7" compares as strings.

Use printf for aligned output: printf "%-10s %5.1f\n", $1, $2.

Example

printf 'name dept salary\nana eng 120\nbo ops 80\ncy eng 95\n' > staff.txt

awk 'NR > 1 && $3 > 90' staff.txt            # ana eng 120 / cy eng 95
awk '$2 ~ /^e/ { print $1 }' staff.txt       # ana / cy

awk 'BEGIN { print "Report" }
     NR > 1 { total += $3; n++ }
     END    { printf "%d people, avg %.1f\n", n, total / n }' staff.txt
# Report
# 3 people, avg 98.3

awk 'END { print NR " lines" }' staff.txt    # 4 lines

Remember

  • pattern { action }: no action prints the line, no pattern matches all lines
  • ~ and !~ match a field against a regex
  • BEGIN runs before input, END after it; uninitialised variables are 0 or empty

Watch out for

  • Forgetting to skip the header line (NR > 1) and having it counted or summed.

Check yourself

  1. What does awk '$2 > 10' file do?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.gnu.org/software/gawk/manual/gawk.html#Patterns-and-Actions