awk semantics: strings, numbers and $0

grep, sed & awk · Expert · 15 min

These rules explain the surprising outputs that bite people once awk programs get serious.

Every awk value is a string, a number, or a strnum: input that looks numeric (fields, getline results, -v values) compares as a number with other numbers. Constants like "10" are always strings, so $1 == "10" is a string comparison and "10" < "9" is true. Force a type with x + 0 (number) or x "" (string). Numbers convert to strings using CONVFMT (default %.6g), which is why large or long fractional values may print rounded.

Fields and $0 are linked:

  • Assigning any field, even $1 = $1, rebuilds $0 by joining the fields with OFS. That is the idiom for changing separators: awk -v OFS=, '{ $1 = $1; print }'.
  • Assigning $0 re-splits it into fields; assigning NF truncates or extends the record.
  • Setting $(NF+2) creates an empty field in between.

Records are separated by RS (newline by default). RS = "" enables paragraph mode: records are separated by blank lines and newline always acts as a field separator. gawk allows a regex RS and sets RT to the matched text.

getline has several forms with different side effects (getline updates $0, NF, NR; getline var < file updates only var) and returns -1 on error, so while ((getline line < f) > 0) is the only safe loop.

Example

echo '10 9' | awk '{ print ($1 > $2) }'            # 1  numeric: both are strnum
awk 'BEGIN { print ("10" > "9") }'                 # 0  string comparison
awk 'BEGIN { x = "3.0"; print (x == 3), (x + 0 == 3) }'   # 0 1

echo 'a b c' | awk -v OFS=, '{ print }'            # a b c   ($0 untouched)
echo 'a b c' | awk -v OFS=, '{ $1 = $1; print }'   # a,b,c   (rebuilt)
echo 'a b c' | awk '{ NF = 2; print }'             # a b

# paragraph mode: one record per blank-line separated block
printf 'ana\n31\n\nbo\n25\n' | awk 'BEGIN { RS = "" } { print NR, $1, $2 }'
# 1 ana 31
# 2 bo 25

Remember

  • Field input is a strnum; string constants always compare as strings
  • Assigning a field rebuilds $0 with OFS: $1 = $1 changes separators
  • RS = "" is paragraph mode; loop on (getline line < f) > 0

Watch out for

  • Writing while (getline line < file) without > 0: a missing file returns -1 and the loop never ends.

Check yourself

  1. Why does echo 'a b' | awk -v OFS=- '{ print }' print 'a b' and not 'a-b'?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.gnu.org/software/gawk/manual/gawk.html#Variable-Typing