The order of expansions and word splitting

Bash · Expert · 15 min

Predicting expansion order explains nearly every 'why did bash do that' surprise.

Bash processes each command line in a fixed order:

  • Brace expansion: {a,b}, {1..5},
  • tilde expansion: ~, ~user,
  • parameter and variable expansion, arithmetic expansion, command substitution and process substitution, left to right,
  • word splitting of the unquoted results of those expansions,
  • pathname expansion (globbing),
  • quote removal.

Consequences: {1..$n} does not work because braces are expanded before variables. Word splitting only applies to the results of expansions, never to literal text, and quoted results are exempt. File names produced by globbing are never split again, so for f in * is safe even with spaces.

Splitting uses IFS (default space, tab, newline). Whitespace characters in IFS are trimmed and runs of them collapse; other characters such as , or : delimit exactly, so empty fields are kept. An empty IFS disables splitting entirely; unset IFS behaves like the default.

"$@" expands to each positional parameter as a separate word; "$*" joins them into one word using the first character of IFS; unquoted $@ and $* are both split and globbed. The same rules apply to "${arr[@]}" versus "${arr[*]}". When building a CSV line, (IFS=,; echo "$*") is a neat idiom.

Example

cd "$(mktemp -d)"
n=3
echo {1..$n}            # {1..3}   braces ran before $n was expanded
echo $(seq 1 "$n")      # 1 2 3
IFS=, read -ra parts <<< "a,b,,c"
echo "${#parts[@]}"     # 4  (non-whitespace IFS keeps the empty field)
data="x  y   z"
set -- $data; echo $#   # 3  (whitespace runs collapse)
IFS=: ; set -- $data; echo $#    # 1  (no colon, one word)
unset IFS
touch a.sh b.sh
pat='*.sh'
echo $pat               # a.sh b.sh  (globbing after expansion)
echo "$pat"             # *.sh
show() { printf '[%s]' "$@"; echo; }
set -- "a b" c
show "$@"               # [a b][c]
show "$*"               # [a b c]
show $*                 # [a][b][c]
(IFS=,; echo "$*")      # a b,c

Remember

  • Order: brace, tilde, parameter/arithmetic/command, word splitting, globbing, quote removal
  • Only unquoted expansion results are split and globbed
  • "$@" keeps arguments separate; "$*" joins them with the first IFS character

Watch out for

  • Expecting {1..$n} to produce a range; use seq or a C-style for loop.

Check yourself

  1. Why does `echo {1..$n}` print {1..3} rather than 1 2 3?
  2. With IFS=, what does splitting `a,,b` produce?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.gnu.org/software/bash/manual/bash.html#Shell-Expansions