TopolearnA pipeline that works on your laptop can fail in Alpine, on macOS or in CI because each ships different implementations.
The same command names hide different implementations: GNU tools on most Linux distributions, BSD tools on macOS and FreeBSD, BusyBox in Alpine and many containers, and mawk as the default awk on Debian and Ubuntu.
Frequent traps:
sed -i: GNU takes an optional suffix, BSD requires one. Use-i.bakor avoid-i.sed:\nin the replacement,Iflag,\U/\L,-z, one-linea textandeare GNU-only. BSD sed needs the escaped-newline form fora\.grep:-Pis GNU-only.\+ \? \|in BRE are GNU extensions;-Eis portable.\w,\s,\band\<are not POSIX;[[:alnum:]_]and[[:space:]]are.awk:gensub,strftime,PROCINFO,--csv,asort,@includeandlength(arr)on older awks are gawk extensions; mawk and BSD awk lack most of them.sort -h,xargs -0,find -print0are widespread but not universal.
Strategies:
- Write to POSIX and test with the target implementation (for example run your script in an
alpineand adebiancontainer). - On macOS, install GNU versions (
brew install gnu-sed gawk grep, available asgsed,gawk,ggrep) if you need them, but don't assume users have them. - When a pipeline needs many extensions, a small Python script is often the more portable choice.
Example
# portable # GNU-only equivalent
grep -E 'a+|b' f # grep 'a\+\|b' f
grep '[[:space:]]' f # grep '\s' f
sed -i.bak 's/a/b/' f && rm f.bak # sed -i 's/a/b/' f
sed 's/x/&\
/' f # sed 's/x/&\n/' f (newline in replacement)
awk '{ print toupper($0) }' f # gawk '{ print gensub(/a/, "b", "g") }' f
# which implementation am I using?
awk --version 2>/dev/null | head -1 || awk -W version 2>&1 | head -1
sed --version 2>/dev/null | head -1 || echo "not GNU sed (probably BSD)"Remember
- macOS ships BSD sed/grep, Debian's default awk is mawk, Alpine uses BusyBox
- POSIX classes like [[:space:]] work everywhere; \s and \w don't
- Test scripts on the implementations you ship to
Watch out for
- Testing a script only with GNU tools and shipping it to macOS developers or Alpine images.