Topolearngrep, sed and awk find and reshape data; these helpers rank, count and tidy it into an answer.
A few small coreutils appear in almost every text pipeline:
sortorders lines.-nnumeric,-rreverse,-hhuman sizes (2K,1G),-uunique,-t ','separator,-k2,2sort by field 2 only. Always write the end of the key (-k2,2):-k2means from field 2 to the end of the line.uniqcollapses adjacent duplicate lines, so it is nearly always preceded bysort.uniq -cprefixes counts,-dshows only duplicated lines.cutextracts columns by delimiter (cut -d, -f1,3) or by character position (cut -c1-8). Unlike awk it treats every delimiter as a separator, so two spaces mean an empty field between them.trtranslates or deletes characters:tr 'a-z' 'A-Z',tr -d '\r'strips Windows carriage returns,tr -s ' 'squeezes repeated spaces.head -n,tail -n,wc -ltake the top, the bottom and the count.
The "top N" idiom combines them:
... | sort | uniq -c | sort -rn | headIt counts each distinct line, then ranks by count.
sort is locale-aware. With LC_ALL=C sort you get plain byte order, which is faster and identical on every machine, important when another tool (like join or comm) expects sorted input.
Example
printf '10.0.0.1 GET\n10.0.0.2 GET\n10.0.0.1 POST\n10.0.0.1 GET\n' > hits.txt
# top client IPs
cut -d' ' -f1 hits.txt | sort | uniq -c | sort -rn
# 3 10.0.0.1
# 1 10.0.0.2
printf 'b,2\na,10\nc,1\n' | sort -t, -k2,2n # c,1 / b,2 / a,10
printf 'b,2\na,10\nc,1\n' | sort -t, -k2,2 # c,1 / a,10 / b,2 (text order!)
echo 'Hello, World' | tr -s ' ' | tr 'a-z' 'A-Z' # HELLO, WORLD
printf 'line\r\n' | tr -d '\r' | od -c | head -1 # no \r leftRemember
- sort | uniq -c | sort -rn | head is the top-N idiom
- uniq only merges adjacent lines, so sort first
- Use -k2,2 with -n for numeric sorting on one field
Watch out for
- Sorting numbers without -n, so 10 comes before 2.