Detecting formats with Sniffer

csv · Advanced · 8 min

User uploads arrive as commas, semicolons or tabs, and you can't always ask which.

csv.Sniffer inspects a sample of text and guesses the format.

  • Sniffer().sniff(sample, delimiters=None) returns a Dialect class with the detected delimiter, quotechar, doublequote and skipinitialspace. Restrict the candidates with delimiters=",;\t|" for more reliable results. It raises csv.Error ("Could not determine delimiter") when it can't decide.
  • Sniffer().has_header(sample) guesses whether the first row is a header, using heuristics such as "the first row's values differ in type or length from the rest".

Typical use:

sample = f.read(64 * 1024)
f.seek(0)
dialect = csv.Sniffer().sniff(sample, delimiters=",;\t")
reader = csv.reader(f, dialect)

Read a sample of a few kilobytes that ends near a line boundary, then rewind with seek(0).

These are heuristics. They fail on single-column files, tiny samples, or data where every column is text (then has_header can't tell a header from data). Use them for user-facing imports with a fallback (default to excel) and, ideally, a preview the user can correct. For files you control, specify the dialect explicitly instead.

Example

import csv, io

files = {
    "commas": "id,name,score\n1,Ada,9.5\n2,Bob,7.0\n",
    "semis":  "id;name;score\n1;Ada;9,5\n2;Bob;7,0\n",
    "tabs":   "1\tAda\t9.5\n2\tBob\t7.0\n",
}
sniffer = csv.Sniffer()
for label, text in files.items():
    d = sniffer.sniff(text, delimiters=",;\t")
    print(label, repr(d.delimiter), sniffer.has_header(text),
          next(csv.reader(io.StringIO(text), d)))
# commas ',' True ['id', 'name', 'score']
# semis ';' True ['id', 'name', 'score']
# tabs '\t' False ['1', 'Ada', '9.5']
try:
    sniffer.sniff("just one column\nno delimiters\n", delimiters=",;")
except csv.Error as e:
    print("csv.Error:", e)          # csv.Error: Could not determine delimiter

Remember

  • Sniffer().sniff(sample) returns a dialect with the detected delimiter
  • has_header() guesses whether the first row is a header
  • Restrict delimiters and keep a fallback; sniffing is heuristic

Watch out for

  • Sniffing a sample and forgetting f.seek(0), so the reader starts mid-file.

Check yourself

  1. What does Sniffer.sniff() raise when it can't decide on a delimiter?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/csv.html#csv.Sniffer

Builds on

Unlocks

Nothing builds on this yet.