Converting types and missing values

csv · Practical · 9 min

Every value from CSV is text; analysis code needs real types and a policy for blanks.

The csv module returns strings only. Conversion is your job, and doing it in one place keeps the rest of the code clean:

def parse(row):
    return {
        "id": int(row["id"]),
        "price": float(row["price"]) if row["price"] else None,
        "day": date.fromisoformat(row["day"]),
    }

Things to decide explicitly:

  • Empty strings: is "" missing (None), zero, or an error?
  • Number formats: thousands separators ("1,234"), currency symbols and decimal commas ("3,5" in many European exports) need cleaning first. Use decimal.Decimal for money.
  • Booleans: map "yes"/"no", "true"/"false", "1"/"0" deliberately; bool("false") is True.
  • Dates: date.fromisoformat for ISO dates, datetime.strptime for anything else.

quoting=csv.QUOTE_NONNUMERIC changes behaviour in both directions: the writer quotes every non-number, and the reader converts every unquoted field to float, raising ValueError if it can't. It only works for files written that way.

For heavy typed analysis, pandas or polars infer types for you; the csv module is ideal when you want control and no dependencies.

Example

import csv, io
from datetime import date
from decimal import Decimal

data = "sku,price,qty,day,active\nA1,19.99,3,2026-10-01,yes\nB2,,1,2026-10-02,no\n"

def parse(row: dict[str, str]) -> dict:
    return {
        "sku": row["sku"],
        "price": Decimal(row["price"]) if row["price"] else None,
        "qty": int(row["qty"]),
        "day": date.fromisoformat(row["day"]),
        "active": row["active"].lower() in {"yes", "true", "1"},
    }

items = [parse(r) for r in csv.DictReader(io.StringIO(data))]
print(items[0]["price"] * items[0]["qty"], items[1]["price"], items[1]["active"])
# 59.97 None False

buf = io.StringIO()
csv.writer(buf, quoting=csv.QUOTE_NONNUMERIC).writerow(["A1", 19.99, 3])
print(buf.getvalue().strip())                       # "A1",19.99,3
print(next(csv.reader(io.StringIO(buf.getvalue()), quoting=csv.QUOTE_NONNUMERIC)))
# ['A1', 19.99, 3.0]

Remember

  • All fields are strings; convert them in one parse function
  • Decide how empty strings map (None, 0, error)
  • QUOTE_NONNUMERIC makes the reader return floats for unquoted fields

Watch out for

  • Using bool(value) on CSV text; any non-empty string, including 'false', is True.

Check yourself

  1. With quoting=csv.QUOTE_NONNUMERIC, what does the reader return for an unquoted field 3?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/csv.html#csv.QUOTE_NONNUMERIC