Missing values and factors

R · Essentials · 10 min

Real data always has gaps and categories, and both behave in ways that surprise newcomers.

NA marks a missing value. It is contagious: any calculation involving NA gives NA, because the true answer is unknown. mean(c(1, NA, 3)) is NA; pass na.rm = TRUE to most summary functions to ignore missing values.

Test with is.na(x), never x == NA (which is NA itself). sum(is.na(x)) counts missing values and complete.cases(df) finds rows with no NA. There are typed versions (NA_integer_, NA_character_), and NaN (not a number, e.g. 0/0) and NULL (the absence of an object, length 0) are different things.

A factor stores categorical data: an integer vector of codes plus a levels attribute with the category names. Factors control the order of categories in tables, plots and model terms, and models create dummy variables for them automatically.

  • factor(x, levels = c("low", "mid", "high")) sets the order explicitly; otherwise levels are sorted alphabetically.
  • table(f) counts each level, including levels with zero observations.
  • droplevels() removes unused levels after filtering.

Since R 4.0, data.frame() and read.csv() keep strings as character by default (stringsAsFactors = FALSE), so you create factors deliberately. Convert a numeric-looking factor with as.numeric(as.character(f)); as.numeric(f) returns the internal codes.

Example

x <- c(4, NA, 10)
print(mean(x))                 #> [1] NA
print(mean(x, na.rm = TRUE))   #> [1] 7
print(x == NA)                 #> [1] NA NA NA
print(is.na(x))                #> [1] FALSE  TRUE FALSE

size <- factor(c("mid", "low", "mid", "high"),
               levels = c("low", "mid", "high"))
print(levels(size))            #> [1] "low"  "mid"  "high"
print(table(size))
#> size
#>  low  mid high
#>    1    2    1
print(as.integer(size))        #> [1] 2 1 2 3

f <- factor(c("10", "5", "10"))
print(as.numeric(f))                 #> [1] 1 2 1 (codes!)
print(as.numeric(as.character(f)))   #> [1] 10  5 10

Remember

  • NA propagates; use na.rm = TRUE and is.na()
  • A factor = integer codes + ordered levels
  • Set levels explicitly to control order in tables, plots and models

Watch out for

  • Converting a factor of numbers with as.numeric(), which returns level codes instead of values.

Check yourself

  1. Which expression correctly finds missing values in x?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: cran.r-project.org/doc/manuals/r-release/R-intro.html#Factors