Testing with CTest

A single ctest command that runs every test is what CI pipelines and IDE test panels rely on.

CTest is CMake's test driver. Two commands register tests:

enable_testing() in the top-level CMakeLists (or include(CTest), which also adds a BUILD_TESTING option).
add_test(NAME parser_test COMMAND parser_test --fast). When COMMAND names an executable target, CMake substitutes the built file's full path.

After building, run ctest --test-dir build. Useful flags:

--output-on-failure shows a failing test's output (set CTEST_OUTPUT_ON_FAILURE=1 to make it the default).
-j 8 runs tests in parallel; -R regex and -E regex select or exclude by name; -L label filters by label.
--rerun-failed and --repeat until-fail:20 help with flaky tests.
-C Debug selects the configuration with multi-config generators.

A test passes when its command exits with status 0. Test properties refine this: WILL_FAIL, PASS_REGULAR_EXPRESSION, TIMEOUT, ENVIRONMENT, LABELS, FIXTURES_SETUP/FIXTURES_REQUIRED for setup and teardown tests.

With GoogleTest, include(GoogleTest) and gtest_discover_tests(my_tests) register each TEST() case as its own CTest test by querying the built executable, so failures are reported per case and -R works on case names.

Example

enable_testing()

add_executable(calc_test calc_test.cpp)
target_link_libraries(calc_test PRIVATE calc GTest::gtest_main)

include(GoogleTest)
gtest_discover_tests(calc_test)               # one CTest test per TEST(...)

add_test(NAME cli_version COMMAND calc_cli --version)
set_tests_properties(cli_version PROPERTIES
  PASS_REGULAR_EXPRESSION "^calc [0-9]+\\.[0-9]+"
  TIMEOUT 5
  LABELS smoke)

# $ cmake --build build && ctest --test-dir build -j8 --output-on-failure
# 1/4 Test #1: CalcTest.Adds ......   Passed    0.01 sec
# 2/4 Test #2: CalcTest.Divides ...   Passed    0.01 sec
# ...
# 100% tests passed, 0 tests failed out of 4
# $ ctest --test-dir build -L smoke

Remember

enable_testing() + add_test(NAME ... COMMAND target)
ctest --test-dir build --output-on-failure -j N runs them
gtest_discover_tests registers each GoogleTest case separately

Watch out for

Calling enable_testing() only in a subdirectory, so ctest from the build root finds no tests.

Check yourself

When does CTest consider a test passed by default?

Official docs: cmake.org/cmake/help/latest/manual/ctest.1.html

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks