test::curious

I’m not checking your code!

test-driven research software engineering

research software engineering

reproducible research compendia
packaged analysis tools
modular functions over scripts
dynamic and html-based reporting
automated tests
containerisation

research software engineering

containerisation

Not minor. I've spent almost an equal amount of time fixing those issues in the past couple of years as I have actually developing code. Containerization freed me from caring about the state of everyone else's laptop, and allowed me to get back to work.
— Dr Mark Greenaway (@certifiedwaif) January 15, 2020

automated tests

‘I started using automated tests because I discovered I was spending too much time re-fixing bugs that I’d already fixed before.’ - from Testing in R Packages (Wickham 2015)

automated tests

context("String length")
library(stringr)

test_that("str_length is number of characters", {
  expect_equal(str_length("a"), 1)
  expect_equal(str_length("ab"), 2)
  expect_equal(str_length("abc"), 3)
})

test_that("str_length of factor is length of level", {
  expect_equal(str_length(factor("a")), 1)
  expect_equal(str_length(factor("ab")), 2)
  expect_equal(str_length(factor("abc")), 3)
})

testing object	description	function
expectation	atom of testing	`expect_*`
test	a collection of expectations	`test_that`
context	a collection of tests	`context`

automated tests in data science

testing the good, the bad, and the ugly

testing the good

testing the good, the bad, and the ugly

# test for fixed positive number
log(3)

## [1] 1.098612

# test for random positive number
x <- rexp(1)

# the exponential distribution only produces positive numbers

log(x)

## [1] -2.978422

testing the bad

testing the good, the bad, and the ugly

# test for fixed negative number
log(-3)

## Warning in log(-3): NaNs produced

## [1] NaN

# test for random netgative number
x <- -rexp(1)

# the exponential distribution only produces positive numbers

log(x)

## Warning in log(x): NaNs produced

## [1] NaN

testing the ugly

testing the good, the bad, and the ugly

# test for ugly string input
log("pig")

## Error in log("pig"): non-numeric argument to mathematical function

# test for random string input
x <- c("cat", "pig", "bird") %>% sample(1)

log(x)

## Error in log(x): non-numeric argument to mathematical function

test-driven development

balancing development with a measure of code::proof

When developing, we won’t test the good, the bad, and the ugly.

a minimal development test

A neet test checks the function outputs a non-empty thing of expected type.

neet tests

a measure of code::proof in data analysis (Gray 2019)

neet test workflow

make a function
check in console it outputs a non-empty thing of expected type
convert this check into a test
run test

getting set up

with usethis:: (Wickham and Bryan 2019) and testthat:: (Wickham 2011)

# make a package in rstudio
usethis::create_package()

# set up testing in rstudio package
usethis::use_testthat()

# create a context file
usethis::use_test()

# add context to first line of context file
context("what these tests test")

example neet test

quack, quack, said the duck!

function

quack <-
  function(says_the_duck = "quack!", 
           greeting = "quack! ") {
    paste0(greeting, 
           says_the_duck, 
           " said the duck")
  }

quack()

## [1] "quack! quack! said the duck"

quack("quack, quack!")

## [1] "quack! quack, quack! said the duck"

test

context("test that the duck quacks")
library(stringr)

expect_neet_quack <-
  function(quack_output) {
    expect_gte(
      str_length(quack_output), 
      1)
    expect_type(quack_output, 
                "character")
  }

test_that("duck quacks", {
  expect_neet_quack(quack())
  expect_neet_quack(
    quack("quack, quack!"))
})

example neet test

repliCATS

context("nas nans infs in confidence scores")


method_nans <- function(method_to_test) {
  cs <-
    expert_judgements_frankenstein %>%
    method_to_test() %>%
    purrr::pluck("cs")

  expect_false(NA %in% cs)
  expect_false(NaN %in% cs)
  expect_false(Inf %in% cs)
  expect_false(-Inf %in% cs)
  expect_false(any(cs < 0))
  expect_false(any(cs > 1))
  expect_false(any(cs == 0))
  expect_false(any(cs == 1))
  expect_is(cs, "numeric")
}

minimal tests

## Set panda = 68 to reproduce this panda.

coda

acknowledgements

## Set panda = 58 to reproduce this panda.

references

Gray, Charles T. 2019. “Code::Proof: Prepare for Most Weather Conditions.” In Statistics and Data Science, edited by Hien Nguyen, 22–41. Communications in Computer and Information Science (Preprint: Https://Arxiv.org/1910.06964.pdf). Singapore: Springer. https://doi.org/10.1007/978-981-15-1960-4_2.

Wickham, H. 2015. R Packages: Organize, Test, Document, and Share Your Code. O’Reilly Media.

Wickham, Hadley. 2011. “Testthat: Get Started with Testing.”

Wickham, Hadley, and Jennifer Bryan. 2019. “Usethis: Automate Package and Project Setup.”