Matrix Testing in Elixir
Matrix, or data-driven, tests apply a list of scenarios to a test template. This technique allows you to test a large variety of scenarios that only differ by their inputs.
Thanks to a bit of macro magic and the way ExUnit works in Elixir, developing a matrix test is very simple. Suppose we have the following simple module to test:
defmodule Temperature do
@doc """
Convert a value from one temperature unit, denoted as `{unit, value}`, to
another temperature unit.
"""
def convert({unit, value}, unit) do
value
end
def convert({:celsius, value}, :fahrenheit) do
value * 1.8 + 32.0
end
def convert({:fahrenheit, value}, :celsius) do
(value - 32.0) / 1.8
end
@doc """
Convert the long atom representation of a unit to the shorthand string
representation.
"""
def short(:celsius), do: "°C"
def short(:fahrenheit), do: "°F"
end
Then you can write a single test that accepts a data table and updates the name of the test based on the scenario:
defmodule TemperatureTest do
use ExUnit.Case
alias Temperature, as: T
describe "convert" do
for {from_value, from_unit, to_unit, expected_value} <- [
# ----------- ------------ ------------ --------------
{0.0, :celsius, :celsius, 0.0 },
{0.0, :celsius, :fahrenheit, 32.0 },
{25.0, :celsius, :fahrenheit, 77.0 },
{40.0, :celsius, :fahrenheit, 104.0 },
{0.0, :fahrenheit, :fahrenheit, 0.0 },
{71.6, :fahrenheit, :celsius, 22.0 },
] do
test "#{from_value}#{T.short(from_unit)} to #{T.short(to_unit)}" do
from = {unquote(from_unit), unquote(from_value)}
to_unit = unquote(to_unit)
expected_result = unquote(expected_value)
actual_result = T.convert(from, to_unit)
assert_in_delta expected_result, actual_result, 0.00001
end
end
end
end
This is what it looks like to run this test:
❯ mix test --trace
TemperatureTest [test/temperature_test.exs]
* test convert 0.0°F to °F (1.3ms) [L#15]
* test convert 0.0°C to °C (0.00ms) [L#15]
* test convert 0.0°C to °F (0.00ms) [L#15]
* test convert 25.0°C to °F (0.00ms) [L#15]
* test convert 71.6°F to °C (0.00ms) [L#15]
* test convert 40.0°C to °F (0.01ms) [L#15]
Finished in 0.03 seconds (0.00s async, 0.03s sync)
6 tests, 0 failures
Randomized with seed 534856
You may be surprised by the use of
unquote
inside the body of the test. The for comprehension
is executing in the module scope at compile time in order to generate code,
the multiple tests. The data table used by the comprehension does not exist
past compilation. The test
macro is generating code for each individual test, and inside it is a block
that is used as part of that generation. In order to use the compilation
values, we must unquote them just like in any macro that uses static data
inputs.