How do we add new capabilities to our codebase while keeping it manageable? How can we continue to build and ship valuable features to our users without getting stuck maintaining what already exists?

In software engineering, one of the most powerful ways we do this is through abstraction.

Being able to see, extract and reuse the common essence of a class of things empowers us to deliver ever more functionality without upping our cognitive load.

Our minds already work this way. We know apples and pears are fruits. Yet when we compare fruits and vegetables, we don’t need to think about the individual fruit types.

By actively viewing our codebase through this lens, we identify and remove duplication, combining and compressing sameness whenever possible. Over time, we’re able to grow the capabilities of our product without a corresponding increase in surface area of the code that delivers it.

In this blog, we’ll explore two coding principles for balancing duplication and abstraction: WET and DRY. Then, we’ll check out two ways to compress Elixir code: functions and macros. We’ll peer into a real example from our codebase where we compressed hundreds of lines of code into just a few.

Let’s dive in!

Writing WET code and DRYing it up

The WET and DRY principles are like the yin and yang of coding, two balancing forces that allow abstractions to sprout organically. Together, they help us avoid the pitfalls of early over-engineering.

What does it mean to write WET or DRY code?

Write Everything Twice (WET)

Don’t Repeat Yourself (DRY)

Write Everything Twice means we are ok maintaining our code with some duplication. We know we can clean it up later, we can maintain two of something without much trouble. When we see a possible abstraction, we resist the temptation to refactor.

Allowing some level of duplication prevents us from abstracting too early, before we have the full picture. Duplication is far cheaper than the wrong abstraction.

Don’t Repeat Yourself means we actively identify and remove duplication.

Once we find ourselves coding the third instance of something, it’s a signal to DRY up our code. Three of something is hard to maintain. Bugs start to emerge. And by this point, we usually have enough information to see what’s the same.

Elixir gives us tools like variables, data structures, functions and modules. Functions are our primary means of abstraction and reuse, so we’ll focus on those next.

Functions

Functions can be composed to build higher and higher levels of abstraction, getting more done with fewer lines of code. A simple example:

1. List of strings

[
  "hello dáire",
  "hello nynke",
  "hello jameela"
]

2. Function calls in list

def say_hello(name), do: "hello, #{name}"

[
  say_hello("dáire"),
  say_hello("nynke"),
  say_hello("jameela")
]

3. Higher-order function (Enum.map/2)

["dáire", "nynke", "jameela"]
|> Enum.map(&say_hello/1)

All three of these expressions return the same output when called at runtime:

["hello dáire", "hello nynke", "hello jameela"]

Notice how each successive example removes additional repetition, further compressing the code.

Elixir’s functions and data structures, augmented with its powerful pattern matching features, offer most of what we need to grow our software.

Enter Macros

Macros are a powerful feature of the Elixir language. They allow us to write Elixir code that itself writes code. This superpower is called meta-programming. It can “generate boilerplate code, add new language features, or build domain-specific languages”.

Macro declarations, on the surface, look a lot like functions. In fact, they are functions.

• They are evaluated at compile time instead of at runtime.
• They take the Elixir Abstract Syntax Tree (AST) of the code as input.

They run in Elixir’s compile phase and expand our code into different code. Macros are harder to write and trickier to debug than functions, so they should be used sparingly.

Example

In the example below from our codebase, we are wrapping all API calls with a version check function. Three instances of the pattern are shown, but hundreds like this littered our codebase. (We didn’t follow WET and refactor after just two, but it’s OK, we can still DRY things up!)

Three examples of ’the same thing’. Can you spot what’s different?

 def get_available_minigames(
        %{
          api_version_requirement: api_version_requirement
        } = attrs
      ) do
    if Versioning.is_version_available(api_version_requirement, api_version = "0.0.1") do
      get_available_minigames(attrs, api_version)
    end
  end

  def get_available_filtered_minigames(
        %{
          api_version_requirement: api_version_requirement
        } = attrs
      ) do
    if Versioning.is_version_available(api_version_requirement, api_version = "0.0.1") do
      get_available_filtered_minigames(attrs, api_version)
    end
  end

  def get_minigame_details(
        %{
          api_version_requirement: api_version_requirement
        } = attrs
      ) do
    if Versioning.is_version_available(api_version_requirement, api_version = "0.0.1") do
      get_minigame_details(attrs, api_version)
    end
  end
# ... 5 more examples like this in this module

We want to preserve the API we have today while removing the duplication in our code. Macros provide means to write code that generates the code above.

We look in our three examples to see what varies.

1. the function name
2. the api version

Next, we define our macro. It’s like a function that returns Elixir code. We use quote/1 to wrap our desired output, and within that, we unquote/1 the parts that vary.

defmacro api_version_wrap(function_name, version) do
  quote do
    def unquote(function_name)(%{
      api_version_requirement: api_version_requirement
    } = attrs) do
      if Lol.Versioning.is_version_available(api_version_requirement, api_version = unquote(version)) do
        unquote(name)(attrs, api_version)
      end
    end
  end
end

To understand quote/1, look to string interpolation as an analogy. In the Elixir expression "#{say_hello(name)}", we are quoting a string literal, and then unquoting code within that string. quote(unquote(...)) does the same thing with Elixir’s representation of our code AST, a data structure that can be transformed via macros.

With our new macro required and imported, we can compress our API declaration code blocks to these one-liners:

    api_version_wrap(:get_available_minigames,          "0.0.1")
    api_version_wrap(:get_available_filtered_minigames, "0.0.1")
    api_version_wrap(:get_minigame_details,             "0.0.1")
    api_version_wrap(:play_minigame,                    "0.0.1")
    api_version_wrap(:get_minigame_details,             "0.0.1")
    api_version_wrap(:complete_minigame,                "0.0.1")
    api_version_wrap(:on_mobile_heartbeat,              "0.0.1")

This is a huge improvement:

• Many 15-line code blocks reduced to one line apiece
• Signal improved, noise reduced
• Single source of truth for the common logic

Even though we’ve cleaned up a lot of duplication, there’s still obvious repetition. We can do even better!

By moving our function names into a list, we can DRY things up even further:

defmacro api_version_wrap(function_names, version) when is_list(function_names) do
  function_names # now takes a list of names and returns a list of quoted `def
  |> Enum.map(fn name ->
    quote do
      def unquote(name)(%{
        api_version_requirement: api_version_requirement
      } = attrs) do
        if Lol.Versioning.is_version_available(api_version_requirement, api_version = unquote(version)) do
          unquote(name)(attrs, api_version)
        end
      end
    end
  end)
end

This enables the following call to generate our version-checked API functions:

Versioning.api_version_wrap([
  :get_available_minigames,
  :get_available_filtered_minigames,
  :get_minigame_details,
  :play_minigame,
  :get_minigame_details,
  :complete_minigame,
  :on_mobile_heartbeat
], "0.0.1")

Much better!

For examples that leverage for comprehensions and generate functions within macros, check out this updated blog post.

Summary

We began by writing WET code and embraced some duplication, avoiding premature optimization or early abstraction. Once we had three or more similar examples, we looked for ways to DRY up our code.

Elixir macros offered the ability to DRY up repeated structures by compressing them in our code, then expanding them at compile time. Because of their power, they should be used sparingly and as a last resort. When employed effectively, macros create clearer, lighter-weight, more maintainable code. They help increase the signal/noise ratio of a codebase. Macros offer custom control over our desired module APIs and stacktraces for easier debugging.