How Is Test Babby Formed?

tl;dr via :test metadata

In a previous post, I mentioned how you could use the test defined via test.check’s defspec macro as a function, calling it directly. I got curious to find out more about how that works.

Turns out, the core of what defspec does is simply defining a function; the secret sauce is that it also sets the :test metadata on the function var. It’s that metadata that clojure.test looks for when it comes along in response to a call to `run-tests`.

You can get a feel for how that works by defining some inline tests.

(ns foo
  (:require [clojure.test :refer [is]]))

(defn
  ^{:test #(is (= "Hello, world!" (hello "world")))} ❶
  hello [someone]
  (str "Hello!")) ❷

1. ❶ The metadata map has a single entry: a :test key mapping to a zero-arity function.
2. ❷ This implementation is not correct.

I can run those tests (with, say, clj -e "(require 'foo) (clojure.test/run-tests 'foo)", and get the expected failing output:

Testing foo

FAIL in (hello) (foo.clj:5)
expected: (= "Hello, world!" (hello "world"))
  actual: (not (= "Hello, world!" "Hello!"))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}

And fixing the implementation and re-running the tests will bring us satisfaction:

 (ns foo
   (:require [clojure.test :refer [is]]))

 (defn
   ^{:test #(is (= "Hello, world!" (hello "world")))}
   hello [someone]
-  (str "Hello!"))
+  (str "Hello, " someone "!"))

Testing foo

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
{:test 1, :pass 1, :fail 0, :error 0, :type :summary}

And this is what macros like clojure.test’s own defest and test.check’s defspec are doing: assigning a function to a var, and then setting the :test metadata of that var to something that can invoke that function in a testing context.

Of course I’m just scratching the surface. Suffice it to say, there’s more to clojure.test’s implementation than just grabbing a function out of a var’s metadata map and invoking it. While at heart clojure.test is working with a map of test information (you can see the summary map in the test output above), there’s a lot of plumbing to hook into the fixture and reporting machinery.

Fortunately, that plumbing is there, so there’s no need to rebuild your testing infrastructure from scratch. The is macro takes care of wiring a simple assertion into all of that plumbing in a very clean way; I’ve gained more respect for the leverage those two characters provide. And, of course, defspec shows that it’s quite possible to extend that plumbing when needed.

Finally, it’s also cool to see a practical use case for metadata. When I’ve reached for metadata in my own projects, it’s often a poor fit, or better expressed as actual data in the domain model.