Editing HUnit 1.0 User's Guide (section)

== Running Tests ==

HUnit is structured to support multiple test controllers. The first subsection below describes the [[#Test Execution | test execution]] characteristics common to all test controllers. The second subsection describes the [[#Text-Based Controller | text-based controller]] that is included with HUnit.

=== Test Execution ===

All test controllers share a common test execution model. They differ only in how the results of test execution are shown.

The execution of a test (a value of type <code>Test</code>) involves the serial execution (in the <code>IO</code> monad) of its constituent test cases. The test cases are executed in a depth-first, left-to-right order. During test execution, four counts of test cases are maintained:

<haskell>
data Counts = Counts { cases, tried, errors, failures :: Int }
  deriving (Eq, Show, Read)
</haskell>

* <code>cases</code> is the number of test cases included in the test. This number is a static property of a test and remains unchanged during test execution.
* <code>tried</code> is the number of test cases that have been executed so far during the test execution.
* <code>errors</code> is the number of test cases whose execution ended with an unexpected exception being raised. Errors indicate problems with test cases, as opposed to the code under test.
* <code>failures</code> is the number of test cases whose execution asserted failure. Failures indicate problems with the code under test.

Why is there no count for test case successes? The technical reason is that the counts are maintained such that the number of test case successes is always equal to <code>(tried&nbsp;-&nbsp;(errors&nbsp;+&nbsp;failures))</code>. The psychosocial reason is that, with test-centered development and the expectation that test failures will be few and short-lived, attention should be focused on the failures rather than the successes.

As test execution proceeds, three kinds of reporting event are communicated to the test controller. (What the controller does in response to the reporting events depends on the controller.)

; start : Just prior to initiation of a test case, the path of the test case and the current counts (excluding the current test case) are reported.
; error : When a test case terminates with an error, the error message is reported, along with the test case path and current counts (including the current test case).
; failure : When a test case terminates with a failure, the failure message is reported, along with the test case path and current counts (including the test case).

Typically, a test controller shows ''error'' and ''failure'' reports immediately but uses the ''start'' report merely to update an indication of overall test execution progress.

=== Text-Based Controller ===

A text-based test controller is included with HUnit.

<haskell>
runTestText :: PutText st -> Test -> IO (Counts, st)
</haskell>

<code>runTestText</code> is generalized on a ''reporting scheme'' given as its first argument. During execution of the test given as its second argument, the controller creates a string for each reporting event and processes it according to the reporting scheme. When test execution is complete, the controller returns the final counts along with the final state for the reporting scheme.

The strings for the three kinds of reporting event are as follows.

* A ''start'' report is the result of the function <code>showCounts</code> applied to the counts current immediately prior to initiation of the test case being started.
* An ''error'' report is of the form "<code>Error&nbsp;in:&nbsp;&nbsp;&nbsp;''path''\n''message''</code>", where ''path'' is the path of the test case in error, as shown by <code>showPath</code>, and ''message'' is a message describing the error. If the path is empty, the report has the form "<code>Error:\n''message''</code>".
* A ''failure'' report is of the form "<code>Failure&nbsp;in:&nbsp;''path''\n''message''</code>", where <i>path</i> is the path of the test case in error, as shown by <code>showPath</code>, and ''message'' is the failure message. If the path is empty, the report has the form "<code>Failure:\n''message''</code>".

The function <code>showCounts</code> shows a set of counts.

<haskell>
showCounts :: Counts -> String
</haskell>

The form of its result is  "<code>Cases:&nbsp;''cases''&nbsp;&nbsp;Tried:&nbsp;<i>tried</i>&nbsp;&nbsp;Errors:&nbsp;<i>errors</i>&nbsp;&nbsp;Failures:&nbsp;<i>failures</i></code>" where <i>cases</i>, <i>tried</i>, <i>errors</i>, and <i>failures</i> are the count values.

The function <code>showPath</code> shows a test case path.

<haskell>
showPath :: Path -> String
</haskell>

The nodes in the path are reversed (so that the path reads from the root down to the test case), and the representations for the nodes are joined by '<code>:</code>' separators. The representation for <code>(ListItem <i>n</i>)</code> is <code>(show n)</code>. The representation for <code>(Label <i>label</i>)</code> is normally <i>label</i>. However, if <i>label</i> contains a colon or if <code>(show <i>label</i>)</code> is different from <i>label</i> surrounded by quotation marks--that is, if any ambiguity could exist--then <code>(Label <i>label</i>)</code> is represented as <code>(show <i>label</i>)</code>.

HUnit includes two reporting schemes for the text-based test controller. You may define others if you wish.

<haskell>
putTextToHandle :: Handle -> Bool -> PutText Int
</haskell>

<code>putTextToHandle</code> writes error and failure reports, plus a report of the final counts, to the given handle. Each of these reports is terminated by a newline. In addition, if the given flag is <code>True</code>, it writes start reports to the handle as well. A start report, however, is not terminated by a newline. Before the next report is written, the start report is "erased" with an appropriate sequence of carriage return and space characters. Such overwriting realizes its intended effect on terminal devices.

<haskell>
putTextToShowS :: PutText ShowS
</haskell>

<code>putTextToShowS</code> ignores start reports and simply accumulates error and failure reports, terminating them with newlines. The accumulated reports are returned (as the second element of the pair returned by <code>runTestText</code>) as a <code>ShowS</code> function (that is, one with type <code>(String&nbsp;->&nbsp;String)</code>) whose first argument is a string to be appended to the accumulated report lines.

HUnit provides a shorthand for the most common use of the text-based test controller.

<haskell>
runTestTT :: Test -> IO Counts
</haskell>

<code>runTestTT</code> invokes <code>runTestText</code>, specifying <code>(putTextToHandle stderr True)</code> for the reporting scheme, and returns the final counts from the test execution.