(In the middle of changing.)
(Add two other approaches to the design.)
Revision as of 06:39, 21 March 2007
- Help library developers ensure that their changes work.
- Enable something like BuildBot to ensure that libraries keep working across all the Implementations.
- Enable test-driven development if a developer wants to use it.
- Provide a series of testcases for implementation developers.
According to Ian Lynagh, SimonM has been moving tests a directory at a time from the testsuite package that comes with ghc to libraries/foo/tests. It looks like he's gotten to unix and network. The tests "assume the package is part of a GHC build tree with the testsuite installed in ../../../testsuite."
Neil Mitchell has a program at http://www.cs.york.ac.uk/fp/darcs/filepath/GenTests.hs to automatically extract tests from Haddock comments like the ones in http://www.cs.york.ac.uk/fp/darcs/filepath/System/FilePath/Version_0_11.hs.
I think the tests should be run with
runhaskell Setup.hs test. Cabal's going to need some work to make this work portably. JeffreyYasskin 06:39, 21 March 2007 (UTC)
3.1 Abbreviated QuickCheck in Haddock comments
Need to recognise parameters. GenTests.hs currently recognises a small range of special variables. What about declaring the parameter names and associated types in a header comment? For instance
-- > :type xs, ys :: [Int]
this would declare that xs and ys are free variables in any following tests and of the type specified. They can then be recognised at the lexical level.
Alternatively, what about special "forall" syntax such as:
forall (xs, ys :: [Int]) . reverse xs ++ reverse ys == reverse (ys ++ xs)
GenTests currently has flag syntax for Posix and Windows tests (necessary for FilePath). Should this be in a general solution? If so, are other flags required?
Do we need special syntax for
3.2 #ifdefed tests in code
http://darcs.haskell.org/packages/base/Data/Map.hs could easily be moved to this form if we replace the
This poses some problems for arranging the compiled output. The #define flags generally don't show up in the name of the output file, but we want to make sure that even if you've run the tests,
runhaskell Setup.hs install installs the non-testing libraries.
dist/test/tree with -DTESTING on. If we have A.hs and B.hs where each exports some implementation functions behind
3.3 Separate test programs
Should be in the style of http://darcs.haskell.org/packages/network/tests/, but with a driver that doesn't depend on living inside a ghc source tree.
I personally dislike the "golden file" approach that these test use, but the existing driver also supports a
run_command_ignore_output test-fn, which we could combine with the
exit_code(0) opt-fn to use any testing framework we want. JeffreyYasskin 06:39, 21 March 2007 (UTC)