SchemeUnit 3 has been released. Although the interface remains compatible with version 2 the underlying philosophy of SchemeUnit has changed in a significant way. The following is extract from the SchemeUnit manual, describing the new approach.

SchemeUnit is designed to allow tests to evolve in step with
the evolution of the program under testing. SchemeUnit
scales from the unstructed checks suitable for simple
programs to the complex structure necessary for large
projects.

Simple programs, such as those in How to Design Programs,
are generally purely functional with no setup required to
obtain a context in which the function may operate.
Therefore the tests for these programs are extremely simple:
the test expressions are single checks, usually for
equality, and there are no dependencies between expressions.
For example, a HtDP student may be writing simple list
functions such as length, and the properties they are
checking are of the form:

(equal? (length null) 0)
(equal? (length '(a)) 1)
(equal? (length '(a b)) 2)

SchemeUnit directly supports this style of testing. A check
on its own is a valid test. So the above examples may be
written in SchemeUnit as:

(check-equal? (length null) 0)
(check-equal? (length '(a)) 1)
(check-equal? (length '(a b)) 2)

Simple programs now get all the benefits of SchemeUnit with
very little overhead.

There are limitations to this style of testing that more
complex programs will expose. For example, there might be
dependencies between expressions, caused by state, so that
it does not make sense to evaluate some expressions if
earlier ones have failed. This type of program needs a way
to group expressions so that a failure in one group causes
evaluation of that group to stop and immediately proceed to
the next group. In SchemeUnit all that is required is to
wrap a test-begin expression around a group of
expressions:

(test-begin
(setup-some-state!)
(check-equal? (foo! 1) 'expected-value-1)
(check-equal? (foo! 2) 'expected-value-2))

Now if any expression within the test-begin

expression fails no further expressions in that group will
be evaluated.

Notice that all the previous tests written in the simple
style are still valid. Introducing grouping is a local
change only. This is a key feature of SchemeUnit’s support
for the evolution of the program.

The programmer may wish to name a group of tests. This is
done using the test-case expression, a simple
variant on test-begin:

(test-case
"The name"
... test expressions ...)

Most programs will stick with this style. However,
programmers writing very complex programs may wish to
maintain separate groups of tests for different parts of the
program, or run their tests in different ways to the normal
SchemeUnit manner (for example, test results may be logged
for the purpose of improving software quality, or they may
be displayed on a website to indicate service quality). For
these programmers it is necessary to delay the execution of
tests so they can processed in the programmer’s chosen
manner. To do this, the programmer simply wraps a test-suite
around their tests:

(test-suite
"Suite name"
(check ...)
(test-begin ...)
(test-case ...))

The tests now change from expressions that are immediately
evaluated to objects that may be programmatically
manipulated. Note again this is a local change. Tests
outside the suite continue to evaluate as before.

2.1 Historical Context

Most testing frameworks, including earlier versions of
SchemeUnit, support only the final form of testing. This is
likely due to the influence of the SUnit testing framework,
which is the ancestor of SchemeUnit and the most widely used
frameworks in Java, .Net, Python, and Ruby, and many other
languages. That this is insufficient for all users is
apparent if one considers the proliferation of “simpler”
testing frameworks in Scheme such as SRFI-78, or the the
practice of beginner programmers. Unfortunately these
simpler methods are inadequate for testing larger
systems. To the best of my knowledge SchemeUnit is the only
testing framework that makes a conscious effort to support
the testing style of all levels of programmer.

Over on LtU I was asked how to help beginning programmers become self-directed learners. I have taught a number of students, but not in a context where I’ve been able to really make a difference in their programming practice, so I don’t have an answer to the whole question (though my instinct is that the apprenticeship method is the right way to go). However I try to teach one process that I think is an essential step towards becoming a self-directed learner. That process is the big idea called science, and I’m not talking about lab coats and chunky glasses

When working with students I always get asked what the result of evaluating some piece of code will be. What I tell the students is toask the computer via a test case or the REPL. Testing ideas by experimentation is science in its simplest and most immediate form, and a crucial step in developing the student’s ability to solve their own problems.

Application of science to programming is not restricted to students;test-driven development is science. So what is science then? I simplify, but basically three things: a theory to test, an experiment to test it, and a standard of proof (note we can never truly prove a theory, just simply not be able to disprove it). This is exactly when a unit test is. For example, a Scheme programmer might pose the theory “the string->number function will convert strings padded with whitespace characters to numbers”, formulate the experiment(equal? (string->number “ 200”) 200), and have as the standard of proof the boolean output of this single experiment.

When most people think of the scientific method they think of the lengthy and expensive double-blind trials used in, for example, medical trials. A really important point is to realise that when you do science, you choose the standard of proof. For example, as most computer programs are deterministic a few tests can be sufficient to show a property holds. When dealing with a concurrent program, or some other non-deterministic system, you may need to be more rigourous.

So there we have it. All programmers are scientists to some extent, though they might not know it. We can extend the use of experimentation to answer other questions, such as determining if productivity is affected by changes to software development process. Doing this in a lightweight way is the intention of the Simple Improvement package, though I haven’t had the time to get that library to a really useful state. Perhaps in a later post I’ll go through the ideas behind it. In the mean time, get experimenting (lab coats optional).