13 Nov 2006

## New Software, For You!

We released two bits of code a little while ago, both at
PLaneT:

• A new minor version of <a
href=”http://planet.plt-scheme.org/300/#schemeunit.plt”>SchemeUnit
that adds a `check-=` form for comparing numbers
within a specified tolerance, and fixes a few bugs.
• A new add-on to SchemeUnit, called <a
href=”http://planet.plt-scheme.org/300/#benchmark.plt”>Benchmark
that, as the name suggests, adds forms for benchmarking
code.

A simple example of using the Benchmark library:

Suppose you were benchmarking some functions that worked
on vectors of numbers, and you wanted to see if the SRFI-43
`vector-map` was faster than writing a loop by
hand. You can test this assumption using the
`check-faster` form:

```(module test mzscheme

(require
(lib "vector-lib.ss" "srfi" "43")
(planet "test.ss" ("schematics" "schemeunit.plt" 2))
(planet "text-ui.ss" ("schematics" "schemeunit.plt" 2))
(planet "benchmark.ss" ("schematics" "benchmark.plt" 1)))

(define big-vector
(vector-unfold
(lambda (i x) (values i x))
1000
0))

(define check-map
(test-case
"Check vector-map is faster than hand-written loop"
(check-faster
(lambda ()
(vector-map - big-vector))
(lambda ()
(let loop ([vec (make-vector 1000)]
[idx 1000])
(if (zero? idx)
vec
(begin
(let ([idx (sub1 idx)])
(vector-set! vec idx (- (vector-ref big-vector idx)))
(loop vec idx)))))))))

(test/text-ui check-map)
)```

On my computer the hand-written loop is a fraction faster
than `vector-map`, so if performance is essential
than the loop is to be preferred.

By formalising assumptions as tests you automatically get
notified when implementation changes render them invalid.
So if changes in the JIT compiler made
`vector-map` faster this test would fail and I
would know to rewrite my performance critical code.

Often it isn’t convenient to keep two versions of a
function around, perhaps because the implementation depends
on many modules. In this case it is useful to benchmark the
implementation against its past performance. You can do
this by creating a `benchmark-case` where you
would otherwise create a `test-case`. An example
is in order: Say you have a complicated function
`foo` and you want to ensure your optimisations
are making it faster. Then you simply write:

```(benchmark-case
"foo benchmark 1"
(foo some-big-input))```

The Benchmark library automatically saves performance
data and fails this test if `foo` becomes slower.
The name of the test case is important, as this is what
the Benchmark library uses to find historical data.

That’s it. As you can see the Benchmark library is quite simple, but I have found it very useful when optimising code. I hope you do as well!

Posted in Code | Comments Off on New Software, For You!