We released two bits of code a little while ago, both at
PLaneT:
- A new minor version of <a
href=”http://planet.plt-scheme.org/300/#schemeunit.plt”>SchemeUnit
that adds a check-= form for comparing numbers
within a specified tolerance, and fixes a few bugs.
- A new add-on to SchemeUnit, called <a
href=”http://planet.plt-scheme.org/300/#benchmark.plt”>Benchmark
that, as the name suggests, adds forms for benchmarking
code.
A simple example of using the Benchmark library:
Suppose you were benchmarking some functions that worked
on vectors of numbers, and you wanted to see if the SRFI-43
vector-map was faster than writing a loop by
hand. You can test this assumption using the
check-faster form:
(module test mzscheme
(require
(lib "vector-lib.ss" "srfi" "43")
(planet "test.ss" ("schematics" "schemeunit.plt" 2))
(planet "text-ui.ss" ("schematics" "schemeunit.plt" 2))
(planet "benchmark.ss" ("schematics" "benchmark.plt" 1)))
(define big-vector
(vector-unfold
(lambda (i x) (values i x))
1000
0))
(define check-map
(test-case
"Check vector-map is faster than hand-written loop"
(check-faster
(lambda ()
(vector-map - big-vector))
(lambda ()
(let loop ([vec (make-vector 1000)]
[idx 1000])
(if (zero? idx)
vec
(begin
(let ([idx (sub1 idx)])
(vector-set! vec idx (- (vector-ref big-vector idx)))
(loop vec idx)))))))))
(test/text-ui check-map)
)
On my computer the hand-written loop is a fraction faster
than vector-map, so if performance is essential
than the loop is to be preferred.
By formalising assumptions as tests you automatically get
notified when implementation changes render them invalid.
So if changes in the JIT compiler made
vector-map faster this test would fail and I
would know to rewrite my performance critical code.
Often it isn’t convenient to keep two versions of a
function around, perhaps because the implementation depends
on many modules. In this case it is useful to benchmark the
implementation against its past performance. You can do
this by creating a benchmark-case where you
would otherwise create a test-case. An example
is in order: Say you have a complicated function
foo and you want to ensure your optimisations
are making it faster. Then you simply write:
(benchmark-case
"foo benchmark 1"
(foo some-big-input))
The Benchmark library automatically saves performance
data and fails this test if foo becomes slower.
The name of the test case is important, as this is what
the Benchmark library uses to find historical data.
That’s it. As you can see the Benchmark library is quite simple, but I have found it very useful when optimising code. I hope you do as well!
