A little while ago we<a
href=”http://www.untyped.com/untyping/archives/2007/06/selenium_code_r.html”>released
an interface to Selenium, a web testing framework.
Since then we’ve learned that Selenium is simply too slow to
use in our work-flow. Hence we started on a faster
reimplementation of the Selenium Remote Control.
A key part of this system is a proxy server, which is
necessary to get around the same origin security restriction in Javascript. I’ve just finished a
large refactoring of the proxy code, and I think the
experience is interesting enough to warrant a blog post.
While there is a large literature on refactoring object-oriented programs, there is rather
less on refactoring functional programs, and what there is tends to concentrate on program
transformation tools (long a FP strength) at the expense
of collecting useful FP refactorings. This post is a small
contribution to redressing the balance.
The code I spent most time on was the HTTP parser. It is
structured as a state machine, so the initial version used
the classic FP pattern ofmutually tail recursive functions. The code for parsing an HTTP request looked something
like this:
(define (parse-request)
(define request-line #f)
(define headers #f)
(define (parse-request-line)
(set! request-line (read-request-line))
(parse-headers))
(define (parse-headers)
(let ([line (read-line)])
(if (end-of-input line)
(begin (set! headers (reverse headers))
(do-something))
(begin (set! headers (cons line headers))
(parse-headers)))))
(parse-request-line))
The code for parsing an HTTP response was very similar:
(define (parse-response)
(define response-line #f)
(define headers #f)
(define (parse-response-line)
(set! response-line (read-response-line))
(parse-headers))
(define (parse-headers)
(let ([line (read-line)])
(if (end-of-input line)
(begin (set! headers (reverse headers))
(do-something-different))
(begin (set! headers (cons line headers))
(parse-headers)))))
(parse-response-line))
The real code was several screens long. I wanted to make
it
simpler by changing to a functional style, and reusing
common code between the request and response parsing
functions. Converting to functional style is simple:
(define (parse-request)
(define (parse-request-line)
(define request-line (read-request-line))
(parse-headers request-line))
(define (parse-headers request-line)
(define headers
(let loop ([line (read-line)])
(if (end-of-input line)
null
(cons line (loop (read-line))))))
(do-something request-line headers))
(parse-request-line))
Reusing common code is not simple. The finite state
machine
pattern doesn’t abstract the next state. For example
parse-headers in parse-request
always calls do-something whereas the
otherwise
identical version in parse-response calls
do-something-different.
I solved this by refactoring the code into<a
href=”http://library.readscheme.org/page6.html”>continuation-passing
style, leading to code that looks like the following:
;; shared between parse-request and parse-response
(define (parse-headers request-line k)
(define headers
(let loop ([line (read-line)])
(if (end-of-input line)
null
(cons line (loop (read-line))))))
(k request-line headers))
(define (parse-request)
(define (parse-request-line k)
(define request-line (read-request-line))
(k request-line))
(parse-request-line
(cut parse-headers <> do-something)))
Note that I’ve used <a
href=”http://srfi.schemers.org/srfi-26/srfi-26.html”>cut
as a short-cut for lambda.
I’ve got code reuse but the code itself isn’t nice. The
arguments lists were quite a bit longer in the real code
and
most of the time arguments are just passed from function
to
function without being used (I’ve seen these sort of
arguments called “tramp data”). I also find
that CPSed code can be quite difficult to read — you
have to construct the control flow graph in your head and
then look at the application site to fill in all the
continuations. Ugh.
One way to get rid of tramp data is to use parameters, and this something we talk about in our experience report. However that solution isn’t appropriate here. It
forces me to stick with CPS so I can set the parameters
in the dynamic extent of the succeeding code, and it
extends the lifetime of the values beyond what is
strictly necessary. This could be an issue if storing,
say, a large request body in a parameter.
Notice that the different versions of
parse-request only differed in which
function
they called with the value they computed. If I separate out
the computation of that value, and the decision of which
function to call I can get code reuse without CPS, and I
don’t have long argument lists! This is what my final solution
looks like:
;; shared between parse-request and parse-response
(define (parse-headers request-line)
(let loop ([line (read-line)])
(if (end-of-input line)
null
(cons line (loop (read-line))))))
(define (parse-request)
(define (parse-request-line)
(read-request-line))
(do-something (parse-request-line)
(parse-headers)))
(define (parse-response)
(define (parse-response-line)
(read-response-line))
(do-something-different (parse-response-line)
(parse-headers)))
It’s short and sweet, and easy to understand.
So let’s recap what I did:
-
I started with the mutually tail-recursive FSM pattern
(that’s a mouthful!)
- I refactored into continuation-passing style.
-
I separated computation and control, and refactored
back to direct style.
So three refactoring (direct style to CPS, separating
computation and control, and CPS to direct style), two of
which are particular to functional languages, and one
pattern. I could do with a better name than
“separating computation and control”. If you’re
aware of some prior work or can think of a better name do
let me know.
Although they use different terminology, the programming
language theory and software engineering communities have
explored a lot of the same ground from different
perspectives. Program transformations are pretty much the
same thing as refactorings, though the former are often
presented in the context of compiler optimisations.<a
href=”http://www.haskell.org/haskellwiki/Research_papers/Functional_pearls”>Functional
Pearls are very similar to design patterns.
If you’re a student of software engineering in
functional
languages it is necessary to familiarise yourself with this
literature. This can be difficult. There are no books
summarising this literature, as you’ll find for OO
languages, and the papers are often terse and are not
focused on software engineering. This means they can be
difficult to read if you don’t have a background in
programming languages, and you have to read between the
lines a bit.
