Untyping

The Untyped Weblog.
Read on or return home.

Posts in the ‘Javascript’ category

7 Aug 2013

by Dave

Writing Documentation using Grunt and Jekyll

Running a startup is a lot of work. All manner of tasks constantly compete for the team’s attention mandating a careful dance to keep on top of everything. We haven’t nailed all the footwork to this dance yet, but we have learned that choosing the right tool for each job can drastically simplify the choreography.

We recently rewrote a large portion of Myna in the march towards version 2 of the platform. Most of the customer-facing software - the dashboard, client libraries, HTTP API, and so on - were completely new and required new documentation. Our existing help pages were in need of an overhaul, so we decided to move them out into their own project using a build system based on Grunt and Jekyll.

The new documentation is still a work-in-progress - you can watch its evolution on the Myna website and its repository on Github (yes, it’s open source - another experiment we’re trying). We’re really happy with the way it’s all working out, so we’ve published the build system as a separate project that you can use to bootstrap your own documentation. Go forth, fork, and profit!

Why Jekyll?

Our old documentation was implemented as a set of templates in the Play 2 web app that runs our marketing site and original customer dashboard. Publishing the app requires a PhD in SBT, Mongo and Redis, and if you’re writing documentation on the train (as we are prone to do) it isn’t uncommon to have your plans abruptly terminated by one of SBT’s frequent unavoidable urges to download the whole internet (unadvisable in the middle of signal-free rural England).

We considered moving to a CMS such as WordPress, but our support team are all developers with their own preferred editors and IDEs. Forcing them to write documentation (painful) in a tiny WYSIWYG editor embedded into a web site seemed like torture. Also, CMSs require internet connections… it kinda goes with the territory.

Grunt and Jekyll, in contrast, run completely offline, and have a number of other advantages too. Plugins like grunt-contrib-watch provide instant previews via Livereload, and Jekyll’s syntax highlighting (provided by Pygments) can highlight any syntax you throw at it (including, to my amazement, HTTP).
Jekyll isn’t completely perfect for the job. We had to work around a few issues. Fortunately, none of them proved insurmountable:

Versioning

We’ve run into versioning problems many times before, some requiring some pretty serious workarounds. In fact, versioning issues are pretty much endemic across all software development platforms. In this toolchain we’re relying on lots of components: Grunt, five Grunt plugins, Ruby, and Jekyll. Fortunately, versioning is pretty much a solved problem these days: NPM and Bower are great package managers for Node, and Bundler normalizes not only the version of Jekyll we’re using, but also the version of Ruby itself.

Static Assets (Say “NO” to Plain CSS)

I may catch some flack for this, but CSS is a silly language riddled with missing features and bizarre design decisions. No way are we going to battle with a new documentation project without tools like Less CSS and Twitter Bootstrap to support us. And if we’re compiling and minifying our CSS, we might do it for Javascript as well. Jekyll doesn’t support support for either process out-of-the-box.

One way of solving these issues would be to use Jekyll plugins - there are many candidates available on Github. However, we prefer to use Grunt for this kind of thing, running Jekyll via grunt-exec and Bundler, and using grunt-contrib-watch and grunt-contrib-connect for preview functionality.

Content Navigation

Jekyll has built-in support for cataloguing and paginating blog posts, but it can’t natively generate navigation for a hierarchical documentation site. Fortunately, this was easy to work around with a couple of custom plugins: one to create a table of contents for the sidebar, and one to generate next and previous buttons at the bottom of each page.

Authenticating Users

The main navigation bar on Myna changes when users log in. Ideally we want like to keep this consistent across the main web site, the blog, and the documentation. Our solution is to built a small Javascript app to monitor the user’s login details and rewrite the navbar on demand. This is a work-in-progress project and it’s not part of the Github repo above.

Conclusion

If you like the idea of writing documentation in Markdown, you can get started in two minutes by cloning our Github repo and following the instructions in the README. We’d love to hear from you if you find our system useful, and we welcome pull requests with improvements.

Posted in Code, Front page, Fun, General, Javascript, Myna, Web development | Comments Off on Writing Documentation using Grunt and Jekyll

6 Mar 2011

by Dave

Javascript compilation for SBT

Over the weekend I knocked up a little SBT plugin to wrap up the Javascript resources in our Lift projects and deploy them as one big minified file. Read on to find out how it works, then grab yourself a copy and take it for a spin.

The plugin scans your webapps directory and looks for files with the extensions .jsm or .jsmanifest. These files, called Javascript Manifests, describe lists of Javascript sources that should be combined into a single file. For example: 

# You can specify remote files using URLs...

http://code.jquery.com/jquery-1.5.1.js

# ...and local files using regular paths
#    (relative to the location of the manifest):

lib/foo.js
bar.js

# Blank lines and bash-style comments are also supported

Manifest compilation happens in two phases: first, the plugin downloads and caches any remote scripts specified using URLs. Second, it feeds all of the sources (remote and local) into Google’sClosure Compiler, which concatenates them and minifies everything (and provides excellent services like static type checking to boot). The output from the compiler is a .js file with the same base name and relative path as the original manifest.

There’s not a lot more to it than that. The plugin hooks into SBT’s standard compile and package phases, so your Javascript gets rebuilt automatically alongside your Scala code. If this sounds useful to you, please feel free to grab a copy and take it for a spin. Full details are available in the README on Github.

I should point out that there are other useful SBT plugins that do a similar job. For example, I plagiarised extensively from Jon Hoffman’sYUI Compressor plugin and Luke Amdor’s Coffee Script plugin when writing my code. These two particular examples don’t do file combination, though, and that was an important feature for our specific use case.

Posted in Code, Front page, Javascript, Scala, Web development | Comments Off on Javascript compilation for SBT

24 Jan 2011

by Dave

Smooth Scrolling for Mobile Safari

I recently wrote a jQuery plugin to do some smooth scrolling on the iPad, and I thought I’d share the code with everyone.

The effect you get is very similar to the iOS home screen. The user touches the screen and drags to scroll. Releasing the screen causes it to spring to the most appropriate page based upon the last dragging position and speed.

Gurus of front end development tell us that pretty much the only way to get smooth transitions on the iPad is to use 3D CSS transforms. After experimenting with jQuery animations and 2D CSS transforms, I pretty much concur: jQuery animations yield one or two frames per second, and 2D CSS transforms aren’t much better. 3D CSS transforms, on the other hand, are hardware accelerated and smooth as silk.

You can get the code from this Gist on Github (contributions and enhancements welcome). Use it with the following HTML: 

 <div id="viewport"> <div>First page</div> <div>Second page</div> <div>Third page</div> </div>

and the following Javascript:

 $("#viewport").scrollpane();

There’s a demo of it in action here. A couple of notes:

  • Because this hooks into touch gesture events and CSS3 3D transforms, it’ll pretty much only work on iDevices and possibly other Webkit-based tablets.
  • It works horizontally and vertically, but I’d recommend only using it horizontally in a regular web page because it interferes with Safari’s natural screen bounce. I had the benefit of a working on an offline brochure where the web page never scrolls naturally. In this environment the plugin really shines. If you are interested in doing something similar, take a look at the iPad app Delivery Site, which lets you customise various things like this.
  • There are a couple of options you can tweak to affect things like dead-zones before a drag will trigger a page transition. See the top of the source code for details.
  • When the first 3D transform is added to a page, Mobile Safari seems to transparently install an OpenGL panel to handle the effects. This causes a rendering glitch that’s just faintly visible if you’re paying attention. The plugin works around this by setting an identity transform on the scroll component on page load. Webkit is presumably frugal about 3D-ification for a reason, so you may find your web pages take more memory and CPU resources with this plugin active than without.
  • Really large (read “many-page, full-screen”) scroll panes can be very heavy on the browser. This is presumably due to the overhead of creating a texture buffer to 3D accelerate the transitions. I’ve managed five-page full-screen scrolling transitions without problems, but your mileage may vary.

Posted in Code, Front page, Fun, Javascript, Web development | Comments Off on Smooth Scrolling for Mobile Safari

4 May 2006

by Noel

The Rumour Starts Here!

Salacious rumour suggests the next version of Javascript will have tail recursion. We can neither confirm nor deny this rumour, but we can rock an old style: w00t!

Posted in Javascript | Comments Off on The Rumour Starts Here!

20 Feb 2006

by Noel

More on the next Javascript

Brendan Eich writes on Python and JavaScript. The first part discusses his plans to add generators and array comprehensions to Javascript. But that’s not the real news! It comes later:

I’m happy to announce that we are now working with Dave Herman, a fourth year graduate student at Northeastern, of PLT and lambda-the-ultimate renown, whom I invited as an expert to help ECMA TG1 develop sound specifications for critical parts of ECMAScript Edition 4 (ES4), also known as JavaScript 2 (JS2).

Go Dave!

Posted in Javascript | Comments Off on More on the next Javascript

27 Jan 2006

by Noel

Ajax: I Fold!

Ok, I fold. It’s time to admit I agree with what Matt has
been saying: Ajax is an immature technology. To save Matt
some effort I’ll list the things that most annoy me with the
current state of the art:

Matt’s already mentioned the (lack of)<a
href=”http://www.untyped.com/untyping/archives/2005/11/rich_web_client.html”>type
system and<a
href=”http://www.untyped.com/untyping/archives/2005/12/printf_in_ajax.html”>poor
debugging support, so I won’t go into them.

Memory leakage is perhaps the biggest obstacle to
creating long-lived Ajax applications. The reasons for
memory leaks are discussed<a
href=”http://www.mozilla.org/scriptable/avoiding-leaks.html”>here
and<a
href=”http://blogs.msdn.com/ericlippert/archive/2003/09/17/53028.aspx”>here.
These leaks are a result of a flaw in the implementation of
(to my knowledge, all) existing browsers. This puts us back
into the bad old days of manual memory management. Either
we be very careful in our programming, or our long lived
application will eat memory till the browser crashes.

Browser incompatabilities and inconsistencies, admirably
documented at Quirks
Mode, are a major hassle. We have to worry about both
browser and version — my code works in Firefox 1.0.4,
but will it work in Internet Explorer 5.0 (probably not)?

With the benefit of experience Javascript could be improved in a number of ways.<a
href=”http://calculist.blogspot.com/2005/12/dynamic-scope.html”>Javascript’s
crazy scoping rules are just bad; there’s no way of
getting around that. Javascript would really benefit from coroutinesfor writing all those animation loops (or heck, let’s get full continuations). Javascript’s syntax is unnecessarily hard to parse, mostly due to the semi-colon insertion rule .

Finally, current implementations of Javascript are slow and
resource hungry, a major impediment to creating really
featureful applications.

Ok, so where does this leave us? We still want to build great Internet applications, but the tools are a bit suck. There are three paths forward:

  • Build libraries. Requires the least amount of time, but offers the least return.
  • Build a better Javascript. This has already started. Just wait two years for the standardisation committee to finish, and then another five Internet Explorer to catch up.
  • Use the existing language as the target, but build a better language that compiles into Javascript. Less effort than standardising Javascript, but more effort that writing libraries, this option has some attractive benefits.

So which path are we going to follow? Tune in next time!

Posted in Javascript | Comments Off on Ajax: I Fold!

22 Dec 2005

by Noel

Printf in AJAX? Sorry, that’s not debugging.

This post is one in a continuing series where we internally debate the merits of the over-hyped promise of AJAX…

Debugging Ajax Requests in Prototype:

How do we debug our Ajax applications? The Rails Weenie has taken the Ajax Responder feature in Prototype…

I’d like to remind Noel that the distance of the Atlantic and business of the Christmas holidays are not going to keep me from pointing out that AJAX is an immature and dangerous platform to build a business on. Yes, I’m glad that GMail is there… and I suppose the Yahoo! Mail beta. However, these are fragile technologies to build upon.

How do we debug things written using the Prototype framework?printf. I mean, I’m glad that with AJAX and Rails I can whip something up quickly that “just works.” However, it doesn’t “just have test cases,” or “just get internationalized”, or “just stand up under load.” They’re rapid-prototyping tools, certainly nothing more. AJAX breaks usability standards, pushes data and computation to an unreliable substrate (the client’s web browser), and as the post above provides some evidence for, there are no good debugging or tracing tools available for developers working in heterogeneous browser environments in Javascript.

So, Noel, riddle me this: why should developers be willing to take eight steps backwards and be shafted with printf as their primary debugging tool when working with AJAX?

Posted in Javascript, Web development | Comments Off on Printf in AJAX? Sorry, that’s not debugging.

4 Nov 2005

by Noel

Rich web clients

Matt likes to provide the dissenting Ajax voice here on untyping.

As overhead on Slashdot:

‘Zimbra’s chief executive, Satish Dhamaraj, says that when he started his company in December 2003, “I really thought that Ajax was just a bathroom cleaner.” Now his San Mateo, Calif., business has amassed $16 million in funding from venture-capital firms including Accel Partners, Redpoint Ventures and Benchmark Capital, the firm that famously funded eBay Inc. Peter Fenton, an Accel partner, says Ajax “has the chance to change the face of how we look at Web applications” and could boost technology spending by corporations, because Ajax is also being used to develop software for big companies, not just for consumers.’”

Ajax is a dangerous and immature technology. It is, in fact, a hack—akludge—to provide rich-client functionality in the browser.

While I’ll acknowledge that Javascript was intended from day one for interacting with the DOM, building entire applications that are predicated on dynamic, untyped tree-transformations is suicide (at best) without significant software development experience and rigor. With that rigor, it’s just dangerous.

What do I mean? Pretend that every HTML element has a shape. If that’s the case, then a document with a series of paragraphs might look like:

Tree1
In this picture, the octagon represents the body tag, and the circles represent the paragraphs. If emphasis tags look like a small square, then (zooming in) our picture might look like:

Tree2
Unfortunately, to make Ajax work, you have to constantly reorganize this tree. The tree is everything: it is the UI, it is the content, and it is the data for the program you are writing. And, I can casually replace any one node with a new node of a different type, or I can insert whole, new subtrees.

Anywhere. Anytime. And, without any validation or safety. So, I can take my first picture, and insert an entire copy of it anywhere I like:

Tree3
The only problem with this is that, if the octagon is a body tag, this iscompletely invalid HTML. But that’s not the worst of it; the tree representing all of my interface, content, and data is a massive, and complex, piece of state. So, after part of my program modifies the tree, I have to forever remember that the state has changed. Keeping track of state is already a dangerous part of programming; it is the cause of crashes in unithreaded programs (memory leaks, for example), and a source of huge problems in multithreaded programs (via problems known as deadlock, livelock, race hazards, etc.). This is why Google Mail and other Ajax-y applications are a miracle of hackery: the fact that they run, without completely buggering up constantly, is a nothing short of a holy intervention on the part of some higher being.

Given that the compiler does nothing to check your typing, and that you have almost no run-time debugging support, Ajax (as currently exemplified by Javascript and XML over HTTP) is certainly only version 0.5 as tool or solution for web-based development goes. Or, perhaps it is just part of a space of tools, where some produce very portable/accessible applications, while others allow the developer to produce richer, more interesting applications (while sacrificing portability).

Spectrum
It may be that Ajax is just the most portable, and least rich/safe way to develop applications for end-users at the moment. Rich clients, likeOpenLaszlo, represent a greater level of safety, as well as a much richer set of tools. Widgets, like Apple’s Dashboard and Yahoo’s Widgets (formerly Konfabulator, an excellent, cross-platform predecessor to Apple’s Dashboard) provide some of the ease of development of the WWW with much of the richness of the desktop—yet there is a run-time engine that provides a modicum of protection and safety for the end-user. And, of course, desktop applications: they sacrifice portability, but give you the full power of the host machine.

I started this post with a quote about people investing millions in Ajaxy technology. People investing millions in Ajaxy startups scare me; they’re sheep, following the herd. However, Ajax (as exemplified by Javascript and HTML) will mature, and we’ll see tools and frameworks develop that will make application development “safer” and less hack-and-slash. Or, it will be abandoned, the <BLINK> tag of the new millennium.

Then again, PHP exists, and it’s a large, steaming turd of a language that grabbed significant market share entirely too easily. So, what can you do?

Posted in Javascript | Comments Off on Rich web clients

15 Sep 2005

by Noel

The Birmingham Course: OpenLaszlo to the Rescue!

We needed to deliver a prototype on-line exam quickly. Despite their simple appearance, on-line exams can have some complex interaction modes that are annoying to deal with in a client/server model.OpenLaszlo provided an excellent framework for quickly going from idea to solution.

 

The Birmingham Course approached us with the idea of
placing online a mock exam for their MRCPsych course. While the time constraints were tight, we thought there was an interesting opportunity in the project, and enjoyed the enthusiasm that The Birmingham Course brought to the table.

I would have liked to do some paper prototyping, and work through the issues that might arise before starting. However, we also wanted students currently enrolled in the course to trial an on-line version of the mock exam. This left us in a bit of a pickle: can you, in one-and-one-half weeks, go from concept to implementation for an on-line examination, and get it right? (I live far enough away from Birmingham that I didn’t have the ability to take a day or two and go up there to meet with the client.)

The mock exam had no data gathering requirements, nor were there any concerns with authentication and identifying users. Therefore, some of the trickier aspects of web-based application development were not concerns. However, we still needed to deliver 133 True/False questions, and 30 multiple-choice questions in a stateful way, so students could easily “go back” and change their answers, as well as get their end-of-exam score.

While there may be AJAX gurus out there who think this would be a piece of cake, I do almost all of this development work in Scheme, and have no particular expertise in HTML/CSS/Javascript interaction on twelve different browser platforms (that’s why I run with a team). I certainly don’t have a lot of patience for developing code in an unstructured and unsupportive environment. Put another way, I think the browser makes a lousy run-time environment; I want my compiler to check for obvious problems up front, and I want my runtime environment to be rich enough that I can go about debugging a running application. Your typical AJAX app fails on these counts far too readily for me to think that any AJAX code I develop in a hurry will be reliable. (I’d like to point out that I’ve never even given AJAX enough of my time to decide if my biases and opinions are right; I’ve only read code and stories from others, and I don’t like what I see.)

Enter OpenLaszlo. It has a compiler, that tells me when I make silly mistakes. It has a rich run-time environment, which includes an interactive debugging REPL. I can create a window as easily as saying 

<canvas><window>Hi there!</window></canvas>

It uses JavaScript 1.5 as it’s scripting language, and has a really sweet little constraint engine built in. The upcoming 3.1 release has some very swish enhancements on the way as well.

I was able to quickly prototype ideas for the customer, push them to the WWW, and let them interact with the application. While it is possible to do all kinds of RPC from OpenLaszlo, I chose to employ another really nice feature of the environment: SOLO application deployment. If you’re willing to give up some (not all) connectivity with remote hosts, OpenLaszlo apps can be compiled to a single, static Flash file. This means that any user with Flash 6 or better can interact with a very rich net-based app, and get a very desktop-like experience.

The end-product of a one-week mad dash can be seen online; I was quite pleased with it. I know I made some choices that were less than optimal, but the code can be refactored and improved in a future iteration of the project. I was also happy working with OpenLaszlo as a development tool—putting together GUIs was no more complex than writing a webpage, the event model and constraint system are both well-behaved and useful, and being able to deliver a static Flash document as an application means I don’t have to worry about a timeouts on client/server interactions, nor do I have to wonder if IE4/IE5/IE6/Opera/Firefox/Mozilla/Safari is going to choke on my particular choice of HTML, CSS, and Javascript idioms. If you’re some kind of “purist” who abhors Flash, then… I can’t help you. But it’s widely deployed, and much better behaved than a gaggle of browsers.

Food for thought. Point is, I liked it. And the customer loved it; the app behaved just the way they wanted. I don’t know what else I can say.

Posted in Javascript, Web development | Comments Off on The Birmingham Course: OpenLaszlo to the Rescue!

18 Aug 2005

by Noel

A little OpenLaszlo lovin’

No doubt I’ll be playing with this technology more in the future, so I thought I’d introduce it now.

OpenLaszlo, released unto the world by Laszlo Systems, is an interesting combination of languages and ideas. It is an XML-based language for specifying the layout and behavior of rich internet applications. For example, I wrote a tabbed slideshow for photographs that you can find on my personal weblog, here….

 

No doubt I’ll be playing with this technology more in the future, so I thought I’d introduce it now.

OpenLaszlo, released unto the world by Laszlo Systems, is an interesting combination of languages and ideas. It is an XML-based language for specifying the layout and behavior of rich internet applications. For example, I wrote a tabbed slideshow for photographs that you can find on my personal weblog, here. I think it would be difficult to write in JavaScript and CSS. Consider:

  1. When you open a slide, audio begins playing.
  2. When you switch slides, the current audio stops, and new audio is started.
  3. I can layer objects (and remove them) over the photo content quickly and easily, when and where I want them.

I imagine this is all possible in JavaScript/DHTML, but… I don’t know how. And, I don’t feel I need to; the OpenLaszlo crew has provided a powerful platform for me to work from—a compiler, GUI toolkit, RPC mechanisms, and more. It seems to me that the whole JavaScript/CSS/AJAX thing is nothing more but a poor reinvention of the technology that OpenLaszlo provides. Granted, for full RPC-functionality, OpenLaszlo currently requires a servlet container, whereas AJAX-apps just need a browser with a JavaScript engine. However, you don’t get much support from the compiler or run-time for debugging AJAX applications, whereas OpenLaszlo apps have a compiler and run-time debugger. That’s worth installing a server to me. (It was a double-click operation on my Mac.)

If I forgo the OpenLaszlo server, I can statically compile my applications; although RPC is no longer available, I can still do HTTP POSTs, and that’s enough to do something RESTful. But, instead of a webpage, I can create a very rich, interactive GUI-based environment that can be delivered to any browser that is Flash-capable. I hear Macromedia has pretty good penetration, and generally things Just Work within that environment, regardless of whether you are on Windows or Mac, IE or Firefox, etc.

So, I think it’s a cool technology. I’ve been poking at the edges of it on-and-off for a few weeks now, and will follow up with some more about what I’ve been doing with it. My tutorial builder/photo slideshow demo just met a need I had; I think OpenLaszlo is capable of much, much more.

Posted in Javascript, Web development | Comments Off on A little OpenLaszlo lovin’