Archive for January, 2006

27 Jan 2006

by Noel

Ajax: I Fold!

Ok, I fold. It’s time to admit I agree with what Matt has
been saying: Ajax is an immature technology. To save Matt
some effort I’ll list the things that most annoy me with the
current state of the art:

Matt’s already mentioned the (lack of)<a
href=”http://www.untyped.com/untyping/archives/2005/11/rich_web_client.html”>type
system and<a
href=”http://www.untyped.com/untyping/archives/2005/12/printf_in_ajax.html”>poor
debugging support, so I won’t go into them.

Memory leakage is perhaps the biggest obstacle to
creating long-lived Ajax applications. The reasons for
memory leaks are discussed<a
href=”http://www.mozilla.org/scriptable/avoiding-leaks.html”>here
and<a
href=”http://blogs.msdn.com/ericlippert/archive/2003/09/17/53028.aspx”>here.
These leaks are a result of a flaw in the implementation of
(to my knowledge, all) existing browsers. This puts us back
into the bad old days of manual memory management. Either
we be very careful in our programming, or our long lived
application will eat memory till the browser crashes.

Browser incompatabilities and inconsistencies, admirably
documented at Quirks
Mode
, are a major hassle. We have to worry about both
browser and version — my code works in Firefox 1.0.4,
but will it work in Internet Explorer 5.0 (probably not)?

With the benefit of experience Javascript could be improved in a number of ways.<a
href=”http://calculist.blogspot.com/2005/12/dynamic-scope.html”>Javascript’s
crazy scoping rules are just bad; there’s no way of
getting around that. Javascript would really benefit from coroutinesfor writing all those animation loops (or heck, let’s get full continuations). Javascript’s syntax is unnecessarily hard to parse, mostly due to the semi-colon insertion rule .

Finally, current implementations of Javascript are slow and
resource hungry, a major impediment to creating really
featureful applications.

Ok, so where does this leave us? We still want to build great Internet applications, but the tools are a bit suck. There are three paths forward:

  • Build libraries. Requires the least amount of time, but offers the least return.
  • Build a better Javascript. This has already started. Just wait two years for the standardisation committee to finish, and then another five Internet Explorer to catch up.
  • Use the existing language as the target, but build a better language that compiles into Javascript. Less effort than standardising Javascript, but more effort that writing libraries, this option has some attractive benefits.

So which path are we going to follow? Tune in next time!

Posted in Javascript | Comments Off on Ajax: I Fold!

25 Jan 2006

by Noel

Yahoo’s Customer Disservice

Sometime in the last two days Yahoo Mail (which I use for my personal email account) broke for me. I can no longer reply to any email; clicking on the Reply button just doesn’t do anything. I searched the web, couldn’t find anything about it, so decided to email Yahoo support. Below is my email and their response.

The ‘Reply’ button is broken on my system (Firefox 1.0.4 on Linux). It was working till a few days ago, now clicking on the Reply button doesn’t go to the ‘Compose’ screen.

I suspect a Javascript error. The following appears in the Javascript console:

… technical stuff …

I’ve tried restarting Firefox and that doesn’t fix it.

Some choice snippets from their response:

Hello,

Thank you for writing to Yahoo! Mail.

We appreciate you bringing this to our attention. Your account appears to be in full working order. We have listed a few troubleshooting steps below which may help resolve this issue:…

  • Add Yahoo! Mail to your list of “Trusted Sites” in Internet Explorer…
  • Switch your browser to Mozilla or Netscape…
  • Finally, shut down and restart your browser.

Thank you again for contacting Yahoo! Customer Care.

Regards,

Danny

Thanks for not reading my email Danny! Had you bothered to read it, you’d have seen what you suggest either I’ve already tried or is inapplicable. In fact you would have seen that the problem is nothing to do with my account, and everything to do with crappy broken code in Yahoo Mail. It’s just too bad I can’t reply to your message to tell you this directly.

It’s amazing that bad customer service is worse than no customer service. Before I contacted Yahoo I merely slightly annoyed. I assumed they’d be aware of the problem (I can’t be the only person facing this) and would fix it reasonably soon. Now, as you’ve guessed, I’m rather more angry. There’s a lesson there.

Posted in Business | Comments Off on Yahoo’s Customer Disservice

22 Jan 2006

by Noel

A Game A Week

At the Experimental Gameplay Project at CMU the participants were challenged to create a game about a particular theme every week. The results are impressive, but more interesting are the insightsderived from observing the process. It’s not about games. It’s about using extreme resource constraints to force innovation. It’s about accepting failure as a necessary part of creativity. It should be read by anyone who wants to turn a big idea into reality.

Posted in Fun | Comments Off on A Game A Week

21 Jan 2006

by Noel

Untyped hosting?

In chatting with Nick about hosting, I began to realize how complex the ISP game can be. These are some thoughts, all provide starting points for further brainstorming and discussion.

We’re building our new machine for ourselves, make no mistake. We took this route because it looked cost effective for the needs we had for ourselves. However, we can, in theory, provide services to others as well—as our server is more than capable of supporting more than just untyped.

But what services would we offer, and at what cost?

We currently run Apache, Exim/Courier, Postgres and MySQL, Subversion… in general, a full assortment of services that users might want to make use of. Through judicious use of LDAP, we’re hoping to expose some of these services (email, WWW) without needing to provide full (shell) accounts on the server. Likewise, we’re planning and building scripts to automate many of our maintenance tasks on the machine; we’re stopping short of CPanel, however.

So if we have the means to support other users, why would I be hesitant? Because we are not an ISP. We are people who enjoy making software. Our machine is like our workbench—it provides the essential services we need to make that software. Therefore, we have no interest in trying to compete with your run-of-the-mill ISPs… you can get a $5/month web account anywhere. So what kind of hosting will we provide?

At the least, I imagine we might offer hosting for “odd” servers and languages. By “odd”, I mean “anything not Apache.” For example, we know that there might be some interest in the PLT Scheme; likewise, I imagine that allowing servers written in Erlang may be of interest as well. Beyond that… how much bandwidth can someone expect to get hosting with us? Do we provide backup services? What kind of quota does a user’s account come with? What kind of response time to user requests are we going to try and make good on?

Bandwidth

We will probably have 50GB of traffic per month in our colo agreement. We will be charged beyond that as per Bytemark’s policies. Therefore, I could simply pass on costs to clients. Really, though, we’re not prepared to deal with the administrative overhead of billing clients for bandwidth, etc.; we’ll barely be in a position to administer billing for basic services. Again, we’re back to the mantra “we are not an ISP.”

We’ll have bandwidth accounting in place; how you charge for bandwidth, I don’t know. Just having to administer this takes time, and time is money (“We are not an ISP…”). Perhaps there’s a simple way to handle this… perhaps not. I need to think more on this one; insights are welcome.

Backup

Backup is expensive. It takes careful scripting on the host, regular movement of data from the server to a remote site (see “Bandwidth,” above), and it takes time to archive that to some kind of storage media. I don’t have a tape jukebox anywhere near the server, nor can I afford one (“We are not an ISP” ). Therefore, backup is an issue of me having a second machine that runs a cron job, and regularly pulls backups off the server. Then, I’ll have to manually burn DVDs and/or make digital tapes archives of that data. There are time and media costs involved in this that are non-trivial.

Backup is something that typically is not offered with a shell account, period. Taking a look at Mythic Beasts, their shell accounts start at £15 per quarter, they offer 200MB of disk space, and there is no mention of backup. I found some backup services while investigating storage (below), and they charge a lot per month for 1GB of backup (£25). This says to me that most hosting facilities assume the following:

  • You backup your own data
  • If the server dies, you’re on your own

I’m not committed to either route at this point, but it’s clear that I should charge money for backup services. I’m willing to spend my time to guarantee my own data’s safety, but I’m not willing to do that for others, for free.

Storage

What kind of quotas should we provide users? What can I get shopping around the internet?

 

 
Site Capacity Monthly
storegate.com 50MB £1.50
box.net 1GB £1.70
box.net 5GB £4.50
storegate.com 1GB £5.00
xdrive.com 5GB £5.60
ibackup.com 5GB £5.60
www.datafort.co.uk 1GB (compressed) £25

 

 

Epinions has more commentary on online storage than I care to duplicate here. I myself have a 1GB Box.net account, and am quite pleased with it—their pricing is competitive, and the interface to the site, as well as features (instant photo albums, passwords, RSS feeds) are great.

Point being? If I’m going to offer space, I have to charge at least £1.75/GB; and since we’re not really in this market space to compete, I’d probably have to charge more than that (due to the administrative overheads, and limited resources on our own machine). Furthermore, that’s assuming it’s a gigabyte of unarchived storage. It looks like sites that actually offer backup of a gigabyte of data charge around £25/month for a gigabyte of content that gets archived monthly. This doesn’t surprise me in the least—offering security is an expensive proposition, both in terms of tools and in terms of time.

Again, comparing, Mythic Beasts charge £30 per month for a “developer” account with 1GB of disk (no backup). The reality is (because we’re not an ISP), we only have one server, with limited disk space. I can resell storage from Bytemark, which eases the stress on my own disks… but as can be seen, having active disk space that you can serve out to the world costs money.

Administration

All these services take time to administer. Yes, we have to do it for ourselves… but it’s another thing entirely to have customers who want guarantees, and support when things go wrong (regardless of whether it is at our end or their end). What does administration cost? I can look at other ISPs and see what they charge for adding a subdomain, or configuring databases, or any of a host of things… and I don’t want to get trapped in a situation where we offer to do these things “for free,” and end up playing sysadmin to a bunch of customers without any recompense.

Again, this is a tough one.

In Conclusion… or, something like that

This isn’t necessarily the kind of laundry a small business should air in public. Typically, this is the kind of uncertainty that would be kept private, resolved off-line, and then published to the world as policy. However, these are probably the kind of questions that others have wrestled with in their time, and it’s possible someone who reads untyping may have some insights into these issues that we don’t have. Given that we only have one machine (“We are not an ISP”), it may be that we should avoid hosting altogether. Or perhaps we can come up with some reasonable terms of service and pricing structures that reflect the quality and nature of services we will offer. Who knows? Either way, they’re challenging questions.

Posted in Business | Comments Off on Untyped hosting?

20 Jan 2006

by Noel

We Have Lift Off!

As we’ve been hinting on the blog our current project has been close to delivery. Truth is, once we got a new server set up the deadline passed without incident. We’ve been waiting a few days to ensure there are no problems, but it’s been long enough so we’re now happy to announce the project is complete.

This work has been done for the School of Biological and Chemical Sciences at Queen Mary University of London. We have implemented a preregistration system for them: a website by which students can be registered for courses and modules, replacing a tedious and error-prone manual system. Unfortunately there is no publically available front-end so we can’t show off our work. However we have a few projects that should be completed in the next months that will have publically visible components.

The whole site runs on PLT Scheme, with SQLite for the database. This is, to our knowledge, the first large site to run the PLT Scheme web server continuously for any length of time. Over the next few days we’ll provide a few more details about the more intricate parts of our setup so other intrepid pioneers can learn from our work.

Posted in Business | Comments Off on We Have Lift Off!

17 Jan 2006

by Noel

Building a server

Currently, untyped lives on a virtual machine provided by Bytemark. The service they provide is excellent, and we recommend them highly. While we are building our own server we hope to continue to host with Bytemark, but instead of being on a VM, we’ll be on our own hardware. We’ll see how that goes.

Today, Christian and I spent a few hours working on the most fundamental part of a server: the filesystem. Before I go into any detail about the decisions we made, I’ll give you a sense for what we’re working with:

Untyped’s new home
Chassis Intel SR1200
Processors 2x PIII 1.4GHz
RAM 2GB 133MHz ECC
Hard disks 2x 250GB IDE

This server has seen use before, but we’ve replaced all the moving parts; we’re quite pleased with its condition, and think it will provide us with a number of years of good service. And, we hope that we don’t need to do a low-level install again in the next few years.

The first thing we did was to grab a Debian 3.1r1 “testing” net installCD image. We had to boot the 2.4 kernel, as the 2.6 kernel fails to load appropriate CD drivers from the install CD; this didn’t really matter. Then, we came around to our filesystem layout; what we knew was that we wanted to partition off different parts of the directory tree (/, /boot, /var, /usr, /tmp, /home); we didn’t know exactly how much space to give each part of the filesystem, however. Do we makehome 40GB, or 60GB? What about usr? The list goes on and on.

We started by setting up a 4GB swap partition at the end of each drive, a 400MB boot partition at the front of each drive, and setting aside the remaining 245GB or so for the main parts of the filesystem. We then used the Debian installer to turn the 400MB partitions into one RAID set, and the 245GB partition into another. This way, both our boot partition and the main part of the drive are in RAID, but we’re guaranteed that our boot partition is at the front of the disk.

(What is RAID? I’ll stick with the Wikipedia on that one. It keeps our two disk drives in perfect sync; this way, if one of them fails, we might be able to replace it before the second one does, thus keeping our system running with little or no interruption. This is a Good Thing.)

Then, we dove into that big, 245GB space. We used LVM—the Logical Volume Manager–to partition the rest of the disk. LVM is great because it essentially ignores the physical layout of your disks, and allows you to dynamically resize partitions without any great gnashing of teeth. So, we laid out 40GB for /home and /data, 20GB for /usr, and 4GB each for /, /var, and /tmp.

Note that this is only 112GB of space; we had (roughly) 245GB of space in that big RAID set. Our intention is to grow those partitions as we need to. For now, we chose large, but reasonable values for each of these partitions. In time, we may decide to increase the amount of space allocated to /home—perhaps from 40GB to 80GB. The point is, we have around 116GB of space to “grow into”, and we can allocate it to any of the partitions we currently have… or we can create new partitions. In either case, these operations don’t require us to shutdown the machine, or even take it entirely offline—we only need to “stop” the logical volume that the partition is on.

While we could do this more quickly the second time around, this took us between one and two hours; we were careful to check our assumptions, and rehashed and discussed a lot of the decisions in light of how we might want to use the server in the future.

Once the filesystem was set up, the rest of the installation went quickly; things were pulled in automatically over the network, and we rebooted into our new machine. The filesystem looks great, and we expect that the decisions we made will serve us well in the future. Now, we’ll work on upgrading the kernel to a 2.6 series (with SMP support), and then begin replicating those services that currently exist on the VM to the server.

Links that came in handy:

Posted in General | Comments Off on Building a server