asana

2/2/2010: Lunascript, our in-house language for writing great web apps

At Asana, we're building a Collaborative Information Manager that we believe will make it radically easier for groups of people to get work done. Writing a complex web application, we experienced pain all too familiar to authors of "Web 2.0" software (and interactive software in general): there were all kinds of extremely difficult programming tasks that we were doing over and over again for every feature we wanted to write. So we're developing Lunascript — an in-house programming language for writing rich web applications in about 10% of the time and code you can today.

Motivation

All of us on the Asana team have deep backgrounds writing rich web applications at companies like Google and Facebook. We've been continually frustrated by how long it takes to write software, and by a nagging feeling that in some deep sense we've been writing the same code over and over. Even when using the latest and greatest frameworks and disciplines, writing fast, highly interacting web applications involves a lot of accidental complexity:

First you need server code to figure out what data the browser needs. Hopefully you have an ORM layer, but you still need to carefully structure your code to minimize your backend dispatches, and you need to carefully keep that in sync with your front-end code lest you don't fetch enough data or hurt performance by fetching too much. If it's a Web 2.0-style app, you re-implement a ton of that server-side code in JavaScript, once for creating the page and then again as controller code for keeping it up to date and consistent. And when the user changes something, you bottle that up -- typically in a custom over-the-wire format -- and send it as an XHR to the server. The server has to de-serialize it into different data structures in a different language, notify the persistence store, figure out what other clients care about the change, and send them a notification over your Comet pipe, which is handled by yet more JavaScript controller code. Offline support? More code.

This is not a one-off task: it's rote work that adds complexity to every feature that you build in every application. By the time that you are done with all this, the important and novel parts of your application are only around 10% of your code. We wondered whether we could build a programming system in which we just wrote that 10% -- the essential complexity -- and a compiler handled the other 90%.

Solution

Inspired by incremental computing, we're building Lunascript as a simple way to write modern web applications. Lunascript has a syntax and ease of use reminiscent of JavaScript, but a powerful pure-functional lazily-evaluated semantics historically confined to academic languages.

A Lunascript application specifes a data model and a function from the model to the view or user interface, annotated with handler functions from user inputs to model mutations. From this the Lunascript compiler produces a functioning Web 2.0 application -- the client-side JavaScript, the server-side SQL, and everything in between -- complete with real-time bidirectional data synchronization. There's no need to write separate code to help the server figure out which values need to be sent to the client: the server can do this by simulating the UI. Because a Lunascript application only specifies how the UI should look given the current data (rather than how the UI should be updated as changes happen) it's impossible to write a UI that loads correctly but does not stay correct as changes are made.

Example

The easiest way to get a sense of how that's possible is to look at some code. Here's how you'd write a simple-but-complete multi-user chat app:

// Some of our currently implemented syntax isn't quite this clean, but it's
// fairly close and this is the direction we're going.

class World {
  // "1." serves the same purpose as "= 1" does in Google protocol buffer syntax.
  1. ChatMessage[] messages;
};

class ChatMessage {
  1. User user;
  2. string text;
};

class Session {
  1. User user;
  2. String new_comment;
};

return fn(world, browser, session) {
  var renderMessage = fn(message) {
    // Most style information omitted to improve readability.
    var bubble_style = { background: '#b7e0ff', padding: 7, ... };

    return <div>   // XML literals are first-class primitives.
      <img src=message.user.small_pic_url />
      <div style=bubble_style>
        <b> message.user.name, ': ' </b>
        <div> message.text </div>
      </div>
    </div>;
  };

  var postMessage = fn() {
    // Only handler functions can request data mutations.
    messages += ChatMessage {
      user: session.user,
      text: session.new_comment
    };
    session.new_comment := '';
  };

  return <table>
    <tr><td>
      messages.map(renderMessage)
    </td></tr>
    <tr><td>
      <img src=(session.user.small_pic_url) />
      <div>
        <input data=session.user.name /> <b>' (your nickname)'</b>
        <form onsubmit=postMessage>
          <input data=session.new_comment hasFocus=true />
        </form>
      </div>
    </td></tr>
  </table>;
};

Q & A

(Thanks to Dion at Ajaxian for sending us some of these initial questions. We'll update with more as we receive questions in comments.)

Writing a general-purpose programming language sounds fun/hard. What interesting problems are left?

We have enough of the system in place that we can use it to build real, powerful web applications, but we're just getting started. Formalizing the type system, batching database requests more efficiently, producing better JavaScript object code, improving security, cleaning up syntax, and all sorts of compiler optimiazation are all on the horizon. New ideas for features are added to the roadmap all the time — always motivated by day-to-day experience using the language to build real applications — and we have plans to borrow some old favorites from other languages (like yield).

How do you debug Lunascript? You must need a new set of tools for this.

Indeed. Debugging can still be somewhat tricky, but we are disciplined about creating and improving debugging tools (and testing frameworks) as we go along writing real code in the language. The most recent advance is a Firebug plugin that provides a visual representation of the dynamic dependency graph of expressions in a Lunascript program. We think there are a lot of exciting opportunities created by the language's pure-functional/lazy semantics; for example, we plan to build a debugger in which you "drill down" and visually explore subexpressions, without ever needing to set breakpoints.

Are you mixing code and presentation?

Lunascript allows you to mix code and presentation as much or as little as you want. It is easy to write code that looks like a Rails template when that's what you want, but we also make it feel natural to combine the two when that makes more sense. Strict separation is also less necessary in Lunascript because you don't need a separate step in which to fetch data; "template" code can pretend it has random access to the full datastore, and the framework will ensure it still executes efficiently. And, Lunascript empowers you to write presentation code in a much more powerful way than you can in existing systems, including CSS. For example, you can define complex styles in terms of inheritance/mixins:

  var header_style = { fontWeight: 'bold', ... };
  var column_header_style = header_style { fontSize: 13px, ... };

Does Lunascript integrate with existing JavaScript code / libraries?

Lunascript supports "JS blocks" similar to asm blocks in C, so whenever we need to interface with existing JavaScript code we can always drop down to raw JavaScript by just including expressions that looks like <?js return someJavaScriptFunction(); ?>.

Does Lunascript restrict the kinds of user interfaces you can create?

Lunascript provides almost full access to the browser's DOM structure. We don't think of the framework as trying to abstract the Document Object Model so much as providing a better API for programming against it. However, browser differences can be annoying, so we're also developing widgets.ls, a library of components that work reliably across browsers.

Lunascript sounds great. Can I use it in my project?

For now we are focused on building the Asana collaboration service, the motivation behind Lunascript, but we hope to release Lunascript more widely once it's more mature and we feel comfortable committing to backward compatibility.

Comments