2. The Jitsu Approach

Two things make Jitsu unique: its compiled Xml language for generating user interfaces, and its rich client-side databinding engine, which supports features like templating, controls and runtime expressions in the client's browser.

This documents provides a quick look at some of the inner workings of the Jitsu framework. We describe how pages are compiled by the Jitsu page compiler. We also introduce the Control Hierarchy, used by the Jitsu runtime. Finally we discuss how data integrates with the Control Hierarchy.

Evolution of the Web

Jitsu represents a third tier of approaches to building dynamic web applications.

Early web applications: Straight HTML

In the first generation of data-driven web applications, illustrated below, the server retrieves data from a database and then uses a server-side "template" processor (e.g. JSP, ASPX, PHP etc) to convert the raw data to HTML that can be displayed to the user:

The main drawback of this approach is that nearly every user action requires a complete round-trip to the server to fetch new data and re-render the page.

Recently: Ajax toolkits

More recent applications have advanced the model: the server obtains data from a database, pushes that data through a set of server-side templates to generate HTML, and then ships the HTML along with custom JavaScript code (often together with an Ajax toolkit or framework) down to the browser.

The JavaScript code then communicates with the server via asynchronous "AJAX" calls and makes in-place modifications to the page via "DOM manipulations".

Using this approach lets the user do many more in-page interaction without having to load a new page.

However, building applications using Ajax toolkits is often tedious: you have to write a fair amount of JavaScript code, and this code tends to be tightly coupled both with the data and HTML - when your data or presentation requirements change, you need to rewrite the code. The JavaScript code and HTML have many deep interdependencies.

Jitsu: Compiled Xml and Databinding

Jitsu is a member of a new breed of Ajax frameworks which use Xml to specify the user interface and data types required by the front end. Jitsu generates most of the JavaScript required to keep a page live by "compiling" the Xml, via the Jitsu page compiler.

In this model, to show the user a page, the server obtains data from the database, serializes the data into JavaScript (JSON) objects, and sends the data together with the compiled Jitsu app down to the web browser.

Subsequently, all template processing happens on the client. The final step of converting data objects into HTML is done by the client, not the server. The Jitsu client-side runtime supports a rich set of UI features, including live two-way databinding, animations, drag-and-drop and so on. This eliminates a large percentage of the "DOM manipulation" code that web application authors typically write.

The Jitsu Compile Process

Consider the canonical Hello World example below. Jitsu is an extension of XHTML, so a Jitsu Hello World looks very similar to a standard XHTML document:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:j="http://www.jitsu.org/schemas/2006">
    <body>
        <j:App>
            <h1>Hello World</h1>
        </j:App>
    </body>
</html>

The one addition here is the <j:App> tag - this indicates that the page contains a Jitsu application

The Jitsu compiler takes this XHTML page, processes it, and spits out a set of output .html/.css/.js files that are deployed on the web server.

The surprising thing about the compiled version of helloworld.html is that, in the compiled HTML, the content of the <App> is gone. Instead, the page looks something like this:

<!-- helloworld.html (compiled) -->

<html xmlns='http://www.w3.org/1999/xhtml'>
    <head>
        <script type="text/javascript" src="/app/jitsu.core.js"></script>
        <script type="text/javascript" src="/app/jitsu.controls.js"></script>
        <script type="text/javascript" src="/app/helloworld.html.js"></script>
        <script>function init() { App_init(); }; </script>
    </head>
    <body onload='init()'>
        <div id="App0"/>
    <body>
</html>

There is an empty <div> tag where the App tag was - the "Hello World" is gone. Also, in this version, <body> has a new onload="init()" attribute - this function does the work of constructing the page on the fly when it is loaded into the browser.

Before the <body> tag there are several <script>...</script> tags. The first two load the Jitsu core runtime and controls. The third loads the generated JavaScript for the page. Finally, there is a script tag to call the framework's initialize method.

The "Hello World" (along with anything else you put in the app) has been moved into the generated JavaScript code.

Generated JavaScript

The "Hello World" content has been moved into helloworld.html.js, as a piece of JavaScript. If you look at the JavaScript, it looks something like this:

    // helloworld.html.js

    // Create the <App> class (line 9)
    function App0() {
	    initializeApp0(this, this, null);
    };
    AppControl.addApp("App0", App0);

    // Function to initialize  (line 9)
    function initializeApp0(target, resourceOwner, data) {
	    target.setInitialValues(
		    "childControls", new SitedList(' Hello World! '));
    };

Looking at the JavaScript for the Hello World example, you see a function, initializeApp0. This is called to instantiate the root control, App0, and it uses setInitialValues to add 'Hello World' as a child to itself.

Crunching

The JavaScript above is quite verbose. To address this, Jitsu includes a JavaScript cruncher, which is run when the compiler is generating "release code". The cruncher strips out extra whitespace and also replaces variable names with much shorter names. After crunching, the above JavaScript looks something like this:

function xB(){var o;o=new Pb();o.Tb("C",new xb([' Hello World ']));return o;};

The cruncher makes JavaScript files approximately 50% smaller. On Internet Explorer, crunched files execute faster too.

We call the App control and its children the Control Hierarchy - its a tree of controls that has its own programming interface. Let's look at this in more detail.

Control Hierarchy

When a page is compiled, it is turned into a set of JavaScript objects. In the web browser these objects form a set of nodes in a graph, very much like the Browser's DOM document graph - except that in our case, rather than being a graph of the final HTML document the user sees, the graph contains the presentation objects that make up the page. We call this the Control Hierarchy.

Why not just use the DOM?

In our performance tests, we found that on most web browsers, accessing and modifying the DOM is slow. By comparison, creating, accessing and updating JavaScript objects is fast. Also, the data binding and templating mechanisms in Jitsu go significantly beyond the DOM's basic capabilities, and it would be hard to retrofit the DOM to add these features. For improved performance and a richer feature set, we implemented a separate control hierarchy.

Consider this more complex page (this is just an illustration):

    <html xmlns='http://www.w3.org/1999/xhtml' 
          xmlns:w='http://www.jitsu.org/schemas/2006'>

        <body>
            <j:App>
                <h1>Hello</h1>
                <j:ComboBox items="#bind(data.folders)"/>
                <div>
                    <j:Button text="Next" /> 
                    <j:Button text="Prev"/>
                </div>
            </j:App>
        </body>

    </html>

In the Control Hierarchy, you will see a graph like this:

Note that the presentation objects in the Control Hierarchy are either Controls or JavaScript strings. The Controls (shown as ovals) do dynamic things like display dynamic lists or manage clicks. The Strings (the "<h1>Hello</h1>" text in the rectangle above) are leaf nodes holding canned snippets of static HTML - bits of markup which don't change when the user interacts with the page.

Controls versus strings

The Jitsu page compiler automatically determines whether to use a simple string or a full control to represent a piece of the source markup. If your source markup has any "dynamic" behaviour (event handlers, data binding, etc), the compiler generates a full control in the Control Hierarchy. Otherwise, to improve both speed and memory usage, it converts "static" markup to strings in the control hierarchy.

If you have multiple consecutive elements that are static, they are converted into a single string in the control hierarchy.

This means the Control Hierarchy does not contain a node corresponding to every node in the final DOM. In some cases, multiple HTML DOM nodes are managed by a single Control Hierarchy node. For example, <h1>Hello <b>Jon</b></h1> requires several nodes in the DOM, but can be represented as a single string in the Control Hierarchy. As another example, the ComboBox control is a single control in the Control Hierarchy, but it generates multiple HTML nodes in the DOM.

Jitsu includes a browser-based Inspector that lets you inspect the Control Hierarchy while your app is running - push F10 on a Jitsu app built with the "Diagnostics" flag turned on, and the inspector pops up, letting you visually explore the Control Hierarchy:

The Jitsu Runtime

The Jitsu Runtime is responsible for taking the Control Hierarchy objects and generating the final HTML that the user actually sees for a page. This process is called "rendering" the Control Hierarchy, and it happens automatically when you load a page containing a Jitsu app. All the HTML that the user sees for a Jitsu app is generated on the fly within the client's browser by the Jitsu runtime.

During rendering, the Control Hierarchy graph is converted into a final HTML graph that the user sees. Each Control spits out a "peer" element that appears in the Dom, and possibly children elements within the peer.

Each Control remembers which peer it generated. For example, during rendering, a ComboBox control might be rendered as a <select> node with a set of <option> children:

In this case, the <select> node in the DOM is known as the ComboBox control's peer.

Jitsu's Control class has a method getPeer(), which returns the control's peer DOM node if it's been created.

Interactivity and Events

When the user moves their mouse, hits keys, or otherwise interacts with the HTML DOM, events are propagated from the DOM element back to the associated control in the Control Hierarchy:

To propagate events, the framework first determines which Control is associated with the DOM element that generated the event. Then it calls the control's onPeerEvent method, which fires event handling code in the Control Hierarchy, possibly "bubbling" events up the Control Hierarchy graph. Any application event handlers are fired on Control Hierarchy nodes. Application logic is written against the Control Hierarchy, not against the DOM.

Control Updates

Event handlers often contain code which modifies the Control Hierarchy in some way. For example, when a user clicks on a button, the application may respond by changing some of the properties of nodes in the Control Hierarchy, or adding or removing Control Hierarchy nodes.

In Jitsu, changes to controls in the Control Hierarchy cause those controls to regenerate the appropriate portions of the user's page (the DOM) using fresh HTML elements (this is done by calling DOM tree manipulation methods, but the process is transparent to the application). Jitsu applications usually let the controls take care of keeping the DOM up to date - they rarely talk to the HTML DOM directly.

Data and Data Binding

Changes to the Control Hierarchy are not only caused by the user clicking or typing something. One frequent reason that the Control Hierarchy is modified is when the underlying data for the page changes.

In Jitsu, each page contains one or more DataSets. DataSets are graphs of JavaScript objects representing application data. Controls in the Control Hierarchy are wired up to DataSets using databinding. Controls use databinding expressions to indicate what data to display e.g. for the items in a list, the text in text fields, the check state for checkboxes, etc.

Two-way "hot" databinding is done in the browser via JavaScript.

Why JavaScript objects for data instead of Xml?

Similar to the DOM, we found that accessing Xml documents via XPath in JavaScript is slow. By comparison, creating, accessing and updating JavaScript objects is fast. Furthermore, we wanted to add support for features not found in Xml, such as reflection, type information, property validators and change notification. To add these features, we implemented JavaScript object-based DataSets.

Xml DataSets using XPath may be added at some point in the future.

Active vs Static DataSets

Jitsu supports two kinds of DataSets: Active and Static.

Static DataSets are used to send data down to the client which is considered read-only - for example a list of counties or states or zip code regions might be sent in a static dataset. You can describe static datasets entirely in Xml - this is compiled by the page compiler into suitable JavaScript notation.

Here is a definition of a static dataset called folders:

    <j:DataSet id="folders">
        <j:di>InBox</j:di>
        <j:di>OutBox</j:di>
        <j:di>Favorites</j:di>
    </j:DataSet>

Active DataSets are used for data that is pulled from a database - data the user can edit and modify. Objects in these datasets include "arefs", or "Active References", which are opaque handles the client uses to refer to objects when communicating with the server. Active datasets typically have a src parameter indicating the endpoint on the server that provides that data:

<j:DataSet id="activefolders" src="http://www.myweb.com/foo/bar" />

We'll come back to active datasets in a moment, first let's look at databinding.

Data Binding

The Jitsu framework contains a rich set of client-side databinding facilities. For example, to get the "folders" static DataSet above to appear in a ComboBox, you write the following markup on the page:

    <ComboBox items="#bind(//data.folders)"/>

This tells the ComboBox that its list of items should be bound to the DataSet called "folders". The ComboBox control will automatically build the HTML <option> nodes for each of the items in the DataSet - from the example above, you can see the folders list contains three items, "InBox", "OutBox", and "Favorites". So the full picture looks something like this:

The DataSet called "folders" contains three items, "InBox", "OutBox" and "Favorites". The ComboBox in the Control Hierarchy binds agains the data.folders - so it receives its items from the "folders" list. When the ComboBox is rendered, it generates a <select> HTML object with three <option>s inside it.

Databindings propagate changes in both directions. That is, if the data.folders DataSet in this example (the "source" of the databinding) changes, then the ComboBox (or "sink") is notified, it receives the changed value, and automatically generates fresh HTML. On the other hand, if the sink (the ComboBox) is modified e.g. because of a user interaction, the framework propagates the changes back to the DataSet. If the DataSet is an "active" dataset, changes are also propagated back to the server.

AJAX and Server Interactions

Jitsu DataSets contain a change notification mechanism that listens for data value changes and posts them back to the server.

For static datasets, the data notifier does nothing when a DataSet changes.

If the DataSet is "active", each object in the dataset has an "aref" property set on it. An aref is an "Active Reference" - its a unique ID generated by the server and associated with a DataObject, to indicate where the object came from. Unlike hrefs, which are well formed URLS, Jitsu treats arefs as opaque strings - arefs may be implemented by the server using URLS, URNS, integers, or any other naming scheme. As far as the Jitsu front end is concerned, an aref is opaque.

Jitsu uses a lightweight JSON protocol for notifying the server of changes and for receiving new data from the server.

When a data object which has an aref is changed, Jitsu issues an AJAX XmlHttpRequest PUT to send the changes to the server. The PUT request has a list of "commands" that the server should perform to make the necessary modifications. For example, here is a PUT request generated by the Bookmarks sample:

{
    "dataSet":"bookmarks",
    "commands": [
        {
            "op":"set",
            "path":["bookmarks","bb7"],
            "key":"title",
            "val":"Hello there"
        },
        {
            "op":"load",
            "path":["bookmarks"]
        }
    ]
}

Jitsu includes both PHP and C# implementations that show show to process commands from the client.

The document on Active Datasets describes this JSON protocol in more detail.

Summary

This document has described a number of powerful mechanisms:

Page Compilation: a server-side technology for pre-processing and crunching pages to produce the final shipping HTML.

Control Hierarchy: Separation of the logical structure of a page from the final rendered markup.

Data and Data Binding: Rapid wiring of controls to application data.

Controls: Let you define reusable controls that can be shared across pages.

Taken together, these mechanisms represent a scalable, modular and fundamentally AJAX approach to building rich web applications.