Zazl Optimizer integrated into Maqetta

The Maqetta project is a great new tool for building HTML5 based user interfaces. One of its features is a "preview" option that allows developers to view the pages they have assembled. This functionality runs as an AMD based webpage loading all of its AMD modules individually. The load time of the preview can be significantly affected when running the preview in a high latency environment as each module load is an individual HTTP request. Typically the fix for this is to run some form of build tool that will concatenate all the modules together so that only one HTTP request is required. However, as the pages are assembled dynamically in Maqetta performing a static build is not really a viable option.

This is where Zazl can help. The Zazl AMD Optimizer supports dynamic optimizations such as module concatenation and can be typically integrated with minimal coding. Maqetta is OSGi based so Zazl must run in its OSGi mode as a set of OSGi bundles.

One of my main goals of the integration was to be as unobtrusive as possible in regard to the Maqetta source modifications. Only 2 core modifications were required :

1. Modify the generated preview URL to include a "zazl=true" parameter when Zazl is required to handle the preview.
2. Ensure that a raw version of Dojo was available for Zazl to use. Zazl requires that the AMD modules it analyzes have not been built with another build tool. Unfortunately the Dojo that Maqetta uses for preview has already been run through the Dojo build tool. Maqetta uses an ajaxLibrary Eclipse Extension Point to register paths to different libraries. A new extension instance for the raw Dojo code was added so that it did not interfere with the existing ajaxLibrary extension for the built version of Dojo.

With the Maqetta modifications in place some bootstrap code is required to setup the Zazl runtime so that it can intecept the preview URL requests and ensure that the Zazl AMD loader is used to load the AMD modules. You can see all of the bootstrap code here.

Modifications have to be made to the Preview's HTML page to ensure that the Zazl AMD loader is configured and loaded. A JEE Filter is a great tool for intercepting HTTP requests and responses. A Filter was written and configured within Maqetta to catch the preview requests and look for the "zazl=true" URL parameter. If matched an HTML parser (written using a Java Library called NekoHTML) is used to parse the HTML looking for the Dojo script tag. The parser switches the script tag with one that loads the Zazl AMD loader and also sets up the configuration.

In addition to creating the JEE Filter for the preview the bootstrap code has to ensure that the Zazl javascript servlet is configured and running and also that Zazl Resource Loading requests can find resources within the Maqetta environment. Both the JEE Filter and the Zazl javascript servlet are registered in an OSGi Activator run within a bootstap OSGi bundle called maqetta.zazl. This Activator also creates an instance of a custom Zazl Resource Loader that understands how to obtain resources from the Maqetta environment. Maqetta provides its own virtual directory API that can be used by this custom Resource Loader to obtain URL's to the resources.

The bootstrap code includes one other component. When the preview webpage is loaded it now has a reference to the Zazl AMD Loader. The Maqetta environment must be able to find this resource which resides in one of the Zazl Optimizers bundles. To achieve this the Zazl Optimizer bundle has to register an ajaxLibrary Eclipse Plugin Extension, I didn't want to contaminate the Zazl code with Maqetta specific references so an OSGi fragment bundle was created to add the required Eclipse Metadata. You see this fragment bundle here.

This integration also had to handle how the Zazl OSGi bundles would be integrated into the Maqetta git repository. The Maqetta git repository use submodules to reference its third-party dependencies. Providing direct submodule links to the Zazl git repositories on github would not work well as Zazl itself has a build step that has to be run. I decided the best way to handle this was to provide Zazl Release git repositories hosted on github.

There are 2 staging repositories:

1. One contains the build output of Zazl with tags marking specific versions.
2. The other contains the binary dependencies that Zazl requires to run.

This provides a nice controlled way for Maqetta to be upgraded to new versions of the Zazl Optimizer.

You can try all of this out by loading Maqetta. Developer setup details can be found here. The Preview7 Release, when available, will contain Zazl. It will be found here.

Using an HTML Filter to insert javascript tags

The code I have produced for my Zazl JavaScript Optimizer works by generating URL's for HTML script tags. Because of this the developer must use some form of server-side support to generate the HTML resource so the script tags can be inserted. As the code is written in Java the obvious choice for the server-side technology is JSP's.  If you are comfortable with writing JSP's and perhaps also plan to use them to insert other dynamic content into the  returning HTML resource then they are a good solution, but if you are only interested in using the Optimizer then writing HTML is a simpler choice to pick.

I decided to write an HTML Filter to make adoption of the Optimizer easier. It can be used to insert the required javascript script tag into the HTML resource before it is returned to the requester. All that the developer is required to do is add the javascript that references the Optimizer's AMD loader entry point. This can be via an embedded script within the HTML or via a "main" javascript resource referenced by the HTML via a script tag with a "src" attribute.

Writing the HTML Filter was made fairly straight forward because of great third party open source libraries that are available, NekoHTML and UglifyJS. The HTML Filter itself is written as a JEE Filter. Filters allows the HTTP requests and responses to be modified before and after the HTTP servlet serving the HTML is executed.  In this particular case the HTTP response is obtained by the Filter and analyzed before being returned back to the requester.

NekoHTML is an HTML parser written in Java. The Zazl Optimizer HTML Filter uses it to parse the HTML response returned from the WebContainer. The parser allows the filter to find and scan embedded javascript within the HTML. It also allows the filter to identify "main" javascript resources attached to the HTML that might contain the Optimizers AMD loader entry point.

Once these javascript snippets have been obtained the filter uses the javascript parser provided by UglifyJS to parse the code and locate the Optimizer AMD loader entry point. The entry point should provide the ids of the AMD modules that can be considered the top level modules for the page. The Optimizer's analyzer is then used to analyze and generate a javascript tag URL that can be inserted into the HTML response.

In a nutshell that's about all there is to it. I should also mention that the Filter attempts to ensure the HTML resource does not get cached because if any javascript resources required for the page are modified a new javascript tag URL would have to be generated. If the HTML resource is cached the javascript changes would never be picked up by the requester. It attempts to do this by stripping out any request headers that the WebContainer might use to indicate caching is possible.

You can see this in action in the Zazl AMD Optimizer sample WAR, also a wiki page is found here that provides some more details. Alternatively, I have a Music Server Application found here that also demonstrates the Filter and Optimizer in action.