Zazl AMD Optimizer and Node.js

When I first started writing the Zazl Optimizer I focused on providing a Java HTTP layer that could be used to support dynamic analysis and optimization of AMD based Web Applications. One of the  core analyzers within the Zazl AMD Optimizer is written in javascript, so with that in mind it made sense to also provide an HTTP layer written to run in Node.js.
Node.js has a number of static HTML libraries available. If one of those is combined with the Zazl AMD Optimizer you have an environment where you can write your AMD based application and serve up an optimized (concatenated and compressed) version of it to your browser based clients.

One of the most well known and used static HTML libraries is Connect. It's actually used within a large number of middleware Node.js based libraries such as Express. For the Zazl Optimizer's  purposes connect is used to serve up any static resource that is not handled by the optimizer. The packaging of the Zazl Optimizer for Node.js provides a Connect based server frontend. The section that starts up the http server looks like this (from the file found here):

    var connectOptimizer = zazloptimizer.createConnectOptimizer(appdir, compress);

    var app = connect()
        .use("/_javascript", connectOptimizer)
        .use(connect.static(appdir))
        .use(connect.static(zazloptimizer.getLoaderDir()));

    http.createServer(app).listen(port);

When creating the optimizer you give it the path to where the JavaScript resources reside and also whether to turn on compression. Also,  you can see above that the typical approach is taken to initialize the connect environment. The path is first checked for "_javascript" and directed to the optimizer to be handled if matched. Otherwise a Connect static handler for the specified application directory is search and also one to handle finding Zazl's AMD loader that the application code references. To take advantage of the AMD loader handler simply reference it as follows in the HTML file :

    <script type="text/javascript" src="loader/amd/zazl.js"></script>

That's more or less all there is to setting up usage. You can see more in two sample github repositories, one with Dojo samples and one with JQuery samples. Also both are hosted here and here.  

Note: The hosting site (Heroku) puts both apps to sleep after being idle for 1hour. Don't be surprised if the first load(s) takes some time. Subsequent loads will demonstrate the full potential. Alternatively you can download the source from the repositories and run them yourself.

Inlining HTML5 WebWorker source content

I have been playing with HTML5 WebWorkers in an attempt to solve a performance problem and came up with a simple way to inline the WebWorker's source code. The Basics of Web Workers tutorial demonstrates inlining via script tags marked as 'type="javascript/worker"' however this means placing the WebWorker source within the HTML page. I use AMD for my module loading and would like to avoid this approach. With that in mind I came up with the following :

    var webWorkerFunc = function() {
        onmessage = function(e) {
            .......
            postMessage(....);
       }
    };

    var content = webWorkerFunc.toString();
    content = content.substring("function () {".length+1);
    content = content.substring(0, content.lastIndexOf("}"));

    var URL = window.URL || window.webkitURL;    var blob = new Blob([content]);
    var blobURL = URL.createObjectURL(blob);
    var webWorker = new Worker(blobURL);
    webWorker.onmessage = function(e) {
        .......
    };

This approach is similar to the one described in the HTML5 Rocks tutorial except that the Function.toString() is used to obtain the WebWorker source code. The function wrapper is stripped and the result passed into the Blob constructor.

Adventures in source map land

JavaScript source maps are a great solution to the age old problem with JavaScript where you have minified your source code but now you need to debug it. Attempting to step through the minified code can push a developer over the edge :-)

Within the Zazl Optimizer there is support to minify the JavaScript responses that are generated. The compressor interface it provides allows for different compression implementations to be configured. Although I do not have the Google Closure compiler implementation available with the Optimizer codebase I have been experimenting with providing one.

With this in mind I decided to see if I could make the minified JavaScript responses Zazl generates include the "//@ sourceMappingURL=" comments and load the source maps when requested. To begin with the source maps themselves have to be generated. When running the Closure Compiler simply setting a non-null value on the sourceMapOutputPath property of the CompilerOptions object will trigger the source map generation. The JSON string representation of the source map can then be obtained from the sourceMap property of the Results object

  CompilerOptions options = new CompilerOptions();
  options.sourceMapOutputPath = "";
  ......
  Result result = compiler.compile(extern, input, options);
  String compressedSrc = compiler.toSource();
  compressedSrc+= "\n//@ sourceMappingURL=_javascript?sourcemap="+path+".map\n";
  StringBuffer sb = new StringBuffer();
  result.sourceMap.appendTo(sb, "sourceMap");
 
Note in the code above the attached URL points to the Zazl HTTP handler to obtain the source map for a given module when the HTTP request contains a "sourcemap" parameter.

At this point I should indicate that for performance purposes the Zazl Optimizer does not run the compresser for each JavaScript response it generates. Individual modules are compressed and cached so that when a response is generated it is simply a matter of concatenating the required modules. This results in a single stream of JavaScript with multiple modules and also multiple "//@ sourceMappingURL=" comments separating them.

And this is where things fall apart with this approach. It appears that the Chrome implementation supporting source maps cannot deal with a single JavaScript resource containing multiple modules and multiple sourceMappingURL comments. When run in Chrome I see the debugger hook up the first module it finds in the resource and then it ignores the rest.

At the moment the only solution I can see is for Zazl to stop compressing individual modules and just compress the single JavaScript response generated. This will result in a single resource listed in the debugger, not individual modules, but the minified code will be hooked up correctly to the unminified source. I really don't want to do this as the performance hit will be substantial. A to-do for me is to find out if there is any way I can get Chrome to handle the multiple modules within the single resource. I'll update the post if I find out more.

Update 9/27/2012 :
After posting a message on the Chrome DevTools google group I was pointed to the source map specification where it describes sections. This is exactly what I needed. Instead of writing multiple sourceMappingURL comments the optimizer writes one URL that gets directed to the optimizers javascript servlet with an identifying key for the contents of the response. When the javascript servlet receives the request for the map it generates a JSON object containing the required sections for each module.

The good news is with these changes in place the Chrome debugger now shows and links to all source files correctly. The bad news is that doing other debug tasks, such as setting breakpoints, do not work.