I have been playing with HTML5 WebWorkers in an attempt to solve a performance problem and came up with a simple way to inline the WebWorker's source code. The Basics of Web Workers tutorial demonstrates inlining via script tags marked as 'type="javascript/worker"' however this means placing the WebWorker source within the HTML page. I use AMD for my module loading and would like to avoid this approach. With that in mind I came up with the following :
var webWorkerFunc = function() {
onmessage = function(e) {
.......
postMessage(....);
}
};
var content = webWorkerFunc.toString();
content = content.substring("function () {".length+1);
content = content.substring(0, content.lastIndexOf("}"));
var URL = window.URL || window.webkitURL; var blob = new Blob([content]);
var blobURL = URL.createObjectURL(blob);
var webWorker = new Worker(blobURL);
webWorker.onmessage = function(e) {
.......
};
This approach is similar to the one described in the HTML5 Rocks tutorial except that the Function.toString() is used to obtain the WebWorker source code. The function wrapper is stripped and the result passed into the Blob constructor.
Friday, December 7, 2012
Sunday, September 23, 2012
Adventures in source map land
JavaScript source maps are a great solution to the age old problem with JavaScript where you have minified your source code but now you need to debug it. Attempting to step through the minified code can push a developer over the edge :-)
Within the Zazl Optimizer there is support to minify the JavaScript responses that are generated. The compressor interface it provides allows for different compression implementations to be configured. Although I do not have the Google Closure compiler implementation available with the Optimizer codebase I have been experimenting with providing one.
With this in mind I decided to see if I could make the minified JavaScript responses Zazl generates include the "//@ sourceMappingURL=" comments and load the source maps when requested. To begin with the source maps themselves have to be generated. When running the Closure Compiler simply setting a non-null value on the sourceMapOutputPath property of the CompilerOptions object will trigger the source map generation. The JSON string representation of the source map can then be obtained from the sourceMap property of the Results object
CompilerOptions options = new CompilerOptions();
options.sourceMapOutputPath = "";
......
Result result = compiler.compile(extern, input, options);
String compressedSrc = compiler.toSource();
compressedSrc+= "\n//@ sourceMappingURL=_javascript?sourcemap="+path+".map\n";
StringBuffer sb = new StringBuffer();
result.sourceMap.appendTo(sb, "sourceMap");
Note in the code above the attached URL points to the Zazl HTTP handler to obtain the source map for a given module when the HTTP request contains a "sourcemap" parameter.
At this point I should indicate that for performance purposes the Zazl Optimizer does not run the compresser for each JavaScript response it generates. Individual modules are compressed and cached so that when a response is generated it is simply a matter of concatenating the required modules. This results in a single stream of JavaScript with multiple modules and also multiple "//@ sourceMappingURL=" comments separating them.
And this is where things fall apart with this approach. It appears that the Chrome implementation supporting source maps cannot deal with a single JavaScript resource containing multiple modules and multiple sourceMappingURL comments. When run in Chrome I see the debugger hook up the first module it finds in the resource and then it ignores the rest.
At the moment the only solution I can see is for Zazl to stop compressing individual modules and just compress the single JavaScript response generated. This will result in a single resource listed in the debugger, not individual modules, but the minified code will be hooked up correctly to the unminified source. I really don't want to do this as the performance hit will be substantial. A to-do for me is to find out if there is any way I can get Chrome to handle the multiple modules within the single resource. I'll update the post if I find out more.
Update 9/27/2012 :
After posting a message on the Chrome DevTools google group I was pointed to the source map specification where it describes sections. This is exactly what I needed. Instead of writing multiple sourceMappingURL comments the optimizer writes one URL that gets directed to the optimizers javascript servlet with an identifying key for the contents of the response. When the javascript servlet receives the request for the map it generates a JSON object containing the required sections for each module.
The good news is with these changes in place the Chrome debugger now shows and links to all source files correctly. The bad news is that doing other debug tasks, such as setting breakpoints, do not work.
Within the Zazl Optimizer there is support to minify the JavaScript responses that are generated. The compressor interface it provides allows for different compression implementations to be configured. Although I do not have the Google Closure compiler implementation available with the Optimizer codebase I have been experimenting with providing one.
With this in mind I decided to see if I could make the minified JavaScript responses Zazl generates include the "//@ sourceMappingURL=" comments and load the source maps when requested. To begin with the source maps themselves have to be generated. When running the Closure Compiler simply setting a non-null value on the sourceMapOutputPath property of the CompilerOptions object will trigger the source map generation. The JSON string representation of the source map can then be obtained from the sourceMap property of the Results object
CompilerOptions options = new CompilerOptions();
options.sourceMapOutputPath = "";
......
Result result = compiler.compile(extern, input, options);
String compressedSrc = compiler.toSource();
compressedSrc+= "\n//@ sourceMappingURL=_javascript?sourcemap="+path+".map\n";
StringBuffer sb = new StringBuffer();
result.sourceMap.appendTo(sb, "sourceMap");
Note in the code above the attached URL points to the Zazl HTTP handler to obtain the source map for a given module when the HTTP request contains a "sourcemap" parameter.
At this point I should indicate that for performance purposes the Zazl Optimizer does not run the compresser for each JavaScript response it generates. Individual modules are compressed and cached so that when a response is generated it is simply a matter of concatenating the required modules. This results in a single stream of JavaScript with multiple modules and also multiple "//@ sourceMappingURL=" comments separating them.
And this is where things fall apart with this approach. It appears that the Chrome implementation supporting source maps cannot deal with a single JavaScript resource containing multiple modules and multiple sourceMappingURL comments. When run in Chrome I see the debugger hook up the first module it finds in the resource and then it ignores the rest.
At the moment the only solution I can see is for Zazl to stop compressing individual modules and just compress the single JavaScript response generated. This will result in a single resource listed in the debugger, not individual modules, but the minified code will be hooked up correctly to the unminified source. I really don't want to do this as the performance hit will be substantial. A to-do for me is to find out if there is any way I can get Chrome to handle the multiple modules within the single resource. I'll update the post if I find out more.
Update 9/27/2012 :
After posting a message on the Chrome DevTools google group I was pointed to the source map specification where it describes sections. This is exactly what I needed. Instead of writing multiple sourceMappingURL comments the optimizer writes one URL that gets directed to the optimizers javascript servlet with an identifying key for the contents of the response. When the javascript servlet receives the request for the map it generates a JSON object containing the required sections for each module.
The good news is with these changes in place the Chrome debugger now shows and links to all source files correctly. The bad news is that doing other debug tasks, such as setting breakpoints, do not work.
Thursday, July 26, 2012
Zazl Optimizer integrated into Maqetta
The Maqetta project is a great new tool for building HTML5 based user interfaces. One of its features is a "preview" option that allows developers to view the pages they have assembled. This functionality runs as an AMD based webpage loading all of its AMD modules individually. The load time of the preview can be significantly affected when running the preview in a high latency environment as each module load is an individual HTTP request. Typically the fix for this is to run some form of build tool that will concatenate all the modules together so that only one HTTP request is required. However, as the pages are assembled dynamically in Maqetta performing a static build is not really a viable option.
This is where Zazl can help. The Zazl AMD Optimizer supports dynamic optimizations such as module concatenation and can be typically integrated with minimal coding. Maqetta is OSGi based so Zazl must run in its OSGi mode as a set of OSGi bundles.
One of my main goals of the integration was to be as unobtrusive as possible in regard to the Maqetta source modifications. Only 2 core modifications were required :
Modifications have to be made to the Preview's HTML page to ensure that the Zazl AMD loader is configured and loaded. A JEE Filter is a great tool for intercepting HTTP requests and responses. A Filter was written and configured within Maqetta to catch the preview requests and look for the "zazl=true" URL parameter. If matched an HTML parser (written using a Java Library called NekoHTML) is used to parse the HTML looking for the Dojo script tag. The parser switches the script tag with one that loads the Zazl AMD loader and also sets up the configuration.
In addition to creating the JEE Filter for the preview the bootstrap code has to ensure that the Zazl javascript servlet is configured and running and also that Zazl Resource Loading requests can find resources within the Maqetta environment. Both the JEE Filter and the Zazl javascript servlet are registered in an OSGi Activator run within a bootstap OSGi bundle called maqetta.zazl. This Activator also creates an instance of a custom Zazl Resource Loader that understands how to obtain resources from the Maqetta environment. Maqetta provides its own virtual directory API that can be used by this custom Resource Loader to obtain URL's to the resources.
The bootstrap code includes one other component. When the preview webpage is loaded it now has a reference to the Zazl AMD Loader. The Maqetta environment must be able to find this resource which resides in one of the Zazl Optimizers bundles. To achieve this the Zazl Optimizer bundle has to register an ajaxLibrary Eclipse Plugin Extension, I didn't want to contaminate the Zazl code with Maqetta specific references so an OSGi fragment bundle was created to add the required Eclipse Metadata. You see this fragment bundle here.
This integration also had to handle how the Zazl OSGi bundles would be integrated into the Maqetta git repository. The Maqetta git repository use submodules to reference its third-party dependencies. Providing direct submodule links to the Zazl git repositories on github would not work well as Zazl itself has a build step that has to be run. I decided the best way to handle this was to provide Zazl Release git repositories hosted on github.
There are 2 staging repositories:
You can try all of this out by loading Maqetta. Developer setup details can be found here. The Preview7 Release, when available, will contain Zazl. It will be found here.
This is where Zazl can help. The Zazl AMD Optimizer supports dynamic optimizations such as module concatenation and can be typically integrated with minimal coding. Maqetta is OSGi based so Zazl must run in its OSGi mode as a set of OSGi bundles.
One of my main goals of the integration was to be as unobtrusive as possible in regard to the Maqetta source modifications. Only 2 core modifications were required :
- Modify the generated preview URL to include a "zazl=true" parameter when Zazl is required to handle the preview.
- Ensure that a raw version of Dojo was available for Zazl to use. Zazl requires that the AMD modules it analyzes have not been built with another build tool. Unfortunately the Dojo that Maqetta uses for preview has already been run through the Dojo build tool. Maqetta uses an ajaxLibrary Eclipse Extension Point to register paths to different libraries. A new extension instance for the raw Dojo code was added so that it did not interfere with the existing ajaxLibrary extension for the built version of Dojo.
Modifications have to be made to the Preview's HTML page to ensure that the Zazl AMD loader is configured and loaded. A JEE Filter is a great tool for intercepting HTTP requests and responses. A Filter was written and configured within Maqetta to catch the preview requests and look for the "zazl=true" URL parameter. If matched an HTML parser (written using a Java Library called NekoHTML) is used to parse the HTML looking for the Dojo script tag. The parser switches the script tag with one that loads the Zazl AMD loader and also sets up the configuration.
In addition to creating the JEE Filter for the preview the bootstrap code has to ensure that the Zazl javascript servlet is configured and running and also that Zazl Resource Loading requests can find resources within the Maqetta environment. Both the JEE Filter and the Zazl javascript servlet are registered in an OSGi Activator run within a bootstap OSGi bundle called maqetta.zazl. This Activator also creates an instance of a custom Zazl Resource Loader that understands how to obtain resources from the Maqetta environment. Maqetta provides its own virtual directory API that can be used by this custom Resource Loader to obtain URL's to the resources.
The bootstrap code includes one other component. When the preview webpage is loaded it now has a reference to the Zazl AMD Loader. The Maqetta environment must be able to find this resource which resides in one of the Zazl Optimizers bundles. To achieve this the Zazl Optimizer bundle has to register an ajaxLibrary Eclipse Plugin Extension, I didn't want to contaminate the Zazl code with Maqetta specific references so an OSGi fragment bundle was created to add the required Eclipse Metadata. You see this fragment bundle here.
This integration also had to handle how the Zazl OSGi bundles would be integrated into the Maqetta git repository. The Maqetta git repository use submodules to reference its third-party dependencies. Providing direct submodule links to the Zazl git repositories on github would not work well as Zazl itself has a build step that has to be run. I decided the best way to handle this was to provide Zazl Release git repositories hosted on github.
There are 2 staging repositories:
- One contains the build output of Zazl with tags marking specific versions.
- The other contains the binary dependencies that Zazl requires to run.
You can try all of this out by loading Maqetta. Developer setup details can be found here. The Preview7 Release, when available, will contain Zazl. It will be found here.
Saturday, February 11, 2012
AMD, jQuery and the Zazl Optimizer
Having got my Dynamic Optimizer running with Dojo 1.7 I decided to take a look at how jQuery works in the AMD world. With the release of JQuery 1.7 it became possible to load and reference the core jQuery code as an AMD module. Also, after looking at the jQuery mobile library (note this is currently only available in the 1.1 version that has not yet been released) I found that it too was AMD enabled. So it seemed like the perfect time to get familiar with the jQuery world. I decided I would write a jQuery mobile based frontend to my Music Server application. It already uses the Zazl Optimizer to load a Dojo 1.7 based desktop and mobile frontend.
The first step was to obtain jQuery 1.7.1 and jQuery mobile 1.1. jQuery 1.7.1 can be downloaded from here and jQuery mobile 1.1 can be obtained by build it from it github repostitory. I should note in both cases the uncompressed versions are used as the compression is handled by the Zazl Optimizer itself.
Once downloaded I placed jQuery 1.7.1 in a directory path of "lib/jquery/jquery-1.7.1.js" and jQuery mobile 1.1 in a directory path of "lib/jquery-mobile/jquery.mobile.js" within my Web Application. I also had to obtain the required CSS file for jquery mobile(also the images it references). This I placed in a directory path of "css/jquery-mobile/jquery.mobile.css" and referenced it in the HTML
<link rel="stylesheet" href="css/jquery-mobile/jquery.mobile.css" />
The HTML front-end code then could simply reference the modules via a call to the Zazl Optimizer's entry point, "zazl". The actual script tag that loads the jQuery code is inserted by an HTML Filter as described here.
<script type="text/javascript">
zazl({
paths : {
jquery: "lib/jquery/jquery-1.7.1",
jquerymobile: "lib/jquery-mobile/jquery.mobile"
}
},
["jquery", "jquerymobile", "app/jqmobile"],
function($) {
});
</script>
Above you can see the main AMD module that handles the application logic called "app/jqmobile". Within that the jQuery core is referenced as follows :
define(['jquery'], function ($) {
.....
$(document).ready(function() {
console.log("ready");
});
});
That's about all there is to it. jQuery is used just as it is normally.
The first step was to obtain jQuery 1.7.1 and jQuery mobile 1.1. jQuery 1.7.1 can be downloaded from here and jQuery mobile 1.1 can be obtained by build it from it github repostitory. I should note in both cases the uncompressed versions are used as the compression is handled by the Zazl Optimizer itself.
Once downloaded I placed jQuery 1.7.1 in a directory path of "lib/jquery/jquery-1.7.1.js" and jQuery mobile 1.1 in a directory path of "lib/jquery-mobile/jquery.mobile.js" within my Web Application. I also had to obtain the required CSS file for jquery mobile(also the images it references). This I placed in a directory path of "css/jquery-mobile/jquery.mobile.css" and referenced it in the HTML
<link rel="stylesheet" href="css/jquery-mobile/jquery.mobile.css" />
The HTML front-end code then could simply reference the modules via a call to the Zazl Optimizer's entry point, "zazl". The actual script tag that loads the jQuery code is inserted by an HTML Filter as described here.
<script type="text/javascript">
zazl({
paths : {
jquery: "lib/jquery/jquery-1.7.1",
jquerymobile: "lib/jquery-mobile/jquery.mobile"
}
},
["jquery", "jquerymobile", "app/jqmobile"],
function($) {
});
</script>
Above you can see the main AMD module that handles the application logic called "app/jqmobile". Within that the jQuery core is referenced as follows :
define(['jquery'], function ($) {
.....
$(document).ready(function() {
console.log("ready");
});
});
That's about all there is to it. jQuery is used just as it is normally.
Sunday, January 15, 2012
Using an HTML Filter to insert javascript tags
The code I have produced for my Zazl JavaScript Optimizer works by generating URL's for HTML script tags. Because of this the developer must use some form of server-side support to generate the HTML resource so the script tags can be inserted. As the code is written in Java the obvious choice for the server-side technology is JSP's. If you are comfortable with writing JSP's and perhaps also plan to use them to insert other dynamic content into the returning HTML resource then they are a good solution, but if you are only interested in using the Optimizer then writing HTML is a simpler choice to pick.
I decided to write an HTML Filter to make adoption of the Optimizer easier. It can be used to insert the required javascript script tag into the HTML resource before it is returned to the requester. All that the developer is required to do is add the javascript that references the Optimizer's AMD loader entry point. This can be via an embedded script within the HTML or via a "main" javascript resource referenced by the HTML via a script tag with a "src" attribute.
Writing the HTML Filter was made fairly straight forward because of great third party open source libraries that are available, NekoHTML and UglifyJS. The HTML Filter itself is written as a JEE Filter. Filters allows the HTTP requests and responses to be modified before and after the HTTP servlet serving the HTML is executed. In this particular case the HTTP response is obtained by the Filter and analyzed before being returned back to the requester.
NekoHTML is an HTML parser written in Java. The Zazl Optimizer HTML Filter uses it to parse the HTML response returned from the WebContainer. The parser allows the filter to find and scan embedded javascript within the HTML. It also allows the filter to identify "main" javascript resources attached to the HTML that might contain the Optimizers AMD loader entry point.
Once these javascript snippets have been obtained the filter uses the javascript parser provided by UglifyJS to parse the code and locate the Optimizer AMD loader entry point. The entry point should provide the ids of the AMD modules that can be considered the top level modules for the page. The Optimizer's analyzer is then used to analyze and generate a javascript tag URL that can be inserted into the HTML response.
In a nutshell that's about all there is to it. I should also mention that the Filter attempts to ensure the HTML resource does not get cached because if any javascript resources required for the page are modified a new javascript tag URL would have to be generated. If the HTML resource is cached the javascript changes would never be picked up by the requester. It attempts to do this by stripping out any request headers that the WebContainer might use to indicate caching is possible.
You can see this in action in the Zazl AMD Optimizer sample WAR, also a wiki page is found here that provides some more details. Alternatively, I have a Music Server Application found here that also demonstrates the Filter and Optimizer in action.
I decided to write an HTML Filter to make adoption of the Optimizer easier. It can be used to insert the required javascript script tag into the HTML resource before it is returned to the requester. All that the developer is required to do is add the javascript that references the Optimizer's AMD loader entry point. This can be via an embedded script within the HTML or via a "main" javascript resource referenced by the HTML via a script tag with a "src" attribute.
Writing the HTML Filter was made fairly straight forward because of great third party open source libraries that are available, NekoHTML and UglifyJS. The HTML Filter itself is written as a JEE Filter. Filters allows the HTTP requests and responses to be modified before and after the HTTP servlet serving the HTML is executed. In this particular case the HTTP response is obtained by the Filter and analyzed before being returned back to the requester.
NekoHTML is an HTML parser written in Java. The Zazl Optimizer HTML Filter uses it to parse the HTML response returned from the WebContainer. The parser allows the filter to find and scan embedded javascript within the HTML. It also allows the filter to identify "main" javascript resources attached to the HTML that might contain the Optimizers AMD loader entry point.
Once these javascript snippets have been obtained the filter uses the javascript parser provided by UglifyJS to parse the code and locate the Optimizer AMD loader entry point. The entry point should provide the ids of the AMD modules that can be considered the top level modules for the page. The Optimizer's analyzer is then used to analyze and generate a javascript tag URL that can be inserted into the HTML response.
In a nutshell that's about all there is to it. I should also mention that the Filter attempts to ensure the HTML resource does not get cached because if any javascript resources required for the page are modified a new javascript tag URL would have to be generated. If the HTML resource is cached the javascript changes would never be picked up by the requester. It attempts to do this by stripping out any request headers that the WebContainer might use to indicate caching is possible.
You can see this in action in the Zazl AMD Optimizer sample WAR, also a wiki page is found here that provides some more details. Alternatively, I have a Music Server Application found here that also demonstrates the Filter and Optimizer in action.
Saturday, January 14, 2012
Optimizing with AMD JavaScript loaders - Part 2
This is the second part, an update might be a better description, for my earlier blog post Optimizing with AMD JavaScript Loaders. Since then Dojo 1.7 is now in GA and I have updated the Zazl Dynamic Optimizer to support optimizing Dojo 1.7 based applications.
I had originally wanted to have the optimizer work generically with AMD compliant loaders and also be able to optimize plugin references using the referenced plugins themselves, however it became obvious that this is not a reasonable goal to achieve with the state that AMD spec is currently in. The result of this is that the optimizer provides its own AMD loader (similar to Almond in that it expects all the modules to be part of the javascript stream) and also supports its own server-side plugin API that allows for custom optimizations to be applied where possible.
The Zazl Optimizer's AMD Loader
As the optimizer ensures all the required AMD modules are part of the javascript stream delivered to the client there is no need for the loader to fully support asynchronous loading. I also found that the loader had to support modules that contained references to the local "require" function. The loader will ensure that modules referenced from "define" calls will be present in the initial javascript stream. If modules contain calls to the local "require" the loader will make additional XHR calls back to the optimizer to load the "required" module along with its dependencies, again in a single response.
Another value-add feature that the loader provides is the ability to preload a cache that is made available to modules via a "cache" property on the require object. The cache property is not part of the AMD spec but the Dojo AMD loader provides this support. This enables optimizations to be made for plugins such as the dojo/text plugin. Using a server-side extension the cache is preloaded with the text value that the plugin will provide, thus avoid and additional XHR call to load the resource.
Configuring the zazl loader
The entry point into Optimizers AMD loader is called "zazl". If follows a similar pattern to how my lsjs AMD loader had defined its entry point. For example :
zazl({
packages: [
{
name: 'dojo',
location: 'dojo',
main:'main'
},
{
name: 'dijit',
location: 'dijit',
main:'main'
},
{
name: 'dojox',
location: 'dojox',
main:'main'
}
]
},
["amdtest/Calendar"],
function(calendar) {
console.log("done");
});
Additionally, an zazl.json file has to be provided on the server-side.
Here is an example of the zazl.json file :
{
"bootstrapModules" : ["loader/amd/zazl.js"],
"debugBootstrapModules" : ["loader/amd/zazl.js"],
"amdconfig" : {
"plugins" : {
"dojo/has" : {
"proxy": "optimizer/amd/plugins/dojo/has",
"has" : {
"host-browser" : 1,
"dom" : 1,
"dojo-dom-ready-api" : 1,
"dojo-sniff" : 1
}
},
"dojo/text" : {
"proxy": "optimizer/amd/plugins/dojo/text"
},
"dojo/selector/_loader" : {
"proxy": "optimizer/amd/plugins/dojo/selector/_loader",
"defval": "dojo/selector/lite"
},
"dojox/gfx/renderer" : {
"proxy": "optimizer/amd/plugins/dojox/gfx/renderer",
"defval": "dojox/gfx/svg"
}
},
"i18nPluginId" : "dojo/i18n"
},
"type" : "amd"
}
Optimizing plugin references
Having given up on loading and running the plugins themselves on the server-side I faced the reality that to support frameworks like Dojo I would have to provide server-side equivalents for some of the plugins the Dojo provides. As the server-side was already providing a commonjs loader environment I decided to write these plugin proxies just as commonjs modules.
A server-side plugin proxy is configured via the zazl.json configuration file.
For example :
1) write(pluginName, normalizedName, callback, moduleUrl) - the return value from the callback is written into the javascript stream.
2) normalize(id, config, expand) - the returned value is used to determine if the plugin has additional dependencies that need to be included in the javascript response stream.
The "config" param for both calls is value specified for the "amdconfig" property in zazl.json.
dojo/has
The dojo/has plugin is used to determine whether to dynamically include other modules based on a "has" configuration. This provides a challenge for how to deal with this in an optimized environment. I chose to provide my own server-side version of the has plugin that used its own configuration. It provides a "normalize" function that is used to direct the optimizer to include these additional dependencies. The "has" configuration is provided in the zazl.json configuration file.
dojo/text
The dojo/text plugin is used to load in text resource dependencies. Ideally, in a optimized environment you would want the optimizer to ensure these text resources are included in the javascript response stream in a form that avoids additional downloads. The dojo/text plugin makes use of a "require.cache" property to populate a cache. The zazl AMD loader supports populating the cache by providing an "addToCache" function. The server-side version of the dojo/text plugin writes calls to this function that are included into the javascript response stream.
This ensures that when the client-side version it will find the cache value and avoid an XHR call to obtain the resource.
dojo/selector/_loader
The dojo/selector/_loader plugin determines what selector engine to use. The client side version expects a true browser environment to be available to determine the required engine type. This not something that can be determined easily in an optimized environment so for now the server side version I have written provides a "normalize" method that simply returns the default "lite" module id.
exports.normalize = function(id, config, expand) {
return "dojo/selector/lite";
};
The optimizer ensures that this module, along with its dependencies, is included in the javascript response stream. I plan to make the value returned configurable.
dojox/gfx/renderer
The dojox/gfx/renderer plugin works in a similar fashion to the dojo/selector/_loader plugin. It currently returns a default value of "dojox/gfx/svg" via a provided "normalize" function.
i18n plugin support
Optimizing i18n plugin support is more complicated that other types of plugins due to the need to provide i18n message bundles based on the locale of the calling client. The other optimized plugins produce output that can be cached and included for ever request for the same AMD module and its dependencies. The i18n output must be generated for each request made.
Because of this I have made the i18n plugin support a special case and there is specialized code that produces the message bundles. The "dojo/i18n" plugin supports the same format of messages as the requirejs i18n plugin does. The zazl configuration files specifies a property that indicates the module id of the i18n plugin. When the optimizer encounters an i18n plugin reference the details are used by the javascript response renderer to include the specified message bundles based on the locale of calling client. When rendered into the client the client i18n plugin finds that the messages bundles have been loaded in and avoid XHR requests to load them.
What's next ?
Update:
I have improved the configuration such that it is not duplicated in zazl.json. I have also written a blog post about the HTML filtering approach to inserting the required script tags
I had originally wanted to have the optimizer work generically with AMD compliant loaders and also be able to optimize plugin references using the referenced plugins themselves, however it became obvious that this is not a reasonable goal to achieve with the state that AMD spec is currently in. The result of this is that the optimizer provides its own AMD loader (similar to Almond in that it expects all the modules to be part of the javascript stream) and also supports its own server-side plugin API that allows for custom optimizations to be applied where possible.
The Zazl Optimizer's AMD Loader
As the optimizer ensures all the required AMD modules are part of the javascript stream delivered to the client there is no need for the loader to fully support asynchronous loading. I also found that the loader had to support modules that contained references to the local "require" function. The loader will ensure that modules referenced from "define" calls will be present in the initial javascript stream. If modules contain calls to the local "require" the loader will make additional XHR calls back to the optimizer to load the "required" module along with its dependencies, again in a single response.
Another value-add feature that the loader provides is the ability to preload a cache that is made available to modules via a "cache" property on the require object. The cache property is not part of the AMD spec but the Dojo AMD loader provides this support. This enables optimizations to be made for plugins such as the dojo/text plugin. Using a server-side extension the cache is preloaded with the text value that the plugin will provide, thus avoid and additional XHR call to load the resource.
Configuring the zazl loader
The entry point into Optimizers AMD loader is called "zazl". If follows a similar pattern to how my lsjs AMD loader had defined its entry point. For example :
zazl({
packages: [
{
name: 'dojo',
location: 'dojo',
main:'main'
},
{
name: 'dijit',
location: 'dijit',
main:'main'
},
{
name: 'dojox',
location: 'dojox',
main:'main'
}
]
},
["amdtest/Calendar"],
function(calendar) {
console.log("done");
});
Additionally, an zazl.json file has to be provided on the server-side.
Here is an example of the zazl.json file :
{
"bootstrapModules" : ["loader/amd/zazl.js"],
"debugBootstrapModules" : ["loader/amd/zazl.js"],
"amdconfig" : {
"plugins" : {
"dojo/has" : {
"proxy": "optimizer/amd/plugins/dojo/has",
"has" : {
"host-browser" : 1,
"dom" : 1,
"dojo-dom-ready-api" : 1,
"dojo-sniff" : 1
}
},
"dojo/text" : {
"proxy": "optimizer/amd/plugins/dojo/text"
},
"dojo/selector/_loader" : {
"proxy": "optimizer/amd/plugins/dojo/selector/_loader",
"defval": "dojo/selector/lite"
},
"dojox/gfx/renderer" : {
"proxy": "optimizer/amd/plugins/dojox/gfx/renderer",
"defval": "dojox/gfx/svg"
}
},
"i18nPluginId" : "dojo/i18n"
},
"type" : "amd"
}
Optimizing plugin references
Having given up on loading and running the plugins themselves on the server-side I faced the reality that to support frameworks like Dojo I would have to provide server-side equivalents for some of the plugins the Dojo provides. As the server-side was already providing a commonjs loader environment I decided to write these plugin proxies just as commonjs modules.
A server-side plugin proxy is configured via the zazl.json configuration file.
For example :
"plugins" : {
"dojo/has" : "optimizer/amd/plugins/dojo/has",
"dojo/text" : "optimizer/amd/plugins/dojo/text",
"dojo/selector/_loader" : "optimizer/amd/plugins/dojo/selector/_loader",
"dojox/gfx/renderer" : "optimizer/amd/plugins/dojox/gfx/renderer"
}
A plugin proxy can provide two exports :
1) write(pluginName, normalizedName, callback, moduleUrl) - the return value from the callback is written into the javascript stream.
2) normalize(id, config, expand) - the returned value is used to determine if the plugin has additional dependencies that need to be included in the javascript response stream.
The "config" param for both calls is value specified for the "amdconfig" property in zazl.json.
dojo/has
The dojo/has plugin is used to determine whether to dynamically include other modules based on a "has" configuration. This provides a challenge for how to deal with this in an optimized environment. I chose to provide my own server-side version of the has plugin that used its own configuration. It provides a "normalize" function that is used to direct the optimizer to include these additional dependencies. The "has" configuration is provided in the zazl.json configuration file.
dojo/text
The dojo/text plugin is used to load in text resource dependencies. Ideally, in a optimized environment you would want the optimizer to ensure these text resources are included in the javascript response stream in a form that avoids additional downloads. The dojo/text plugin makes use of a "require.cache" property to populate a cache. The zazl AMD loader supports populating the cache by providing an "addToCache" function. The server-side version of the dojo/text plugin writes calls to this function that are included into the javascript response stream.
function jsEscape(content) {
return content.replace(/(['\\])/g, '\\$1')
.replace(/[\f]/g, "\\f")
.replace(/[\b]/g, "\\b")
.replace(/[\n]/g, "\\n")
.replace(/[\t]/g, "\\t")
.replace(/[\r]/g, "\\r");
};
exports.write = function(pluginName, moduleName, write, moduleUrl) {
var textContent = require('zazlutil').resourceloader.readText(moduleUrl);
if (textContent) {
write("zazl.addToCache('"+moduleName+"', '"+jsEscape(textContent)+"');\n");
}
};
return content.replace(/(['\\])/g, '\\$1')
.replace(/[\f]/g, "\\f")
.replace(/[\b]/g, "\\b")
.replace(/[\n]/g, "\\n")
.replace(/[\t]/g, "\\t")
.replace(/[\r]/g, "\\r");
};
exports.write = function(pluginName, moduleName, write, moduleUrl) {
var textContent = require('zazlutil').resourceloader.readText(moduleUrl);
if (textContent) {
write("zazl.addToCache('"+moduleName+"', '"+jsEscape(textContent)+"');\n");
}
};
This ensures that when the client-side version it will find the cache value and avoid an XHR call to obtain the resource.
dojo/selector/_loader
The dojo/selector/_loader plugin determines what selector engine to use. The client side version expects a true browser environment to be available to determine the required engine type. This not something that can be determined easily in an optimized environment so for now the server side version I have written provides a "normalize" method that simply returns the default "lite" module id.
exports.normalize = function(id, config, expand) {
return "dojo/selector/lite";
};
The optimizer ensures that this module, along with its dependencies, is included in the javascript response stream. I plan to make the value returned configurable.
dojox/gfx/renderer
The dojox/gfx/renderer plugin works in a similar fashion to the dojo/selector/_loader plugin. It currently returns a default value of "dojox/gfx/svg" via a provided "normalize" function.
i18n plugin support
Optimizing i18n plugin support is more complicated that other types of plugins due to the need to provide i18n message bundles based on the locale of the calling client. The other optimized plugins produce output that can be cached and included for ever request for the same AMD module and its dependencies. The i18n output must be generated for each request made.
Because of this I have made the i18n plugin support a special case and there is specialized code that produces the message bundles. The "dojo/i18n" plugin supports the same format of messages as the requirejs i18n plugin does. The zazl configuration files specifies a property that indicates the module id of the i18n plugin. When the optimizer encounters an i18n plugin reference the details are used by the javascript response renderer to include the specified message bundles based on the locale of calling client. When rendered into the client the client i18n plugin finds that the messages bundles have been loaded in and avoid XHR requests to load them.
What's next ?
- Currently the optimizer is only usable in a Java JEE WebContainer environment that requires you to write your frontend HTML resources as JSP's. I now have a working HTML filter that enables developers to write HTML resources instead. The HTML is parsed for AMD module references. The HTML filter inserts the required javascript tag to load the module and all its dependencies based on the parser results. My next blog post will provide more details on how the HTML filter works.
- I plan to also provide a version of both the AMD optimizer and HTML filter for node.js.
- So far I have focused on making the optimizer support Dojo based applications. As jquery and other javascript frameworks are now supporting AMD I plan to ensure the optimizer supports applications using these frameworks too.
Update:
I have improved the configuration such that it is not duplicated in zazl.json. I have also written a blog post about the HTML filtering approach to inserting the required script tags
Subscribe to:
Posts (Atom)