Saturday, October 29, 2011

AMD - Loading from HTML5 localstorage

One of the best advantages of using AMD as your javascript loader architecture is that it allows for different AMD implementations to be "plugged-in" without requiring modifications to your application code. As more devices provide HTML5 compliant browsers as part of their bundled software the need for more specialized loaders becomes important, a tailored loader will ensure your application loads in the most efficient manor.

HTML5 can play an important part in all of this as it provide a number of "localstorage" specifications that can be leveraged by AMD loaders to load locally within the device if possible.

Over the summer I decided to try this out and the result is an AMD compliant loader that will load its modules from a "localstorage" implementation. It's available here on github.

One goal I had was that the "localstorage" implementation used should be configurable. The HTML5 localStorage API works great however it has a hard limit of 5MB of storage. This limit is quite easy to reach, especially if you are using a large set of uncompressed modules. In addition to the default localStorage implementation the loader provides a Web SQL storage implementation that can specify the size of the storage to be used. While Web SQL is no longer active in the HTML5 spec it is still currently available in all webkit based browsers. That means Google Chrome, Safari and iOS based browsers can use it. Also there is now an IndexedDB storage implementation available too that works in Chrome and Firefox. Alternative custom storage implementations can be written by following the storage implementation spec.

Another goal I had was to make the loader work independently of any server-side requirement. It is usable just by referencing it via a script tag in your HTML file. There is a caveat with using it this way though, when modules need to be updated the whole storage area must be cleared. To overcome this limitation the loader optionally uses a configured "check timestamp" URL to check timestamps on each of the modules it loads. If a returned timestamp for a module is different than the one stored locally for it the module is reloaded from the server. The protocol for the "check timestamp" API is very simple and allows for a variety of server-side technologies to be used to provide this functionality. In fact the lsjs project provides a Java based servlet implemenetation and also a node.js based implementation.

So how does it perform ? When compared with regular AMD based loaders that load modules asynchronously its performance benefit can be seen in higher latency environments where the time taken to load the modules or perform validation based cache calls for the modules can affect load time. The lsjs loader does still have to load each module independently via script injection or eval calls and  this can affect load performance, especially in browsers with slower Javascript engines. One thought I have on improving this is to allow the lsjs loader to load all the required modules in a single script injection call or eval call. That would require the loader storing knowledge of the dependency chains involved.

In addition to the load performance improvements I have some other ideas on how the loader can emulate HTML5 appcache. Using AMD plugins other required resources such a text, css and images can be loaded from localstorage too. The advantage over appcache is that the loader has complete control on if and when the cache is used versus loading from the server.

Overall it been a very interesting experiment that has allowed me to learn how to write an AMD compliant loader. As I have found with my work on dynamic server-side optimizers the ability to write specialized AMD loaders (my next blog post will cover more on this) has become a hard requirement.