Compiling / Optimizing A Subset Of A RequireJS Application
In my last few blog posts, I explored the lazy-loading of a RequireJS module within a modular JavaScript application. I don't know if such an approach is really worthwhile - I'm not convinced that it wouldn't just be better to concatenate everything into a single, front-loaded file; but, I thought it was a fun mental exercise. To continue on in the same vein, I wanted to think about how this approach would work in a build script. It's one thing to lazy load a module when all modules remain independent; but, what happens to lazy-loading when you need to start building and optimizing your scripts?
Life is simple when you keep your modules in individual files - you make requests to load one module and RequireJS magically handles all the dependencies. This is easy, but it can incur a large numer of costly HTTP requests back to the server. To cut down on this cost, RequireJS provides a build tool - r.js - that can concatenate and compact all your modules and dependencies into a single file.
r.js works by analyzing and walking your dependency tree, inlining and compacting each module along the way. If you're building a single file, this couldn't be easier. If you want to compile a lazy-loaded sub-system of your RequireJS application, however, your build script gets a little bit more complicated. But, after some trial an error, I found out that the complications are actually quite minimal.
To start off this exploration, I used the same lazy-loaing FAQ code that I had in my previous two blog posts. The directory structure for this code looks like this:
- / build / [RequireJS build tools]
- / css /
- / js /
- / js / lib / [...]
- / js / templates /
- / js / templates / faq.htm
- / js / views /
- / js / views / faq.js
- / js / main.js
- demo.htm
In this application I want to lazy load the FAQ module - views/faq.js. That is, I don't want to bring it down to the client until it is actually requested by the user. Now, if you look at the faq.js module, you'll see that it has several dependencies:
NOTE: I am leaving out most of the file for brevity.
faq.js - Our FAQ Module
// Define the module.
define(
[
"jquery",
"text!templates/faq.htm",
"util"
],
function( $, faqHtml ){
// I provide access to the FAQ modal window.
function FAQ(){
// .... excluded for brevity.
}
// Define the class methods.
FAQ.prototype = {
// .... excluded for brevity.
};
// -------------------------------------------------- //
// -------------------------------------------------- //
// Return the module constructor.
return( FAQ );
}
);
As you can see, the FAQ module has four dependencies:
- jquery
- text
- templates/faq.htm
- util
NOTE: The dependency, "text!templates/faq.htm" is actually a dual dependency for both the "text" plugin and the "faq.htm" markup page.
Of these dependencies, three of them are what I consider "global" dependencies; that is, the following three modules are ones that we'll want to front-load in the application so that they can be used by other front-loaded modules:
- jquery
- text
- util
Only the third dependency:
- templates/faq.htm
... is uniquely coupled to the FAQ module. As such, when we compile the FAQ module down into its own, lazy-loaded sub-system, we'll want to inline the "faq.htm" markup, but exclude the other three modules, assuming they've already been loaded.
To do this, we have to be a bit more explicit in our RequireJS build script configuration, telling it which modules to include and exclude when compiling the dependency tree.
Before we look at the build configuration, however, let's take a quick look at our main bootstrap file - the one we really want to compile and optimize for our application:
main.js - Our Main Bootstrap File
// Set up the paths for the application.
requirejs.config({
paths: {
"domReady": "lib/require/domReady",
"jquery": "lib/jquery/jquery-1.7.2.min",
"templates": "templates",
"text": "lib/require/text",
"util": "lib/util",
"views": "views"
}
});
// Run the scripts when the DOM-READY event has fired.
require(
[
"jquery",
"util",
"domReady!"
],
function( $, util ){
(function(){
var faq = null;
var loaded = false;
var body = $( "body" );
var helpLink = $( "p.m-help a" );
// Handle the help link click - this will lazy-load the
// FAQ module when it is needed for the first time.
var handleHelpClick = function( event ){
event.preventDefault();
require(
[ "views/faq" ],
function( FAQ ){
// Initialize the module on the first
// request for its use.
if (!loaded){
loaded = true;
faq = new FAQ();
}
faq.open( body );
}
);
};
// Bind click handler for the FAQ link.
helpLink.click( handleHelpClick );
})();
}
);
There's a couple of important things to note when looking at this bootstrap file. For starters, notice that the FAQ module is being lazy-loaded when the user clicks on the actual "help" link (see the click-handler).
Secondly, notice that this main file contains a call to require.config(). This is important because we can use this main.js file to help define our build configuration file. Rather than redefining all of the Paths in our build file, we can simply tell it to use the paths defined in the main.js file.
Thirdly, notice which dependencies have been explicitly defined in this bootstrap file:
- jquery
- util
- domReady
- views/faq
Do you see something missing? This bootstrap file makes no mention of the "text" module. In our mini application, the "text" module is currently only required by the FAQ module; however, since "text" is such a powerful and commonly used module, I want to front load it as part of the primary compilation.
So, to recap, before we look at the build script, we want our main.js file to compile with the following modules:
- jquery (implicit)
- util (implicit)
- domReady (implicit)
- text (explicit) ***
... and, we want it to ignore the "views/faq" module, as that will be lazy loaded.
Ok, now that we see that there's a mix of implicit and explicit compilation behavior, let's take a look at the build configuration. For my demo, this is in a top-level directory called, "build."
app.build.js - Our RequireJS Build Configuration
({
// Define our base URL - all module paths are relative to this
// base directory.
baseUrl: "../js",
// Define our build directory. All files in the base URL will be
// COPIED OVER into the build directory as part of the
// concatentation and optimization process. You should use this
// so you don't override your raw source files.
dir: "../js-built",
// Load the RequireJS config() definition from the main.js file.
// Otherwise, we'd have to redefine all of our paths again here.
mainConfigFile: "../js/main.js",
// Define the modules to compile.
modules: [
// When compiling the main file, don't include the FAQ module.
// We want to lazy-load FAQ since it probably won't be used
// very much.
{
name: "main",
// Explicitly include modules that are NOT required
// directly by the MAIN module. This allows us to include
// commonly used modules that we want to front-load.
include: [
"text"
],
// Use the *shallow* exclude; otherwise, dependencies of
// the FAQ module will also be excluded from this build
// (including jQuery and text and util modules). In other
// words, a deep-exclude would override our above include.
excludeShallow: [
"views/faq"
]
},
// When compiling the FAQ module, don't include the modules
// that have already been included as part of the main
// compilation (ie. jquery, text, util). This way, we only
// include the parts of the FAQ dependencies that are unique
// to the FAQ module (ie. its HTML).
{
name: "views/faq",
// If we don't exclude these modules, they will be doubly
// defined in our main module (since these are ALSO
// dependencies of our main module).
exclude: [
"jquery",
"text",
"util"
]
}
],
// Turn off UglifyJS so that we can view the compiled source
// files (in order to make sure that we know that the compile
// is working properly - for debugging only.)
optimize: "none"
})
When r.js runs, using a build configuration file, it creates a "build" directory into which it will copy all of the optimized files. In this case, our build directory is called, "js-built". It is parallel to the "js" directory that contains our raw JavaScript source code.
The r.js build tool uses the "modules" property to figure out what to actually compile. In this case, you can see that we're defining two modules:
- main
- views/faq
When we define each module, r.js will automatically walk the dependency tree of the given module, inlining all of the dependencies. As part of the module definition, however, you can tell it to include modules that are not found in the dependency tree. You can also tell it to exclude modules that are part of the dependency tree.
Including a module is straightforward. Excluding a module is a little bit more complex. When you exclude a module, you have two choices:
- exclude
- excludeShallow
The first choice - exclude - will exclude a module and all of that module's dependencies. Essentially, it excludes an entire dependency tree. This is important to understand because the exclude will override any modules in the include. Meaning, if you include module "A" and then exclude module "Z", which has a dependency on module "A", r.js will end up excluding module "A" from the final file. Exclude takes precedence over include.
Since this is not always a desirable effect, r.js provides the second exclude option: excludeShallow. excludeShallow will exclude the given module, but will ignore the modules dependencies. This is perfect for our scenario because we want to exclude the "views/faq" module; but, we don't want to exclude its dependencies: jquery, text, and util.
If you look at the build configuration, you can see that our main module needs to explicitly include the "text" module and explicitly excludeShallow our lazy-loaded FAQ module.
The "views/faq" module, on the other hand, has to exclude most of its dependencies since they will be part of the main module. Remember, RequireJS doesn't know how we are going to be using these files. As such, its job is to build them as stand-alone files. In order to do so, it will attempt to inline jquery, text, and util into the FAQ module. But, since these will be part of our front-loaded main module, we have to explicitly exclude them from the FAQ module so that they are not doubly-defined in both main.js and "views/faq.js".
Once you have the build configuration file in place, you can call it from the command line (I am doing this from within the "build" directory):
node r.js -o app.build.js
When this runs, it outputs (to the terminal), which files it merged:
main.js
----------------
lib/jquery/jquery-1.7.2.min.js
lib/util.js
lib/require/domReady.js
domReady!
main.js
lib/require/text.jsviews/faq.js
----------------
text!templates/faq.htm
views/faq.js
As you can see, the appropriate files were included into the two different compiled files.
All of the above files were copied into a directory called, "js-built", as defined in our build configuration file. In order to test the build code, I had to put some switch logic in my demo.htm file. If you look at the HEAD of the following page, you can see that it looks for a URL parameter in order to figure out which "main" file to include as part of the RequireJS bootstrap:
demo.htm - Our Application Demo
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<title>Compiling A Sub-System With RequireJS</title>
<link rel="stylesheet" type="text/css" href="./css/demo.css"></link>
<script type="text/javascript">
// Default to using the raw modules.
var appDirectory = "js";
// Check to see if we have a flag in the URL to use the build
// modules compiled by RequireJS.
//
// NOTE: If I were to move this to production, I would change
// this whole set of logic with a server-side rendering switch
// that would check to see if I was on Dev or Production.
if (/useBuild=true/i.test( location.search )){
appDirectory = "js-built";
}
// Define the attribute and Script source for the RequireJS
// loader and bootstrap file.
var srcAttribute = (appDirectory + "/lib/require/require.js");
var mainAttribute = (appDirectory + "/main.js");
var tagHtml = (
"<\script \
type='text/javascript' \
src='${src}' \
data-main='${main}'> \
<\/script>"
);
// Write the RequireJS include to the browser.
document.write(
tagHtml
.replace( "${src}", srcAttribute )
.replace( "${main}", mainAttribute )
);
</script>
</head>
<body>
<h1>
Compiling A Sub-System With RequireJS
</h1>
<p>
This is the main page. It contains modules that may never be
invoked in standard usage. I am experimenting with compiling
complete sub-systems of detached modules with RequireJS.
</p>
<!-- BEGIN: Lazy Loaded Module Sub-System. -->
<p class="m-help">
Need help? <a href="#">Check out our FAQs</a>.
</p>
<!-- END: Lazy Loaded Module Sub-System. -->
</body>
</html>
As you can see, if the URL contains "useBuild=true", the demo will load our compiled file:
- / js-built / main.js
... otherwise, it will include the uncompiled version:
- / js / main.js
It took me a little while to wrap my head around how the build configuration works for RequireJS and r.js; but, once I figured out what everything meant (at least the parts that I looked at), it turned out not to be all that complicated. Again, I'm not completely sold on lazy-loading modules; but at least it's a relatively easy task to accomplish.
Want to use code from this post? Check out the license.
Reader Comments
Yeah, lazy-loading is controversial. Depending on the case it's better to combine everything into a single file even if that implies that some of the pages won't need all the JS that is loaded (cache between pages, single request). For these cases I would recommend using [almond.js](https://github.com/jrburke/almond) to distribute a single file. RequireJS is only needed in production if you plan to lazy-load scripts.
There are cases where it makes total sense to bundle only the common modules together and lazy-load the other stuff based on the user interaction/navigation. - In some cases I had 30+ JS files for production (300+ source files) since I had a large amount of JS and a good part of it was unique to some pages. So I loaded ~120KB of JS of shared scripts upfront and than each "section" was bundled into files that had 10-200KB, that way I avoided loading ~800KB of JS upfront. (the user flow was linear, so I could also start preloading the next page as soon as all the JS/img of current section finished loading) - For these cases I usually keep a [single entry point](http://blog.millermedeiros.com/single-entry-point-redux/) since I find it easier to handle which files to load and I can make changes on the JS structure without affecting the markup.
The best approach will vary based on the project. Need to profile it (use net panel on firebug and chrome) and check what makes more sense for your app. Usually if I have under 200KB of *minified & gzipped* JS I don't worry about lazy loading it.
PS: Even if you have a single file there are still benefits on doing an async load of the script (unless the app can't work without JS). Use the google maps/facebook/twitter async snippets as reference. I would avoid
in all cases since it affects performance.
Cheers.
@Miller,
Re: document.write(), I am only using that in this case since I have no server-side code. Typically for switching includes, I would do something on the server-side (ex. in ColdFusion):
That said, since I have yet to really use too much of this in production, I don't really know what my requirements will be. We are in the process of rebuilding / revisioning some software that I think we'll put together as a "collection of single-page apps." So, rather than having one giant page (which has parts that are VERY memory intense), we'll create a collection of very rich, interactive pages.
Of course, that's not set in stone either. But, I think it might make sense to have a set of "common" features that are loaded on each page; and then, a page-specific set of features (in its own compiled file).
Probably, we'll start off with one large file and just see if performance becomes a problem. If for no other reason, this makes it easier to put together and then use in production.
I've heard of Almond, but am not familiar with it - I'll have take a look. And, I'll also take a look at your blog article. Thanks!
Hi, could you tell me. How to make so that files with templates not included in main.js.
?.?. Sorry for my English
@Lomiren,
Understanding the r.js build tool is definitely difficult! It took me a good deal of trial and error to even get this far. But, if you look at this demo, I am compiling a file that is NOT included in the main.js file; the "FAQs" stuff is simply listed as a "module" in the build configuration file.
This was really helpful - thanks
Ben, thank you for writing this post - it answered a bunch of my questions. I just wrote a post regarding creating applications with independently tested and versioned modules and wanted to reference it here because it's related. http://naheece.wordpress.com/2013/08/31/independent-marionette-js-modules-using-require-js/
Hi Ben, great article which makes a complex topic look almost easy!
I have applied the concept of modules to my build, now I have less popular sections of my app lazy loaded, avoiding to force a massive 600K onload on to the user.
But somehow this article still didn't answer the original reason why I came to read your blog in the first place :
Let's say you wanted to optimise all your code with RequireJS. How do you avoid having also Jquery optimised for example? (or any other 3rd party lib) which has already been minified/optimised by the vendor!
thanks
Hi Ben,
This is a really useful post on optimizing RequireJS based apps.
A small suggestion about faq.js file.
Please use
// I provide access to the FAQ modal window.
var FAQ = function(){
// .... excluded for brevity.
};
instead of
// I provide access to the FAQ modal window.
function FAQ(){
// .... excluded for brevity.
}
Because "var" keyword keeps FAQ restricted to scope of your module and doesn't expose it globally.
@Abhishek, that is wrong. FAQ will not be visible outside the anonymous function that it is defined in.