ÆFLASH

NOTE: this guide is out of date. It is only vaild for grunt-browserify@~1.3.0 and its included versions of browserify and browserify-shim. An updated guide is here.

It's been a year since I wrote the long Javascript Module Systems article, in which I concluded that CommonJS and Browserify was the best way to write modular javascript for the browser. Over the last year, I've had a chance to test that conclusion. I've used Browserify for every client side project since then, as well as the planned migration of a large application that previously used a concatenation-style build. Along the way, I've learned a lot about the whole Browserify process, some tricks, and some pitfalls.

Grunt is awesome

Although Browserify's command-line interface is pretty good, it's programmatic interface is much more powerful. Grunt in conjunction with grunt-browserify is perhaps the best way to set your configuration. While overkill for a simple case, it is invaluable as your config gets more complicated.

//...
browserify: {
  main: {
    "dist/main.js": ["src/main.js"],
    options: {
      transform: ["brfs"],
      alias: [
        "node_modules/lodash/lib/lodash.compat.js:lodash",
      ],
      external: ["jquery"]
    }
  }
}
//...

I will explain everything going on here in more detail later.

Grunt's watch mode -- where it monitors the file system for changes and runs tasks based on what has updated -- has revolutionized development. Coupled with live-reload, you could not have a more efficient front-end code/build/run/debug loop. Having your application refreshed automatically in the time it takes to alt-tab to the browser can't be beat.

Node Modules are awesome

The biggest advantage of Browserify is that you can use Node modules from NPM right out of the box. npm install module-foo --save and require("module-foo") is available to you. A lot of utility libraries, such as Lodash, Underscore, Async, and now even jQuery 2.x are available through a simple npm install, and require()able with no extra configuration. If bundling these heavyweight libraries isn't desireable, you can also include smaller, more-focused modules from Component as well. There is a decomponentify transform that can convert a Component module to a node-style module, but many components are also dual-published to NPM as well. Since both Component and Browserify use the CommonJS module style they are very inter-operable.

The real advantage comes from writing and using your own node modules. Encapsulating some functionality in a module allows you to completely isolate it from the rest of your application. It helps you think about defining a clear API for whatever part of your system a module may cover. You can also test it in isolation from the rest of your system. You do not have to publish it to NPM to use it, if it is very specific to your application -- you can simply refer to it using a git url in your package.json:

//...
  "my-module": "git+ssh//git@bitbucket.org/username/my-module#0.3.0",
//...

You can then require("my-module") as you would expect. You can also refer to a specific branch or tag by using the hash at the end. While it does not use the semver rules, you can at least freeze a dependency to a specific version and upgrade when ready. If you do not care about semver and always want to use the latest version, you can just use my-module#master.

Managing the Menagerie

Scaffolding and Project Templating

Using many node modules can be cumbersome, but there are extra steps you can take to make the whole process more manageable. First of all, to make creating new modules as friction-free as possible, I'd recommend using a tool like ngen to be able to quickly create scaffolding for a project. Customize one of the built-in templates to your liking. You can also pull commonly-used Gruntfile snippets and other tools into a common build-tools repository that every project uses as a devDependency. For example, we have a template that includes:

• a lib/.js
• a lib/.test.js spec file
• a README.md with a customizeable description and the basics on how to develop the module
• a Gruntfile.js with watch, browserify (to build the spec files for testing), jshint, and release (for tagging)
• .editorrc and .jshintrc files for maintaining code consistency.
• a testem.json for configuring testem for running the tests continuously in multiple browsers.
• a Makefile that serves as the catalog for all tasks that can be run and for bootstrapping development quickly. a make dev will install all the dependencies, build, and launch testem in watch mode with a single command. It also contains a make initialize command that will create the repository on Github/Bitbucket, and add all the hooks for things like Campfire and Continuous Integration.

You get all of this out of the box with a single ngen command. It is somewhat similar to Yeoman, except not opinionated. Instead, you can tailor your ngen templates to your own opinions.

Npm Link

Developing a sub-module concurrently with a parent module can be cumbersome. Oftentimes you do not want to have to push/publish a new version to test a child module in a parent module. This can be easily solved with npm link. If your child module is checked out in the same directory you can simply npm link ../child-module and it will create suitable symlinks for you.

Meta-Dependency Management

If you have dozens of modules it may become tedious to keep everything up to date with the latest versions. I would recommend writing a tool or script that reads the package.jsons of all your modules using the Github/Bitbucket APIs and detects out-of-date packages. If your project is open-source on Github, you can use David for free. A more advanced version of this tool would automatically update the dependencies, test the project, and publish a new version.

Continuous Integration is a must. Use something like Travis CI and put the badges on your README. I will also note that Bitbucket has more favorable pricing than Github for many small private repositories, and near feature parity otherwise. Bitbucket charges per collaborator, while Github charges per private repository.

Speeding up your build

So you are sold on Browserify. You set up a new project, and start including all your dependencies. You include jQuery, Lodash, Backbone, Marionette, Async, some snippets from Foundation, a couple jQuery plugins from Bower, a handful of private modules, some Components, and a few dozen classes from your app's core. You then notice that Browserify takes ten seconds to build and you spend an eternity waiting for your tests to re-run each time after you hit save in watch mode. How can you improve things?

Shim It

Browserify's default behavior when it encounters a source file is actually pretty involved. It has to parse the file and generate an AST. It then has to walk the AST to find any relevant require() calls as well as other Node globals. It then has to require.resolve each module and recursively parse those. However, this is not needed if you know a library doesn't contain any require() statements. Parsing the 100k lines of jQuery only to find it doesn't have any require()s is a waste. Instead what you can do is use browserify-shim, which is automatically included in grunt-browserify,

//...
browserify: {
  main: {
    "dist/main.js": ["src/main.js"],
    shim: {
      jquery: {
        path: "node_modules/jquery/jquery.js"
        exports: "$"
      }
    }
  }
}
//...

Since jQuery exports a global variable in lieu of a module system, we can take advantage of this to avoid expensive parsing. We just create an on-the-fly module named jquery and make sure that its known global exports ($) end up being the module.exports. Every module in the bundle can just var $ = require("jquery"). Also note that window.$ and window.jQuery will still be created in this case unless you call jQuery.noConflict(true) somewhere in your code.

If a shimmed module relies on other modules you can just add a depends object to the config:

shim: {
  jquery: {path: "node_modules/jquery/jquery.js", exports: "$"},
  backbone: {
    path: "app/vendor/backbone-1.1.js",
    exports: "Backbone",
    depends: {
      lodash: "_",
      jquery: "jQuery"
    }
  },

The key of the object is the name of the module to Browserify, and the value is the global name the shimmed module expects for the dependency. In this example, Backbone expects Lodash and jQuery to be available as window._ and window.jQuery. window.Backbone is then captured and made available as if there was a node module named backbone.

Shimming is usually faster that using a transform like debowerify or decomponentify, since those both involve parsing. (This only works if those modules export a global as a fall-back.) If a module does not export anything (such as a jQuery plugin), set exports: null to not export anything. You will have to call require("jquery.plugin.foo") somewhere in your code for it to be mixed in to the main jQuery module. Gluing together disparate module systems can get a bit ugly, I'm afraid.

Splitting up the build

You may also notice that re-bundling all your dependencies for every small change in your core app code is a bit inefficient. It is strongly recommended to create a separate build for your large, seldom-changing libraries.

One of the interesting features of the Browserify require() function is that it will defer to a previously defined require() function if a module can't be found in its bundle. Step through a Browserified require() using a debugger and you will see the logic. If you include two Browserify bundles on a page, module thats cant be found in the second will be looked up in the first. Very handy for splitting up the build and making everything work.

This is what the grunt config would look like:

//...
browserify {
  libs: {
    "dist/libs.js": ["src/libs.js"],
    shim: {
      jquery: {path: "node_modules/jquery/jquery.js", exports: "$"}
    }
  },
  main: {
    "dist/main.js": ["src/main.js"],
    external: ["jquery"]
  }
},
//...

The shim makes the libs build register jquery, and the external parameter instructs the main build to not include jQuery. On its own, the main bundle would throw an error on load, but when you load both dist/libs.js and dist/main.js on a page the main require function wont find the module named jquery, and defer to the libs require function, where jquery will actually be found. Now you can configure your grunt-contrib-watch to only build browserify:main when your JS in src/ change, as opposed to building everything all at once. This is actually quite speedy -- parsing a few dozen src/ files is generally 2 to 5 times faster than bundling all the libraries a typical application would include. This means your dev build and tests can be refreshed in one to two seconds.

Also, if you still want a single JS file in the end, you can just concatenate the libs.js and main.js -- it works equivalently to including two scripts.

Collapsing dependencies

Once you fall in love with private node modules, you may find it conflicts with your love for handy utility libraries like Lodash. You may find that your private-module-a depends on lodash@1.3.x and private-module-b depends on lodash@1.5.x, and your parent project depends directly on lodash@2.4.1. You inspect your bundle and find that three different versions of Lodash are included, needlessly adding to your app's file size.

While I would argue that this might be desirable in certain cases for certain modules (according to semver, a major version increment could include backwards-incompatible changes), you probably only want to include one version of Lodash in your final build. There is a clever and counterintuitive way to fix this in your browserify config:

//...
  alias: ["lodash:lodash"]
//...

By aliasing Lodash to itself, it guarantees that any require("lodash") encountered while parsing will resolve to the parent project's version of Lodash. We basically just short-circuit the module lookup process. Normally Browserify would use the version of Lodash local to private-module-a, but aliasing creates a global name that will override the default module lookup logic.

Circular Dependencies

In my previous article, I recommended temporarily using a global namespace or deferred require()s as a way to work around circular dependencies. However, I quickly came to realize that neither solution was ideal. Global namespaces are a form of JS pollution, leak your internals to 3rd party code, and can be overwritten. They also don't show up in the myriad of tools that can do dependency analysis on CommonJS modules.

Deferred require()s in theory can work, but you have to be certain that the second module in the cycle wont actually need the first module until the next tick in the JS event loop. If the second needs the first before that deferred require, it will be undefined and create a subtle bug.

I concluded that it was better to re-factor away the circular dependencies than deal with these two problems. I eliminated them using a few techniques, depending on the nature of each dependency cycle: pulling shared functionality into a 3rd module, using event-driven programming to decouple components, and dependency injection.

A really common pattern I use is when something like a "controller" needs something from a main "application" class, but the main application needs to instantiate the controller. In this case, I just use dependency injection:

//app.js
//...
var controller = require("./controller.js")(app)
//...
module.exports = app;

//controller.js
//...
module.exports = function (app) {
  var controller = new Controller({
    // do something with `app`...
  });

  return controller
}
//...

Using non-relative paths

There are cases when you don't want to use relative paths for everything and just want to use a path relative to your project root. It would be nice to simply require("myApp/views/buttons/foo_button") from src/controllers/foo.js rather than figuring out how many ../s to add to the front of your path. Luckily you can do this by dynamically creating aliases for every module in the core of your application using grunt-browserify's aliasMapping option. Here's what it looks like:

aliasMappings: [{
    cwd: 'src',
    dest: 'myApp',
    src: ['**/*.js']
}]

What this tells Browserify to do is to take every file in your src/ directory and create an alias based on the file path, prefixed with myApp. src/views/buttons/foo_button.js does become available everywhere as require("myApp/views/buttons/foo_button"). Very handy.

However, I will say that if you need crazy relative paths or deeply nested folders, its either a sign that parts of your app are becoming too tightly coupled, or your app might need to be split up into smaller, more autonomous and manageable modules. Some view needs to talk to a model way on the other side of the application? Rather than call it directly, use a global event bus/Pub-Sub. Another classic telltale sign is require("../../../utils/foo"). Just make utils/foo.js into its own private module, write some tests, and refer to it in your package.json. Then it's available everywhere as require("utils-foo").

Other tips and tricks

Dont have grunt-contrib-watch listen to your entire node_modules directory for changes to rebuild your libraries bundle. You can quickly run into file-system limits this way. Instead, only listen to the first-level package.json's -- they will be updated by npm installs. For your own npm linked modules, have those watchers touch their package.json's when their source files are changed -- as a way to signal that the parent needs to rebuild.

Colony is a handy little tool for generating a dependency graph of your code. If your code is a spider a web, its time to decouple and re-factor. Colony was very helpful in detecting dependency cycles as well -- I was able to feed its intermediary JSON into graphlib. Some cycles were 10 links long. I never would have found them, otherwise. Once caveat of Colony is that it doesn't use your Browserify config, just the default node require() logic, so it can be slightly inaccurate if you use aliases. The author of Colony also has a tool called Disc that can monitor filesizes, albeit with stricter CJS module lookups.

The brfs transform in-lines fs.readFileSync() calls - replaces the function calls with a string containing the contents of the file. This is a convenient way to load templates. Keep in mind that it can only use static paths to files -- the only variable it can evaluate is __dirname.

Finally, here is an annotated Gruntfile for a sample project. It follows all of the recommendations laid out in this article, if you want to see what everything looks like in action.