justjs: node.js tutorials

New here? You might want to start at the beginning.

12/31 '12

Creating reusable Express modules with their own routes, views and static assets

Every developer eventually creates something nice they'd like to reuse in other projects. And Node is generally great at that, thanks to the npm repository.

But there is one area where things get sticky. While you can release any code you'd like as an npm module, it is more difficult to package up a collection of Express routes, template views and static asset files in such a way that other developers can just drop your module into their project, initialize it and go.

Adding Routes: Finding A Strategy That Works

Express has a somewhat under-documented mechanism that sounds really useful. I was excited when I first found it: you can make a second app and "app.use" it from the first. This seems a natural way to add routes specific to a module.

var express = require('express');

// My main app
var app = express();
app.get('/', ... home page ...)

// A second app object created in my cats module
var catApp = express();
catApp.get('/', ... main cats page ...)

// Mount the second app into the first one at the prefix '/cats'
app.use('/cats', catApp);
 
Unfortunately, this doesn't work as well as you'd hope, because routes of the first app that are attached after this point still get considered first. Which means you can't have catch-all routes and sub-apps at the same time:
// My main app
var app = express();
app.get('/', ... home page ...)

// A second app
var catApp = express();
catApp.get('/', ... main cats page ...)

// Bolt the second app into the first one at '/cats'
app.use('/cats', catApp);

app.get('*', ... default page always wins, even for /cats ...)
 
An alternate strategy is to pass the main app object to the cats module and allow it to add routes directly, always with a prefix. And this does work. The catch-all route is matched last as you might expect:
 
// My main app
var app = express();
app.get('/', ... home page ...)
// Call a method of the cats module to add its routes
cats.addRoutes(app);

// In cats.js, add the /cats route
app.get('/cats', ... main cats page ...)

// Back in the main app, add the default route last
app.get('*', ... default page ...)

Adding Views: View Folders And Template Languages Are Not What You Think They Are

OK, so we've managed to add routes in our module. Great. Now we need views for those routes to render...
 
Oh, geez! What template language is the main project even using? Who knows? It might not be the one we've chosen for our module. 

 

Worse, Express has a single setting for the folder where views reside. There is no support for loading views from an alternate location.

What's our way out of this mess? Simple: pick the template language we want and use it directly. Most template languages for Node have their own mechanism to load a file, compile a template, cache that template if it has been asked for before, and execute it on demand. And we're writing this module, so we can pick the template language we want and point it at the folder where our views are kept.

Here's how my Jot module renders the views that come with it. I use the Nunjucks template language, because it's familiar to folks who know Twig, Jinja, Jinja2 or any of several other fully featured template languages. But you can write similar code for other template engines:

 

var nunjucks = require('nunjucks');
var nunjucksEnv = new nunjucks.Environment(new nunjucks.FileSystemLoader(__dirname + '/views'));

function partial(name, data) {
  if (!data) {
    data = {};
  }

  if (typeof(data.partial) === 'undefined') {
    data.partial = partial;
  }

  var tmpl = nunjucksEnv.getTemplate(name);
  return tmpl.render(data);
}
 
Here I require nunjucks, then create a nunjucks "environment" pointing to a subdirectory of my module, then write a simple "partial" function that I can use to render any template. What's more, I also inject "partial" itself into the template so that I can have templates that invoke other templates. This is nice if, like me, you like clean namespaces for every template and dislike variables being shared by included files. I think that cuts down on bugs and results in cleaner code, but you can skip it if it doesn't excite you. Why not? This is your own module. You're not taking away my ability to do it elsewhere in the project.
 
I also have a simple "render" function, which I call when I'm ready to render a template as a response to the browser:
 
function render(res, template, info) {
  return res.send(partial(template, info));
}
 
All I have to do now is put it to work, passing data to the template just as I would if I were calling res.render normally:
 
return render(res, 'page.html', { name: 'bob' });
 

Static Assets: get the timing right

 
We're almost there! There's one more thing our reusable module needs before it can do everything your project-level code can do: a way to serve up static asset files. (What? Your module doesn't need any? Great. But mine often have images, CSS and browser-side JavaScript code they need to deliver.)
 
Let's say we want static files to be delivered if they appear in the "public" subdirectory of our "cats" module and a matching URL starting with "/cats" is seen. This is similar to what the "express.static" middleware typically does at the top level of a project.
 
But if we just call:
 
app.use('/cats', express.static(__dirname + '/public'))
 
In the "addRoutes" method of our module, after routes have been added to the project, and there are any wildcard routes in the project, we're out of luck. Why? Because by default, as soon as a route is added to an Express app, the Express "router" is added to the app, which means in a nutshell that all routes will be considered first before our static files are considered. And I do mean all routes, not just those in our module.
 
There are two reasonable workarounds for this:
 
1. Provide two different methods in your module, "addStatic" and "addRoutes." Require developers to call "addStatic" before they add any routes to their Express app objects. This will work, but it's confusing and people get it wrong a lot. It can be difficult to guarantee that no code has added a route yet in a big project.
 
2. Consider working around the problem with your own route to deliver static files. We're not really avoiding Express, because we'll take advantage of a function provided by Express to send static files efficiently. Here's a working example:
 
    app.get('/cats/*', function(req, res) {
      var path = req.params[0];
      // Don't let them peek at /etc/passwd
      if (path.indexOf('..') === -1) {
        return res.sendfile(__dirname + '/public/' + path);
      } else {
        res.status = 404;
        return res.send('Not Found');
      }
    });
 
What does this route do exactly? It matches anything beginning with /cats/, captures the rest of the URL as req.params[0], and uses the handy res.sendfile method to deliver the file. Boom! Almost as simple as an express.static call.
 
There's one catch though: the user could submit a devious URL containing sequences like /../ that, if taken literally, would allow them to view files anywhere on the system. We address this by rejecting all URLs that contain two periods together (".."). If your static files actually do that on purpose, you can use more subtle code to validate the URL before treating it as a path.
 
My personal preference is for solution #2, because it makes the process of adding modules to a project more foolproof and predictable. Routes match in the order they were added, including the route that delivers static files. And it becomes  intuitive that a generic wildcard route should be added last after all modules have been added to the project. There are no nasty surprises.

No, I'm Not Crazy, or Try These Test Programs

Here are some test programs that demonstrate the issues I ran into above. Be sure to consider these (and try them) if you have an alternative solution in mind. Then, if you think there's a more elegant way, definitely drop me a line. Thanks!
 
 
 
 

 

blog comments powered by Disqus