justjs: node.js tutorials

New here? You might want to start at the beginning.

5/24 '12

Page Templates With EJS: Finally, We Can Have Nice Things

Page templates, partials, and other attractive concepts

So far, we've been sending web pages to the browser with the following marvelous technique:

var s = "<title>Oh come on</title>\n";
s += "<h1>Oh come on!</h1>\n";
s += "<h2>You have GOT to be kidding.</h2>\n";

Yeah, not so marvelous. It's too much work, it's tough to maintain, and it's scattered all over our "app" module. Which really ought to be our "controller" layer. Not our controller and view layers all mashed up like a Reese's Peanut Butter Cup. Except not yummy.

Clearly, we need page templates! Folks who have programmed in almost any web framework have used templates of one sort or another. What they all have in common is that they put HTML first and code second, allowing you to write HTML with some code in it, as opposed to the other way around.

Really good template systems also let you have layouts. Sure, you can write a complete HTML page - starting with the DOCTYPE, continuing through <html> and <head>, plowing through <body> and going on to that final </html> - for every single interaction. But that's a waste of effort, and it pretty much guarantees your pages will get out of sync with each other as you improve your design for one page and fail to apply it to 600 others.

The best template systems also permit you to have partials. Partials are snippets of HTML usually not intended to render an entire page. Instead they are responsible for some part of the page - the tabs, the breadcrumb trail, the side navigation, or a single blog post in a pageful of post summaries.

Good template systems also let you pass data into a template, but don't automatically assume every variable should be visible to the template. That's because good separation of concerns is a virtue in every part of your code, whether it's part of the model layer (the database), the view layer (the templates) or the controller layer (the routes). Many developers have come to feel that it's better to be explicit rather than doing things "by magic." Things that happen by magic are also things that lead to a lot of "WTFs per minute" and "spooky action at a distance" - situations in which you can't figure out why something is happening. Usually something bad.

At the same time, it's important that we be able to communicate things from a page template back "up" to the layout. In particular, the page title, the breadcrumb trail and the subnav (just to take common examples found on many sites) usually have consistent markup that can profitably be shared amongst pages via a shared layout template. But the actual text of the page title or list of breadcrumbs is most intuitively set in the page template. That's why great template systems allow us to set "slots" that are intentionally visible to other partials (a concept I learned from the Symfony framework).

Picking a Template System for Node

Which template system for Node.js best exemplifies these virtues? Well... none of them, really. Express has template support built in, which is great, but all variables visible to the page template are (as near as I can tell) shared with partials, which I consider to be a drawback. There's also support for layouts in Express 2.0, but no clear mechanism to share data between partials and layouts on purpose. Express 3.0 (which we're not using yet, as it's not the current stable release) defers the issue of layouts to the template language... and not all template languages have them built in.

But in this confusion lies an opportunity. Node developers tend to want to pick and choose from different systems and build apps their way, for maximum performance and control. So in that spirit we'll write our own view module to wrap our template engine and provide page templates, layouts and partials that work the way we want.

But will we write our own template language? Not on your life, buddy! Creating the right "glue" for our needs is one thing. Totaly reinventing the wheel is another. That way lies madness.

So what template language should we use? One that suits our philosophy of putting JavaScript front and center when possible and not learning additional languages to gain relatively small benefits. And one that builds on our existing knowledge of HTML. So while Jade and Haml have some neat characteristics, we're going to use EJS. In particular, the really sweet implementation of EJS from visionmedia which is available in npm.

EJS is right for us because it makes mixing JavaScript into HTML simple. EJS has just three really important features:

1. To print the value of a JavaScript expression as part of your HTML, while escaping any markup characters like <, >, and & to prevent XSS attacks, just use <%= ... %>, like this:

<h3><%= post.title %></h3>

As a good rule of thumb, this is the safest way to output variables in your markup.

2. To print the value of a JavaScript expression as part of your HTML without escaping markup characters (because you are intentionally printing out HTML markup), use <%- %>, like this:

<div class="post-body">

<!--- I hope you validated this HTML against XSS attacks! -->

<%- post.body %>

</div>

3. To simply run some JavaScript inside your template, use <% and %>, like this. Notice that we can shift back into "HTML mode" inside the callback function of our call to "_.each()". We'll switch back to "JavaScript mode" to close the _.each call:

<% _.each(posts, function(post) { %>

  <h3><%= post.title %></h3>

  <div class="post-body">

    <%- post.body %>

  </div>

<% }) %>

The _.each() function, as you've probably guessed, is an Underscore function that acts a lot like jQuery's each() function. It loops over all of our posts and invokes a function (usually written inline with the "function" keyword) once for each post.

EJS has more features, but these are the features we depend on. They provide almost enough power to finish the job.

I say "almost" because we still need a way to get page templates, layouts and partials to behave the way we want. And that's where our own "view.js" module comes in.

Housekeeping: Setting Up the Fourth Version of the Blog

First, make sure you've copied your blog-3 project to a new blog-4 folder.

Then install the ejs npm module:

cd node-apps/blog-4

npm install ejs

Changes to server.js: initializing the new view module

Now, to get started, we'll load view.js as part of our initialization in server.js. Just add one more callback to the async.series() call:

async.series([setupDb, setupView, setupApp, listen], ready);

And here's the setupView function:

function setupView(callback)
{
  // Create the view object
  context.view = require('./view.js');
  // Serve templates from this folder
  context.view.init({viewDir: __dirname + '/views'}, callback);
}

Like our other modules, the view module has an init() method that takes a callback, just in case we want to do time-consuming things. Unlike our other modules, we don't pass the context object into the module. Instead we just pass the options the module really needs. That's because I'm making an effort to keep this module "clean" of justjs-specific features, with an eye to releasing it as an npm module. And the more we can keep our modules independent of each other, the easier it is to debug, refactor and reuse code later.

The call to context.view.init() is interesting because we it deals with directories on the server. The special global variable __dirname is always set to the name of the folder in which the current source code file (server.js) is located. We can take advantage of this fact to reliably locate views (that is, ejs template files) that are kept in a subdirectory of the main project called views.

Introducing view.js: our view layer

Now that we've seen the configuration code, it's time to check out the view module itself.

We'll kick off by requiring the other modules we need:

var ejs = require('ejs');
var _ = require('underscore');
var fs = require('fs');

We need the ejs module for templates, Underscore to fill JavaScript gaps too fundamental to go without even inside a template, and the fs module to load template files.

Next we'll declare two variables that are global to the module:

var options;
var templates = {};

"options" is just a reference to the options we pass to the init() function. We'll use templates to cache each compiled ejs template so that we don't have to load each one from disk and compile it to a JavaScript function again on every page view. If the templates object already has a property matching the name of the template, we don't have to compile it again.

The structure of the module and the init() function are familiar:

module.exports = view = {
  init: function init(optionsArg, callback)
  {
    options = optionsArg;
    if (!options.viewDir)
    {
      throw new Error("options.viewDir is required, please tell me where the views are");
    }
    callback();
  },
  // more methods here
};

As before, we have a callback so that we have the option of doing time-consuming, asynchronous things in the init() function later.

The "page" method: rendering complete pages with layouts

Now we can check out the good stuff. Let's look at the "page" method. Our "page" method takes two parameters: a template name and a "data" object.

  page: function page(template, data)

The "data" object is passed to the ejs template we'll use to render the main body of this particular page. Any properties set on the data object become visible as local variables in the ejs template. So if we write this in a route:

res.send(view.page('post', { post: post }));

Then we'll be able to access the post object in post.ejs:

<h3><%= post.title %></h3>

There is a special property that is always set on the data object. That property is called "slots." This property is special because it is always passed on to any additional partials you call from inside your page template, as we'll see. It is also passed on to the special "layout" partial that renders the outer part of the page (the head element, the title element, the body element and so on). Since variables are always passed by reference in JavaScript, this means that the layout can see any changes you make to the slots property.

One great use for the slots property is creating a breadcrumb trail. The page template might add a "crumb" linking to the current page, and the layout itself might add another "crumb" pointing to the home page. Since this is a common operation, the "slots" object is created with an empty "crumbs" property to save us the trouble of checking whether it exists yet in every partial that touches it. We do the same for "title," which page templates use to pass the title of the page back to the layout, and "bodyClass," which page templates use to set a CSS class on the entire "body" element. Setting defaults here saves us many calls to _.defaults inside our templates:

    _.defaults(data, { slots: { crumbs: [], title: '', bodyClass: '' } });

We've seen _.defaults() before. We use it to provide default values for any properties that are still undefined on the data object and otherwise leave it alone. In this case we use it to provide a slots property if one was not explicitly passed in. We'll use _.defaults() in many places throughout justjs.

Now that we've set up the data object for this page, we'll call the "partial" method to actually render it, and stuff the result into the "body" slot. Notice that a page template is really just a partial like any other, with a thoughtfully set up data object:

    data.slots.body = view.partial(template, data);

Wrapping the Page Template in a Layout

Once we've done that, we're ready to tuck the output of the page template inside the layout by rendering the "layout" partial. But first we provide the option to override the layout or shut it off entirely and just return the output of the page template. This is helpful when certain pages require radically different layouts or are being rendered as part of an AJAX response:

    if (!data)
    {
      data = {};
    }
    _.defaults(data.slots, { layout: 'layout' });
    // ... Or cancel the layout from a partial
    if (data.slots.layout === false)
    {
      return data.slots.body;
    }
    return view.partial(data.slots.layout, { slots: data.slots });

Notice that we don't pass the entire data object from the page template to the layout - only the slots object is shared, as we intended. This makes it easier to insulate the layout from unintended side effects.

Of course, the real bottom line of the view module is the "partial" method. This is the all-important method that:

1. Compiles the ejs template, if it hasn't been compiled already.

2. Injects a function into the data object that allows partials to embed other partials. This allows us to split things like the breadcrumb trail and the rendering of a single blog post summary out to separate files, which makes code easier to maintain and read and promotes reuse.

3. Ensures that every partial has access to the shared slots object, and also to Underscore (which I consider essential to responsible JavaScript programming because of core features like _.each and _.defaults).

4. Actually renders the ejs template.

Here's the code for step one:

    if (!templates[template])
    {
      templates[template] = ejs.compile(fs.readFileSync(options.viewDir + '/' + template + '.ejs', 'utf8'));
    }

This code first checks whether we already have a compiled template function for this template name. If not, we call ejs.compile() with the text of the template. In order to do that, we first need to fetch the text of the template from the appropriate template file.

The fs.readFileSync function reads a file from the filesystem and returns a string, provided that we pass the filename and the encoding we want. The path to the template is simple enough: we already know what directory the views are in, so we just add the template name and a .ejs extension. The encoding is...

"Hold up there. What's this encoding business?" Historically there have been many ways to encode text on the Internet so that many different human languages can be represented. The fs module does not assume that everything is in English, and boring "plain ASCII" English at that. The most common way to encode text is the "utf8" encoding, part of the Unicode standard. "utf8" starts out with the plain ASCII character set but can also use multiple bytes to represent characters in just about any language, as long as your fonts cover them. In a nutshell, utf8 is the right encoding choice for the modern web.

"You're calling a 'sync' (synchronous) function. Isn't that against the spirit of Node?" Calls that load small files from the local filesystem occupy a middle zone between "clearly too slow to do without callbacks" and "often too fast to worry about," which is why synchronous versions of them are available. However, in this case, notice that we only load the template file if we haven't loaded it already since the server was launched. So there is really no meaningful performance penalty here. If you're concerned, you can refactor my code to compile everything in the "views" folder in the init() function. Heck, you can do it asynchronously with async.parallel and invoke the callback function of init() when it's all done. Knock yourself out.

Let's look at step two: making a "partial" function available inside the template.

    if (!data.partial)
    {
      data.partial = function(partial, partialData) {
        if (!partialData)
        {
          partialData = {};
        }
        _.defaults(partialData, { slots: data.slots });
        return view.partial(partial, partialData);
      };
    }

Notice that we don't force the issue. If you really want to pass in a different "partial" function, you can, and that will be communicated through to all of the partials called by your page template.

The function we assign to "data.partial" is a simple wrapper around a recursive call to view.partial (a call that a function makes to itself). Our wrapper function expects a partial name and allows us to also pass a data object to the partial. As before, we set partialData.slots to the same data.slots object that is shared with the calling partial if it has not been explicitly overridden.

Here's step three:

    _.defaults(data, { slots: {}, _: _ });

Now our "data" object is guaranteed access to a "slots" property (usually already present by this point unless it was intentionally cleared), and also has an "_" property (which allows us to call Underscore functions from JavaScript code in the template).

Finally, step four:

    return templates[template](data);

Once the template has been compiled, rendering it is very easy. A compiled template is just a JavaScript function that expects a data object. We pass in the data and the function returns the result.

Writing EJS Templates

That covers it for the view module. We're ready to render templates on demand. Now let's look at the actual templates.

We'll start with the layout, because the layout provides the basic structure of every page on the site. Here's the beginning of layout.ejs:

<!DOCTYPE html>
<% slots.crumbs.unshift({'title': 'Home', 'href': '/'}) %>

What's going on here? The first line is an HTML doctype for HTML5; it tells the browser to expect modern markup.

The next line adds a breadcrumb. Actually, it prepends a breadcrumb to the front of the breadcrumbs array. That makes sense because the "Home" crumb should always be the first one. If you're not familiar, the "unshift" method is a standard JavaScript method that inserts a new element at the beginning of an array (much as push() appends it at the end).

Then we tackle the "head" element:

<html lang="en">
<head>
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>justjs: <%= slots.title %></title>
  <link href="/static/css/justjs.css" rel="stylesheet" />
  <link href="/static/bootstrap/css/bootstrap.css" rel="stylesheet" />
  <link href="/static/bootstrap/css/bootstrap-responsive.css" rel="stylesheet" />
</head>

If you're basically familiar with HTML this isn't too new to you. The <meta name="viewport" ... /> line is a little unusual; it's part of the standard boilerplate that comes with Twitter Bootstrap, an amazing library of styles, buttons and functionality that makes brand new web apps look and feel ever so much better fresh out of the box. This particular line tells mobile browsers not to try to automatically "zoom out" like iPhones usually do if they suspect a page was not specifically designed for them.

The next line sets the title element. Totally familiar stuff, but it demonstrates how to use EJS to insert one of our slot properties:

    <title>justjs: <%= slots.title %></title>

After that we bring in stylesheets from Bootstrap as well as one of our own. We bring them all in from a directory called "static." Later on I'll show you how to make files in the "static" folder visible much like a regular Apache webserver would.

Our "body" element kicks off by using a slot to optionally set a CSS class on the body. This is very handy for writing CSS rules scoped to entire pages:

<body class="<%- slots.bodyClass %>">

Now a little barrage of HTML to set up the standard Bootstrap navigation bar, which we'll fill with tabs later. For now we just have a "brand" element with a link to the homepage:

  <div class="navbar navbar-fixed-top">
    <div class="navbar-inner">
      <div class="container">
        <a class="brand" href="/">justjs</a>
      </div>
    </div>
  </div>

Next we'll create a container div to wrap the main content of the page as well as the breadcrumb trail. We'll insert the breadcrumb trail by calling the "partial" function to render the breadcrumb.ejs template. We don't have to pass the crumbs explicitly because they are part of the "slots" object:

  <div class="container">
    <%- partial('crumbs', {}) %>

And we'll output the title of the page via slots.title:

    <div class="row">
      <div class="span12">
        <h1>justjs: <%= slots.title %></h1>

What did we forget? Oh yeah, the content of the page, as rendered by the page template! That will be in slots.body, thanks to the page() method ot the view module:

        <%- slots.body %>

Now let's close things up and bring in jQuery (we're bound to need it soon, and loading scripts at the end of the body is a good habit if possible).

      </div>
    </div>
  </div>
  <!-- Scripts at the end, for faster page load -->
  <!-- Pretty soon we're bound to want jquery! -->
  <script type="text/javascript" src="/static/js/jquery-1.7.2.min.js"></script>
</body>
</html>

That sums it up for the layout. But how does that breadcrumb partial work exactly? Here's crumbs.ejs:

<div class="row">
  <div class="span6">
    <ul class="breadcrumb">
      <% _.each(slots.crumbs, function(crumb, i, crumbs) { %>
        <% var last = ((i + 1) === crumbs.length) %>
        <% var crumb = slots.crumbs[i] %>
        <li class="<%= last ? 'active' : '' %>">
          <a href="<%= crumb.href %>"><%= crumb.title %></a>
            <%- last ? '' : '<span class="divider">&raquo;</span>'%>
        </li>
      <% }) %>
    </ul>
  </div>
</div>

The divs here are just Bootstrap structural stuff. The interesting bit is the JavaScript in the middle.

Here we loop over all of the crumbs found in slots.crumbs and create a list of links. Notice that this time when we use _.each() we take advantage of the opportunity to receive the current index in the loop and the list of crumbs itself in the callback function. This is a highly convenient feature of _.each() that makes it a good substitute for a 'for' statement in most situations involving an array.

Each link points to the "href" property of its crumb and takes its label from the "title" property:

<a href="<%= crumb.href %>"><%= crumb.title %></a>

The additional complexity comes from the desire to set a class on the last crumb (the active page), and also to insert a nice divider character between crumbs:

<li class="<%= last ? 'active' : '' %>">

...

<%- last ? '' : '<span class="divider">&raquo;</span>'%>

Note the use of the "?" operator, JavaScript's "ternary" operator. This is a handy way to avoid an "if" statement for simple cases. It works like this:

condition ? "value if true" : "value if false"

To set the "last" variable on each pass through the loop, we need to test whether our current index in the array is the last element in the array:

<% var last = ((i + 1) === slots.crumbs.length) %>

Now let's look at the "page templates" (templates that render a page on behalf of various routes). Here's index.ejs. Notice how similar it is to our previous hardcoded HTML. The layout does most of the work. All this template has to do is set the title and worry about its content. The markup has changed slightly to take advantage of Bootstrap's buttons. The button-box CSS class is found in our own justjs.css and just provides a little padding.

<% slots.title = "My Blog" %>
<div class="button-box"><a class="btn" href="/new">New Post</a></div>
<ul>
  <% _.each(posts, function(post) { %>
    <li><a href="/posts/<%= post.slug %>"><%= post.title %></a></li>
  <% }) %>
</ul>

The new.ejs template, which presents the form for adding a new blog post, is simple too. The markup has changed a bit more to provide a decent look and feel via Bootstrap. An error message is passed in by the route via the data object parameter, as we'll see:

<% slots.title = "New Post" %>
<% if (message) { %>
  <%= message %>
<% } %>
<div class="row">
  <div class="span9">
    <form class="well" method="POST" action="/new">
      <fieldset>
        <label>Title</label>
        <input name="title" class="post-title" value=""/>
      </fieldset>
      <fieldset>
        <label>Body</label>
        <textarea name="body" class="post-body"></textarea>
      </fieldset>
      <fieldset>
        <input class="btn btn-primary btn-large" type="submit" value="Save" />
      </fieldset>
    </form>
  </div>
</div>

And the post.ejs template is very simple indeed:

<% slots.title = post.title %>
<% slots.crumbs.unshift({'title': post.title, 'href': '/posts/' + post.slug}) %>
<%= post.body %>

One interesting thing we do here: prepend a crumb to the breadcrumb trail, just like we did in the layout. Since the layout template executes last, it prepends last, so the Home link winds up first in the breadcrumb trail.

"Hey, you're still not formatting that post body!" Yes, in truth, the way we output post.body here is still half-baked. Escaping <, > and & is good because it means the site won't play host to XSS attacks, but we really need to give consideration to letting the user style what they type. Right now we don't honor newlines or present links, let alone allow for rich text editing. In a future installment we'll see how to use a rich text editor such as CKEditor to add a user-friendly rich text editing experience while still protecting the site against XSS attacks and bad markup. One could also use a module like node-markdown to allow a text-based markup language more convenient than HTML.

Rendering Page Templates in app.js

Almost there! We still need to actually call our lovely new page templates from our routes. That's where app.js comes in.

The code at the end of each route that used to build the entire page with "s +=..." statements goes away. And this call comes in:

res.send(view.page('index', {posts: posts}));

That's all it takes to render a page template for the home page route (app.get('/')), passing all of the posts along to the page template.

Rendering the page template for an individual post (app.get('/posts/:slug')) is quite similar:

res.send(view.page('post', {post: post}));

Both versions, GET and POST, of the /new route still share a "newPost" function that renders the form. That function just got a lot shorter:

    // Send the "new post" page, with an error message if needed
    function newPost(res, message)
    {
      res.send(view.page('new', { 'message': message }));
    }

And that's all there is to it!

We could modify the notFound function to call view.page too rather than just passing a string, if we wanted to create a fancy 404 error page. It would work exactly as it does now, with 404 still being passed as the second argument.

Delivering Static Files (When Node Acts Like Apache)

However, there's one more important change to app.js. We need a way to serve static files, most importantly the Bootstrap CSS and icon files, as well as our own justjs.css.

That's why we add this line to the init() method to leverage the built-in support for delivering static files in Express:

    app.use('/static', express.static(__dirname + '/static'));

Again we take advantage of __dirname to locate the folder this source code file is located in, and add '/static' to point to a subdirectory by that name.

As for the contents of /static, I've divvied it up into folders like so:

/static

  /bootstrap

  /css

  /js

/bootstrap should contain the css, img and js folders from the current release of Twitter Bootstrap. If you look at the justjs project on github, I provide a tidy little subset that works with my markup.

/css contains justjs.css, a simple stylesheet that adds a little padding here and there:

body
{
    /* Without this (borrowed from one of the bootstrap samples) the bootstrap navbar
      clobbers the top of the content. */
    padding-top: 60px;
}

h1
{
    padding-bottom: 10px;
}

.button-box
{
    padding: 10px 0px 10px 0px;
}

.post-body
{
    width: 500px;
    height: 200px;
}

And /js contains just contains jQuery for now.

Ain't it pretty? (Well... relatively speaking)

That's truly it - we're ready to play with our new view layer. Fire up the site and have a look.

Bootstrap definitely makes a dramatic difference. And that's just a small down payment on what can really be done with Bootstrap by a skilled (or even competent) designer.

More generally speaking, the power of layouts, partials and slots sets developers free from repetitive coding and accidental mistakes and enables much better front end coding practices. Code is code, and all code affects the user experience your application delivers. It pays to take the view layer at least as seriously as the model and controller layers.

Next Steps: Locking the Front Door

The look and feel of the blog has improved a lot, but one big omission is easy to spot: security. There's absolutely nothing to prevent any random visitor from writing a new blog post. In the next installment I'll show you how to fix that by logging users into the site with the flexible Passport module, which supports many types of authentication, from a local database of passwords to Facebook Connect.

blog comments powered by Disqus