Testing the front end with CasperJS

Background

After the first few releases of MSNBC, we found we needed a way to check that the core parts of the system were working as expected, and that no regression bugs crept in to the system. We started writing tests to cover the most critical and complex parts of our codebase. Ideally we'd have been writing tests from day one, but sometimes we have to wait until the project matures to justify it to the client.

Choosing a testing framework

We agreed on the following requirements for a testing framework:

• We wanted everyone in the team to write tests for the tickets they were working on.
• We did not want the learning curve to be too steep.
• We wanted to focus on testing the front end (we have a considerable amount of logic decoupled with AngularJS), although we also needed to test the back end.
• Our tests would simulate a user navigating the site (functional testing). We wouldn't write unit tests. At least, not in the short term.
• Tests would run on local, development and testing environments. These environment's databases get wiped out daily with a sanitized copy of the production database, so it is safe to run tests there.

First attempt: Behat

Given those requirements, we discarded Drupal's Simpletest because it is not the best tool to test JavaScript. Our first choice was Behat + Mink. We started writing features and custom steps; and configured our Jenkins server so it would run tests against a headless Chrome browser when a GitHub Pull Request was merged in. Headless comes from the fact that there is no display since the browser runs in a server.

Using Behat achieved some momentum for a few weeks but we found the following pain points:

• Some feature steps were difficult to write. specially the ones involving JavaScript. Having to write JavaScript code embedded in PHP code was a tricky task.
• Tests would take a very long time to complete and sometimes fail because of race conditions. We had to add wait(10000) statements everywhere to make sure something like an image was loaded at a certain point while a test was running.
• The team struggled to write Behat features and custom steps.

Meet CasperJS

CasperJS is a navigation scripting & testing utility for PhantomJS and SlimerJS written in Javascript. The fact that seemed most appealing to us was that tests were written in JavaScript. For us, it felt just like opening the browser console and typing JavaScript commands to check if a variable is present, or a CSS3 selector returns the right result, or a triggered event would return the right data. We also found out that it was very easy to install.

We starting by porting our Behat tests to CasperJS and immediately realized that PhantomJS (the browser that CasperJS uses) was way faster and that running these tests in a Continuous Integration fashion required a simpler setup at our Jenkins server.

Even more important, both our front end and back end developers got excited with this technology and started to write tests, making suggestions along the way about how to structure them better. They found in CasperJS a new challenge to dive deeper into their JavaScript skills.

A test example

Here is the test that we wrote for the homepage. It checks that the header (AngularJS powered) has loaded properly and that there are 10 articles listed:

  
/**
 * homepage.js - Homepage tests.
 */

casper.test.begin('Tests homepage structure', 7, function suite(test) {

  casper.start('http://www.msnbc.com', function() {
    // Verify that the main menu links are present.
    test.assertExists('a.j-signin-label', '"Sign in" link is found.');
    test.assertExists('a.j-register-label', '"Sign up" link is found.');
    test.assertExists('li.main-nav__link--explore a', '"Explore" link is found.');
    test.assertExists('li.main-nav__link--watch a', '"Watch" link is found.');
    test.assertExists('li.main-nav__link--join-in a', '"Join In" link is found.');
    test.assertExists('li.main-nav__link--speak-out a', '"Speak Out" link is found.');
    // 10 articles should be listed.
    test.assertElementCount('article', 10, '10 articles are listed.');
  });

  casper.run(function() {
    test.done();
  });
});
  

That's it. Simple, huh? This test will ensure that our homepage works. In order to run it, save it anywhere, make sure that CasperJS is installed and run the following command:

  
$ casper test homepage.js
Test file: tests/homepage.js                                                    
# Tests homepage structure
PASS "Sign in" link is found.
PASS "Sign up" link is found.
PASS "Explore" link is found.
PASS "Watch" link is found.
PASS "Join In" link is found.
PASS "Speak Out" link is found.
PASS 10 articles are listed.
PASS 7 tests executed in 5.064s, 7 passed, 0 failed, 0 dubious, 0 skipped.    
  

That's it. You can see above that all of our assertions above passed.

The sky is the limit when you are about to define how thoroughly you want to test a particular page. Our advice is just to test what is critical or what has many dependencies and it is therefore more prone to fail. Then slowly build up your test suite as you go by, using a test to affirm that a logged bug actually exists, and then fix the code so that the test passes.

Next steps

We created a repository with the basics of how to install, configure and structure tests within your project. It also contains some useful links to help you ramp up on how to navigate pages and how to run assertions in your CasperJS tests. You can find it here.

Happy testing!