Automated Frontend Testing


Or: How to Automate the Process of detecting when you Break Things

Why do I need frontend testing?


There are untold number of subtle
errors that can occur on the frontend.


  • Minor CSS changes that throw things off
  • Changes to JS files that break things
  • Aggregates changing when not necessary
  • Performance regressions


Additionally, frontend development is
becoming more critical as the trade matures.


We need the same testing abilities
that the backend has had for years.


  • Testing page load times
  • Testing render speeds
  • Sticking to a performance budget
  • Verifying visual changes
  • Accountability for code changes

workflow_alter()


In order to deliver the best, fastest site possible, we have to change our development processes.


Performance is not a checklist, it's a
continuous process.
Ilya Grigorik

…don’t take measures without measuring them.
Maximiliano Firtman

YES


The slides AND code are on GitHub, I will tweet the link. Yes, you may take code from the slides and use it for any purpose :)


This presentation is using a fresh clone of these slides, using the many examples I provided in the repo.


Functional testing

CasperJS

Casper allows for scripted actions to be tested. It uses PhantomJS under the hood.


  • Run the same test with multiple screen sizes.
  • Test complex features or components.
  • Automate complex user actions.
  • Test content creation, transactions, other features.
  • Keep an eye on problematic pages.

Test frontend components


# test the canonical picturefill demo
$ casperjs test picturefill.js

Read the blog post or watch a screencast
describing this test in detail.

Simulate user actions


# test simple user actions like clicking,
# keyboard navigation, filling forms
$ casperjs test user-actions.js

Read the blog post describing this test in detail.

Test author
workflow in Drupal


# test a Drupal demo site, log in, add content
$ casperjs test drupal.js

Read the blog post or watch a video.


More examples


Keep checking our blog post series on the Four Kitchens blog to learn more about CasperJS.


Performance Testing

Automating PageSpeed


Google has a service called PageSpeed Insights that grades your site and boils down tons of factors into a "speed index"


Testing sites can be automated. Get your API key first.

grunt-pagespeed


PageSpeed API is documented quite
thoroughly, but there's also a grunt plugin.


# examples/grunt/pagespeed
$ npm install
$ grunt # runs default task
$ grunt pagespeed:mobile # runs mobile task

View code on GitHub

gulp + pagespeed


Una Kravets wrote up an excellent walkthrough for gulp + pagespeed + local development



See the Four Kitchens
frontend performance training
for an alternate implementation

Phantomas


Phantomas is a PhantomJS-based
web performance metrics tool


It gives you loads of data about how the
frontend of your website is performing.


The usage guide is extensive.


# run a basic report
$ phantomas --url http://gruntjs.com

# set viewport dimensions, generate images of rendering process
$ phantomas --url http://gruntjs.com --viewport=320x480 --film-strip

# assert a test for total number of requests
$ phantomas --url http://gruntjs.com --assert-requests=20

grunt-phantomas


This grunt plugin is not just a wrapper for running the tool.


It provides detailed reports that track your data over time, allowing you to identify trends using dynamic charts that update themselves each time you run the grunt task


Graph displaying a spike in CSS size.

# examples/grunt/phantomas
$ grunt phantomas:default

# run report and generate screenshot
$ grunt phantomas:screenshot

# test for certain values. this might cause failure!
$ grunt phantomas:requests

View code on GitHub

Performance Budgets


The idea is simple: performance budgets are just like a monthy expense budget. We should keep track of how fat our sites grow over time.


grunt-phantomas has performance budget features that visualize over-budget metrics that you set.

grunt-perfbudget


Tim Kadlec, who first suggested performance budgets, released this tool to help teams meet their goals.


grunt-perfbudget relies on the immensely useful WebPageTest API to enforce a budget.


WebPageTest.org and its API are much more flexible than PhantomJS tools, because it can leverage multiple browsers, geographic locations, and network speeds.

# examples/grunt/perfbudget
$ npm install

# run report
$ grunt perfbudget
View code on GitHub
Output of grunt-perfbudget task

CSS Regression testing


CSS regressions? say it ain't so!


Having no scope at all, CSS is
the easiest thing to nudge out of place.


It's also easier to prevent than you think.

Wraith

Wraith is the easiest way to take screenshots of two environments, producing a visual diff.

Basic usage of Wraith


# examples/wraith
$ gem install wraith

# run the capture process
$ wraith capture demo

# view results in the browser
$ open shots/gallery.html

Multiple tests


Wraith handles one comparison per config file.


However, it has support for multiple configs, so several config files in one repo allows for multiple comparisons.


Read more on GitHub


Automating Tasks with CI

The Basics


Everything outlined in this section
requires two key ingredients:




Four Kitchens uses Jenkins and GitHub WebHooks
in our workflow, and you can use whatever you wish.

Trigger Jenkins builds
by pushing to GitHub


An oldie but goodie: check out our how-to from 2011


  1. You push to GitHub master branch (or merge PR)
  2. GitHub pings your CI server using post-receive hook
  3. "Yo Jenkins, the repository was updated!"
  4. CI server pulls the new code to your staging area


Although fairly simple in application, the post illustrates the basic concepts underlying all tasks involving Jenkins.

Use git hooks to test before pushing code


Git can be configured to automatically run
tasks before or after many of its operations.


pre-commit hook that runs jshint on your JavaScript.


pre-push hook that runs performance tests on your local. Helps enforce performance budgets.

Travis CI


  • Simple config via YAML file
  • FREE for open-source projects


Example using Four Kitchens' frontend performance training repo.

Click the red × to see Travis log



Automated Testing Services


Ghost Inspector


If you have a team willing to code up these examples, great! I like writing tests to complement code.


If not, Ghost Inspector can record a user's actions as they browse normally, and turn those actions into Casper code.


Ghost Inspector


Further reading



Thanks for getting this far!



hire me for frontend performance consulting

github.com/rupl

twitter.com/rupl

drupal.org/u/rupl

chris.ruppel ❀ gmail