Would You Hire This Guy?

(Based on a true story)

• Project 1: Wrote code with minimal errors.
• Project 2: Wrote code with no errors.
• ...
• Project n: Wrote code with no errors.

Grass? I don't see any grass!

What Is Protractor?

• An end-to-end test tool.
• Based on Selenium WebDriver.
• Aimed at testing AngularJS applications.
• Find more information at http://angular.github.io/protractor

Why Do We Need End-To-End Testing?

Different kinds of tests for different purposes:

• Unit Tests
• Integration Tests
• Performance Tests
• End-to-End Tests
• Manual Tests
• ...

Why is Protractor great?

• Built on top of Selenium.
• Integrates into your build tool chain (e.g. grunt).
• Tailored to testing AngularJS applications.
• Natural fit for your application's architecture: Everything is JavaScript.
• Developers enjoy writing test cases with Protractor.

Writing Protractor Tests

Install

Use the grunt-protractor-runner plugin:

npm install grunt-protractor-runner --save-dev

Automatic download of the WebDriver binaries as part of npm install (in package.json):

"scripts": {
  "install": "node node_modules/grunt-protractor-runner/node_modules/\
        protractor/bin/webdriver-manager update"
}

Create a configuration file

protractor.conf.js:

exports.config = {
  specs: ['test/**/*.spec'],
  baseUrl:'http://localhost:9000/',
};

Point it to your test files.

Define the URL to test.

Create a Grunt configuration

Gruntfile.js:

protractor: {
  options: {
    keepAlive: true,
    configFile: "protractor.conf.js"
  },
  run: {}
}

Implement your test

Use standard Jasmine syntax

describe('my app', function() {
  beforeEach(function() {

  });

  it('should do something', function() {
    expect(...);
  });
});

Run Your Tests

Deploy/Start your web application

Run the tests

grunt protractor:run

Watch in awe!

DOM Access

Select DOM elements by ID, CSS, or by Angular model:

var fooForm = element(by.id('foo'));

var barElements = foorForm.all(by.css('.bar'));

var input = fooForm.element(by.model('user.name'));

Angular Integration

Each test step is executed asynchronously, waiting for the Angular digest cycle to finish.

You can also use promises:

fooForm.evaluate('user.name').then(function (name) {
  expect(name).toBe('Foo');
});

The evaluate function allows you to take a look at the element's Angular scope.

Testing non-Angular apps

Protractor is primarily used to test Angular apps, since it integrates really well with the Angular lifecycle and scope.

Since it is just a wrapper around Selenium WebDriver, it can also be used to test non-Angular apps. Simply apply this change before running your test:

browser.ignoreSynchronization = true;

Protractor will now no longer wait for Angular to load, and you can use it to test any web application.

Lessons Learned

Unique ID Values

Use unique ID values on your controls to allow easy identification of the screen elements.

Make sure the ID values are consistent and don't change across test invocations.

Try to auto-generate the ID values or work with conventions.

Test on multiple browsers

Tests can be run in parallel against multiple browsers:

multiCapabilities: [{
  'browserName': 'firefox'
}, {
  'browserName': 'chrome'
}]

Browser configuration documentation: https://github.com/angular/protractor/blob/master/docs/browser-setup.md

Understand OS-specific behavior

Keyboard Input: Some operating systems (e.g. Mac OS X) handle keyboard input in a non-standard way.

Animations: Take rendering speed of the OS/browser into account.

var webDriver = protractor.getInstance().driver;

webDriver.sleep(500);

or - better:

browser.wait(element.isDisplayed);

Reusable test libraries

Since Protractor is written as a Node package, you can (and should!) create your own reusable test libraries.

var fooApp = require('foo-app-test');

fooApp.session.login('john.doe', '12345');

Create reusable libraries for

• Framework functionality used in multiple applications
• Application features to be reused across test suites
• Common use cases

Measure Code Coverage

Just like with regular unit tests, get a feeling for how much of your code is covered by your tests.

Remember: 100% code coverage does not mean that your code is error-free!

Continuous Integration Approach

Create xUnit Report Output

To create xUnit-compatible output, which is understood by most Continuous Integration systems, install the jasmine-reporters package:

npm install --save-dev jasmine-reporters

protractor.conf.js:

onPrepare: function() {
  require('jasmine-reporters');

  jasmine.getEnv().addReporter(
    new jasmine.JUnitXmlReporter('xunit-reports/', true, true));
}

Then point your Continuous Integration tool to the xunit-reports directory for reading the test results.

Check-in/Nightly build?

Trade-off:

• Running end-to-end tests on every commit will prevent regression issues early on.
• False positives due to incomplete commits.
• Overhead of deploying the application every time.
• Execution time of the end-to-end test suite.

Headless tests: PhantomJS

• Headless WebKit browser, ideal for testing in headless environments.
• Works well with unit tests.
• Some minor differences to Chrome --> make sure your tests work in all browsers.
• Doesn't play well with Selenium WebDriver, e.g. keyboard events.

Headless tests: Xvfb

• X Virtual Frame Buffer, available for all major Linux distributions.
• Simulates an X environment.
• Allows you to run standard browsers like Chrome or Firefox without an attached display.

Xvfb Configuration

npm install --save-dev grunt-env
npm install --save-dev grunt-shell-spawn

Gruntfile.js:

grunt.initConfig({
  // ...
  shell: {
    xvfb: {
      command: 'Xvfb :99 -ac -screen 0 1600x1200x24',
      options: { async: true }
    }
  },
  env: {
    xvfb: { DISPLAY: ':99' }
  }
  // ....
});

grunt.registerTask('protractor-xvfb', [
  'shell:xvfb',
  'env:xvfb',
  'protractor:run',
  'shell:xvfb:kill'
]);