Headless Testing – Demo Time!

View Github Repository Open presentation in a new window

mdb

See all presentation from mdb

Headless Testing – Demo Time!

2 4

testing-with-xvfb

Headless testing against real web browsers with Xvfb!

On Github mdb / testing-with-xvfb

Headless Testing

Against Real Web Browsers with Xvfb

Mike Ball / @clapexcitement

Mike Ball

mikeball.info

github.com/mdb

twitter.com/clapexcitement

Prologue

Why are we talking about this?

My Beliefs

Automated Tests

  • ensure software works as it should.
  • Helps team move fast
  • Helps us understand how change impacts the larger system.
  • Helps ensures customer quality

Continuous Integration

  • ensures a codebase's health is continuously monitored with each change
  • methodically consistent, easily repeatable
  • as fast and cheap as possible, while still honoring its responsibilities
  • feedback cycle!

Servers are cheap!

Virtualization, Containers, the Cloud, etc.

Headless testing helps!

  • phantomjs
  • EnvJS
  • Rhino
  • RubyRacer
  • Nodejs

The Problem

What if your technology can't run in one of these headless runtimes?

Huh?

  • Flash
  • NW.js
  • Google Polymer web-component-tester

Let's focus on Google Polymer

Google Polymer

Google technology for authoring web components

web-component-tester

github.com/Polymer/web-component-tester

But...

web-component-tester assumes a GUI

and virtualized CI build agents don't have a display

Candidate Solutions

Travis CI

Sauce Labs

Travis CI

docs.travis-ci.com/user/gui-and-headless-browsers

  • Xvfb
  • Firefox

Sauce Labs CI

saucelabs.com

  • Selenium cloud
  • Access to over 100 device/OS/browser combos

But...

What if your team can't use these?

  • budget
  • SLA
  • etc.

Enter X Virtual Frame Buffer

AKA Xvfb

run GUI applications (like web browsers) with no GUI!

Sidebar: let's talk about the X Window System

how GUI displays are rendered on UNIX-like systems

X11 Display Server

  • current release of the X Window System
  • windowing system for bitmap displays used to build GUIs
  • how visual elements are rendered on a screen and available for user interaction via GUI in Mac OS and Linux

X Virtual Frame Buffer

  • behaves like any other X server, yet no graphical output is shown
  • performs graphical operations in memory without showing screen output
  • does not require the computer running it to have a screen

polymer-testing-box

github.com/mdb/polymer-testing-box

Core technologies

  • Xvfb
  • ansible
  • Google Chrome (with Flash!)
  • Firefox
  • Node.js
  • bower
  • web-component-tester

Demo Time!

provision a Vagrant Ubuntu box ansible will install and configure all necessary dependencies we'll run some tests against a Polymer web component

Vagrant

(not JavaScript)

  • built on VirtualBox
  • tool to spin up lightweight, headless VMs
## Vagrantfile ... config.vm.network 'forwarded_port', guest: 5900, host: 5901 config.vm.provision :ansible do |ansible| ansible.playbook = 'playbook.yml' end ...

ansible playbook.yml

(not JavaScript)

install and run Xvfb on display port 0 install Node.js, bower, web-component-tester Install Google Chrome and Firefox

Provision the box

ssh into the box

## clone core-ajax $ cd /vagrant $ git clone https://github.com/Polymer/core-ajax.git
## install bower components $ cd /core-ajax $ git checkout 0.5.6 $ bower install

run web-component-tester

But...

How do we know Chrome and Firefox really ran?

What about debugging?

VNC!

The Ubuntu box has no GUI but your Mac does!

## On the Ubuntu box $ sudo apt-get install x11vnc $ x11vnc -display :0 &
## On your Mac $ brew install Caskroom/cask/tigervnc-viewer # Start Tiger VNC Viewer on `localhost:5901`

The code

  • github.com/mdb/polymer-testing-box
  • github.com/mdb/nw-testing-box

Next steps

  • spin up an cloud instance during your builds?
  • AWS, Digital Ocean, OpenStack?
  • as-testing-box?
  • advanced web scraping?
  • headless functional testing?

Thanks!

  • twitter.com/clapexcitement
  • github.com/mdb