How do you know your site is working?
You don't. Unless you monitor it
Challenge - I
- Changing the style on a page might break others
- We’re often unaware of issues caused - unrelated pages, only on specific devices etc
- This happens on changes introduced post-launch as well
- Manual QA not a feasible solution
- Automated testing doesn’t catch visual problems
Challenge - II
- Confirm live site uptime
- Validate key features are functional
- Embedded elements we have no control on, might provide key functionality (StripeJS, Facebook comment, Google login)
Shoov - Gizra's testing toolkit
Solution I - Visual Monitoring
- Automatic process constantly takes screenshots in various browsers and devices
- Compares to expected page format (baseline image)
- Alerts dev team when the page changed
Solution I - Visual Monitoring
Visual Monitoring (Mobile Emulation)
it('should show the home page',function(done) {
client
.url(baseUrl)
.webdrivercss(testName, {name: '1'}, shoovWebdrivercss.processResults)
.call(done);
});
Visual Monitoring (Cross Browsers)
Solution II - Live Monitoring
- Test code written in any language (e.g. Behat, CasperJs, etc') runs against live site
- Alerts dev team when a test failed
- Keeps log of test actions - devs can immediately reproduce the error and start debugging
Live Monitoring (One Click Setup)
Live Monitoring (Real Time Logs)
Live Monitoring (Functional Tests)
# .shoov.yml
addons:
- selenium
env:
# GitHub username
- secure: "8a0e4139d27a8f1cec757c485eaa3ca5a038074ced07af46a90959db331a2dd8a7c8"
# GitHub password
- secure: "8a0e4139d27a8f1cec757c485eaf2eb3a1210953ec078f41ab037d93261332d58ed51849fce6258a"
before_script:
- cd behat
# Instead of composer install, we save time by getting
# a gzipped file with the vendor and bin directories.
- tar zxf behat-vendor.tar.gz
- cp behat.local.yml.shoov behat.local.yml
script:
- ./bin/behat --tags=~wip
