visual-regressions-talk

View Github Repository Open presentation in a new window

dunckr

See all presentation from dunckr

visual-regressions-talk

0 0

visual-regressions-talk

Mini talk about visual regressions workflow

On Github dunckr / visual-regressions-talk

Visual Regression Testing

@dunckr

what do I mean by that? best way to explain why it is needed

Test Stack

  • Comprehensive Coverage
    • Unit
    • Integration
    • Acceptance
    • ...
we have our test stack: however, we are missing something...

Think about the views

  • Why?
  • Where?
  • What?
example I have dogecoin - why not? you could argue that these are the most imporant part of the frontend they dont care if you have a bunch of spaghetti code or not but they are not being tested anywhere - where should they go? only place maybe would be the acceptance tests but they would be very brittle the acceptance tests the various users scenarios Yes, does this button exist and can I click it But what we are NOT testing how does it appear on screen and if it changes we dont want to start muddling that with css not practical if we are trying to quickly iterate or A/B test end up with alot of rewritten tests - not giving us much value

Look and Feel

  • Want
    • Notified of appearance changes
    • Find regressions
  • Don't want
    • Overhead
    • Brittle
Want: to be notified when something in the view changes to catch changes the break existing functionality/usability simple things such as hiding other elements are going to break the experience for the user prevent these simple mistakes and reduce the number of bugs cost benefit analysis DONT want: overhead of manual work constant of correcting broken tests one solution that we could achieve this is to take screenshost during different points of our app

Tools

number of tools that are available to help us do this huxley - slightly different and allows you to record macros adv - easy to create dis - moving elements around would cause them to break others using Selenium or PhantomJS to take screenshot adv - already written acceptance tests so less overhead

Git?!

  • Screenshots are included in Repo
  • MRs will include image diffs
  • Part of Peer review
real value is when we moved this process into git so our workflow is that screenshots of our app are taken during scenarios and committed then if any subsequent change causes them to differ then they will appear as a diff this is then included in the MR and then these become part of our peer review dis - cost associated with checking these images adv - spot visual issues that are not covered by existing tests dont make there way into production faster QA cycle as fewer bugs are found customers have more faith in product quality side benefit - higlights leaks if random screens are breaking when you are working on separate feature

Next Step

Image Comparsion

problem is small changes in the UI will be difficult to distinguish when we are looking at the image diffs something like a colon can will break the screenshots and we dont want to strain our eyes solution is to use image processing to visually highlight the differences between images making it more obvious and faster to check future different screen resolutions set in browser for testing responsive design ultimately these can be run as part of our CI notified of any broken screenshots just as you would any other breaking test emails/growls etc

Further Information

recommend explains this workflow in depth