Test the Web Forward – Better tests for a better web – Learning

View Github Repository Open presentation in a new window

rhauck

See all presentation from rhauck

Test the Web Forward – Better tests for a better web – Learning

0 0

testtwf-2013

On Github rhauck / testtwf-2013

Test the Web Forward

Better tests for a better web

Sacramento HTML5 Meetup - September 17, 2013

About me

Rebecca Hauck

Computer Scientist

Web Platform

@rebeccahauckhttp://testthewebforward.org

"If I had taken a proprietary control of the Web, then it would never have taken off. People only committed their time to it because they knew it was open, shared: that they could help decide what would happen to it next..."

—Tim Berners-Lee

The web is defined by the community

Standards are the root of communication

Standards bring people together

But what exactly is a standard?

  • Defines how something should work
  • Based on the consensus of the community
  • It's theoretically possible to implement
  • Can actually be implemented more than once
  • Implementations pass a common test suite

How do standards come to be?

Draft

  • A work in progress
  • Multiple (many) updates
  • Prototyping & early implementation begins
  • Advances on consensus
  • Tests are needed for proof of concept and resolution on details

Candidate Recommendation

  • More firm than a draft
  • Working group believes the spec has everything it needs
  • Implementations are needed to validate that belief
  • Tests are needed for implementations to be interoperable from the start

Recommendation

  • Final official version
  • Reached when there are two or more implementations
  • All implementations pass the same suite of tests
  • Future implementations also prove conformance using common test suite

15000+ more tests needed

6000 tests needing review

How do we get there??

More people

More tests

Better Web!

A movement is born...

Test the Web Forward!

First event: June 2012, San Francisco 2012: Beijing, Paris 2013: Sydney, Seattle, Tokyo (x2), Shanghai Upcoming: Shenzhen Hosts & Sponsors: Adobe, Google, Microsoft, Intel, Facebook, Baidu More coming soon!

So what happens at these events?

Learning

Kris Krueger, HTML5 Test Facilitator
Robin Berjon, HTML5 Spec Editor
Elika Etemad, CSS Flexbox Spec Editor

Learn from the experts

  • How to Read a W3C Spec
  • Testing with testharness.js
  • How to Write a W3C Reftest
  • Filing the Best Bugs
  • The CSS Testing Framework
  • The current testing status of your favorite spec!

How to write a Reftest

(Excerpt)

Full tutorial at http://testthewebforward.org under 'Resources'

Reftest Overview

  • Two files: Test file + Reference file
  • Test file uses the feature you're testing
  • Reference file is an exact visual match without using the feature you're testing
  • Self-describing (works both manually and with automation)
  • Cross-browser & cross-platform

Sample Test Case

Basic test for the transform property described in the CSS3 Transforms spec using the translate() function

Completed Reftest

Test Metadata

                    	    
<html>
<head>
<title>CSS Transforms Test: transform property with translate function</title>
<link rel="author" title="Your Name" href="mailto:youremail@address.com">
<link rel="help" href="http://www.w3.org/TR/css3-transforms/#transform-property">
<link rel="help" href="http://www.w3.org/TR/css3-transforms/#two-d-transform-functions">
<link rel="match" href="reference/css-transform-translate-ref.html">
<meta name="assert" 
  content="This transform moves the element by 100 pixels in both the X and Y directions.">
  . . .

Test File

Add an element with the transform applied

        			    
   <style type="text/css">
       .greenSquare {
           position: absolute;
           background: green;
           top: 0;
           left: 0;
           width: 100px;
           height: 100px;
           transform: translate(100px, 100px);
       }
    . . .
   </style>

Reference File

Add an element that renders identically

        			    
           <style type="text/css">
               .greenSquare {
                   position: absolute;
                   background: green;
                   top: 100px;
                   left: 100px;
                   width: 100px;
                   height: 100px;
               }
            . . .
           </style>

Test File

Add element only visible when the test fails

        				    
            <style type="text/css">
                . . .
                .redSquare {
                    position: absolute;
                    background: red;
                    top: 100px;
                    left: 100px;
                    width: 100px;
                    height: 100px;
                }
            </style>

Test & Reference File

Simple statement of how the page renders when the test passes

                    	    
  <body>
     <p>The test passes if there is a green square and no red.</p>
     . . .
  </body>

Test File

Add the red & green elements

                    	    
  <body>
     <p>The test passes if there is a green square and no red.</p>
     <div class="redSquare"></div>
     <div class="greenSquare"></div>
  </body>

Reference File

Add the green element

                      	    
  <body>
      <p>The test passes if there is a green square and no red.</p>
      <div class="greenSquare"></div>
  </body>

Passing test

Failing test

Hacking

Specs

  • HTML5
  • CSS Transforms
  • CSS Backgrounds & Borders
  • CSS Values & Units
  • Pointer Events
  • IndexedDB
  • CSSOM
  • Dom Events
  • WebRTC
  • CSS Fonts, Writing Modes
  • CSS Flexbox

Contributing

  • Over 2000 new tests written
  • Interoperability bugs found at each event
  • Several spec bugs also found
  • Converting existing test suites
  • Side hacking on test infrastructure
  • Tutorials & documentation

Building Community

Building a Better Web!

Join Movement!

  • Come to an event
  • Become a Sponsor
  • Host your own
  • Write tests!

Visit

http://testthewebforward.org

Subscribe

public-testtwf@w3.org

Follow

@testthewebforward#testtwf

Me

@rebeccahauck

Colophon

  • reveal.js by Hakim El Hattab
  • Fonts ChunkFive Adelle Source Sans Pro
  • Images Community: http://pitchfork.com/festivals/chicago/2012 Sir Edmund Hlliary: brewbooks http://www.flickr.com/photos/brewbooks/318391886 Standard: chmeredith http://www.flickr.com/photos/chmeredith/4560746937 Club party: Ellen Ordóñez http://commons.wikimedia.org/wiki/File: Colombian_Party_Cartel._Photo_by_Ellen_Ordóñez_for_Colombian_Party_Cartel.jpg Binary code: http://commons.wikimedia.org/wiki/File:Binary_Code.jpg Jump for joy: http://www.psxextreme.com/ps3-wallpapers/910.html