Scenery

A Scene Graph for Modern Browsers

Jonathan OlsonPhET Interactive Simulations | University of Colorado Boulder

PhET Interactive Simulations

• 128 free educational simulations in Java and Flash
• Previously seen in Sam Reid's talk
• Run over 110 million times, translated to 74 languages
• Moving to HTML5 to be available on more devices(iOS, Android, IE9+, FF4+, etc.)

Graphics Performance

• WebGL not widely available for mobile devices
• Optimized Canvas and SVG perform better in different situations (and on different devices)

Most libraries focus on either Canvas, SVG or WebGL

Abstraction over Renderers

• Changing the renderer (Canvas/SVG/WebGL/DOM) and optimizations used should be easy
• Allow parts of the scene to use different renderers
• In the future, automatically choose the best renderers based on performance hints and browser profiles

Most libraries only support one renderer at a time

Additional Desired Features

• Bounds tracking
• Input event system (hit regions, multi-touch support, cursor support)
• Clipping, Masking, Opacity, Gradients, Patterns, Visibility
• Text, embedded DOM, smooth shapes, CSG
• Integration with arbitrary content, like 3D with three.js
• DAG (directed acyclic graph) support
• Debugging utilities
• Accessibility

Most libraries have few of these

Scenery

A Scene Graph (like the DOM)

Layers

Higher performance by splitting content into layers

• Only need to redraw (or update) changed layers
• Each layer can use the best-performing layer for that content

greenContainer.layerSplit = true; // will be isolated in a layer

Subtree Layers

• Flags can change the renderer/layer for an entire subtree
• Nesting is fine, and Scenery combines layers where possible

Renderers

• Canvas: Ideal for many objects changing every frame. Slower when full-resolution on Retina-style devices.
• SVG: Performance varies, but large objects with CSS transforms are fast. Text looks 'crisper' than Canvas, and SVG can be a better choice on high-resolution devices.
• DOM: Supports text, images and arbitrary DOM, and may be faster for some cases (untested).
• WebGL: Anticipated highest performance for sprite-like cases, but currently lacks mobile browser support.

Canvas Layers

• Tracks the bounding box of changes (only repaint 'dirty' areas)
• Filters duplicate state change calls to the Canvas API
• Soon: Paint only nodes whose bounds are in the dirty region
• Soon: Off-screen subtree rendering for faster drawing

A Tour of Scenery

// Create a scene graph over a positioned block-level element
var scene = new scenery.Scene( $( '#hello-world-scene' ) );

// Add our text
scene.addChild( new scenery.Text( 'Hello World', {
  centerX: 200, // the center of our text's bounds is at x = 200
  centerY: 50, // the center of our text's bounds is at y = 50
  font: '40px sans-serif', fill: '#eee'
} ) );

// Paint any changes (in this case, our text).
scene.updateScene();

Shapes

// simple shapes can be accelerated
scene.addChild( new scenery.Rectangle( 10, 10, 100, 80, {
  fill: '#f00', stroke: '#fff', lineWidth: 2
} ) );

scene.addChild( new scenery.Path( {
  shape: kite.Shape.regularPolygon( 8, 30 ),
  fill: '#0f0', x: 150, y: 50
} ) );

scene.addChild( new scenery.Path( {
  // Canvas-like Path API. Supports SVG Shape( 'M0 90Q155 50 0 10Z' )
  shape: new kite.Shape().lineTo( 0, 90 )
                         .quadraticCurveTo( 155, 50, 0, 10 )
                         .close(),
  fill: '#00f', x: 190
} ) );

Images

var imageUrl = 'http://phet.colorado.edu/sims/energy-skate-park/' +
               'energy-skate-park-basics-thumbnail.png';

// direct URL reference, but cannot position based on bounds
scene.addChild( new scenery.Image( imageUrl, {
  x: 100, y: -30, rotation: Math.PI / 6,
} ) );

// can also pass in HTMLImageElement or HTMLCanvasElement references
var img = document.createElement( 'img' );
img.src = imageUrl;
scene.addChild( new scenery.Image( img, { opacity: 0.25, scale: 2 } ) );

DOM Elements

var element = document.createElement( 'span' );
element.innerHTML =
  '<label style="display: inline;">Type in me:</label>' +
  '<input type="text">';
scene.addChild( new scenery.DOM( element, {
  x: 120, rotation: Math.PI / 6
} ) );

// HTML, with the same styling and positioning as Text
scene.addChild( new scenery.HTMLText( '<em>Italic</em> and <b>Bold</b> ', {
  fontSize: 30, bottom: 230, centerX: 200, fill: '#9aa'
} ) );

Fills

first.fill = 'rgba(255,127,0,0.8)'; // full support for CSS colors

// linear gradients
secondBottom.fill = new scenery.LinearGradient( 0, 0, 100, 0 )
                               .addColorStop( 0, '#000' )
                               .addColorStop( 1, '#666' );

// radial gradients
secondTop.fill = new scenery.RadialGradient( 32, 32, 5, 64, 64, 60 )
                            .addColorStop( 0, '#0ff' )
                            .addColorStop( 1, 'rgba(0,127,255,0)' );

// patterns
var transform = dot.Matrix3.rotation2( Math.PI / 6 )
                           .timesMatrix( dot.Matrix3.scale( 0.3 ) );
third.fill = new scenery.Pattern( img ).setTransformMatrix( transform );

Strokes

first.stroke = 'rgba(255,127,0,0.8)'; // full support for CSS colors

// linear gradients
secondBottom.stroke = new scenery.LinearGradient( 0, 0, 100, 0 )
                                 .addColorStop( 0, '#000' )
                                 .addColorStop( 1, '#666' );

// radial gradients
secondTop.stroke = new scenery.RadialGradient( 32, 32, 5, 64, 64, 60 )
                              .addColorStop( 0, '#0ff' )
                              .addColorStop( 1, 'rgba(0,127,255,0)' );

// patterns
var transform = dot.Matrix3.rotation2( Math.PI / 6 )
                           .timesMatrix( dot.Matrix3.scale( 0.3 ) );
third.stroke = new scenery.Pattern( img ).setTransformMatrix( transform );

Stroke Styles

Supports all Canvas line styles

Directed Acyclic Graphs

Axis-aligned Bounds

// vertical layout
path.top = 20;
fastText.top = path.bottom + 20;
accurateText.top = fastText.bottom + 20;

// horizontal layout
fastText.left = 20;              // offset from the left
path.centerX = fastText.centerX; // center the path above
accurateText.left = path.left;   // align the lower text with the path

User Input

Multi-touch abstraction over DOM3/Touch/Pointer/MSPointer

section.cursor = 'pointer'; // change the cursor when the mouse is over a section

_.each( scene.children, function( section ) {
  // move each section individually
  section.addInputListener( new scenery.SimpleDragHandler() );
} );

Scenery In Use

Under development:

And 2 more: Build an Atom and Balloons and Static Electricity

Require.js for modularization

Mostly feature-complete, working on performance

Kite

2D Shape Handling | phetsims.github.io/kite

• Create shapes using Canvas-like API or with SVG paths
• Computes: stroked shapes, shape bounds and shape-line intersection for hit testing, transformed shapes, arbitrary subdivisions, tangents and curvature
• Soon: CSG, line-dash computation, adaptive discretization, simplication, tesselation, inversion

Dot

Math, with a focus on linear algebra | phetsims.github.io/dot

• Core types support mutable and immutable operations
• Matrices use typed arrays, 3x3 optimized for Scenery's usage
• Transform3 lazily evaluates inverses and transposes for calculation of normal/inverse transformation of points, vectors, normals, bounds, shapes and rays

var v = dot( 10, -5 ) // shorthand for Vector2, used for points and vectors
                      // v.x and v.y are the components

v.normalized().times( 5 ).plus( dot( 1, 2 ) ).x  // immutable, matches formulas
v.normalize().multiply( 5 ).add( dot( 1, 2 ) ).x // mutable, minimizes GCs

dot.Matrix3.rotation2( Math.PI / 6 ).timesVector2( v )    // immutable
dot.Matrix3.rotation2( Math.PI / 6 ).multiplyVector2( v ) // mutable

Text Bounds

• Canvas/DOM/SVG methods are all inaccurate
• This prevents us from only repainting dirty Canvas regions
• We also have an accurate Canvas pixel-inspection method

Browser Differences

• Many Canvas inconsistencies exist
• SVG is more consistent
• Scenery works around these, or constrains parameters

Memory

• Currently each canvas layer takes up the entire width and height of the scene (a lot of memory).
• Mobile devices are very memory-constrained, too many layers can cause crashes.
• Layers will by default be more "fitted" soon, but we don't want to be constantly resizing canvases!

Canvases and Events

• No good way to allow DOM events to pass through a Canvas and fire on a DOM element
• We use SVG layers with 'pointer-events: none' (works on IE) above DOM layers

Antialiasing

• Antialiasing differences between Canvas/SVG/DOM will cause a slight change when switching the renderer on a node

Opacity Across Layers

If opacity is applied to a node whose children are across different layers, they will need to be handled differently:

<!-- layers before -->
<div style="opacity: 0.6;">
<canvas><!-- red circle layer --></canvas>
<svg><!-- blue circle layer --></svg>
</div>
<!-- layers after -->

Accessibility

Mixed Canvas/SVG/DOM not accessible by default

Soon: Creating DOM peers for each interactive node, with subtree control over the tab order

Questions or Comments?

(or catch me after)

• Scenery: phetsims.github.io/scenery
• These slides: jonathan-olson.com/slides-scenery

Jonathan Olson

github.com/jonathanolson  |  jonathan.olson@colorado.edu

Thanks to the PhET team (phet.colorado.edu), especially Sam Reid, Chris Malley, John Blanco and director Kathy Perkins