The BioDigital Human

Opportunities and Challenges of WebGL

Tarek Sherif

BioDigital

BioDigital's Goals

• Use interactive 3D to make anatomical content engaging
• Move beyond the anatomy atlas
  • Tell stories
  • Bring content to life
  • Map data to and from 3D models
• Make it all widely accessible through the Web

The BioDigital Human

• 3D content library
  • 5000+ anatomy objects
  • 500+ health conditions
• Rendering engine built on SceneJS
• User-created annotations and custom views
• API for embedding Human content into external websites

The BioDigital Human

• Medical device manufacturers
• Pharmaceutical
• Educational
• Medical students

What is WebGL?

• A low-level JavaScript API that provides access to the GPU
• Allows for high-performance, interactive 3D graphics on the Web

Why WebGL?

• It's everywhere!
• Use HTML/CSS/JavaScript for easy UI
• Integrate into client web sites
• Leverage existing web services
• (Relatively) easy distribution
• (Relatively) easy deployment of updates

In a nutshell...

• Leverage the entire infrastructure of the Web as a platform

Why WebGL?

• Web UX expectations
  • It should be fast
  • It should be easy
  • It should just work

Why WebGL?

• JavaScript!!!
  • Memory management is hard
  • Common JS patterns are bad for performance

          
            object.transform({
              rotate: {
                axis: vec1.cross(vec2).normalize(),
                angle: Math.PI / 4
              } 
            })
          
        
      

The BioDigital Human

• Content
• 3D Rendering
• Integration and API

Types of Content

• Static content
• Animated content
• Tours

Content Challenges

• Artists want it beautiful: big textures, detailed geometry
• Engineers want it to not crash on an iPhone
• Tension between anatomical detail and application stability

Rendering

• Navigating anatomical content is difficult
  • Deeply nested
  • Hierarchical
  • Scale (from cells to whole body)

Rendering

• Focus on giving users ways to view, emphasize, interact with information of interest:
  • Highlight object
  • Highlight region (defined by texture)
  • Dissect (remove an object)
  • Isolate (remove all other objects)
  • Transparency
  • Annotations
  • (+ text, audio, video, etc.)

Example

Rendering Challenges

• Rendering performance and memory can be severely limited on mobile devices
• Possible to query some properties:
  • GL variable limits
  • Shader precision
  • Available extensions
• Not others:
  • GPU memory

Rendering Challenges

Human API

• Embed Human content in an external web page through an iFrame
• Basic interactions built-in
  • Mouse movement
  • Dissection, highlight, annotate, etc.
• UI can be customized through URL parameters

Human API

• For more customized interactions, use the JavaScript API
  • Communicates with iFrame using the window messaging API

          
              var human = new HumanAPI.Human("iFrameID"); 

              human.camera.flyTo({
                  eye: { z: -25 },
                  velocity: 20
              });

              human.pick.on("picked", function(e) {
                console.log(e.worldPos);
              });
          
        

Example

• About.com Diabetes Page

API Challenges

• Want the average web developer to understand it all
  • 3D terminology and concepts
  • 3D navigation
  • Anatomical concepts
  • Architecture of the Human

Solution

• Documentation
• Tutorials
• Support

Get involved!

• Try out the API! https://developer.biodigital.com
• BioDigital summer internships!

Thanks!

• tarek@biodigital.com
• @thsherif
• www.biodigital.com