Web Audio

• Multimedia has always been important on the web.
• Audio is very popular on the web.
• Sites like soundcloud.com and archive.org provide web audio
• Web audio is often streamed like "radio stations".

Web Audio

• Currently lots of web audio is still served by plugins such as:
  • flash
• These plugins are typically missing on mobile platforms
• Flash is being poorly maintained and flash on Linux
• Flash has been a security issue for browsers for some time:
  • http://helpx.adobe.com/security/products/flash-player.html
  • http://helpx.adobe.com/security.html#shockwave
• Is flash dead? https://www.google.ca/#q=is+flash+dead+2014

Web Audio

• So I won't be covering flash or any plugins.
• HTML5 ushered in the <audio> tag.
  • We're going to use that.

HTML5 Web Audio

• <audio> tag was welcomed by many
• allow any device to play videos if they can support it
• No proprietary nonsense*
• http://caniuse.com/audio
• http://www.jplayer.org/latest/demos/ (nice looking player for the web)

<audio>

• The audio tag can play encoded audio files streamed over HTTP.
• If the container allows streaming it will be streamed in the browser.
• Some browsers support RTP and RTSP -- it doesn't work for most.
  • Besides HTTP is guaranteed to work!
  • RTP is often over UDP -- not guaranteed!

<audio>

• The audio tag can play encoded audio files streamed over HTTP.
  • The audio tag allows multiple sources
  • The audio tag allows multiple formats
  • In fact there's not 1 format that will play on all browsers

<audio>

Use the <audio> tag, and specify 2 formats one could use and ask the audio tag for controls.

• autoplay - automatically play the audio (annoying)
• controls - show UI controls for the audio
• loop - loop it
• muted - mute it by default
• preload - should the audio be preloaded (ready for play)
• src - url (use source tags instead!)

    <audio id="basicaudio" controls>
      <source src="music/KIMIKO_ISHIZAKA_-_Goldberg_Variations_BWV_988_-_05_-_Variatio_4_a_1_Clav.__44k-24b.mp3"
      type="audio/mpeg" />
      <source src="music/KIMIKO_ISHIZAKA_-_Goldberg_Variations_BWV_988_-_05_-_Variatio_4_a_1_Clav.__44k-24b.ogg" type="audio/ogg" />
    </audio>

<audio>

• Notice the source tags within the video tag?
  • 3 file types
    • Ogg Vorbis (Theora)
    • Mpeg1 Layer 3
    • Wav
• We need to provide different formats for different devices and different license holders

<audio>

• Different Codecs
  • Wav
    • Runs in Chrome
    • Runs in Firefox
  • OggVorbis - royalty/patent free. Ogg is behind it.
    • Runs in Chrome
    • Runs in Firefox
  • MP3 - Patent encumbered in USA and somewhat in Canada
    • Runs in Chrome
    • Runs in Safari
    • Runs in IE
    • Runs on Android
    • Runs on iOS
    • Decoded and Encoded often in hardware

<audio>

• What do Different Codecs mean?

  • You have to encode each audio file at least twice:

• Theora isn't that popular.

• Best quality regardless of size: Wav or FLAC
• Best quality with low size? Ogg
• Best quality for medium size? Ogg
• Most compatible? mp3

<video> files

• Recommendations:
  • .ogg -- Ogg container with Ogg Vorbis audio
  • .mp3 -- MPEG1 layer 3 encoded audio (heavily patented)

How do we make ogg?

• oggenc

   oggenc soundfile.wav

How do we make mp3?

• lame

   lame soundfile.wav

WATCH OUT!

• Be sure to return proper mime-types for each video type in the HTTP headers!

• Also ensure that you use the type attributes for source videos so that browsers can choose the appropriate one!

<audio> tag and Javascript

There are many relevant events to fill in onpause onplay onerror onended oncanplay onseeking

<video> tag and Javascript

Important methods and properties video.play() - play video.pause() - pause video.volume -- volume 0.0 to 1.0 video.currentTime - current playback time (or sets it!)

Resources

• Web Audio spec https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html and http://www.w3.org/TR/webaudio/
• HTML5 Video from HTML5Rocks http://www.html5rocks.com/en/tutorials/video/basics/
• Chrome audio examples https://chromium.googlecode.com/svn/trunk/samples/audio/index.html
• Video tag events http://www.w3schools.com/tags/ref_eventattributes.asp
• MDN: Using HTML5 Audio and Video https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Using_HTML5_audio_and_video

LICENSE

Copyright 2014 (C) Abram Hindle

The textual components of this slide deck are placed under the Creative Commons Attribute-ShareAlike 4.0 International License (CC BY-SA 4.0)

LICENSE

The source code to this slide deck is:

Copyright (C) 2013 Hakim El Hattab, http://hakim.se

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN