Creating Web Audio – Building useful abstractions over the web audio api – Loading Sounds

View Github Repository Open presentation in a new window

meandavejustice

See all presentation from meandavejustice

Creating Web Audio – Building useful abstractions over the web audio api – Loading Sounds

1 5

creating_web_audio

Slides for a talk I gave at a pdx-node meetup group

On Github meandavejustice / creating_web_audio

Creating Web Audio

Building useful abstractions over the web audio api

meandavejustice.github.io/creating_web_audio

Nodes

  • AnalyserNode
  • AudioBufferSourceNode
  • AudioDestinationNode
  • AudioListener
  • AudioNode
  • BiquadFilterNode
  • ChannelMergerNode
  • ChannelSplitterNode
  • ConvolverNode
  • DelayNode
  • GainNode
  • MediaElementAudioSourceNode
  • MediaStreamAudioDestinationNode
  • MediaStreamAudioSourceNode
  • OscillatorNode
  • PannerNode
  • ScriptProcessorNode
  • WaveShaperNode
  • analyserNode: get back data
  • audioBufferSourceNode: This is actually where the audio lives, this is what we will connect to other node to add effects and create visualizations.
  • AudioDestinationNode: This is what we will connect our source or analyserNode to. This is usually your speakers

Loading Sounds

WebAudio vs audio tag

Audio tag

  • Perfect for simple playback
  • can provide it's own playback (control attr)
  • lacking most of webaudio spec
                <audio src="/path/to/file.ogg" autoplay=""></audio>
                <audio controls="controls">
                  <source src="/path/to/file.mp3">
                </audio>

WebAudio XHR

              var req = new XMLHttpRequest();
              req.open("GET",url,true);
              req.responseType = "arraybuffer"; // <<<<<
              req.onload = function() {
                audioContext.decodeAudioData(req.response, function(buf) {
                  source = audioContext.createBufferSource(); 
                  source.buffer = buf;
                  source.connect(audioContext.destination);
                  source.start(0);                                    
                });
              };
              req.send();

Audio Tags are cool but..

...they're not enough!

We need more

  • visualize audio data
  • effects
  • manipulate, cut and copy
  • portable between devices
  • collaboration

Controlling Playback

            
              source.start(0);
              source.stop();

what about pause/replay?

            
              var trackTime = 0;
              function pause () {
                trackTime = audioContext.currentTime;
                source.stop();
              }
              function play() {
                source.start(audioContext.currentTime + trackTime);
              }

Effects

                    
// lowpass filter
var bufferSize = 4096;
var effect = (function() {
    var lastOut = 0.0;
    var node = audioContext.createScriptProcessor(bufferSize, 1, 1);
    node.onaudioprocess = function(e) {
        var input = e.inputBuffer.getChannelData(0);
        var output = e.outputBuffer.getChannelData(0);
        for (var i = 0; i  < bufferSize; i++) {
            output[i] = (input[i] + lastOut) / 2.0;
            lastOut = output[i];
        }
    }
    return node;
})();
http://noisehack.com/custom-audio-effects-javascript-web-audio-api/

Creating Visualizations

                  
                    var fft = ctx.createAnalyser();
                    fft.fftSize = samples;
                    fft.connect(ctx.destination);
                    // setup canvas and call update()

                    // in load function
                    ctx.decodeAudioData(req.response, function(buf) {
                    var src = ctx.createBufferSource(); 
                    src.buffer = buf;

                    play(src);
                    //connect them up into a chain
                    src.connect(fft);
                    }

                    // inside update function
                    var data = new Uint8Array(samples);
                    fft.getByteFrequencyData(data);
fft visual demo
play stop

Making Music

My Problem

“Solution”

Simple Sampler

Hacking

  • paul's extreme sound stretch
  • baudio/asynth
  • node webaudio
  • clojure
  • 4minutesAnd33Seconds

Web Components

Polymer

sampler-polymer

Components.FM

Web Audio Components

Complimentary Apis

  • getUserMedia (recorder.js)
  • webRTC
    • audio channel-- Real Time Collaboration
    • webRTC data channel-- Real Time Sharing
  • Web Components

DAW

DEMO

Get Involved

Thank You

Twitter: @meandave2020

IRC: JSON_voorhees

GH: https://github.com/meandavejustice