Building a collaborative audio editor based on the Web Audio API – Kolloquium – DAW Analyse
Building a collaborative audio editor based on the Web Audio API – Kolloquium – DAW Analyse
0 2
kolloquium
On Github janmonschke / kolloquium
Building a collaborative audio editor based on the Web Audio API
Kolloquium
Jan Monschke
Audio im Web
- bgsound
- object, embed
- < audio />
- Audio Data API
- Web Audio API
first three points mainly for playing and varying in features
bgsound is kind of the blink aka marquee element of the web ;)
Audio Data API gave raw audio access and some ways tom manipulate audio, but API was nut sufficient
Newest addition to the family: web audio API
Web Audio API
- low-level Audio Kontrolle (Buffer)
- Tonerzeugung (Oszillator)
- Tonmanipulation (Filter)
- Präzsises Timing
- Audio-Analyse, räumliche Sounds...
create: literally! create sounds from waves
manipulate: a rich api to create custom filters easily
precision: timing is everything when dealing with more than one sound
Who feels comfortable with his knowledge about the physical basics of audio?
Experimente → Tools
DAW Analyse
- Viele Best Practices
- Interface Patterns
- Minimalumfang
Garageband
Ableton Live
Reason
DAW
- Timeline (links nach rechts)
- Vorschau (waveform, patterns)
- Spuren als Zeilen
- Aufnehmen, Importieren, Drums, Synthesizer
Web Audio Editor
Demo: show track layout, each track has different type, name the three types
Control buttons on the top, timeline, skip things
control elements for each track
canvas previews in the track elements
Aufnahme
- Audiosignal aufnehmen
- Visualisierung des Signals
- Ausschnitte auswählen
- Aufnahmen hochladen
- Dateien importieren
Aufnahme (Screenshot)
Aufnahme (Node Graph)
Web Audio nodes in combination with other new web tech
This is why you can't use it in IE, all modern web APIs in one place ;)
Drums
- Kontrolle über Geschwindigkeit und Beats
- Komplexe Drum Loops
- Verschiedene Töne
- Exaktes Timing!
Drums (Screenshot)
Web Audio Timing
- setTimeout/setInterval
- Eigenes Zeitsystem in der Web Audio API
- Timing in eigenem Thread
var node = context.createBufferSource(); node.start(when, offset, duration);
Synthesizer
consists of oscillators, filters and gain control, all present in web audio api
Minimoog
Oszillator
Oszillator
there are several forms of waves: sine, sawtooth, triangle
Synthesizer (Node Graph)
Envelope / Hüllkurve
Synthesizer Screenshot
Scheduler
- Web Audio Timing ist primitiv
- Spur-Playback mittels eigenem Scheduler
- Spuren und Stücke in Editor Node Graph
Scheduler-Anforderungen
- Effizient (Performance)
- Exakt (keine Teile auslassen)
- Intelligent (Offsets)
- lookahead = 3 * tick (à 100ms)
- lokales Scheduling vs. globales Scheduling
- offsets werden an lokale Scheduler gegeben
Kollaboration
- Fern-Brainstorming
- Melodie-Kollaboration
- Auto-save und Auto-Backup
Locking
- Entitäten bei Bearbeitung sperren
- Hält kreativen Prozess auf
- Frustriert
- Inkonsistenz bei Paketverlust
Operational Transformation
Operational Transformation
- De-facto Standard (Google Drive, SubEthaEdit...)
- Viel Forschung
- Sehr kompliziert, viele Ausnahmefälle
- Anpassung für JSON notwendig
Differential Synchronization
Differential Synchronization
- Ring-Synchronisierung
- Konsistent bei Paketverlust (Versionierung)
- Diff/Patch JSON
- Gut geeignet für AngularJS
Differential Synchronization (Implementierung)
- Eigenes Protokoll auf Websockets
- Unabhängiges Modul im Frontend
- Erste JS Implementierung
- Push > Pull (Erweiterung)