The Lieder Project – Exploring Poetry and MusicThrough Computer-Assisted Statistical Modeling – Kris Shaffer, Ph.D. & Jordan PyleCU–Boulder

View Github Repository Open presentation in a new window

kshaffer

See all presentation from kshaffer

The Lieder Project – Exploring Poetry and MusicThrough Computer-Assisted Statistical Modeling – Kris Shaffer, Ph.D. & Jordan PyleCU–Boulder

0 0

liederprojectpresentation

On Github kshaffer / liederprojectpresentation

The Lieder Project

Exploring Poetry and MusicThrough Computer-Assisted Statistical Modeling

Kris Shaffer, Ph.D. & Jordan PyleCU–Boulder

kris.shaffermusic.com / @krisshafferliederproject.shaffermusic.comview presentation at kris.shaffermusic.com/liederprojectpresentationfork presentation at github.com/kshaffer/liederprojectpresentation

kris.shaffermusic.com/liederprojectpresentation

I want to suggest that talk of fricatives and plosives and open vowels can be extended beyond the private voice studio and mingled productively with discussion of modulatory schemes and metric dissonances and structural melodies. Indeed, I would argue that the expressive power of musicalized phonemes is as responsible for our emotional response to song as the semantic content of the words and the musical expression of that content. For listeners and composers alike, how words sound matters no less than what they mean. Stephen Rodgers, "The Fourth Dimension of a Song," Music Theory Spectrum 37/1

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille Brust. Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! Holy night, you sink down; Dreams too float down, Like your moonlight through space, Through the silent hearts of men. They listen with delight; Cry out when day awakes: Come back, fair night! Fair dreams, come back! "Nacht und Träume," by Matthäus von Collin

Schubert & Collin, "Nacht und Träume," mm. 4–9

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille Brust. Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! Holy night, you sink down; Dreams too float down, Like your moonlight through space, Through the silent hearts of men. They listen with delight; Cry out when day awakes: Come back, fair night! Fair dreams, come back! "Nacht und Träume," by Matthäus von Collin

Schubert & Collin, "Nacht und Träume," mm. 10–15

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille Brust. Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! Holy night, you sink down; Dreams too float down, Like your moonlight through space, Through the silent hearts of men. They listen with delight; Cry out when day awakes: Come back, fair night! Fair dreams, come back! "Nacht und Träume," by Matthäus von Collin

Schubert & Collin, "Nacht und Träume," mm. 16–23

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille Brust. Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! Holy night, you sink down; Dreams too float down, Like your moonlight through space, Through the silent hearts of men. They listen with delight; Cry out when day awakes: Come back, fair night! Fair dreams, come back! "Nacht und Träume," by Matthäus von Collin

Can we use software to identify songs worthy of a closer look? Can we use software to learn about general trends in a large collection of songs?

  • Create a corpus of phonetically encoded text and music
  • Mathematically model what Stephen has already found
  • Write software to search for those structures in the corpus

Encoding the data

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille Brust. Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! 'ha:Il.gə 'naχt du 'zIŋ.kəst 'ni.dəʁ 'ni.dəʁ 'va.lən 'a:ʊχ di 'trɔ:Y.mə 'vi da:In 'mont.lIçt 'dʊɾç di 'ɾɔ:Y.mə 'dʊɾç deʁ 'mɛn.ʃən 'ʃtI.lə 'bɾʊst 'di bə.'la:ʊ.ʃən 'zi mIt 'lʊst 'ɾu.fən 'vɛn deʁ 'tak ɛɾ.'vaχt 'ke.ɾə 'vi.dəʁ 'ha:I.lgə 'naχt 'hɔl.də 'tɾɔ:Y.mə 'ke.ɾət 'vi.dəʁ "Nacht und Träume," German text and IPA transcription

Schubert & Müller, "Die liebe Farbe," Lines 1–3

The statistical model

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille, stille Brust. Die belauschen sie mit Lust; Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! Holde Träume, kehret wieder! Collin's "Nacht und Träume," as set by Schubert

International Phonetic Alphabet (IPA) vowel chart

Heil'ge Nacht, du sinkest nieder; Nieder wallen auch die Träume Wie dein Licht durch die Räume, Durch der Menschen stille, stille Brust. Die belauschen sie mit Lust; Die belauschen sie mit Lust; Rufen, wenn der Tag erwacht: Kehre wieder, heil'ge Nacht! Holde Träume, kehret wieder! Holde Träume, kehret wieder! Collin's "Nacht und Träume," as set by Schubert

"Nacht und Träume," vowel category probability by line

"Nacht und Träume," vowel probability difference as Euclidean distance

The Lieder Project corpus, vowel probability difference as Euclidean distance (ranked)

Z-normalization

  • What is the standard deviation for line-to-line distance in each individual song? (the standard rate of change)
  • Z-normalized distance = raw distance / standard devation (how big a deal is the change in the context of the song)
  • Z-normalized distance allows us to interpret contextually and compare globally

The Lieder Project corpus, vowel probability difference as Z-normalized distance (ranked)

Den Morgenwinden möcht ich's hauchen ein, Ich möcht es säuseln durch den regen Hain; Oh, leuchtet' es aus jedem Blumenstern! Trüg es der Duft zu ihr von nah und fern! Ihr Wogen, könnt ihr nichts als Räder treiben? Dein ist mein Herz und soll es ewig bleiben. I’d like to breathe it into the morning breezes, I’d like to blow it through the stirring grove; Oh, if it could only glow from every starry blossom! If the scent could carry it to her from near and far! You waves, can you only push wheels? My heart is yours and will ever remain so. "Ungeduld," by Wilhelm Müller Stanza 3

Schubert & Müller, "Ungeduld," Stanza 3 refrain

The Lieder Project corpus, vowel probability difference as Euclidean distance (ranked)

Ach Bächlein, liebes Bächlein, Du meinst es so gut; Ach Bächlein, aber weißt du, Wie Liebe tut? Ach unten, da unten Die kühle Ruh! Ach Bächlein, liebes Bächlein, So singe nur zu. Oh brook, dear brook, You mean so well: Oh brook, but do you know What love does to you? Ah, below, down there, The cool repose! Oh brook, dear brook, Just sing to me. "Der Müller und der Bach," by Wilhelm Müller Stanza 3

Schubert & Müller, "Der Müller und der Bach," Lines 29–32

The Lieder Project corpus, vowel probability difference as Euclidean distance (ranked)

Heran, heran, Was wiegen kann, Woget und wieget den Knaben mir ein! Wenn ein Jagdhorn schallt Aus dem grünen Wald, Will ich sausen und brausen wohl um dich her. Come closer, come here, Whatever can soothe, Lull and rock my boy to sleep. If a hunting horn sounds From the green forest, I’ll rumble and thunder all around you. "Des Baches Wiegenlied," by Wilhelm Müller Stanzas 2–3

Schubert & Müller, "Des Baches Wiegenlied," mm. 1–8

Can we use software to identify songs worthy of a closer look?

Can we use software to learn about general trends in a large collection of songs?

Data to explore

  • words/syllables
  • specific phonemes
  • phoneme categories
  • poetic stress
  • pitch height
  • duration
  • metric placement
  • beat strength

What's next?

The Lieder Project

Exploring Poetry and MusicThrough Computer-Assisted Statistical Modeling

Kris Shaffer, Ph.D. & Jordan PyleCU–Boulder

kris.shaffermusic.com / @krisshafferliederproject.shaffermusic.comview presentation at kris.shaffermusic.com/liederprojectpresentationfork presentation at github.com/kshaffer/liederprojectpresentation

The Lieder Project Exploring Poetry and MusicThrough Computer-Assisted Statistical Modeling Kris Shaffer, Ph.D. & Jordan PyleCU–Boulder kris.shaffermusic.com / @krisshaffer liederproject.shaffermusic.com view presentation at kris.shaffermusic.com/liederprojectpresentationfork presentation at github.com/kshaffer/liederprojectpresentation