Monday, May 27, 2013

Composing Music with PHP

I'm not an expert on probability theory, artificial intelligence and machine learning, and even my Music 201 class from years ago has been long forgotten. But if you'll indulge me for the next 10 minutes, I think you'll find that even just a little knowledge can yield impressive results if creatively woven together into an application. I'd like to share with you how PHP can be taught to compose music.

Here's an example:

generated melody

You're looking at a melody generated by PHP. It's not the most memorable, but it's not entirely unpleasant either. And surprisingly, code to generate such sequences is rather brief.

So what's going on? In short, musical data is analyzed by a script to "learn" which intervals make up pleasing melodies and then creates a new composition by selecting pitches based on the possibilities it's observed. Technically speaking, the script calculates a probability map of melodic intervals and applies a Markov process to generate a new sequence.

Standing on Shoulders

Music composition doesn't happen in a vacuum. Bach was fond of Buxtehude and Vivaldi; Chopin influenced Lizt and Wagner; Mozart and Hayden instructed Beethoven. Even the same melodic phrase can be found in different pieces of work. Orfeo ed Euridice by Gluck and the hymn tune Non Dignus for example both share a common phrase.

similar melody in Orfeo ed Euridice by Gluck and hymn tune Non Dignus

But if you ask PHP to compose blindly, the results aren't pretty. To prove this, here's a melody generated by mapping random values returned by successive calls to rand() to notes on a staff.

$notes = ['C4','D4','E4','F4','G4','A4','B4','C5','D5','E5','F5','G5'];
$melody = array_rand($notes, 12);
foreach ($melody as $note) {
    draw($notes[$note]);
}

random melody

Unless you're keen on twelve-tone, it's better to draw from earlier compositions for inspiration and understanding.

To do just this, I transcribed the melody of several pieces of music using Scientific Pitch Notation. I didn't concern myself with note duration, rather I focused on the notes themselves. A middle C on paper was entered as 'C4' (C is the note name and 4 is its octave), a semitone above that was 'C#4', the next semitone 'D4', and so on until a melody (the first 8 measures of Tantum Ergo by Bottazzo shown here) was encoded:

first 8 measures of Tantum Ergo by L. Bottazzo

A4 C5 G4 A4 G4 A#4 D5 A4 A#4 A4 C5 D5 C5 A4 B4 B4 C5

With a sequence that's easily parsable we can perform some basic analysis. For example, given any instance of A4, what is the next probable note to follow?

A4 C5 G4 A4 G4 A#4 D5 A4 A#4 A4 C5 D5 C5 A4 B4 B4 C5

or:

C5 G4 A#4 C5 B4

There is a 40% chance that the next note would be C5, a 20% chance it will be G4, a 20% chance it will be A#4, and a 20% chance it will be B4.

This process translates warm flowing music into something the computer, understanding things only within the context of cold, unfeeling mathematics, could comprehend and reason about.

Paging Doctor Markov

You're probably familiar with deterministic systems—systems where the same input will always generate the same output. Addition is a deterministic function, with inputs 2 and 4 always yielding 6. Stochastic systems on the other hand behave with some level of randomness. Identical inputs can result in wildly different outputs, such as the function array_rand(). There is an element of randomness in composition, or else all compositions starting on F4 would end up the same, making generations of composers irrelevant and filling the coffers of the RIAA. But the randomness is tempered, even if at a subconscious level, by the composer with the knowledge of what combinations are pleasing.

A prime example of a stochastic system, one which is also relevant to the composition script, is a Markov process (named after the mathematician Andrey Markov who not only studied them but also had an amazing beard). As Nick Didkovsky explains:

Markov analysis looks at a sequence of events, and analyzes the tendency of one event to be followed by another. Using this analysis, you can generate a new sequence of random but related events, which will look similar to the original. A Markov process is useful for analyzing dependent random events - that is, events whose likelihood depends on what happened last.

The traditional example used to illustrate the concept is a weather predicting graph. Suppose the day following a sunny day has a 90% chance of also being sunny and the one following a rainy day has a 50% chance of being rainy. The graph looks like this:

Markov Sunny/Rainy graph

Walking the graph for 5 iterations we might find ourselves transitioning with Sunny the first day, Rainy the next, Sunny after that, then Sunny, and Sunny, or we might find ourselves transitioning Sunny, Sunny, Sunny, Rainy, Rainy another.

Hopefully it's obvious where I'm going with all of this; it's possible to constrain the random process of "next note selection" using the weighted probabilities learned by analyzing melodies for a better sounding result. This process allows us to generate passable melodies in infinitely less time than it would take for a monkey hitting random keys on an organ to play the complete works of Messiaen.

Robot Composers (the singularity is near)

At this point you hopefully have a cursory understanding of the key concepts. Even if you don't, you've survived long enough and will now be rewarded with some code.

<?php
namespace Zaemis;

class Composer
{
    private $pitchProb;

    public function __construct() {
        $this->pitchProb = [];
    }

    public function train($noteData) {
       $numNotes = count($noteData);
       for ($i = 0; $i < $numNotes - 1; $i++) {
           $current = $noteData[$i];
           $next = $noteData[$i + 1];
           $this->pitchProb[$current][] = $next;
       }
   }

   public function compose($note, $numNotes) {
       $melody = [$note];
       while (--$numNotes) {
           $i = array_rand($this->pitchProb[$note], 1);
           $note = $this->pitchProb[$note][$i];
           $melody[] = $note;
       }
       return $melody;
   }
}

<?php
require_once '../include/Zaemis/Composer.php';
use Zaemis\Composer;

$noteData = trim(file_get_contents('../data.txt'));
$noteData = explode(' ', $noteData);

$c = new Composer();
$c->train($noteData);
$melody = $c->compose($_GET['note'], $_GET['count']);

echo '<img src="img/notes/clef.png" alt="Treble Clef">';
foreach ($melody as $note) {
    echo '<img src="img/notes/' . urlencode($note) . '.png" alt="' .
        $note . '">';
}

The learning process takes place in the train() method which accepts an array of training notes (the encoded melody string split on spaces). The code is simple, quick, and dirty; the notes are pushed to a 2-dimensional array with their probabilities indirectly implied by the quantity of elements themselves. When populated, the array looks similar to:

array(9) {
  ["A4"]=> array(13) {
    [0]=> string(2) "C5"
    [1]=> string(2) "G4"
    [2]=> string(3) "A#4"
    [3]=> string(2) "C5"
    [4]=> string(2) "B4"
    [5]=> string(3) "A#4"
    [6]=> string(2) "G4"
    [7]=> string(2) "A4"
    [8]=> string(2) "D5"
    [9]=> string(2) "G4"
    [10]=> string(2) "C5"
    [11]=> string(2) "C5"
    [12]=> string(2) "G4"
  }
  ["C5"]=> array(11) {
...

Looking at the data, a randomly selected note to follow A4 has approximately a 31% chance of being C5 since 4 out of the 13 members of the list hold that value. Maintaining a list like this can be memory-exhausting for large sets, and there are better ways to perform weighted selection. You can find an excellent write up (using Python) at electricmonk.nl.

The compose() method encapsulates the logic to generate the melodic sequence. A starting note the desired length is given, and the method randomly selects a value for the following note from the array until the desired number of notes has been retrieved.

Of course we humans would rather see the result notated on a staff as opposed to a list of note values, so I created a set of note images to accompany the script. Each image displays a note on the appropriate position on a staff, and the files are named according to the note name. Looping through the melody to emit some IMG elements was an effective rendering method for my needs.

Harder, Better, Faster, Stronger

It is impressive that such simple concepts can be used to create a script capable of emulating a composer. Of course, there is infinitely more that can be done to build and improve. Consider this your first exploration into musical intelligence. David Cope, who has been exploring computer composition since 1981, has this to say:

Simply breaking a musical work into smaller parts and randomly combining them into new orders almost certainly produces gibberish. Effective recombination requires extensive musical analysis and very careful recombination to be effective at even an elemental level.

Beyond the obvious changes, such as changing the pitch matrix to maintain probabilities, how would you improve things? Maybe replace this naive approach with a completely different mechanism for analyzing music? Parse input from MIDI files? What would be needed to identify harmonies? How about chord progressions? Note durations? Composer "signatures"? Could your script learn from itself by analyzing and feeding pleasing melodies it produced back into its knowledge base? In what ways could you recombine samples to form new works?

I look forward to hearing about your own experiments in AI-driven composition in the comments below.

Update 6/10/13: I've tossed some code up on GitHub if anyone's interested.