Monday, May 27, 2013

Composing Music with PHP

I'm not an expert on probability theory, artificial intelligence and machine learning, and even my Music 201 class from years ago has been long forgotten. But if you'll indulge me for the next 10 minutes, I think you'll find that even just a little knowledge can yield impressive results if creatively woven together into an application. I'd like to share with you how PHP can be taught to compose music.

Here's an example:

generated melody

You're looking at a melody generated by PHP. Sure it's not the most memorable melody, but it's not unpleasant either. And surprisingly, the code to generate the sequence of notes is relatively brief.

So what's going on? In short, musical data is analyzed by the script to "learn" which intervals make up pleasing melodies, and then the script creates a new composition by selecting pitches based on what it knows. Speaking technically, the script calculates a probability map of melodic intervals and applies a Markov process to generate a new sequence.

Standing on Shoulders

Music composition does not happen in a vacuum. Bach was fond of Buxtehude and Vivaldi; Chopin influenced Lizt and Wagner; Mozart and Hayden instructed Beethoven. Even the same melodic phrase can be found in different pieces of work. Orfeo ed Euridice by Gluck and the hymn tune Non Dignus for example both share a common phrase.

similar melody in Orfeo ed Euridice by Gluck and hymn tune Non Dignus

If you ask PHP to compose blindly, the results aren't pretty. To prove this point, here's a melody generated by mapping random values returned by successive calls to rand() to notes on a staff.

random melody

Unless you're keen on twelve-tone, it's better to draw from earlier compositions for inspiration and understanding.

To do just this, I first transcribed the melody of several pieces of music using Scientific Pitch Notation. I didn't concern myself with note durations, instead focusing on the pitches themselves. A middle C on paper was entered as 'C4' (C is the note name, 4 is the octave), a semitone above that was 'C#4', the next semitone 'D4', and so on, until a melody (the first 8 measures of Tantum Ergo by Bottazzo shown here) was encoded like:

first 8 measures of Tantum Ergo by Bottazzo

A4 C5 G4 A4 G4 A#4 D5 A4 B4 A4 C5 D5 C5 G4 B4 B4 C5

I now had a sequence that was easily parsable and on which I could perform some basic analysis. For example, given an instance of A4, what is the next probable note to follow?

A4 C5 G4 A4 G4 A#4 D5 A4 B4 A4 C5 D5 C5 G4 B4 B4 C5

or:

C5 G4 B4 C5

There is a 50% chance that the next note would be C5, a 25% chance it will be C4, and a 25% chance it will be B4.

This process translated warm flowing music into something the computer, understanding things only within the context of cold, unfeeling mathematics, could comprehend and reason about.

Paging Doctor Markov

We're familiar with deterministic systems – systems where the same input will always generate the same output. Addition is a deterministic function, with inputs 2 and 4 always yielding 6. Stochastic systems on the other hand behave with some level of randomness. Identical inputs can result in wildly different outputs, such as with the function array_rand(). There is an element of randomness in composition, or else all compositions starting on F4 would end up the same, making generations of composers irrelevant and filling the coffers of the RIAA. But the randomness is tempered, even if at a subconscious level, by the composer with the knowledge of what combinations are pleasing.

A prime example of a stochastic system, one which is also relevant to the composition script, is a Markov process (named after the mathematician Andrey Markov who not only studied them but also had an amazing beard). As Nick Didkovsky explains:

Markov analysis looks at a sequence of events, and analyzes the tendency of one event to be followed by another. Using this analysis, you can generate a new sequence of random but related events, which will look similar to the original. A Markov process is useful for analyzing dependent random events - that is, events whose likelihood depends on what happened last.

The traditional example used to illustrate the concept is a weather predicting graph. Suppose the day following a sunny day has a 90% chance of also being sunny, and the one following a rainy day has a 50% chance of being rainy. The graph would look like this:

Markov Sunny/Rainy graph

Depending on the mechanism used to resolve probabilities, walking the graph for 5 iterations we might find ourselves transitioning with Sunny the first day, Rainy the next, Sunny after that, then Sunny, and Sunny, or we might find ourselves transitioning Sunny, Sunny, Sunny, Rainy, Rainy another.

Hopefully it's obvious where I'm going with all of this; it's possible to constrain the random process of "next note selection" using the weighted probabilities learned by analyzing melodies for a better sounding result.

A4 graph

This simple process allows us to generate passable melodies in infinitely less time than it would take for a monkey hitting random keys on an organ to play the complete works of Messiaen.

Robot Composers (the singularity is near)

At this point you hopefully have a cursory understanding of the two key concepts I employed to generate music. Even if you don't, you've at least survived my humor and deserve to be rewarded by seeing some code.

<?php
namespace Zaemis;

class Composer
{
    private $pitchProb;

    public function __construct() {
        $this->pitchProb = [];
    }

    public function train($noteData) {
       $numNotes = count($noteData);
       for ($i = 0; $i < $numNotes - 1; $i++) {
           $current = $noteData[$i];
           $next = $noteData[$i + 1];
           $this->pitchProb[$current][] = $next;
       }
   }

   public function compose($note, $numNotes) {
       $melody = [$note];
       while (--$numNotes) {
           $i = array_rand($this->pitchProb[$note], 1);
           $note = $this->pitchProb[$note][$i];
           $melody[] = $note;
       }
       return $melody;
   }
}

<?php
require_once '../include/Zaemis/Composer.php';
use Zaemis\Composer;

$noteData = trim(file_get_contents('../data.txt'));
$noteData = explode(' ', $noteData);

$c = new Composer();
$c->train($noteData);
$melody = $c->compose($_GET['note'], $_GET['count']);

echo '<img src="img/notes/clef.png" alt="Treble Clef">';
foreach ($melody as $note) {
    echo '<img src="img/notes/' . urlencode($note) . '.png" alt="' .
        $note . '">';
}

The learning process takes place in the train() method which accepts an array of training notes (the encoded melody string split on its spaces). This code is simple, quick, and dirty; the notes are pushed to a 2-dimensional array with their probabilities indirectly implied by the quantity of elements themselves.

When populated, the array would look similar to:

array(9) {
  ["A4"]=> array(13) {
    [0]=> string(2) "C5"
    [1]=> string(2) "G4"
    [2]=> string(3) "A#4"
    [3]=> string(2) "C5"
    [4]=> string(2) "B4"
    [5]=> string(3) "A#4"
    [6]=> string(2) "G4"
    [7]=> string(2) "A4"
    [8]=> string(2) "D5"
    [9]=> string(2) "G4"
    [10]=> string(2) "C5"
    [11]=> string(2) "C5"
    [12]=> string(2) "G4"
  }
  ["C5"]=> array(11) {
...

Looking at the data, a randomly selected note to follow A4 has approximately a 31% chance of being C5 since 4 out of the 13 members of the list hold that value. (Yes, I know maintaining probabilities instead of a memory-exhausting list of all of the notes in a composition is the "right way", but I wanted to keep things simple for illustrative purposes.)

The compose() method encapsulates the logic to generate the melodic sequence. A starting note the desired length is given, and the method randomly selects a value for the following note from the array until the desired number of notes has been retrieved.

Of course we humans would rather see the result notated on a staff as opposed to a list of note values, so I created a set of note images to accompany the script. Each image displays a note on the appropriate position on a staff, and the files are named according to the note name. Looping through the melody to emit some IMG elements was an effective rendering method for my needs.

Harder, Better, Faster, Stronger

It is impressive that such simple concepts can be used to create a script capable of emulating a composer. Of course, there is infinitely more that can be done to build and improve. Consider this your first exploration into musical intelligence. David Cope, who has been exploring computer composition since 1981, has this to say:

Simply breaking a musical work into smaller parts and randomly combining them into new orders almost certainly produces gibberish. Effective recombination requires extensive musical analysis and very careful recombination to be effective at even an elemental level.

Beyond the obvious changes, such as changing the pitch matrix to maintain probabilities, how would you improve things? Maybe replace this naive approach with a completely different mechanism for analyzing music? Parse input from MIDI files? What would be needed to identify harmonies? How about chord progressions? Note durations? Composer "signatures"? Could your script learn from itself by analyzing and feeding pleasing melodies it produced back into its knowledge base? In what ways could you recombine samples to form new works?

I look forward to hearing about your own experiments in AI-driven composition in the comments below.

Update 6/10/13: I've tossed some code up on GitHub if anyone's interested.

5 comments:

  1. Interestingly, I was doing something similar last year, but instead focused on chord progressions. The script outputted working MIDI files (just a piano, but could of course be extended to include bass and drums or so) and generated "music" by matching chords to possible next chords in a sequence - obviously, not good for generating something completely bizarre, but that goes for any algorithm. The resulting snippets were perhaps nothing spectacular, but nearly always sounded passable as a basis for a composition.

    I am SO going to combine what you have here with that as soon as I have time - the note progression generated could also determine the probability of the next chord, which in turn influences the next progression etc. Having it learn from MIDI files seems relatively simple (it's basically the reverse from what I was outputting before, although the MIDI files would presumably need some massaging first, but still), and in that way it could also be self-learning. Awesome post! If anything comes of this I'll be sure to report back.

    ReplyDelete
    Replies
    1. Thanks for sharing, Marijn!

      I've hardcoded a static lookup table to generate harmonic progressions since there's "rules" that are generally followed, and had somewhat decent results with the following for the major tonic:

      $lookup = array('ii', 'IV', 'V', 'V⁷', 'vii°'),
      'ii' => array('I', 'V'),
      //'iii' => array('vi'),
      'IV' => array('I', 'V'),
      'V' => array('I', 'V⁷', 'vi'),
      'V⁷' => array('I', 'vi'),
      //'vi' => array('ii', 'iii'),
      'vi' => array('ii'),
      'vii°' => array('I')
      );

      $chord = 'I';
      $num = 10;

      $harmony = array($chord);
      while ($num--) {
      $chord = $lookup[$chord][array_rand($lookup[$chord])];
      $harmony[] = $chord;
      }

      print_r($harmony);

      Example:

      Array (
      [0] => I
      [1] => IV
      [2] => V
      [3] => vi
      [4] => ii
      [5] => I
      [6] => V
      [7] => I
      [8] => V⁷
      [9] => I
      [10] => IV
      )

      I've left iii out of the lookup because it sticks out like a sore thumb. vii° seems to be hit or miss, but I've kept it in as a substitute for V to mirror ii as IV. I suspect a bit of analysis could be applied to massage the results, or perhaps even jump ahead in the sequence to determine a chord and then fall back to insert it's secondary dominant to convey a stronger sense of purpose.

      Generating a harmonic progression, and then influencing the melody generation using the chord spellings (targeting smallest/reasonable interval jumps) might be an interesting exercise.

      All my training data was hand-coded, and so my initial input to train with is small. Reading in MIDI files would definitely help that situation. I haven't used it, but PHP MIDI Parser (http://phpmidiparser.com) looks interesting. It might also be an interesting exercise to attempt a harmonic analysis across tracks to weed out leading tones, passing tones, appoggiatura, etc. of the melody (something I didn't do in my input).

      Delete
    2. Cool - that looks not dissimilar to my approach. The real challenge will be to generate "non-obvious" progressions, since they are typically what make or break a tune. I can imagine - if the sample set becomes high enough - you could also optionally specify a "style" to generate in. E.g., selecting "Nirvana" would have it tend more towards what I (non-classically trained guitar player) call "angular" progressions, e.g. from A to C to E to G. Selecting "McCartney" would tend more to C Am F G style stuff (that's what my original POC did), and if you go for "Brian Wilson" (assuming Pet Sounds era :)) it would try to throw in less common jazz chords (diminished and stuff) and generally be more accepting of note sequences in the harmony (ever look at the bars for God Only Knows?).

      There are also a few patterns you can notice, based on composers' styles. The "angular" approach typically modifies 2 or all 3 notes in a chord when progressing (Nirvana, but also Lennon had a penchant for that, and lots of punk or other guitar-based music), McCartney goes for 1 or 2 at most (typically focusing on the melody itself, really the basis of most modern pop music) whereas the Pet Sounds-style does 1, 2 or 3 (though often in 4 or 5 note chords :), making it sound more jazzy - something done also by people like Gershwin and more recently Joe Jackson), but is much more likely to progress to notes and chords in a slightly different key mid-melody.

      I'm sitting with my business partner tomorrow, she's also totally obsessed with this kind of thing. This one's definitely on our "to discuss" list!

      Delete
  2. Combining music and computers, loved it.

    ReplyDelete
  3. Markov Analysis. Haven't seen that in an article since college. I hope the nightmares don't start again. Very impressive. But it is entirely possible that you know too much about too many things to be allowed to roam free. The Committee will have to consider your case carefully.

    ReplyDelete