Friday 02/05/2016 by SimpleMike

PHISHING FOR SETLIST PATTERNS

This guest post is written by Michael Mastroianni, user SimpleMike.

We’re into 2016, between tours and with plenty of time to reflect on the music we’ve enjoyed for the past 7 years. Instead of focusing on the great jams of the “3.0 era,” we turn our attention to the data in hopes of revealing what has become the band’s unspoken preferences of what to play and when.

The following analysis is restricted to the 285 shows between 2009 and 2015, since I think these years essentially comprise modern Phish and, in all probability, patterns prior to 2009 would be different and less relevant to make current predictions than those in the present, so-called “3.0 era.”

Set Bias

Are there certain songs Phish prefers to play during set 1? Similarly, are there certain songs Phish usually reserves for set 2? The graphs below show the top 20 songs by number of plays where the dominating set represents at least 80% of the shows in which the song was played. In other words, the graph for set 2 shows the top 20 songs with a set 2 bias, i.e., songs that were played in set 2 in at least 80% of the shows they were played in. "Tweezer," for example, was played in 80 of the 285 shows (28% of the shows). Only 10 of these were played during set 1.

Top 20 Songs...
Top 20 Songs...
Top 20 Songs...
Top 20 Songs...

It is interesting that “Divided Sky,” “Funky Bitch,” “Lawn Boy,” “Gumbo,” “Poor Heart,” “Ya Mar,” “Foam,” “Train Song,” “Alumni Blues,” “My Sweet One,” “Brian and Robert,” “Beauty of a Broken Heart,” “Vultures” and “Crowd Control” have only been played in Set 1 since 2009. Similarly, “Piper,” “Golden Age,” “Crosseyed and Painless,” “Waves” and “Drowned” have only been played in Set 2 since 2009. (Note, however, that I have excluded encores and any third sets.)

Transitions

What are the most common song transitions? I restricted my analysis to only include sets 1 and 2, and to consider one song following another within the same set as a “transition.” I also only examined transitions that occurred in at least 6 shows. The graph below shows these top transitions in descending order:

Some transitions are no surprise to see, of course, but it did surprise me that “Harry Hood” has led into “Cavern” so frequently. Similarly, “Down with Disease” into (or followed by) “Free” isn’t something I would have anticipated to have occurred in so many shows.

I visually examined a heat map of the “transitions” and their frequencies and there were virtually no patterns to be found. With the exception of the “transitions” shown in the graph above, there were mostly just a scattering of equally frequent events. This shows that Phish truly finds a way to make these happen on the fly. They do not pick certain songs that are easier to transition between seamlessly.

Predicting Songs During the Show

You have almost certainly been at a show and wondered, if they play song X, then what song (Y) will I most likely hear later on in the show? If I hear “grind, grind, grind, grind, grind, grind, grind, bum bum bum …” then will I hear my favorite song “You Enjoy Myself” in the same show? It turns out that the answer is “most of the time.” I calculated every probability for every pair of songs played from 2009 to 2015 and collected the most relevant for you.

In the table below, “Likelihood” is the chance that the second song listed in each row in the “Implication” column will be played during the show at some point before or after the performance of the first song listed in the “Implication” column (e.g., the actual timing of the song’s appearance in the show is not considered, which is why you’ll see that “Tweezer Reprise” implies that “Tweezer” will be played, even though “Tweezer Reprise” would never precede “Tweezer” in the show). “% of Shows With Antecedent” is the portion of shows during the 2009-2015 period in which the first song was played (ie. “Weekapaug Groove” was played in 27% of the shows and “Mike’s Song” was played in every one of these shows). The higher this percentage, the more likely the information in this table will be relevant to you at a future show, so keep it handy. The lower this percentage, the more likely that the information is coincidental (However, note that 4% here is at least 10 shows):

Other Useful Observations

The following also applies to shows between 2009 and 2015:

Most frequent set 1 openers: “AC/DC Bag,” “Kill Devil Falls,” “Chalk Dust Torture,” “Sample in a Jar,” “Runaway Jim,” “Crowd Control”

Most frequent set 1 closers: “Run Like an Antelope,” “David Bowie,” “Character Zero,” “Walls of the Cave,” “Suzy Greenberg”

Most frequent set 2 openers: “Down with Disease,” “Rock and Roll,” “Chalk Dust Torture,” “Carini,” “Crosseyed and Painless,” “Golden Age,” “Mike’s Song,” “Drowned,” “Backwards Down the Number Line,” “Tweezer.”

Most frequent set 2 closers: “You Enjoy Myself,” “Run Like an Antelope,” “Slave to the Traffic Light,” “Character Zero,” “Weekapaug Groove,” “Harry Hood”

Most frequent encore songs: “Tweezer Reprise,” “Loving Cup,” “Character Zero,” “Good Times Bad Times,” “First Tube,” “Sleeping Monkey,” “Suzy Greenberg,” “Show of Life,” “Julius,” “A Day in the Life”

Closing

It would appear, from all of the juicy results above, that we can make some solid predictions at our next show. It is true. I calculated the results in the Predicting Songs During the Show section above halfway through Saturday’s (1/16/16) show and correctly predicted not only that “Harry Hood” would close the second set, but that “Character Zero” would be the encore.

Despite some successful predictions, the most glaring property of the data was the chaos within it. This study looked at 285 shows. Among these 285 shows there were 99 unique set 1 openers, 60 unique set 1 closers, 68 unique set 2 openers, 52 unique set 2 closers and 102 unique songs played in encores. Phish evidently makes an effort to surprise us with the opening song of each show. Furthermore, 57% of the shows contained a song that was only played in 1 or 2 shows throughout all 7 years. Hats off to our favorite band for making every show special!

Good luck calling songs at your next show, but don’t blame me when they open with “Mexican Cousin.”

Happy touring,
Michael Mastroianni

If you liked this blog post, one way you could "like" it is to make a donation to The Mockingbird Foundation, the sponsor of Phish.net. Support music education for children, and you just might change the world.


Comments

, comment by Jay1318
Jay1318 This article is 100% awesome! We all make out predictions and have out thoughts and what will be played and when and having some try Phish sabremetrics is very cool. Props for researching this.
, comment by paulj
paulj Solid netting, and I've often wondered about the same thing. Like my cellphone, though, I'm still gonna leave these lists at home and not worry about what comes next!
, comment by BigPimpinNYC
BigPimpinNYC I wonder how high sleeping monkey -> tweezer reprise would have ranked if we included encores
, comment by pikepredator
pikepredator This is awesome!! I love it. I have one question regarding the data in the Implication table (FWIW I don't fully understand the Implication column, 73% for BOABH and Zero?) Did you control for the frequency of the songs? For example, it's more likely that commonly played songs will appear in the same show. If Gin and Hood are both played in 50% of shows, we'd expect to see them together in 25% of shows (I think). If they appeared together in 10% or 40%, that would be . . . interesting to nerdy people such as myself.

57% of the shows contained a song that was only played in 1 or 2 shows throughout all 7 years.

That makes my head spin just thinking about it. Every other show has a unique rarity/bust-out.
, comment by kernalforbin1983
kernalforbin1983 I'd be interested to see, a tease variant. I.e. if they tease a song does it get played the next night? or i.e what song has the most likelyhood for a manteca tease.
, comment by CadillacRainb0ws
CadillacRainb0ws @SimpleMike, what BA software did you use for this?
, comment by uctweezer
uctweezer Good stuff @SimpleMike. I'd be interested in seeing the covariance data for 1.0 -- one song pairing (besides the obvious ones like Mike's Groove, Horse > Silent, etc.) that I saw popping up in the same shows repeatedly throughout my year-by-year listening exercise from '97 - '00 was Taste and Theme from the Bottom. Especially in '97, those two had an insanely high coincidence in the same show.

One nit to pick:

They do not pick certain songs that are easier to transition between seamlessly.

You'd be surprised (and I'd be interested in an analysis of this as well) -- Piper and Twist are both in G, so they flow together nicely. DWD and Free? They're both in D. While other pairings may not be in the same key throughout the whole song -- and many of Phish's songs have multiple sections in multiple keys and modes, and that's not even including Type II jams that may modulate entirely to another key -- there are other pairings that lead into one another via some other musical pattern, whether a walk-up or walk-down (like the end of AC/DC Bag, for example, whose walk-up may be extended chromatically for an arbitrary amount of time until they reach the desired key for the next song), or dropping from the I at the end of the first song to the minor sixth at the start of the next, or a whole-step up or down to give a different effect, or my favorite, a jam that modulates effortlessly at some point and suddenly finds itself in the key of another tune (think Tweezer (in Am) -> Heavy Things (in C) from 10/31/14). When these decisions by the band (read: Trey) are made wisely and executed precisely, we as fans say things like "that set had good flow". So I think it's important to realize that, while predicting the next song might be near impossible at times, it's not completely chaotic -- there is a method to the madness. It certainly helps that their repertoire is vast -- so while they may like to go from songs in G to songs in Em, they've got loads of options for each and any patterns would be invisible to us when just looking at the songs.

I do understand your point though, but I might rephrase it a bit: one of the best things about Phish is their ability to take sets of songs seemingly out of thin air and -- when they're at their best -- effortlessly weave them together as if they were meant to be one longer song. Think Magna Tweezer > Caspian (-> Tweezer Jam in G!) or the Blossom "Cheezer", or that entire second set from 8/12/15.

Thanks again for the nice write-up!
, comment by E_Wolfe
E_Wolfe @kernalforbin1983 said:
I'd be interested to see, a tease variant. I.e. if they tease a song does it get played the next night? or i.e what song has the most likelyhood for a manteca tease.

THAT one's easy; in 3.0 anyway, Light = Manteca-Tease 😏👍
, comment by The_Ghost
The_Ghost This is absolute treasure. One of the best things I've read in a while. A post about songs with no opinions. Just the facts. Great work!
, comment by philanthropist
philanthropist Great work! Very appreciated! Further affirmation of why I love this sight! Your data confirms some of my humble suspicions. While there are the 'likely' openers and set-closing songs, there is a concerted effort to work new song placements, and unique or rare songs. It's nice to see this data!
I wonder if anyone has done a similar analysis of 1.0? Maybe it would be more accurate break it up into two sections, maybe pre-90 and post-90. Anyhow..huge props!
Love this community!
, comment by SimpleMike
SimpleMike First, thank you for all of the love everybody. It was a treat for me to write this article and I'm glad others are finding it as interesting as I did.

@BigPimpinNYC: If we included set 3s and encores, Sleeping Monkey -> Tweezer Reprise ranks #5 with an astounding 13 shows (just under Mike's Song -> Simple and the same 4 seen at the top of the graph in the article). Nice observation!

@pikepredator: "(FWIW I don't fully understand the Implication column, 73% for BOABH and Zero?) Did you control for the frequency of the songs?" BOABH was played in 4% of shows (about 11 shows probably) and 73% of those shows (8 shows?) contained "Character Zero". This list was generated by picking the implications with the best % in the second column with at least 4% in the third column. You rarely see high second column percentages if the third column is high (ie. the song implying the other is frequently played). I didn't think it was necessary to control for anything. Likely the most prediction power will be when you hear a fairly rare (but not so rare that it is barely played) song which almost always comes with another more common song. The more common songs tended to have several second column percentage implications in the 30% range but, like you said, this is expected and maybe not so interesting.

@ucpete: Thanks for providing a musician's prospective. I don't have this knowledge so these ideas would not have occurred to me.

Another interesting little fact to leave you with: Tweezer always has a Tweezer Reprise. 92% of the time they are done in the same show and 8% of the time Tweezer is played in one show and Tweezer Reprise is saved for a future show which doesn't have a Tweezer.
, comment by FreeRoses
FreeRoses You just made my Phish fantasy team so much stronger! :) Thank you for this killer analysis.
, comment by MagnAnt
MagnAnt I would like to propose 2 things for 2016 season:
1) Couch Tour season tickets

2) Phantasy Leagues -
similar to fantasy baseball or football leagues
Basic structure something like this:
1 pt for picking any song played that night (choose 10 songs)
3 pts for picking a set opener (pick one song, good for either set)
3 pts for a set closer (pick one song, good for either set)
5 pts for an encore (pick one song)
7 pts for show opener (pick one song)
20 pts for calling Harpua, but -10 pts if it's not played!

"Draft" songs starting at 11am the morning of the show.

Advanced leagues could add in points for picking songs with increased song gaps, jam modulations, other things over my head. Maybe over/unders on song lengths (pick 15 min tube and the play 6 min tube, 0 points awarded), etc. the possibilities are endless!

Now I just hope some one with the capabilities and ambition to set something like this up reads this and agrees!
, comment by johnnyd
johnnyd So, like a Phantasy draft maybe for a whole Tour's worth of shows.

hmmm.
very interesting.
Very interesting indeed.
, comment by MagnAnt
MagnAnt I was thinking of drafting different songs each show, but that actually sounds more interesting. You draft YEM to bat clean up for the year and your gonna have a good season...
, comment by pikepredator
pikepredator @simpleMike thanks for the explanation, I get what you are doing now with the Implication.
, comment by Dressed_In_Gray
Dressed_In_Gray DWD-> Free was all the rage in 1996.

Kinda like Runaway Foam in 1992
You must be logged in to post a comment.
Support Phish.net & Mbird
Fun with Setlists
Check our Phish setlists and sideshow setlists!
Phish News
Subscribe to Phish-News for exclusive info while on tour!


Phish.net

Phish.net is a non-commercial project run by Phish fans and for Phish fans under the auspices of the all-volunteer, non-profit Mockingbird Foundation.

This project serves to compile, preserve, and protect encyclopedic information about Phish and their music.

Credits | Terms Of Use | Legal | DMCA

© 1990-2024  The Mockingbird Foundation, Inc. | Hosted by Linode