The Music Data Problem

John Loken’s blog post a couple weeks back got me thinking about a post I’ve been meaning to do for months, but have been too swamped to actually spit out. The post, titled “More Metrics Please,” is, as the title suggests, a plea for metrics on par with the TV business.

Some of this data has been collected in the past, but for various reasons (the biggest of which being the market and channel control by the majors) almost all data points beyond Soundscan and Billboard charts seem to have been ignored. Now, however, there is a perfect storm for music data: more artists than ever before, more listeners than ever before with easier access through more channels than ever before, and more technology for data collection than ever before.

Put simply: music data is both more important and more available than ever before.

This shouldn’t come as a surprise, and John is not alone in asking for more data. Companies are springing up left and right to help answer her prayers. Companies like Next Big Sound, BandMetrics, and Music Metric are busy collecting third party data from around the web in single dashboards. Companies like Bandize and Artist Data are helping bands organize their own data. Companies like HypeMachine and Elbo.ws are aggregating marketing/A&R data in the form of blog and Twitter charts. Companies like Songkick and Gigulate help fans track basic concert data and music news, respectively, creating interesting data sets that are sure to surface soon.

The issue isn’t in the data collection or availability, it’s in the connecting, processing, and understanding. Like all data, the points themselves are completely irrelevant without context – a MySpace play doesn’t inherently mean anything, nor does a friend on Facebook. Data can only be understood with relevant ratios.

Connecting the data might be the biggest obstacle at the moment, but I’m hopeful that will change. It’s primarily an issue of a few major players (namely, the retail channel) not having open APIs, or sharing any of the data with the artists – when a fan buys an album on iTunes, that fan data belongs to iTunes and iTunes doesn’t want to part with it.

The other obstacle in connecting the data is the breadth to which music discovery, engagement, and purchasing have spread. You used to get your discovery metrics solely from radio numbers, your engagement metrics from concert tickets, and your purchasing metrics from Soundscan. Now discovery can span blogs, P2P, internet radio, and beyond, engagement can be a play on a blog, a MySpace page, an iPod, or anywhere else, and while record purchases are still largely covered by Soundscan, records are an increasingly small portion of the overall business picture for an artist.

Pulling all that data together is not easy, but I think it can be done as long as those collecting the data are ensuring it’s clean and are willing to make it accessible. Purchases should be easiest, as long as retail channels come around. Last.fm and Twones are handling engagement fairly well (though neither has all the necessary channels covered, by any stretch). Discovery is incredibly tough without having the engagement piece nailed – if you don’t have a view of all a fan’s engagement data, how do you know the first time/place they interacted with an artist?

Progress is coming, but only as a stream of one-dimensional data points. More radical change is needed in order to get to the ratios that are truly relevant and key to making informed business decisions. We need to be able to close the loop on fan data – how fans move down the funnel from discovery to engagement to purchasing. Knowing that you had a spike in MySpace plays around the same time you had a spike in iTunes sales shouldn’t be surprising, and is thus practically worthless (if you didn’t see a spike in sales, it would indicate a strong PR effort but not a high enough quality product for fans to take out their wallets, so the data isn’t totally worthless). Knowing that the majority of the folks who purchased your deluxe edition CD through Amazon first heard your song on Pitchfork and downloaded your EP through P2P is highly valuable.

The technology to make this happen is not expensive, nor particularly difficult to implement, with APIs and OpenID or OAuth. Thanks to the web's ever-expanding processing power, we have far easier access to more data than ever before – it’s no longer reserved for the major corporations who can pay hundreds of thousands of dollars to market research firms to run surveys. We just need to have the right technology implemented properly in order to make the data connection easier.

If we can connect all the data collection points properly, the processing isn’t horribly difficult – it’s merely implementing techniques that economists and social scientists have used for ages to slice and dice large data sets. Everyone should be able to be armchair statisticians (as Google Analytics has enabled web hosts to be), the winners will be the ones who have the deepest understanding how to find the most relevant data and take the most appropriate actions as a result.

The understanding is the place where I fear the music business will have the most catching up to do. From my personal experience, data analysis seems to be a new concept to many industry veterans. Many are excited by the prospect of data being available, but few seem to know what to do with it – it serves as little more than eye-candy, another blurry trend chart at a corporate meeting (to be fair, it's not yet easily available in relevant and easy-to-read format without some tech savvy and excel skills). Layer on top of that the fact that people in the music business seem to be among the world’s biggest optimists, and you’ve got a recipe for disaster when it comes to biased or outright incorrect interpretation of data.

The winner of the data wars among the music startups will be the one who can provide the deepest insights, not just the most data. They will preemptively answer both questions of “What does this mean?” and “What should I do?”

The biggest winner will also have the tools to power the “What should I do?” actions. This means marketing, ecommerce, and access to relevant channels. These tools need to empower the artist, but also add value. In the end, I believe success may be measured as some variation of the following equation (would love to hear your feedback on improvements to the equation), and the winners among the startups will be the ones who can prove to consistently impact the quotient the most:

(Engagement + (Investment x Demand Generation Execution))^Quality = $$$

Note that quality is hyperefficient and falls almost entirely on the artist. Even the best data analysis in the world can’t make up for a product that doesn’t appeal. But it can tell you why your product isn’t selling as well as your competitors, and what you can do to make incremental improvements from a business perspective. And that makes data incredibly valuable, and a worthwhile pursuit -- it will undoubtedly unlock a plethora of new business models for music, and allow artists to figure out their own optimal model.

Bands vs Fans--Who's responsible for spreading the word?

The following was an e-mail I sent to my friend Adam a week or two back, so excuse any typos:
I was thinking yesterday, the niche of folks who are active in pushing music to others is pretty small, but they're pretty much the only ones who are truly engaged with the music. The average music listener might buy a cd, or might get something from a friend, and will just throw it on the iPod and be done with it. They don't check MySpace, iLike, etc, they don't generally buy merch, they go to concerts when their friends go or when it's one of their top 5 favorite artists. Is there any way to engage those folks more? Or are they a lost cause as they are just not predisposed to get more involved with the music they listen to?
If we count them as a lost cause, how do bands best use their fan support to grow? There used to be a whole market of folks to help manage street teams and such. Now everyone just uses MySpace (one such person came up to my friends in scenes from a movie at warped tour this year, as soon as he left they threw his card in the trash saying "unless your name is MySpace, you're worthless to us"). But let's face it--as much as bands are businesses, and as well as bands generally know their audience, precious few are truly marketing geniuses themselves. That opens up a new market of providing marketing tools to the bands (I know you know all this, I’m just thinking out loud). They have tens, hundreds, thousands of folks who would happily perform easy tasks for them, particularly if those tasks are fun. In the traditional method, fans get rewarded with tickets or t-shirts for passing out flyers, emailing the most people, etc. those kinds of rewards are great and appreciated, but they aren't a constant. Bands need their fans to CONSTANTLY be pushing the marketing for them. Thus, the fans can't always be physically rewarded. Instead they either need to have fun in the process or feel good at the end--or better yet, have it be so mindless that they don't even have to think about it.
Putting a band in your favorites on Facebook or MySpace rarely does any real good. The very very very few people who notice it generally already know the band. There really needs to be a PUSH of information (or whatever it is) for any promotion to have an effect.
Speaking from my own experience, my pushing happens a few ways. First off, I RARELY push music to people who I’m not at least 70% sure will take the time to listen and will enjoy it. When I do push music, it generally happens in one of three forms: cd, imeem, blog. For cds, I send out mixes to about 10 people every month or so (it started with my college roommates and has grown a bit as others have asked to join). I plan the cds as though I were making a mix tape for a girl in middle school (while I feel I can push my friends a little musically, I know what they like and tend to play to that), and even design cover art for them. Imeem I use for only a couple friends (I think I only have one or two on there). If I hear a song I like on my iTunes or on some band's MySpace page, I go to imeem, do a search, click the song, and send it along. That is the only thing I use imeem for. Since the usability design is so bad (almost has to be as they need to serve a whole lot of ads to make money), I don't find value in any other site functionality at this point. Finally, if I see a particularly good concert, I’ll blog about it. Each story on my blog averages about 20-25 reads, and only a few people click the links to listen to tracks (they more readily watch embedded videos, but even then it's maybe 20% of readers).
all this results in my pushing of music more than once a year to maybe 20 unique people, and probably 200 unique tracks end up in peoples' iTunes as a result. knowing my friends, I probably influence about 2 cd purchases a year--they already have the songs they like and don't often feel the need to explore a band further ("if they're really that good, Ty will send me another cd with another track or two from them"). Case-in-point: I got my college roommates obsessed with Kaddisfly. Not an easy task, considering one's favorite music is jazz, one's is rap, and the third is top 40 through and through. But they now LOVE Kaddisfly. Yet I’m the only one who owns a cd or t-shirt or has been to a concert. How do I make their love translate to revenue for Kaddisfly?
And there's the big key: sharing is all well and good, but how much sharing has to happen before the band can actually make money off of it? the process needs to be refined on both ends--fans need easier, more fun ways to spread their favorite music (and perhaps a better sense of who might be open to listening to it?), and the people they spread it to need a fun, easy, relatively inexpensive (at best, free--work on that) way to generate revenue for the bands.
Part of what I think might help this is if bands truly take it upon themselves to build their brand beyond the music. I used to blog a bunch about the need to think of a band as a small startup business, and I truly believe that. Of course, in both Seth Godin's (marketing guru) concept of a "Purple Cow" is incredibly relevant--if you have a product that is truly "remarkable" (his term), it's infinitely easier to market. remarkable doesn't just mean unique or fascinating, because different is not always good or pleasing to the masses; instead, it means something that is innovative and interesting, but is generally relatable to things we already are familiar with (why Panic(!) at the Disco's first cd got so huge--it was new instrumentation of very traditional pop punk music, so people thought it was new and different but were still universally comfortable with its conventions).
Regardless of how "remarkable" a band is, though, they can always do more to build their brand. Merch is one extension, and touring is one outlet/marketing tool, but there has to be more. Videos, blogs, hotlines, etc help, but can also be overdone--fans want to feel a like a part of the music and the band, but there also has to be some allure left. It’s like if a company was so into creating a "team" environment that they completely do away with hierarchy and put the management in the same cubicle as the entry level folks--teams are good, but they also need leaders who garner the respect of their peers partially by having closed-doors meetings and such. It’s a fine line between encouraging fan involvement and pandering to them. Ultimately, the best thing to have happen is to have the fans work with each other in a sort of community setting that you can oversee and occasionally communicate with to give some direction and encouragement. That being said, you have to make sure the conversation in the community is constant. If you think of it as an internet message board, if people run out of things to talk about related to your band, they'll stop coming to the site, and it happens as a snowball effect. If they stop coming to the site, reengaging each of them is infinitely difficult, and the longer you wait, the more folks you'll have to try to reclaim. Lil Wayne is a decent example of keeping fans engaged--he keeps releasing songs on MySpace rather than waiting every 2 years to do a cd. He doesn't have to do much in the way of talking to fans or anything really besides constantly providing the content for them to enjoy and buzz about.
The traditional thinking is that the music is the product. Now, it's sort of a product (people still buy it, but iTunes overtaking Wal-mart as top music reseller is evidence that people are more comfortable with digital music, and digital music has a marginal cost of zero, thus market forces will push price towards zero). In the future, it may only be a tool. Google started as a search engine. Now their search engine is a tool for selling ads, and they have a plethora of other free services that would traditionally have been considered products but are really just tools for building a brand and generating revenue through other areas. Can a band mirror that? I think so. They just need the tools to do so.

Huge Tool: The LinkedIn Answer

Last Tuesday I was at a They Might Be Giants concert in Dallas, and amidst the usual witty banter between the Johns, John Flansburgh asked the audience “So do you think we should get on this Facebook thing?” There were some strong boos and some strong yeas, but to my surprise the majority of the crowd remained silent, or merely chuckling at the question itself.
The truth is, Facebook isn’t something people associate with music (though Facebook is trying to change that with their partnership with iTunes). I guarantee there were multiple people in the audience turning to their friends and whispering “Isn’t that what MySpace is for?” Just like it might be weird to get a Facebook Friend request after taking a business meeting with someone, but it might be appropriate on LinkedIn. Each site is, when all the initial excitement wears off, a tool for a specific purpose.
MySpace is for entertainment. It allows full creativity in making your profile as ugly as you dare, and is a hub for up-and-coming musical and comedy acts to share their material and plug themselves. Gone are the days of promoters and street team managers—bands manage all that by messaging their MySpace “friends.” It used to also be for people, but that was only when people on the internet were a form of entertainment, rather than an extension of real life.
Now Facebook is for people. People being the majority of people who don’t go to the internet looking for new relationships necessarily, but just want to keep track of what their real life friends are up to. Therein lies the beauty of the Social Graph—Facebook is a tool for keeping track of real life friends. Facebook has thus focused on communication and photos.
LinkedIn has survived in the face of possibly the worst design in internet history (recently upgraded to workable) because ultimately it was a very good tool for keeping track of business contacts (it’s syncing to Outlook was a fan favorite). In fact, one could argue that the poor design and difficulty in navigation may have been something of a comfort to business people who often see computers that way in the first place. If Facebook or MySpace (or Google's Orkut) buys LinkedIn and tries to integrate it, they will need to focus on the business-specific aspects of the site and be sure not to alienate long-time users by taking the focus away from that functionality.
Look at some of the other big guns: Google’s homepage is famously simple, focusing entirely on its search tool; Craigslist does nothing but provide a tool for online classifieds; eBay went through a craze, but now gets most of its traffic from stores and most of its revenues from PayPal (a smart pickup when they were on top of the world), both tools for facilitating ecommerce between existing merchants and everyday consumers; YouTube won in video not because it was the best in a lot of ways, but because it was the easiest tool to share videos with friends.
In the end, every truly successful website will boil down to being a tool—the others will have their fads, but will die off relatively quickly if they don’t evolve into valuable tools. While we have a fascination with this internet thing as though it were in an infomercial on tv, in the long run it’s going to boil down to a new set of tools for humans to get around their everyday lives. But perhaps by then we’ll be entertaining ourselves by taking family vacations to Mars.