Component is a section of Aux.Out. for one-off pieces, special editorials, and lost orphans of the music discussion. Today, another cultural landslide creates the biggest Aux.Out. ever on music discovery and the services that help us navigate our musical terrain.
There’s an Elephant in the Music Room.
You may not be able to see it, but it’s there.
A few people actually see the elephant; a few more can’t see it but have noticed that something incredibly huge is taking up a lot of space in the room; yet, most people haven’t even noticed that the room is getting smaller – or that there’s this humongous, pissed-off elephant just sitting there, scowling at them.
Oh. Nobody really wants to talk about it, either.
Instead, we hear about lots of divisive discussions between musicians, labels, tech start-ups, writers, bloggers, legislators, and music advocacy groups over issues like streaming royalty rates, copyright infringement, file sharing, fair use, declines in record sales and digital downloads, termination rights, pay-for-play, venues taking larger cuts out of touring bands’ payouts, legislation to force radio stations to pay their fair share for music use on-air, and so on…
But as those battle lines get further entrenched and fortified, the damned elephant just keeps getting bigger; and unless we all deal with this elephant – pretty quickly, too – none of those problems are going to matter.
It’s a freakin’ huge elephant, too; hell, just look at the amount of real estate it covers: as of December 2013, Spotify claimed 20 million songs; in September 2012, iTunes claimed to have over 26 million songs; and in November 2013, Deezer (a service not yet in the U.S. but available in 150 more countries than Spotify) announced a library of over 30 million songs. A more staggering number comes from The Echo Nest, a leading music intelligence company: at the time of this writing, they claim to have identified well over 35 million known songs by nearly 2.7 million artists that have now generated in excess of 1.168 trillion data points. (If you have nothing better to do, you can actually go to their site and watch those numbers grow by the second…)
And yet, the most astounding figure to date appeared in the December 23rd announcement that the Tribune Company had agreed to purchase Gracenote from Sony for $170 million; hidden within their PR boilerplate was the statement that Gracenote – which started in 1998 as CDDB and now provides music identification & recommender systems for clients like Apple, Google, and Amazon – now tracks over 180 million songs.
Meanwhile, the growth in recorded music is expanding exponentially due to order-of-magnitude changes in recording technology, lowering the nut required for musicians to produce and distribute their music; and musicians are taking advantage of those changes at an astounding rate. Soundcloud just reported that people are uploading 12 hours of new material every single minute…
And yet, realistically speaking, there’s no way to accurately quantify how much music is getting released right now, nor how many musicians there really are; and if you start to include the boatloads of older catalog materials and artists that have never been released in a file-based format, it’s nearly impossible to get an accurate grasp of how much music really exists out in the wild.
But wait: that massive mountain of music isn’t the Elephant. That’s just the real estate sitting under its shadow.
Now, even if we apply Sturgeon’s Law (90% of everything is crap) to just those middle-ground Echo Nest numbers, that would mean there’s at least 3.5 million songs by 270,000 artists that have generated nearly 117 billion data points of not-crappy music, and defining crap is often just a matter of taste…
Speaking of taste definition, each day your average, dedicated music writer is besieged by hundreds of promotional e-mails/press releases/one-sheets practically begging the writer to pay attention to a new release, all while the writer struggles to find an outlet that will accept their article pitches from an ever-diminishing number of outlets – outlets that are themselves struggling to reach eyeballs in hopes of increasing advertising revenues and must therefore tighten their coverage to articles that will draw the most eyeballs: that is, what they believe will be popular to their readership.
A cursory scan from opposite ends of the music writing spectrum reveals that Pitchfork published roughly 1,300 reviews over the last year (not including features or aggregate columns), while a mainstream media outlet like the New York Times featured weekly playlists of five or six non-classical releases and three larger reviews per week – roughly around 450 to 500 reviews.
So… there’s all that music out there, just waiting for a set of ears to hear it, and yet all you ever read or hear about are maybe a few thousand releases?
Say hello to The Music Room Elephant.
Let’s call it “Lack of Discovery.”
Wait – that’s a terrible name for an elephant…
And it doesn’t sound very threatening, does it? You might even find yourself thinking, “Shit, that’s the way it’s always been!”
Match that mind-boggling volume of available music with staggering shifts in technology and listener habits, and Lack of Discovery rears its ugly head as an alarming and incredibly complex set of problems with potentially catastrophic ramifications for most musicians, and for the recorded music industry itself.
And there are no easy answers.
Why Discovery Matters
You’ve probably noticed the heightened use of the term “discovery” within just the last few months; and now that Beats Music has gone live with its “curated by trusted sources” discovery system – and with YouTube due to debut its own dedicated music service and Deezer expected to open up shop in the U.S. later this year – the discussion of music discovery and recommendation systems is about to get a hell of a lot louder.
So, really, is discovery that important?
From the business side of things, you need go no further for an answer than to look at the March 6th pre-emptive purchase of The Echo Nest by Spotify. As reported by Ben Sisario of The New York Times, this deal not only gives Spotify a programming advantage over its competition, but it also seems “to set up a possible conflict for The Echo Nest’s other clients, which include some of Spotify’s competitors” – like Rdio, Deezer, iHeartRadio, SiriusXM, Microsoft’s XBox Music, and Rhapsody – potentially depriving each of them of their primary source for discovery and recommendation data. According to the Times report, The Echo Nest’s CEO Jim Lucchese said his company “would honor its current contracts with these services, but gave no further details.” (Later in this article, you’ll see why this acquisition matters so deeply.)
What’s more interesting: back in November, Spotify raised $250 million in new funding, pegging its total funding since its formation in 2006 at $538 million; if Techcrunch is correct in reporting that Spotify paid $100 million for The Echo Nest – even if 90% of that is in Spotify equity – that’s still an enormous outlay for a company that may have a total valuation north of $4 billion… but has yet to post a profit.
Another measure of discovery’s value to business interests: on January 15th – the day before Spotify & Rdio preemptively announced that they’re now really really free in their attempts to blunt Beats Music’s initial PR push (now the only advantage in subscribing to either Spotify or Rdio is killing the ads in your streams) – Pandora quietly announced that it had added a recommendations function to the engine that runs its limited library of a little over 1 million songs; within 36 hours, Pandora’s stock price spiked up nearly 5% – reaching a market capitalization of $6.7 billion. (To give you some perspective, Universal Music Group, the world’s largest record label group, generated revenues just a little south of $6 billion in 2012.)
These services and their investors see discovery’s effect on user engagement – and more importantly, keeping users engaged – as absolutely vital to the long-term health of their business models; and if you go back though the last six months of music-service press announcements, read them in chronological order and parse them carefully, you might pick up on a sense of increased urgency for these services to emphasize their discovery features.
For musicians, how can discovery be a more important problem than the issues of low royalty rates paid for music streaming, or the claims that streaming is cannibalizing sales?
Well, using simple horse-in-front-of-cart logic, how are you going to collect royalties, or sell your music, or even have a remote chance at a sustainable life as a musician if nobody even knows you exist? Faced with the current overwhelming abundance of music, unless your potential listener has even the slightest clue to look for you, their odds of discovering your music is similar to your odds of plucking a single perfect snowflake out of a raging blizzard.
And if you’re an established artist and you think discovery won’t affect you, then you might be overlooking the long-term impact that this lack of discovery will have on the overall ecosystem of recorded music.
As a listener, discovery could become the deciding factor of whether you stay with downloads or physical product, move to a free streaming system, adopt a subscription model, dump a service entirely, or even give up on looking for anything new to listen to – all because of what or how something is recommended to you.
In his January 3rd essay, The Wall Street Journal’s John Jurgensen writes about facing this abundance of music without some form of easy and meaningful recommendation as “one reason why we fall back on the same stuff we’ve been listening to since senior year in high school,” and even coining a great new term for it: “search-bar paralysis.” Basically speaking, you are overwhelmed by choice… and yet, with so many choices available to you, you have no way of differentiating the signal from the noise. So, in the end, you don’t choose anything at all.
If what you’re listening to is what you’ve always listened to, you are much less likely to plunk down money for a new album or, in the case of streaming services, a subscription; and if you listen to a free version of a service with a narrow spectrum of choice – leaving the vast number of musicians out in the cold, looking in – and are subjected to an ever-spiraling number of ads to make up for that service’s lack of subscription revenue, then you might as well be listening to radio.
And coincidentally, the market that many of these outlets and services now seem to be targeting just happens to be the same turf mainstream radio lives within – a decision that may well be their biggest mistake. (More about that later, too.)
Yet bad recommendations can often be more problematic than search-bar paralysis; in the three months we’ve been working up this article, we’ve seen dozens upon dozens of remarkable tweets about problematic service recommendations – for example, this classic from Stephen T. Erlewine, senior editor of All Music Guide:
Or this one by writer Michele Catalano:
Clearly, there are some big problems facing music discovery and recommendation systems; but now, if it’s so damned important, how do we fix “discovery?”
But That’s the Way It’s Always Been…
If we want to tackle the problems of “discovery,” we need to clear away some long-standing preconceptions and recognize that the technological shift occurring over the last several decades represents something much more than a change of distribution formats…
So, let’s take a very brief look at music’s history as a way to gain some perspective. (Don’t worry, history-phobes. This all ties together in the end. Besides, a little history is good for you. Don’t forget to eat your peas.)
Although phonographic reproduction was invented in the late 19th century – and even at that time, a vigorous format battle was being waged between vertically grooved cylinders and laterally grooved discs – the era of recorded music didn’t really get under way until around 1901 with the introduction of a 10” disc that could play back a whopping 3 minutes of music recorded “acoustically”; i.e., a large horn collected sound, channeled it to a diaphragm that in turn vibrated a needle that etched those vibrations into a solid, wax disc spinning beneath it, thus cutting the master recording directly to the disc. (Now that’s seriously analog audio.)
So, for the sake of argument, let’s use 1901 as the true advent of The Era of Recorded Music: this means the primary method we use for listening to music – recorded audio – is a little over 110 years old.
Pretty long time, eh?
With hard evidence provided within a report presented in the June 2012 Journal of Human Evolution, after re-analyzing the carbon dating of a mammoth bone flute and a swan bone flute (one that plays a clearly-tuned series of C, D, F, and B notes), their creation now clocks in at somewhere around 42,000 to 43,000 years ago.
Why make this point?
1) Human beings have been predisposed to music for a very, very long time. You might even say it’s as if we are prewired for it. (That’s the good news.)
2) Although we face several important musical dilemmas at this time, it’s important to realize we view our dilemmas subjectively, as did our ancestors; to the cave dweller who carved that flute out of a swan bone, the possession of that instrument might have been pretty damned important at the time. (Or, then again, if they’d lost that flute while getting chased by a very hungry bear, maybe not.)
3) In a 2010 interview with The Guardian, Brian Eno stated that “the record age was a blip.” The age of these flutes demonstrates exactly just how small of a blip The Era of Recorded Music constitutes in music’s historical timeline: 0.026%.
That’s 26/1000th’s of all evidence-based music history; and music itself is thought to be much, much older. Many ethnomusicologists and archaeologists think it predates the use of language itself.
4) Therefore, “that’s the way it’s always been” is not how it’s always been. It’s just how things have been for 26/1000th’s of mankind’s total evidenced-based music history.
5) And just because something worked (somewhat) for the last 110 years out of 42,000 doesn’t mean shit. (You have to admit: that’s not much of a track record.)
6) Most importantly: Things Change. If they didn’t, we’d all still be playing C, D, F, and B notes on swan bone flutes. (And we’d still be getting chased by very hungry bears.)
Sometimes these changes occur rapidly – and sometimes, gradually. For example, the Era of Recorded Music didn’t start off with a bang but with a gradual shift away from the predominant form of popular music distribution of the time (sheet music), with recorded music not really taking off until the widespread introduction of radio – roughly 20 years after the initial introduction of the 10” disc.
Some music scholars have cued to that last analogy, equating our current societal shift to digital music technologies to the societal shift from sheet music to recorded music…
But the changes we’re experiencing today are not those of a simple change in format or distribution technology; in fact, we’d argue that the shift to digital technologies – across all creative fields – is similar to the technological and societal shift brought about in 1450 by Gutenberg’s invention of a printing press that used a moveable type system; in other words, something that may happen once or twice every thousand years or so.
Take a look at the parallels:
Prior to Gutenberg’s innovations, there were very few books; each book was either hand-copied or at best etched to wood block and block-printed, both limiting quantity and time-consuming to manufacture; control of information was in the hands of the few who could actually afford these books – the same people who also wielded power over an illiterate, information-poor populace, forcing this populace to accept their elite interpretations of knowledge…
And then things changed.
By 1500, there were over 20 million volumes, with printing presses numbering in the hundreds; by the mid 1500’s, that figure rose to 150 to 200 million volumes from thousands of presses.
With the advent of this revolutionary technology, information could be shared quickly and easily; and with this ability to distribute new ideas across borders and languages came the advent of mass communication. Education became available to many, as literacy erupted across Europe…
As such, this printing technology threatened the power that political/business and religious authorities held over the populace, as it allowed new, revolutionary ideas to spread like wildfire. There was lots of wailing and gnashing of teeth about the unwashed masses taking control (as well as book burnings, bloodshed, political and religious persecution); and yet, printed information just kept growing and spreading unabated.
Now, does any of this sound vaguely familiar to what we’re experiencing today?
Summing up The Renaissance and The Reformation in this manner is a gross over-simplification (be thankful, we could go on about this for days), but much like those who lived within those periods, it’s difficult for our society to understand the absolute enormity of the changes these new technologies have thrust upon us; it’s hard to see the forest for the trees… especially if the forest is huge and dark, the trees are massive and menacing, and there may be some hungry bears around and we can’t lull them to sleep with our swan bone flutes. (Yes, bears and flutes are a theme here.)
And massive change is never a pleasant experience to the people stuck within it; with all the wars and persecution surrounding them, we doubt many people living during the Renaissance said, “Man, this is great!” Yet changes of the magnitude we are experiencing today are not unprecedented: they just don’t happen that often.
So, in essence, we are living within an event that occurs once an eon – wrestling 21st century technologies exhibiting an exponentially accelerating rate of change as coupled to an equally exponentially enlarging volume of music and data…
… and trying to do that using barely-out-of-the-19th century methodologies.
Historically speaking, that’s a #fail.
When facing a change of this magnitude, the primary tenet of the 2,500-year-old I Ching – the Book of Changes – rings true:
We can act with change, or be a victim to it.
And since we doubt anyone involved in music really wants to be a victim, how do we “act” with this change?
Well… first off, we need to start thinking differently.
Rule No. 1: The Music Comes First.
Let’s explore three basic concepts starting off with a simple, yet often overlooked fact:
No two human beings ever hear the exact same song.
Once recorded, the song itself exists in a locked form: manipulations of sounds, silence, instrumentation, and timbre, held within time and space as an unchanging object. Viewed objectively, The Song Is The Song.
And yet, it’s not.
Think of it this way: at an arena show, even though we might be surrounded by thousands of other human beings hearing the same exact musical performance, we still view and listen to that music alone; this experience is individual to us, based upon our own unique set of life experiences up to the moment of that particular musical experience. This is why a performance may be a life-changing experience to one person, while, to another person, it was “just another show.”
The same principle applies to recorded music: each individual’s perception of that recorded object is different – not only based on their life experience up to the time the music is first heard, but also affected continually after it’s heard…
For example, think of a song you once loved that – unfortunately – you also relate to a relationship that ended in a not-too-great manner (it was a train-wreck, they ran your heart through a shredder, and so on). Your primary experience of the song has been tainted by the secondary experience you’ve associated with it; and depending on the depth of your personal disaster, it may take one hell of a long time before you can even listen to that song again – that is, disassociate the painful second experience from your initial experience of the song.
The song didn’t change: your perception of it changed.
The same principle applies to the passage of time: think of the songs you played over and over and over as a kid, and now when you hear them, you wonder, “What the hell was I thinking?”
Therefore, Basic Concept No. 1: A musical experience – and thus music discovery itself – is a subjective, not objective, experience. (One Size Does NOT Fit All.)
To demonstrate our next basic concept, let’s say you come across an album. It could appear on your stream, it could be something you rifled from a remainder bin, you could stumble across it on Bandcamp, or a vinyl copy appears at your door. You know nothing about this release; the artwork could be amazing, or it could be seriously meh-inducing, but what the hell, you decide to listen to it anyway…
And suddenly you have a HOLY FUCK experience – that moment when the music you’re hearing silences everything else in the world and the only thing you can concentrate on is how fucking good this music is and how it has already rewired each and every one of your neurons and possibly even your entire subatomic structure and then the sky rips open above you and a gigantic glowing hand descends from the heavens and gives you a BIG “thumbs-up” and you happily nod in absolute agreement and give it a BIG “thumbs-up” back and none of this exchange seems the slightest bit unusual to you since you aren’t thinking about anything at all except how this music is washing over you like a cleansing and renewing wave of cool, fresh water except it’s not water washing over you but a whole-body sweat generated from you dancing around the room ecstatically since you’ve lost control over your body to this music that now owns you and has already shaken you to the very core of your being and now you just can’t imagine living life without it and you just can’t listen to it loud enough and when it’s over all you want is more more MORE…
After you’ve recovered from this experience (or not), what’s the very first thing you instinctively want to do (other than listen to it again)?
You want to tell somebody about it.
You want to share that experience.
This Is How Music Works.
Basic Concept No. 2: Discovery is excitement; it is passion realized; it’s a HOLY FUCK experience to be shared.
And being able to tell somebody about it? That’s Word-of-Mouth.
Word-of-Mouth has always been music’s ace in the hole; it’s still the most powerful (and yet, due to Basic Concept No. 1, the most unpredictable) marketing technique known.
If a good friend with whom you share similar musical tastes were to show up at your front door dripping in sweat and clutching a phone, iPod, disc, or vinyl, spouting, “HOLY FUCK YOU HAVE GOT TO HEAR THIS SONG” – well, most likely, you’re going to be intrigued enough to give it a listen. (Or call the police, depending on the friend.)
And that sweaty friend who’s clutching their phone, iPod, disc, or vinyl spouting, “HOLY FUCK” at your front door illustrates the second part of a word-of-mouth experience: they want to receive a reciprocal outcome from sharing their experience, since they’re at your door hoping you’ll like it, too. (And also hoping you won’t call the police.)
In other words, Musical Word-of-Mouth is a two-part, shared process: a) being able to share a musical experience and, after having shared it, b) achieving a shared consensus about the music that can or will be shared again (Basic Concept No. 3, if you’re still counting).
So… when we talk about building a successful discovery or music recommendation methodology, we’re talking about building a system that can automate or facilitate a subjective word-of-mouth experience – one that actively creates excitement or engagement.
Strangely enough, it’s been built before.
The First Online Music Recommendation System
In the early 1990’s, Pattie Maes – an AI researcher at MIT’s Media Lab Software Agents Group, and one of the pioneers of software agent technology – felt frustrated when listening to Boston’s radio stations. A native of Brussels, Boston’s commercial radio scene seemed way too bland and restrictive – not nearly eclectic enough; worse, she couldn’t find any other way to discover new music to fit her tastes…
So, Maes and a group of research assistants at the Software Agents Group decided to prototype a new form of software agent named HOMR (Helpful Online Music Recommendation, later renamed “Ringo”) designed to function as a form of “electronic word-of-mouth.”
The idea was simple (but the work to build it wasn’t): create a software agent that would ask its user to scale-rate a list of artists, with users specifically advised to rate these artists for how much they liked to listen to them, thus creating a basic user profile. The software agent would then search its database, find like-minded users, compare this new user’s ratings to those of the like-minded user’s pool, and email back personalized recommendations based on that comparison.
This type of agent technology is now known as “collaborative filtering.” Limited forms of collaborative filtering existed at the time, but none had been put to use in such a novel and openly user-based manner. And it wasn’t “intelligent” software; it was an artificial intelligence built on a database of the total knowledge and choices made by its user base. As the user base grew and each user rated more albums, not only would the agent’s database grow in size – its accuracy in recommending new music would grow as well.
In 1994, Ringo went live as an email-based system – later introduced to the nascent World Wide Web using a bare-bones graphic user interface – and was enormously successful, especially for a quietly introduced academic project… but creating this agented instrumentality was only a part of Maes’ thinking: what really intrigued Maes was how a collaborative filtering agent could be used to foster and build a community.
In January 1996, Firefly – the first commercial music recommendation system – went live and blossomed immediately; but Firefly stood apart from its previous Ringo iteration in allowing its users to set up their own profile pages, write reviews, and most importantly, not only offering to connect the user to those other like-minded users to see what they were listening to, but allowing users to actually contact and communicate with each other, thus automating and fully facilitating the electronic word-of-mouth experience.
For those who were lucky enough to have used Firefly, it was an amazing experience. Talk to former users of Ringo or Firefly and they’ll tell you how much their record-buying skyrocketed as they were introduced to new bands they didn’t know existed and how it widened their musical horizons; and since the user base was made up of music geeks and active listeners (we’ll be using that term quite a bit shortly), by giving users a forum to expound on their enthusiasm, they then made new recommendations well beyond the ken of the recommender agent – all of which were fed back into the recommender engine itself. Firefly’s database grew rapidly and exponentially – as did the user base itself.
In its initial commercial iteration, Firefly worked like a charm. At its height, it boasted 3 million user profiles; again, this was pre-broadband, pre-download, and pre-Internet Bubble 1.0, so that number represented an outstanding installed user base when compared to the prime movers of that period (AOL with over 4.6 million users and Compuserve with approximately 2.6 million users)…
And, true to Maes’ thinking, it became a community – one made up entirely of new connections bonding over music they loved…
But nothing good lasts forever.
As the site boomed, the collaborative filtering technology behind Firefly caught the eyes and ears of advertisers and retailers of all sorts, and suddenly there was a mad rush to either license or develop competing collaborative filtering platforms. Firefly became a victim of its own success, diversifying the recommender engine toward other areas of interest, like appliance recommendations, causing Firefly to lose its primary identity – and many users as well.
In 1998, Firefly was sold to Microsoft during their Engulf and Devour phase for $40 million (pretty decent money pre-bubble). With the filtering technology much more meaningful to Microsoft than a hardcore gaggle of music geeks geeking out over music, Microsoft eventually shuttered the site in 1999, porting its technology into their new Passport software.
At that time, all that mattered to Microsoft was the agent technology used to aggregate and filter user data for purchasing recommendations – with the human interactive/community element that actually created that data completely removed from the equation.
And, in our opinion, that’s the exact point where automated music recommendation technology went horribly wrong.