Posts tagged: Recommendation Models

Rediscover Discovery

By , August 10, 2009 10:07 am

Discovery seems to be one of those things that we don’t really spend a lot of time doing, but when it happens, it’s like our world could not exist as it once did. Whether it’s great discoveries of mankind (oh snap, the world is round) or small discoveries made by each of us everyday (Baoguette – Vietnamese sandwich shop on 25th and Lexington), the result is the same … totally mind changing.

How we find things on the web, or rather how we think the best way to find things on the web has changed dramatically.  The initial thoughts were to build web portals, where anything and everything you wanted to know would be conveniently located on a single web destination.  Our one stop shop for discovery.  That quickly changed when consumers were not able to find the best information in one place, and instead scoured the web for sources they personally found valuable.  As content became easier to publish, resident experts were now making places on the web to get very specialized information, and we needed a new way to find things.

Hello Yahoo, I’d like to introduce you to Google.  Google – “We’ve got this great algorithm that scours the entire web and returns the best result. You can buy us for $1M.”  Yahoo – “Well that’s silly, why would we want people to leave our site?  We want people to stay on our site for as long as possible.  Carry on now, nothing to see here.”  (I’m still shocked that Yahoo would pass on this; if not for the not so obvious revenue model, but just to use Google’s search to discover great content to add to their own portal!)

So spurned by Yahoo, Google acquires a small company called AdSense and the rest is somewhat recent history.  Google really capitalized by making a better way for users to find things.  They served up what people were looking for better (PageRank) and linked it with people providing what users were searching for.  Awesome, so now people can find out where to get things that they are looking for.

Often discovery is more than just finding where to get what you are looking for.  Discovery is often a question of “what do I want,” versus “where do I get it.”  Search does a great job of solving the “where”question (Where is Boaguette located?), but it’s not so great for the “I could really go for something sweet & spicy, but not served in a sauce with rice.”   To solve that, we look to the experiences of others, and we count on their recommendations to discover new things.  Search will always have it’s place, but taking good ol’ word of mouth and placing that in a useful platform on the web will open up all kinds of new doors of discovery for everyone.

Recommendation models have come a long way, but there is still work to be done.  NetFlix recently ended their competition to improve their recommendation algorithm and will pay the winning team $1M.  There is a huge amount of value in this space, and companies are recognizing that more and more each day.  How do we capture that on the web?  A difficult problem to solve, but one with great rewards.

I’ll leave you with a quote from Greg Linden, the man behind Amazon’s recommendation system, that pretty much sums it up.  He said, “Whoever manages to change the nature of content display on the Web from a search problem to a recommender problem will reap tremendous rewards.”  It makes a lot of sense when you think about where you got your last great restaurant recommendation.

Recommendations on Facebook?

By , May 17, 2009 12:51 pm

Has Facebook finally gotten it?  With the amount of personal data and interactions that they are able to collect, it should be a no-brainer for them to work on a recommendation framework.  With the advent of user-generated content, we are in information over-load at this point (just look at how many unread emails you have, or how many article feeds you end up ignoring).  The next big move needs to be a way to sort through all of this information and bubble up things that we care about as individuals.

Facebook is in a great place to do this because of their unique position in that people see a ton of value in declaring a lot about themselves over their platform (maybe a bit too much when you are “no longer in a relationship” – broken heart).   The popular a/s/l (age, sex, location) back in the day in AOL chat rooms was one of the first forms of public declaration on the web and this basic desire/need to share information about ourselves has not changed as the web has evolved.  Facebook has created a place where people want to do this, and that could be the most important and valuable thing they have to offer.

I recently met with some folks at Yahoo and noted a particularly interesting thing about users inputting accurate information.  There are tens of thousands registered users at Yahoo that have the zip code 12345.  Now the funny thing about this zip code is that it is for a small county in New York where there are 10 registered business and way less than 10,000 inhabitants!  A Yahoo user doesn’t care to share where they live because there is no good, compelling reason to do it, while a Facebook user wants to share that information because it is valuable to their social network.  I wonder how many people on Yahoo live in 90210.  (On a side note, I found this information on Wolfram Alpha which is a statistical gold mine for anyone who loves numbers.)

Since Facebook has already gotten people to buy into sharing their information, they would have to be crazy not to work with this information to provide better services, including personalized recommendations.  But alas, it appears that this feature is just a way to condense duplicate posts so users can’t spam their friends newsfeeds.  Another tease.  It’s pretty sad to say that the best analysis of online social interactions, and the most entertaining, has been Dateline NBC’s To Catch a Predator.  Yea, I’m sure you were just there to help with some homework.

Pandora’s (Beat)Box

By , May 13, 2009 12:04 pm

The story of Pandora and her box goes a little something like this.  Zeus gets pretty pissed off at this guy Prometheus for throwing fire in a game of RoShamBo, and in retaliation creates this woman Pandora to punish all of mankind (OK, I’m speculating about the game of Rock-Paper-Scissor, but I’d get pretty upset if I got beat by someone using their once in a lifetime throw of fire).  Pandora was given many seductive gifts from the gods and one in particular, the gift of curiosity, led her to open a box releasing all the awfulness into the world (including the credit crisis).  Realizing what she had just done, Pandora quickly slams the box shut, trapping only Hope inside … or maybe not.  After using, it is easy to see why President Obama sees so much hope in the world.

Pandora is an online, streaming music player where users can “customize” their own radio stations.  It’s absolutely brilliant since the only thing you really have to do is enter in a song or artist, and Pandora will automatically stream music that is similar to that song or artist.  Now, instead of spending hours customizing the perfect 80s party playlist (or mix tape/CD for the romantics out there), we can just tell Pandora what song fits our mood at the time and get hours of music delivered too us.  Best of all, if a song comes on that we’re not sure why it was there in the first place (Blame it on the Rain made it on all my mix tapes for some reason), we can easily skip it, and Pandora will exclude songs like it.

So how does Pandora do this?  Well, they’ve hired a team of music analysts who essentially measure each song on 100+ musical characteristics, an idea inspired by the Music Genome Project.  These characteristics, or metrics, make up the “genes” of a song, and their measurements are used to construct a song vector, a mathematical attempt to value the essence of a song.  The similarity of two songs is figured out by measuring the differences between all the musical characteristic of the two songs.  To do this well, Pandora uses a complex distance function, which is essentially saying “how far apart, or different, are these two songs.”  The shorter the distance, the more similar the songs are, and the more likely that song will be played next in your Pandora station.

This is a very powerful framework, but there is one important assumption that shouldn’t be overlooked, and could be a major drawback to implementing this particular recommendation engine.  That critical assumption is that we have identified every factor that captures the je ne sais quoi of a song, which for the non-French speaking means an intangible quality that makes something distinctive.  Do you smell the conundrum brewing?  How does one measure the intangible?  Can you find all the right factors to accurately describe Kris Allen’s performance of Kayne West’s Heartless?  Now while it might be next to impossible to figure out everything that makes a song click, it is very important that you catch the most influential ones in your recommendation model.  Failure to do this could get you voted off.

Pandora is doing a pretty damn good job recommending songs using this framework and they understand that there are a lot of factors that make a song a unique piece of work.  They have developed a framework where they have identified a lot of the measurable, tangible metrics, and have used them to effectively relate songs to each one another.  The next big step in recommendation models would be to understand how each individual values a song, what aspects are more important on a case by case basis, and eventually delivering a personalized, Rathan and Rathan’s Infinite Playlist just for me.

Panorama Theme by Themocracy