Theory vs. Practice: Thoughts On The Netflix Prize

I always wondered whether the solution proposed by BellKor’s Pragmatic Chaos, the winners of the $1 million Netflix Prize would actually solve the company’s fundamental problem: that of making what we (at work) call the Next engine – the recommendations that come attached to your movie selections – work more effectively. This article, and the associated research paper, is very useful in understanding the logic behind it all. I was also fascinated by how Chris Anderson’s Long Tail theory is woven into the whole thing, as well as Pareto’s 80-20 rule. According to Wharton professor Serguei Netessine, the costs associated with investing in a company like Netflix or Amazon, such as renting out warehouses to store the DVDs and items that need to be shipped, mean that they DO need to find a way of shipping more of everything – not just the ‘hit’ products. 

Netessine says the research indicates that “primitive” recommendation systems are likely to blame for the delay in lesser known products becoming available and consumers finding their way toward them. “Many recommendation systems are not terribly smart,” Netessine states, adding that recommendations about films are made to Netflix subscribers who view similar films. But in order for a film to be recommended, it must be viewed in the first place. “If you want to see the long tail effect — consumers going into those obscure products — you have to be sure consumers learn about them, and that’s not easy. Current tools may not be good enough.”

Ergo the need for the $1 million Netflix prize.

Anderson’s counter-argument is equally well-put, however:

“Although academics are free to do all the relative analysis they want, it is incorrect to apply it to my theory,” he writes. Anderson argues that defining the head and tail of demand in percentage terms is meaningless in a market with unlimited inventory, such as a retailer with digital distribution. For example, take a company with 1,000 different items in which the top 100 — or 10% — account for 50% of sales. If 99,000 more items are added to the catalog and sales of the top 100 fall to 25% of the total, it may take another 900 items to make up the next 25%. In this case, Anderson would argue that sizable demand has shifted down the tail toward more people selecting fewer products.

In relative terms, however, 1% of the products now constitute 50% of the revenues, which would make it appear that there was a greater importance of the hit products. But since real people experience the world in absolute numbers, not percentages, this is a statistical illusion, he states. The truth is that people are choosing a wider array of titles. “Nobody in the business world is confused about this, thankfully,” Anderson adds.

Hmmm…as Benjamin Disraeli said, there are three kinds of lies: lies, damned lies, and statistics. We’ll need to wait a couple of years to see if the $1 million investment was worth it, I suppose. 

More power to Google

On Thursday, Google’s latest project that will allow people to read books that have gone out of print, will go live. In partnership with On Demand Books, the Espresso Book Machine will print in 4 minutes flat, on demand, any book of upto 300 pages from a list of over 2 million books whose copyright period has expired or whose authors have given permission for them to be used by Google. All this at a cost of less than $10 to the user and $3 to the bookshop that owns the vending machine, which is to cover the cost of the materials. The machine costs a whopping $100,000 but quite a few locations already have it, including Blackwell’s in London. Here’s how it works:

I was really glad to read about this, and hope that it does not run into stormy weather like the Google Books deal has, where the US Justice Department has urged a New York court to reject this deal. Just as with the continuing debate over free, I wonder how long anyone can prevent it from coming to life. Dan Brown’s ‘The Lost Symbol’ is selling more e-copies on the Kindle than hard copies via Amazon, and that’s because there are enough people are willing to buy the e-version though they have access to the hard copy. If the situation is reversed and enough people decide they’d prefer to read out-of print books in the flesh rather than online, where Google Books will likely have an e-copy available, can Amazon, Microsoft and Yahoo, who are among the companies that object to the Google Books deal, continue to stand their ground? THAT will be an interesting conversation to follow.