Solve Information Overload

What is the "million or none" problem? It's when an Internet query is answered with either a uselessly long list of results or "no results found" and no help on what to do next.

Look at your search logs and you'll find quick proof that your customers face this information overload problem frequently. The bulk of their queries are broad and general - just a word or two - and are answered with a uselessly long list of results. Many other queries will be overly specific, and return "no results found." After examining the search logs of many of our customers, we found that 25% to 30% of queries return nothing. What's worse, another 35% to 40% return a list too long to digest. There was probably something in the catalog of interest to these 60% to 70% of your shoppers, but most leave when they get either of these responses.

Help Your Customers Buy
Technology can solve the "million or none" problem by putting all query results in a precise navigation context. This shows shoppers exactly how to refine their search or explore further. After all, the very heart of browsing is in telling your customers how to ask smart questions. But how? You've already invested heavily in detailed product information, but you can realize far more value from that data by exposing it to your shoppers in a way that will help them ask their next question. For example, at most DVD stores, if a shopper searches for John Wayne, they simply get a John Wayne list, a long set of titles they must page through. This squanders all the useful, structured information that exists about the available titles in that list. This could include information like genre, director, and price. What the shopper wants is a "John Wayne store," where all the product information is organized in a meaningful way based on this information. This organization helps the shopper whittle that long list down to a single title.

The technical difficulty of generating a "John Wayne store" varies depending on how much functionality you want to give your users. Just breaking the list into genres can be a big help to users, and that can be accomplished with a modest increase in load to your search engine. As the search engine returns the list of results, it simply looks up the genre associated with each title and breaks up the list into smaller segments.

The Complexities Of Scale
The problem becomes increasingly complex as we scale up the number of users, the size of the catalog, the number of ways we classify an item, and the frequency that our inventory changes. For example, if we want to navigate that same John Wayne list by actor as well, we now extend the complexity of our lookup from dozens of genres to thousands of actors. After we find all the valid actors, we need to organize them. Perhaps this is done in alphabetical chunks, or perhaps by ranking the list of actors so it leads with the most popular stars. For catalogs with inventory that falls in many departments, it also becomes necessary to control the sequence in which navigation options appear. For example, a shopper searching for the brand Calvin Klein at the top page of a department store site probably wants to pick next from a department or a price range. That shopper would most likely be overwhelmed if they were also offered "thread count" from the bedding aisle, "belt size" from men's pants, and so on.

Showing the "perfect store" that organizes and ranks every valid navigation option for a result list of any size can quickly bump against the practical limits of a search engine or relational database. Successful solutions will require in-memory indices that hold the relationships between both structured and unstructured data, which will make it possible to show users the questions they can ask at interactive speeds. While a difficult IT challenge, there is immediate ROI in the solution. Customers invariably view more unique items, and reach them from more unique paths, when this rich information is exposed to them in a meaningful way.