On the bias of news aggregators
I follow developments in news aggregators because they often serve as a bellwether for how we are collectively dealing with the task of finding important things in the vast information infinity of the web. Strategies differ considerably between the many companies in this space, but most offerings fall on what is becoming a well-defined continuum. In one corner, there is Google News representing the robots. On the other end is Digg, representing thousands of bored-at-work geeks. Both ends of the spectrum offer clear reflections of an important shift in journalism. While once upon a time there were editors to determine tone and journalistic "brand," the critical factors in determining the editorial vision of an aggregator are decisions about its technology.
My pattern of news checking throughout the day takes both ends of the spectrum into account. In the morning, I do a quick sweep of something like Google News. At this point, I'm not looking for angles, just the big stories. Once I've absorbed those, I switch gears. I'm now looking for perspective and small surprises. This is the kind of thing it's much harder for an automated reader to pick out. Google's traffic ranking used for picking out news wasn't designed to troll the long tail. Finding something unique and surprising is pretty hard for even a human to do - which is why sites like The Drudge Report get huge amounts of traffic. As it turns out though, it's something that thousands of people can do if their responses are filtered and weighted correctly, and that's what Digg is all about. Digg is not about to tell you exactly how they do it, but it's reported to be a calculation between momentum (tons of votes) and influence (user often submits highly ranked links).
The important difference is that while Google News presumably reflects the vox populi, it seems voiceless. The top stories are repeated there as elsewhere, and since they're all redistributing the same AP stories, whether you get the news there or on CNN is relatively unimportant. Digg, by contrast, has as much of a voice as single-editor news blogs like Drudge Report. Digg can surface anything, but the news it does surface is a clear reflection of a particular community's interests. Digg's voice is not the best thing to listen to if you want deep political insight or serious cultural critique. It is, however, extremely good at picking out stories that are just interesting. It's very rare that I go to Digg without clicking on something.
Last year, bbc.com held a contest to inspire their redesign. The winning idea was something called "BBC Malkovich" - which (taking its cue from the movie) contained a slider at the top that would shift from news suited to your perspective, over to news from the perspective of someone very different than yourself. The design is hypothetical, but if it were possible, it would avoid the danger of overpersonalization that Andrew Shapiro warns about, while preserving the ability to find the kind of things you want to know about. Shapiro's idea, that the middleman we've eliminated online might have a valuable perspective, gets complicated in a situation like Digg in which we're all essentially middlemen.
I'd like to see a news aggregator that took the same idea as BBC Malkovich and applied it across the spectrum of news readers we've been referring to. On one end, you'd get the automated headlines, but then are able to slide some control over to get increasingly vernacular, community-surfaced views on the news. Users would be able to apply the bias of particular technologies like filters over the news. Thinking about it in this way might also encourage us to develop more niche algorithms, such as highly unpopular news, to add as a point along the slider. Dipping a toe into science fiction, imagine if we could do a sort of motion capture on great editors now, translating their vision into algorithms that we can preserve forever. Will the day come when we can do an algorithmic reconstruction of Horace Greeley? Not that he would have much to say about the iPhone and the latest Linux distros, but who knows?
