Editor or Algorithm?

THE NATURE OF AN ALGORITHM: THE AUTOMATION OF JUDGEMENT

Algorithms, as closely guarded as state secrets, buy and sell stocks and mortgage-backed securities, sometimes with a dispassionate zeal that crashes markets. Algorithms promise to find the news that fits you, and even your perfect mate. You can’t visit Amazon.com without being confronted with a list of books and other products that the Great Algoritmi recommends. Its intuitions, of course, are just calculations — given enough time they could be carried out with stones. But when so much data is processed so rapidly, the effect is oracular and almost opaque. Even with a peek at the cybernetic trade secrets, you probably couldn’t unwind the computations. As you sit with your eHarmony spouse watching the movies Netflix prescribes, you might as well be an avatar in Second Life. You have been absorbed into the operating system.

George Johnson. “An Oracle for Our Time. Part Man, Part Machine.” The New York Times September 23, 2007.

OUTSIDE.IN

“If it’s out there, it’s in here” Outside.in claims in a paragraph describing refinements to its service. If I were to make a bold “I have seen the future of the media … ” claim, I’d direct readers to Outside.in. The year old website hits every mark: It’s conceptually profound, based on co-founder Steven Johnson’s concept of “the long zoom”, taking the Google maps ability to be in a frame of mind, event or place that can be tagged with its geographic location and then “zoom” out to connect this dot to the rest of the world, and (in time, perhaps) the universe. The design is a timelessly sharp expression of form uniting with function that’s of the tradition that Charles and Ray Eames brought to the link between science, art and design. (Steven Johnson was inspired by the “zooming out” effect of their “Powers of Ten” movie.) The advertising has been rolled out slowly and in a way that doesn’t leave the deep fang marks of predatory marketing tactics seen on other news sites. It elegantly and clearly aligns all scales of information, from local bloggers to global news organizations. And it comes with an unofficial “media critic in residence”.

To mark the first anniversary of Outside.in Steven Johnson has published an essay called “The Pothole Paradox: Why Building The Geographic Web Is Hard, and Why It’s Worth Doing” on his blog.

At outside.in, we believe the answer is to build an information system modeled not after traditional newspapers or search engines, but rather the way that people intuitively think about the communities they live in. First, people have an extraordinary innate capacity for organizing their world spatially, which is precisely why pothole repair five blocks away is not interesting to us. And part of that spatial organization involves anchoring people and events in specific places. Think about the people you know socially, and the implicit place-based social networks that you carry around in your head: these are the folks I know from the local school, and these are the ones I know from the coffee shop, and these are the ones from my office…

I started writing this little essay in Brooklyn, but I’m finishing it in Barcelona. The day I arrived I wandered out into the Born neighborhood, a kind of Soho grafted onto a medieval street layout. There’s a distinctive feeling you get walking around a new city on your own — the guide books and review sites can tell you where the best restaurants and bars are, and give you the architectural history. But there’s always a feeling that you’re missing something, that the neighborhood is filled with another kind of data: all the debates and rumors and breaking news that make up the real social information of a community, from the true experts. Right now that layer is almost inaccessible to us — assuming we can’t always sit down and talk to an actual neighborhood maven. We can search a million servers scattered across the globe for a specific text string and get results within seconds. But we can’t do a search that tells us what people are saying about the street we’re currently standing on. It’s about time we changed that.

Steven Johnson

EMULATION PROVES SUCCESS

A new service called Your Street strips the functional components from Outside.in and wraps regular opportunistic word-association Google ads around the content. The difference is in character and depth of community. Readers and writers provide and burnish the links to stories through Outside.in and Your Street compiles the stories through RSS feeds.

CEO and founder James Nicholson says that what sets YourStreet apart is its extensive news service: the site collects 30,000 to 40,000 articles a day from more than 10,000 RSS feeds, mostly from community newspapers and blogs. “We’re not relying on the users to provide us with articles,” Nicholson says. The stories featured on the site aren’t of a specific type, and users will find the locations of murders marked alongside the locations of upcoming music shows. Stories featured on the site are teasers, and, if a user clicks to read further, she will be directed back to the source of the information.

Erica Naone. MIT Technology Review. November 9, 2007

Outside.in co-founder John Geraci is quoted in the Technology Review story. He says that Outside.in is focused on information and not maps because maps are interesting to people after information has drawn them in. Outside.in has the complex jumble of information that gives life its context. A life lived in search of meaning, not “meaning” as a definition that can be effectively pinned to an advertising category. The reliance on human input, not algorithms is what sets it apart from other local news portals.

Geraci says that Outside.in is built to rely heavily on human intervention, rather than on natural-language search algorithms, because, in his opinion, the algorithms don’t work well enough at this phase, and, with this type of service, stories are only useful if mapped accurately. “When you’re talking about location, there’s a low tolerance for noise,” Geraci says. “We believe you need people, that you always need that discernment.”

Technology Review

Your Street is underpinned by powerful algorithms.

The site’s main technological advance lies in its ability to mine geographical information from news stories. Using natural-language-processing algorithms developed in-house, as well as supplementary algorithms provided by the company MetaCarta, the site searches the text of regular news stories for clues about associated locations. The system searches particularly for entities within cities such as hospitals, schools, and sports stadiums, Nicholson says, relying on databases of entities created by the U.S. Geological Survey. YourStreet is currently working on some improvements to the system’s ability to recognize nicknames; for example, it should be able to interpret “GG Bridge,” as many bloggers refer to it, as the Golden Gate Bridge.

Technology Review

SEXY MAPS

Buzztracker is software that visualizes frequencies and relationships between locations appearing in global news coverage. Buzztracker tries to show you how interconnected the world is: big events in one area ripple to other areas across the globe. Connections between cities thousands of miles apart become apparent at a glance. Buzztracker currently only tracks English-language news sources.

Buzztracker.org website

Buzztracker.org is software that’s a version of land art on the internet. The founders describe it as an art project. “Buzztracker is our attempt at adding some depth and beauty to the experience of engaging the news.”

Outside.in’s buzzmap is dynamic, with bloggers represented in orange and traditional news media in gray. The circles dynamically grow and recede as news gathers and subsides around a particular location. “The idea of requiring geographic metadata for information might strike some people as excessive, but I suspect in a few years we will look back at the first decade of the web and be amazed that we went for so long without it,” wrote Steven Johnson on his blog.

YAHOO! NEWS

Way, way back in 2004 searchenginewatch.com reported on Yahoo!s news search setting itself up to compete with Google’s news search, and in a report a year later noted that the difference between the service is Google’s reliance on algorithms: Google says in its news blog, that it directs “…readers to the professionally-written articles and news sources our algorithms have determined are relevant for a topic.” On September 28 the Google news blog reported that media organizations already tied into Google News can augment the algorithms and submit news articles directly “…via News sitemaps in all the languages we support. You can also specify keywords for each article to tell us more about them so we can better place them in the appropriate news section.”

The Google News homepage is fusty. It looks like it was designed by algorithms to appeal to algorithms. Yahoo!s news page has a vaguely retro newspaper front page design with big images running alongside story excerpts and soft features — photo of the day, cartoons — running down the side, while Google has a list style design separating the news into categories and listing them with thumbnail images. The battle for the news search engines reminds me of the battle between Apple and Microsoft / IBM at the dawn of the computer age. Once computers were machines designed by machines to appeal to the logic and operating needs of machines. Microsoft and IBM took the high ground and their machines and software implied that the logic of machines and the needs of machines were paramount and design was irrelevant. It was a long way from IBM’s glory days of linking design and science. Apple was smart, sexy and human focused. We know how this story ended.

Yahoo! appeared to be sinking fast a few months ago, forcing founder Jerry Yang to come back to take control of the company. In September Yahoo! bought Buzztracker (which seems to have no connection with Buzztracker.org): In the “About” section on Buzztracker’s home page it says: “Our goal has been to launch a news site that leverages the power of the “head of the long tail” of the blogosphere to automatically generate news pages for a multitude of topics, both broad and narrow. We currently are tracking 1,000 topics and 90,000 blogs.”

Considering the froth, general factoids and celebrity gossip published below the toolbar (the online equivalent of “above the fold”) on Yahoo!s home page, the solid and serious tone of the Yahoo! News page is a pleasant surprise. The news is broken into categories, but there are tabs listing stories grouped by the media source from the Associated Press, Reuters, the Christian Science Monitor, McClatchy newspapers and NPR, among others. The OP-ED section links to The Nation, The Huffington Post and The Weekly Standard.

Yahoo executives told The Associated Press that The Columbus Dispatch and 16 regional newspapers owned by The New York Times Co. have joined the consortium, bringing its total number to about 415 dailies and another 140 weeklies. The New York Times itself, however, hasn’t joined. Lem Lloyd, who runs the consortium for Yahoo, said the partnership has already been bearing fruit both for newspaper publishers as well as Yahoo, but he declined to provide specific dollar figures.

Seth Sutel. Associated Press. 17 November, 2007.

While this sounds like a step forward for the year-old local news ad sharing project, it comes in the wake of wider disarray. Newspapers are based on an ethic of competition: competition for readers, for stories, and for advertising revenue. This competitive spirit was encouraged to drive papers to spend money on the improvement of content, to produce news most pertinent to its readers, and to keep newsstand prices as low as possible. In an era when the medium must integrate into the “new media” paradigm, the competition that once drove newspapers to improvement is only driving their integration projects into the ground.

Tim Conneally. Associated Press. November 19, 2007.

I suspect that the Yahoo! News homepage is the ground that the New York Times ceded while it was hiding its premium content behind a subscription wall. I still go to The New York Times homepage but I suspect that’s mostly because I’m sentimental about newspapers. If The New York Times and International Herald Tribune were a part of Yahoo!’s news service I’d probably go to their stories through the portal. Right now I can only select The New York Times as one of “My Sources” if the stories fall into the Yahoo! News categories. So now The New York Times faces a new conundrum, does it protect its brand by holding out? Or does it isolate itself further? The motto on the print edition of The New York Times is “all the news that’s fit to print.” On the website it’s “all the news that’s fit to remix.” The World Wide Web can sometimes seem like a massive covalent bond, held together by permalinks to New York Times articles posted in blogs and stories from other news organizations. The bringing down of the subscription wall also opened up the archive to The New York Times and provided permalinks for stories going back a couple of decades for stories that previously needed to be purchased. It’s a phenomenal resource.

YAHOO!S TREASURES

It’s self-evident that all successful online media businesses grow by acquiring companies as well as developing their own innovations. Yahoo! purchased two services I can no longer work without: the bookmarking site, de.licio.us, and the photo sharing website, Flickr. De.licio.us is now four years old. In 2004 Steven Johnson wrote a story about emerging web services for Discover magazine. Using an example of a poodle breeder being alerted to a new book, by Amazon.com, and posting a review on her blog, Steven Johnson follows a chain of information to de.licio.us.

Del.icio.us’s creators call the program a social bookmarking service, and one of its key functions is to connect people as readily as it connects data. When our poodle lover checks in on the dog-breeding tag, she notices that another del.icio.us user has been adding interesting links to the category over the past few months. She drops him an e-mail and invites him to join a small community of poodle lovers using Yahoo’s My Web service. From that point on, anytime she discovers a new poodle-related page, he’ll immediately receive a notification about it, along with the rest of her poodle community, either via e-mail or instant message.

Now stop and think about how different this chain of events is from the traditional Web mode of following simple links between static pages. One small piece of new information—a review of a book about poodles—flows through an entire system of reuse and appropriation within hours. The initial information value of the review remains: It’s an assessment of a new book, no different from the reviews that appear in traditional publications. But as it ventures through the food chain of the new Web, it takes on new forms of value: One service uses it to help evaluate the books with the most buzz; another uses it to build a classification schema for the entire Web; another uses it as a way of forming new communities of like-minded people. Some of this information exchange happens on traditional Web pages, but it also leaks out into other applications: e-mail clients, instant-messenger programs.

Steven Johnson. 10.24.2005. Discover.

It’s rare to find a story published online by a major media organization these days that doesn’t have a de.licio.us tag button in its “share this story” menu. And during the fire in Griffith Park that started on May 8 this year the Los Angeles Times ran links to photographs of the fire taken by residents and posted on Flickr. Citizen journalism is a great, ancillary resource but it doesn’t replace traditional journalism. On October 10 last year The Los Angeles Times ran an opinion piece by Susan D. Moeller, director of the International Center for Media and the Public Agenda at The University of Maryland, and Moises Main, the Editor-in-Chief of Foreign Policy Magazine.

OVER THE WEEKEND, at almost the same time that the world was informed that Google was vying to pay $1.65 billion for YouTube, a 2-year-old video-sharing website, famed Russian journalist Anna Politkovskaya was gunned down in Moscow. Politkovskaya covered human rights abuses in Chechnya. She was also a vocal critic of Russian President Vladimir V. Putin, and Russian authorities consider her death a political assassination.

YouTube’s acquisition and Politkovskaya’s killing are unrelated events. Yet both offer powerful clues about the forces shaping the way information is produced, distributed and consumed in today’s world. YouTube epitomizes “new media” — their immense potential and surprising effects. Politkovskaya represents “old media” — their literal struggle for survival and also their historical, indeed indispensable, value….

YouTube, Google, Flickr and many other websites offer valuable tools for keeping the world informed. But they are not a substitute for Politkovskaya and her colleagues. Societies are judged on how they treat their most vulnerable citizens. We suggest that added to that calculation should be whether journalists have been threatened, assaulted and killed. Tell us how many journalists were assassinated in your country last year, and we will tell you what kind of society you have.

Susan D. Moeller. Moises Naim. Los Angeles Times. 10 October, 2006.

Flickr and de.licio.us haven’t been integrated into the Yahoo! News website but it would be a bold and profound editorial decision to find some way of making them part of the reader’s experience and toolkit alongside the solid news resources they already have.