Smarts Update. Why Journalists Will Earn Less Than Two Cents a Word and a Pulitzer Prize Could Go to an algorithm

The one thing I never imagined when I became a freelance journalist in the late 1970s, is that one day, someday, a smart machine might take my job. 

Back then, only a few humans were supposed to have the ability to find a good story and write it down. While it was understood that some animals are kind of clever, none, so far as I knew, went out into the world with a reporter’s notebook and came back with well written copy. Okay, we knew that dolphins and whales sing to each other, possibly about the big events in their lives, but that wasn’t the same as Woodward and Bernstein bringing down Richard Nixon. Story-telling was high on the list of unique human attributes, right up there with belief in God and knowledge of the facts of death.  Journalism was considered to be a craft requiring years of hard training by a well educated, possibly talented human being, and, when practiced by some, an art form.

However, as doing the research for Smarts has taught me, smartness has broken through all the old boundaries that used to divide us from animals, plants, slime molds and machines. About ten years ago, journalism professors and computer scientists teamed up to create algorithms (instructions) enabling computers to trawl through heaps of data to find patterns and turn it into prose.   Companies like Narrative Science (their story- spinning software is called Quill) and Automated Insights (theirs is called Wordsmith) have since partnered with publishers like Forbes and the renowned journalism cooperative, the Associated Press, to write stories involving Big Data. The software can express raw sports scores and financial reports in something they call Natural Language, meaning sentences employing active verbs describing victory, defeat, humiliation, growth, shrinkage, merger, and looming bankruptcy. According to a March 8 opinion piece in the New York Times (“If An Algorithm Wrote This, How Would You Even Know?” by Shelley Podolny, a director of a Search company called H5), the AP produces 3,000 financial reports per quarter using Automated Insights’ Wordsmith. Forbes uses Narrative Science’s Quill for writing various stories. These are allegedly almost impossible to distinguish from those written by human hacks. How do we know? Studies have been done. 

But here’s the thing that matters more than whether the prose is good or bad. Back in 2011, according to a story by Steve Lohr, also published by the New York Times, the cost per 500 words for such stories, produced within seconds of data collection, had already fallen below $10. That’s right, just two cents per word and nobody has to wait for the journalist to come back from lunch. Costs will continue to shrink as more publishers fire their sports and financial writers and use software instead.  And lest you think that a quarterly report or a sports story is all that such systems will do, be advised that one of the inventors, journalism and computer science professor Kris Hammond, aims at big investigative stories done faster and better than human journalists.  In 2011, he predicted that within five years, Narrative Science’s software would win a Pulitzer. 

That’s next year.

If you think that’s scary, come back to this site next week and I’ll tell you about DARPA.