SMARTS Update/ DARPA, Narratives as Weapons, and You , Part II

Human beings need stories. We learn who we are, what we think, what we care about, not to mention what to be afraid of, from stories. My very first memory is rolling off a couch. Everybody screamed. Somebody was blamed. I have been telling that rudimentary story ever since I was old enough to talk. Another early memory is of my mother reading the story of Hansel and Gretel who laid a trail of breadcrumbs to find their way home. When I grew up, my job as a journalist required me to accumulate facts, secrets, insights, arguments, and arrange them like breadcrumbs to guide readers through a forest of possibilities.  It’s a wonderful life, hauling home the fixings for a narrative, and shaping it. It’s like making play dough sculptures, only better, because stories don’t break or gather dust. How and why I shape them as I do remains a mystery to me. How do I know which form will work best? How do I know what a reader will take away? I don’t. And for that, I am grateful. I like surprise. Besides, what, if anything, is done with my stories is up to readers, not me.

So: the fact that the greatest military power in history wants to reliably turn narratives into the equivalent of weapons of war keeps me awake at night.

As I reported last week, I first heard about DARPA’s Narrative Networks program back in 2012 when I was researching SMARTS. I learned that DARPA wanted to revolutionize what is known about effective story-telling; to measure, quantify and to predict human responses to narratives; and to use narratives to modify human behavior. There is nothing new about the desire to capture the hearts and minds. Political leaders, political parties, CEOs, generals, teachers, lawyers, parents, have always told stories to shape attitudes. Also,DARPA cast its motives in the best light possible-—soldiers and police dropped on hostile ground need to tell the right stories to get their opponents to stand down, not to mention to counter the dark stories spun by terrorists. 

But there is a difference between pushing stories and remodeling brains.

DARPA really wants to move far beyond the iffy statistical measurements of effectiveness offered by pollsters. DARPA wants  neurological certainties. 

DARPA has been working since the reign of President Richard Nixon (felled, you will recall, by the power of stories) on Brain-Computer Interfaces. This work began in the 1960s when cyberneticist Grey Walter used human EEG (electroencephalogram) signals to guide the operation of a slide projector.  Since 1974, DARPA-funded work has enabled alterations in the way neurons fire, using both implanted devices to send signals, and signals from non-invasive sources.  This work aimed in many directions at once, and each success led to more, up to and including restoration of signal transmission between muscles and brains after spinal cord injuries. Recent studies have shown that by monitoring brain, heart, and other signals, and feeding them back to the brain, soldiers can learn how to shoot twice as fast as with the usual methods.

Brains are made of cells---neurons--that chatter to each other via electrochemical signals. The transmission of signals back and forth from brains to sensory systems is how we learn, how we remember, how we move.  The way neurons signal, and the architectures of their relationships, generate representations of the world-- otherwise known as meaning. Neurons change the strength of their connections to each other: this is a reflection of experience over time. Deliberately changing connections between neurons has effects more long- lasting than showing someone a picture and asking how it makes them feel.

DARPA-funded work with Brain-Computer Interface devices has shown that it is possible to change, alter, and modulate neuronal connections.  In this way, memory functions can be restored to brains that have been damaged. People can learn to ‘feel’ with prosthetic hands. Some of these advances have spread already from the lab to the market: that’s why you can wave your arms and make your avatar behave, and steer through virtual environments.

DARPA is not shy about taking credit for its works. Yet when I inquired about what the Narrative Networks program has achieved, the media relations person at DARPA sent me back to where I started—-their website. When I pushed for more, he stopped responding.

That website told a very threadbare story. But by googling Justin Sanchez and William Casebeer, the successive managers of the Narrative Networks project, I came across a fascinating report on DARPA’s Brain-Computer Interface program which mentioned Narrative Networks. It was published in the Journal of Neuroscience Methods in 2014. The title is “DARPA-funded efforts in the development of novel brain-computer interface technologies.”  The lead author is Robbin A. Miranda. I can’t do its complexity justice here, so please read it yourself at www.ncbi.nlm.nih.gov/pubmed/25107852.

The revelations unfold, page after page. For example, something called the REPAIR program involves the use of light of different wavelengths to alter neuronal behavior, “modeling…the brain’s dynamic responses to perturbations of neural activity,” as the paper puts it. The point of REPAIR is to record neurons and alter their behavior at the same time. This work has already succeeded in mice, rats, and an “awake, behaving non-human primate.” Will humans be far behind?

The achievements of the Narrative Networks program are recounted near the end of the article.

 “Through an improved basic understanding of narrative effects,” says the report,  “ tools are being developed to detect brain activity associated with narrative influence and to emulate this activity in the context of larger environmental factors with models of narrative influence on individual and group behavior…One goal of the program is to create BCI technologies that close the loop between the story-writer and consumer, allowing neural responses to a narrative stimulus to dictate the story’s trajectory. In this way, moment-by-moment neural activity would drive the subsequent story outcome, resulting in an individualized narrative tailored by neural signatures associated with cognitive processes such as attention and empathy.”

I took two lessons away from this paragraph.

The first is this: DARPA hopes to strip the mystery out of story-telling entirely,constantly re-shaping them according to the feelings of “consumers.” 

Be careful what you wish for, DARPA. When I was a magazine editor, we learned, after many attempts to find out what readers wanted so we could give it to them, that this is a recipe for doom. We used information from polls and focus groups to find out what readers cared about. Yet when we gave readers what they said they wanted, they vanished in smoke. We figured out-- some of us-- that what they really wanted was something they didn’t ask for, and that was to be surprised,or, to be offered a picture of the Queen. Magazines with the Queen on the cover sold really well. 

You can’t know what will surprise before the fact, not with all the measurement devices in the world.

The second lesson I took home is this: DARPA is interested in hearts, not minds. It wants to know how to generate stories that will grab us by our emotions and drive us to behave as required. It wants stories that will turn people into robots.

This is what the report said about that: “N2 researchers have explored how narratives can reinforce in-group and out-group memberships and induce profound empathy gaps between members of these groups…Having detected a number of neural states associated with narrative influence, investigators are using this information to develop novel brain-in-the-loop systems to improve narrative creation and delivery.” 

The study they referred to involved playing videos with positive and negative characteristics to a group of human subjects, measuring their neural responses, and using that to predict their behavior. They wanted to know which story would induce the subjects to give money to charity, and which would not.

Apparently, they were able to predict the outcome with a fair degree of reliability.

The company that did this work calls itself Advanced Brain Monitoring. In this, the Edward Snowden era, such a moniker conjures up unpleasant thoughts. (They should put EEGs on some study subjects and run their name by those neurons.)

Learning how to reliably generate stories that provoke empathy gaps is a dangerous business. Besides, it is quite unnecessary. Humans are really good at this already. Last week, in Afghanistan, a female professor of religious studies who raised doubts about the value of amulets sold in a market, was beaten, kicked, run over, and then burned to death by a mob of men. Why? Someone made up a story that she had burned a Koran. The Nazi’s also employed empathy gap stories to brilliant effect. They didn’t need devices to tell them when they were on to a really good one, either. They could gauge that by the noise at their public rallies.

Perhaps, then, using machines to shape this kind of story-telling better is in no one’s interest.

As I mentioned last week, a really smart woman, Regina Dugan, was in charge of DARPA when the Narrative Networks program was set up.She gave a TED talk. What about unintended consequences, her interlocutor wanted to know. Do you worry about that?

She ducked the question.