Did you ever notice when you see a close up of Julia Roberts smiling on the big screen, you have an urge to smile back? That’s an effect of your brain’s “Mirror Rule” according to Dr. Jeff Zacks of Washington University.
Watching movies in a theater stimulates the signals in our brains more than almost any other activity.
Dr. Zacks investigates the various ways your brain is being manipulated while you are watching movies – including how propaganda movies embed into your memory more powerfully than books or any other medium. Plus – how “the talkies” changed us!
Phil Stieg: Hello I’d like to welcome Dr. Jeff Zacks, professor of psychology and Brain Sciences at Washington University. Today he is here to discuss his book Flicker – Your Brain on Movies. Filmmakers might not know the Neurosciences around movies and brain function, but they have a tremendous understanding of perception, cognition, and emotion related to movies. Let’s learn how movies can be more powerful than the written word in creating our memories. How does propaganda in films work and how producers take advantage of our brains to achieve this? Finally, we will learn where our expert thinks movies are going. Jeff, thanks for being with us today.
Jeff Zacks: It’s a real pleasure. Phil, thanks for having me.
Phil Stieg: When my producers brought this idea up, I was just ecstatic. You probably don’t know this, but I’m a movie buff / addict. I pride myself in picking up the little editorial cuts that get missed, the pocket square that was in, and then it was out, on the next scene.
Jeff Zacks: Continuity errors are a tremendous source of entertainment for those aficionados among us.
Phil Stieg: So I have to ask you, before we start talking about the meat of the matter, is were you a movie buff first or a neuroscientist first? What happened?
Jeff Zacks: So, we’re all movie buffs, right? This is the 21st century, and we as a world culture spend billions and billions on filmed entertainment. And I grew up in the thick of that. And I think even more than most, I was a fan of the movies.
I didn’t really think much about movies technically until after I started as a cognitive neuroscientist. And in cognitive neuroscience, we’re interested in how the mind works, how people think, perceive, remember, talk, decide, judge, control action, and how the brain enables those functions. And in my laboratory, we got interested in how people understand complex every day, dynamic activity. And so to give people experiences that were complex and naturalistic. But repeatable, we turned to movies. And we started filming like the world’s worst, least entertaining movies and showing them in the lab. And my idea was that we would make “anti-movies”, that we would film from a fixed head height perspective with no editing, so I wouldn’t have to think about cinematography per se. But of course, I just described a particular viewpoint and a particular editing choice. So even if you’re going to try that route, you’re just thrust into it. And we quickly realize that there are just so many interesting things going on. And we started looking at the dynamics of brain activity in relation to what filmmakers have access to. I just realized there’s way too many interesting questions.
Phil Stieg: I’m curious about the nuts and bolts of what you do. What techniques do you use to assess how I might respond to a movie?
Jeff Zacks: I would say the most important techniques are behavioral measures. We ask people to make judgments while they watch movies. For example, we can ask people about how tense or relaxed things feel moment by moment. We can ask people afterward, what do they remember? We can show them pictures and ask them, Was this picture a frame from the movie or a similar picture that you actually never saw. And we can measure their eye movements. So using high-rate infrared cameras and computers, we can track where you’re looking while you’re watching. All of that stuff can be super informative. And then when we go to measure brain activity with functional MRI, we’re in a much stronger position to understand what is the significance of that brain activity.
Phil Stieg: Let’s face it, evolution. Our brains have been developing for hundreds of millions of years. Movies have been around for maybe 100 or so. Our brains weren’t developed for movies, as you’ve said, multiple times in your book. But then you talk about some of the things that the brain does. This mirror rule. Tell us how that works.
Jeff Zacks: Sure. Sure. The mirror rule says that other things being equal, if you see somebody doing something, you might want a prep a response to do that same thing. So for example, if you see someone’s face 20ft tall on the screen, smiling or frowning or crying, you may notice your own face forming itself into that same pose. If you turn and look at the people sitting next to you, you may notice that as well. And if you see an actor flinching from an approaching object, even if the object itself isn’t depicted on the screen, you may find yourself flinching. If you look at little kids watching movies, they’ll be fully up and out of their seats, jumping around, performing the actions that they’re seeing. And some of that is this fast, automatic processing.
Phil Stieg: I’m only going to ask you one kind of “science-y” question here, because otherwise I think it would be too deep. But can you tell us what parts of the brain are tweaked say with mirroring, emotion and memory.
Jeff Zacks: Sure, let me start by saying however you want to measure the strength of the signals being generated. There’s no stimulus that I’ve encountered that drives bigger signals than commercial cinema. If we want to make somebody upset or happy in the scanner, the quickest, most efficient way to do it is to show them a movie. If we want to drive big signals in the brain, the most powerful way to do it is with movies. Now, having said that, the structure of that that activity is highly structured. So you find areas, including the amygdala and the orbit of frontal cortex. These are parts of the brain that we know are important for adaptively processing emotion in the real world and performing emotional memories. They’re highly activated at appropriate moments when perceiving emotional cinema. The visual parts of the brain show massive signals rising and falling rapidly in response to changes in the visual information. And you see activity in the motor cortex that is time-locked to actions that people are observing. And that’s related to this mirror rule that we talked about before. So that’s just a few of the big effects that you see.
Phil Stieg: Thinking about action movies, what are these directors doing to suck me into that action movie other than just cars blowing up and guns going off and loud noise and intense music? I got that.
Jeff Zacks: That’s not enough?
Phil Stieg: But there’s something they’re doing with the camera. All right, I’m being sucked in. Let’s enjoy the ride.
Jeff Zacks: Yeah. So sound effects and motion are intrinsically attentionally engaging, Anything that’s a transient signal in your environment, your perceptual systems will tend to Orient towards. So, like, if something starts moving, your brain will Orient toward that. If there’s a new sound, your brain will orient toward that. So in an action movie, you’ve got that stuff happening at a really high rate. And that helps, too.
That’s tightly integrated with the way those sequences are edited. And the editing can do things like direct your attention spatially. So if I have an onset, like a new object appearing or something exploding or something starting to move on one side of the screen, then I’ll orient there. And if you have more editing, I’ll reorient more toward the middle of the screen more frequently. And so all of these things the director and the editor can use to control my attention over time and to reengage my attention when it might be flagging.
If you track people’s eyes and you compare tightly edited sequences to more static sequences where you have a camera pointed at a scene and the action unfolding. That kind of filmmaking allows different people to explore the scene differently, whereas if you cut frequently and you keep things in close up, everybody’s going to be looking in the same place at the same time.
One of the striking things about editing is how invisible it is. A demonstration that I often do when I give talks is to show an action sequence and then ask the audience to estimate how many edits there were in the 30 second clip that I showed them. And typically people underestimate by about a third. But it can be even higher in some situations.
One powerful mechanism of that missing of edits. What Smith and Henderson have called edit blindness is if the edit is followed by a big transient signal. So, for example, if you have an edit and then an explosion or an edit and then a car driving in front of the scene, that dramatically reduces the probability that you’ll notice that there was an edit. And even when we deliberately monitor and try to detect edits as they’re happening, it’s kind of surprising how many you can miss.
Phil Stieg: Earlier, you referred a little bit to this concept of bottlenecking, where you get so much information that your brain can only process. What did you say? Most people can remember at most seven things and more likely to be four to five. So if you overload them, you can get them to focus on what you really want them to focus on.
Jeff Zacks: Yeah. Well, I would say it’s not even a matter of needing to overload them. It’s just that the world is an overload. The world is more than our central processors can deal with by orders of magnitude. And part of what makes us so smart is our ability to just toss the vast majority of what’s impinging on our sense organs and retain the good stuff or retain what we need. The stuff that we’re holding on to isn’t the fine details of the pixels or of the sound waves. It’s not the fine details of the colors and the shapes. It’s like these are the objects and these are the people. And here’s what they’re doing. And here’s why.
Phil Stieg: I was fascinated also that you suggest and have proof to show that memories generated by reading a book are managed differently than memories managed by watching a movie. And in fact, you suggest that movies are more powerful in terms of our memory retention. Am I correct in that?
Jeff Zacks: I wouldn’t say that movies are more or less powerful than books. What I would say is that the stimulus itself affords very different things when you talk about reading a story versus watching a story. So on the front end, you’re looking at a relatively visually homogeneous set of characters on a page. You’re looking at a static stimulus that changes only when you advance the page or turn it if it’s paper. And in the case of a movie, you’re looking at something that is changing rapidly over time and also is multimodal, includes visual and auditory information. In the face of that, what’s really striking is how much commonality you see across those two.
So you find these time-locked responses in the auditory cortex, and time-locked responses in areas of the brain that process spatial information that are very similar, independent of whether it came from a movie or from books.
Phil Stieg: So if the brain responds in a similar fashion to both books and movies – but just more powerfully to the movie form – is that how those propaganda films worked – like Frank Capra’s “Why We Fight Series” that you cite in your book?
Jeff Zacks: In the Capra movies, there were actually psychologists on the team, and they found that G.I.s who were quite skeptical of these training movies right after they saw them nonetheless accepted as fact information from those. If you just waited a few weeks, they didn’t distinguish well between information that they’ve gotten from the movies and information they’ve gotten from their buddies or from the newspaper. And so propagandists have these powerful tools. They’re well aware that if you repeat something often enough, I am quite likely to accept it more, in part because of the affective response associated with it, and in part because I may not remember to be able to distinguish whether that was repeated from a credible or non-credible source. So what can we do as consumers? Well, we can keep track of our sources as well as we can, and that often requires doing a little extra work, maybe taking some notes, maybe doing a little bit of extra research. These days, they’re doing a little bit of extra research part is just so much easier than it was for those GIs back in the 40s.
(Interstitial Theme Music)
Narrator: Nearly a century ago, these words galvanized the movie going public:
Excerpt from the Jazz Singer
“Wait a minute … wait a minute – you ain’t heard nothing yet!…”
Narrator: By combining Al Jolson – America’s number one singing sensation – with brand new technology from Western Electric called “Vitaphone”, Warner Brothers created their now famous blockbuster hit “The Jazz Singer”.
Jazz Singer excerpt:
“toot-toot-tootsie, don’t cry…”
Narrator: But not everyone welcomed the introduction of talking pictures. Gossip columnist Louella Parsons described the Jazz Singer as a passing fad saying, “I have no fear that the screeching sound film will ever disturb our theaters,” while Irving Thalberg, head of production at MGM dismissed the film as “a good gimmick, but that’s all it was.”
To be fair, many of the early sound films varied between technically inept and downright dreadful.
Movie trailer excerpt
Ladies and gentlemen, I am privileged to say a few words to you in this most modern and novel manner. This is the first living Vitaphone announcement ever made announcing the coming of ……
Narrator: Noisy film cameras had to be locked away in sound proofed chambers described looking like giant ice boxes, forcing the actors to restrict their movements and stand as close as possible to the giant-sized microphones. Many performers and producers resisted the change as long as they could.
In his 1931 masterpiece, City Lights, (one of the last feature films released as a silent) Charlie Chaplin included a brief sound scene mocking the low fidelity of the western electric system by dubbing the sound of a politician’s speech — using a kazoo!
Excerpt from “City Lights”
Zzz-zzz- zzzzz zzzz! …
Narrator: Historian Robert Sklar in his classic book “Movie Made America” documented an interesting change in how the audience responded to movies with the coming of the talking picture. In the silent film era it was considered acceptable to talk during the movies. Audience members would share their reactions to the story on the screen, and bond with their friends over the experience. With talkies, however, audiences had to be quiet to listen to the dialog coming from the screen … losing that sense of camaraderie and substantially changing the movie going experience.
As Sklar ironically observed – “talking audiences for silent pictures became silent audiences for talking pictures”.
Phil Stieg: I can’t imagine that high-end producers and movie directors haven’t studied what you are generating from a data perspective, or they just understand what sells tickets through experience. They’ve looked at thousands of movies and learned empirically rather than scientifically.
Jeff Zacks: This is what I think of as the “Moneyball” issue. Right. So people like me are not primarily focused on how to sell more tickets. There are lots of tickets being sold. There are people who are really good at that, and that’s just not a focus of our research. The other piece of the puzzle is that people who really are experts and who have great intuitive experience in making and promoting movies often feel disconnected from quantitative research. And so to them, even if there might be value there, if there might be applied research that really would be directly relevant for totally legitimate reasons, lots of folks in the business just don’t see that as the most productive route to their end.
Phil Stieg: Somehow I just can’t imagine Scorsese not just going underneath that rock and trying to find out what’s going to tweak me.
Jeff Zacks: And I can tell you one of the most fun, rewarding aspects of having drifted into this line of work is the calls and emails I get from time to time from filmmakers. I’ve had the opportunity to interact with some folks whose work I respect just tremendously and love, and that’s been super rewarding. So if there are any filmmakers out there listening to this, feel free to get in touch. I always love to talk to artists.
Phil Stieg: And speaking of Scorsese, you gave a number of examples in the book also about the violence in the movie Taxi Driver and the obvious effect that that must have had on John Hinkley and his emotions What’s going on there?
Jeff Zacks: So for those listeners who might not be familiar with this case, Hinckley was a mentally ill person who shot President Reagan and his press Secretary. And in the time leading up to that, he repeatedly watched the film Taxi Driver, which features Robert De Niro as a violent, vigilante character. And the first thing to say about that is that all of our actions are caused by multiple, multiple causes. And so it’s not like watching one movie is going to turn someone into a mentally ill, homicidal person.
Particular events that we experience in our lives may influence us. And whether they’re in film or in real life, experiences are going to serve as triggers, serve as cues, unmask ideas. And we learn things socially. People get ideas of behaviors that wouldn’t have occurred to them to engage in, but they might read about it or learn about it from the news or see it, and that changes their behavior. So all of these things do happen. And it’s also the case that exposure to violence chronically has modest but measurable influences on people’s behavior.
The consensus in the field at this point is that chronic and acute exposure to violent media can increase aggression in people. And so the chronic piece is very difficult to study causally. Right. Because you can’t just lock somebody in a room and make them watch Taxi driver over and over again for 20 years and then look to see how they behave. But you can look at it correlationally with very thoughtful statistical controls from lots of angles. And there seems to be a small but reliable association. And then you can do acute things, you can do experiments. And again, you can’t get too extreme about this. Right. It’s not quite ethical to do experiments in which we deliberately provoke people to violence, but we can put people in situations where they have the opportunity to behave nicely or behave a little more aggressively, and we can see acute effects of watching violent media. So together those things tell a pretty compelling story.
Phil Stieg: You equated movie regulation with regulation of nutrition. I was curious about that analogy and what your thoughts are. Obviously, when movies first came out, there is an incredible governance in terms of what was decent and indecent. And now that obviously has changed considerably. What are your thoughts?
Jeff Zacks: For me, the analogy is like, no, don’t take away my Full Metal Jacket, my Quentin Tarantino. Those are important parts of my mental life, just like I would say I live in St. Louis. So don’t take away my Ted Drew’s custard. But if I were eating custard every day, three meals, that would not lead to good outcomes for me. And I think when we think about our media diets, we want to think about a very healthy diet that’s not all loaded up on empty sugar.
Phil Stieg: As an educator and a scientist, do you think that there should be more media training in grade school and high school to teach people how to selectively watch things and what to carry away from it?
Jeff Zacks: Absolutely. I think these initiatives to improve media literacy are super important to get people to, especially kids as they’re growing up and their parents, to understand where these stories and other media are coming from and to interpret them in light of where they’re coming from. The organization Common Sense Media does a fantastic job helping guide families through media choices. They’re not doctrine error, and they take age and development and situation into account really well in their guidance. And I think those kinds of resources are super valuable.
Phil Stieg: Finally, where is this all going? 3D versus stereoscopic, sound technology, vibrating seats. I found it hard to believe the smell is going to affect my perception of a movie. I know it would, but probably not in a positive way. And then I dread the day that I’m going to have to go and do a video game to interact with my movie. I like to sit back and be entertained. So what’s all happening?
Jeff Zacks: You know, display technology for presenting immersive environments and allowing people to control their viewpoint more. That technology is becoming increasingly ubiquitous and increasingly high fidelity. Now a simple version of that is just having a really big screen with a really wide field of view. Right. So if you go to an Omnimax theater, you have much control over your viewpoint because the screen is much bigger than your field of view in that case. And so you really are looking around in order to control your viewpoint. If you’re wearing a goggle display that tracks your head and updates your view based on where you’re looking, that’s an even more extreme version of that. So those technologies are going to continue to evolve.
The coming together of narratively oriented gaming and what we think of as movies and TV is continuing to happen. And one of the really interesting questions there for me is how will our culture and our psychology adapt to having shared experiences that are not quite shared? Because everybody makes slightly different choices in whatever you want to call it, the game or the interactive movie. So I think there’s lots of interesting questions there.
Stereoscopic displays that present really vivid depth cues seem to keep like having a Renaissance and then dying away again over and over and over again. I think that’s really interesting. These augmentations – like shaking the seats or blowing air at you. There was a big push a few years ago that seems to be petering out. I’m sure it’ll be back.
I end the book with some kind of more speculative stuff about there’s lots of wonderful science fiction about jacking stories directly into our brains. And what’s the scientific, what’s the neuroscience that corresponds to that. And is that ever something that could be recreationally relevant? And that, I think, remains to be seen.
Phil Stieg: That’s certainly not my lifetime. At least. I think I’m happy about that as well. I don’t want a probe on my head nor invasively put in to help me appreciate a good movie.
Dr. Jeffrey Zacks, I don’t know if I’ll ever watch a movie in the same way. I hope that I can continue to enjoy movies and not think about how I’m being manipulated and how my mind is being played. But thank you for enlightening us all on the techniques that directors and producers use to create the emotions that we experience at a movie. Thanks so much for being with us.
Jeff Zacks: Thanks for having me.
Additional Resources
Dr. Jeff Zacks Bio