Artificial intelligence is ushering in a new era of mind reading, with advanced brain scans revealing much of what we’re thinking about. Dr. Marvin Chun, Professor of Psychology and Neuroscience at Yale, explains how researchers in his lab are using fMRI and other new technologies to see what’s going on in your brain, even as you sleep. This holds great promise for those in persistent vegetative states, even as it raises ethical questions about just who gets to read your mind.
Dr. Stieg: I’m really excited about our topic of mind reading today because it seems right out of sci-fi movies. In fact, as our esteemed guest, Dr. Marvin Chun will explain, it is now possible to read people’s minds with a startling degree of accuracy. Dr. Chun is a cognitive neuroscientist who wears many hats at Yale University. He is the Richard M. Colgate Professor of Psychology and Dean of Students at Yale. He also teaches in the department of neuroscience at Yale University School of Medicine. Dr Chun’s cognitive neuroscience lab is pioneering the use of brain imaging and machine learning to study how people see, attend, remember, and perform. Marvin, thanks so much for being with me.
Dr. Chun: I’m happy to be here.
Dr. Stieg: In reading through your papers and watching your TED talks, it’s obvious that you’ve really focused on the utility of functional MRI in terms of determining what’s going on in our brains. Can you, number one, describe what a functional MRI is?
Dr. Chun: Yes. FMRI is a technology that allows us to study what’s happening in the brain while people do different kinds of tasks. It is based on the same MRI technology that is very commonly used all around the world and hospitals to scan different structures and abnormalities within the body, and it can do so noninvasively that is safely. The adaptation that allows us to study brain activity is something called functional MRI and basically what it does is it, it allows us to measure correlates of which parts of the brain are more active when people do different tasks. The way I like to think about it, if MRI is like looking at a map of a city, it’s like looking at structures of the brain. What FMRI allows us to do, it’s like looking at traffic on those maps. It tells us where there is more activity on these different brain areas.
Dr. Stieg: And we’ll get into it through example, but you truly believe that we will at some point be able to accurately read people’s minds as a result of this technology.
Dr. Chun: I do believe there will come a day when we can read people’s minds. However, I’m not sure if FMRI is the technology that will allow us to read it as accurately as we would like. FMRI has some limitations that are physical and physiological. Basically they don’t allow us to measure neuronal activity at a very precise way, so I’m not sure we would be able to read people’s minds very accurately using FMRI alone. That said, the kind of progress that my lab and other labs have made, it’s still pretty exciting and promising.
Dr. Stieg: What’s amazing to me is that I look at your functional MRI studies and it tells me that a certain activity is there, but then I have a patient that comes in with a brain injury or a tumor in that area and they still have that function and want to emphasize the brain’s adaptability to certainly to chronic injury, but even sometimes to acute injury like head trauma.
Dr. Chun: Again, as you know better, one of the most remarkable traits of the brain is that it is plastic. It is resilient. It allows for distribution of its processing, so when you have a tumor or acute brain damage with time, the brain can rewire itself to an extent to make up for the lost function.
Dr. Stieg: I’m so glad you said that. Cause most people after they’ve had a stroke or something like that, they feel like I’m doomed. It’s over. And they’d need to realize that they are going to get better.
Dr. Stieg: Your work looks at the difference in the way we see a scene versus the way we see a person’s face. And this activates different parts of the brain. What’s the functional and what’s the structural significance of that in terms of our individual brains.
Chun: 04:06 So fMRI is this remarkable technology that only started to be developed in the 1990s and during that first decade, which is when I entered the field, a lot of our efforts were devoted to mapping different brain functions to different brain areas. And one of the most robust forms of this specialization has been seeing processing in the scene area and face processing in a separate face area of the brain. And the fact that they mapped onto different parts of the brain allowed researchers to use that as their first hook into doing some rudimentary mind reading. A very famous paper by Nancy Kannwischer at MIT, who was one of my mentors, she and her colleagues demonstrated that just by imagining a face, you’ll see activity in the face area and just by imagining a scene you’ll see activity in the scene area and because these brain areas were distinct from each other, it allowed her to guess what people were thinking just based on fMRI data alone.
Dr. Chun: So that’s the importance of functional activity versus the structural location and mapping onto the brain. The complication is that not all brain functions equally map onto different brain areas. Scene processing and face crossing are pretty fundamental forms of visual perception and so they happen to have evolved at different brain areas, but most brain functions, let’s say you’re looking at a cat versus looking at a dog, versus looking at a chair that’s not going to really elicit notable differences in brain areas, and so we need more sophisticated techniques like machine learning, artificial intelligence techniques to decode that level of brain activity differences.
Dr. Stieg: It reminds me a little bit about that book, The Man Who Mistook His Wife For a Hat. What’s going on in our brains that allows that to happen? And, you know, where it happens.
Dr. Chun: In fact, famously, Dr. Oliver Sacks actually had face blindness — prosopagnosia. He had deficits in his ability to recognize faces and recognize friends from strangers and that is likely due to some dysfunction in the part of the brain that processes faces and then we have a safe area and then the more extreme cases that leads to those kinds of intriguing patient cases.
Dr. Stieg: So the parts of our brain that perceives shoes and dogs and cats and things like that… I presume then, that there via the human connectome are these neuro networks is how we can then intellectually conceptualize something as being a shoe or a cat, correct.
Dr. Chun: That’s right. It really emerges out of a pattern of activity rather than specific activity on one particular brain area per se.
Dr. Stieg: I’m also interested in the concept of the persistent vegetative state where the patient is unable to communicate with us other than through yes or no questions. Have you learned anything with functional MRI in regard to how the brain works in that patient population?
Dr. Chun: So this would be work not from my lab, but from Adrian Owen’s lab back when he was in Cambridge, England. He published a landmark paper, one of the most exciting papers for me in 2006 published in science where basically he had the brilliant idea of putting patients in persistent vegetative state into brain scanners. And of course, by definition they are not responsive to any verbal prompting or questions. However, what he found was that when in the scanner, uh, when he asked these patients different kinds of questions that were designed to elicit different patterns of brain activity, he found that their brains were responsive to these different kinds of questions, suggesting that they were indeed conscious and not only conscious, but they were able to answer yes/no like questions by pure thought.
Dr. Stieg: But we don’t really have any understanding. I mean the concern is that it’s only yes or no. We don’t really at this point know whether they have higher function. Meaning they can emote, they can sense and they can rationalize and show judgment in relation to whatever you’re questioning them on. And the next thing we’ll be able to do is put a microchip in and hopefully link their brain up to a computer so they can communicate with us more effectively.
Dr. Chun: That’s the future. Yeah.
Dr. Stieg: In that regard, you’ve combined computer vision and computer technology with the application of functional MRI to get more specific information about how our brain works. Can you discuss that briefly?
Dr. Chun: A study, uh, that I’m very proud of from my lab was conceived by a undergraduate student, Alan Cohen and a postdoc. Three of us did a study published in 2014 where we were able to decode what faces people were looking at. So the way we did this study with, we put people in scanners in the fMRI scanner and then we measured their brain activity while they were looking at different face stimuli of, either real pictures of real faces. And we showed them one by one and, basically, Alan and Bryce, they developed an algorithm that allowed us to convert the fMRI responses to actual drawings on computer renderings of the faces that we thought that the computer was guessing that people were looking at in the scanner.
Dr. Stieg: Fascinating. And in addition to that, I understand that there’s been some work where you’re able to determine what video somebody’s actually looking at.
Dr. Chun: That’s work actually by Jack Gallant at UC Berkeley. It was pioneering work published in 2011 where he showed people videos and basically did the same thing. He measured fMRI activity. He used AI to learn the relationship between the brain activity and the particular video being shown. And then based on brain activity alone, he could guess what kind of videos people were looking at. That was published in 2011, and in fact it was the inspiration for the study that we did. We were, Alan Cohen, looking at that study said, “Oh my gosh, wouldn’t it be cool if we tried to extend this to draw human faces based on what people were looking at in the scanner,” and that’s what he did three years later.
Dr. Stieg: Even more frightening for me is your ability now to put a person in a scanner, let them sleep, and you can actually determine what they’re dreaming about?
Dr. Chun: That’s correct. This is all you know, work done around the world. That particular study of dream reading was produced by a group in Japan and they trained people to fall asleep in the scanner and then people are able to determine when someone is dreaming or not, and just based on the scan of their brain, while they were dreaming, they were able to make a pretty good guess of what the participants were dreaming about.
Dr. Stieg: You indicated that, you through the studies, were able to determine which people that were in prison would have a higher rate of recidivism in this day and age. When I’m in New York city now they’re decided for economic reasons that they would like to release people. This is concerning. What can you tell us about your ability to do that?
Dr. Chun: I was citing work by Ken Keel(?) who studies prisoners. He actually scanned them in the prisons. He has a mobile scanner that he wheels into the prisons to study incarcerated individuals. And what he found was that, based on the scans that he had conducted on them in the prison, he was able to predict the likelihood of their coming back to prison after they were paroled. It is somewhat frightening. It may raise ethical concern, but one way to think about it is, you know, this is something that parole officers do all the time based on behavior, based on other types of assessments. And so the hope is that if this technology is developed, it will allow for a more accurate, prediction of recidivism or rehabilitation into society.
Dr. Stieg: So that’s always the question that arises when I read through these papers in nature, neuroscience or whatever about functional MRI. What’s the n, are we talking about two people? Are we talking about a hundred people? What reaches the level of scientific significance?
Dr. Chun: That’s a great question. And, and it really taps into it. Big evolution in the field of fMRI research. So certainly in the first 10-20 years maybe a lot of the research was based on averaging data across 10, 20, 30, 40, 50 people. You were not really able to say something about an individual because you had to rely on what is essentially averaged data, but what are the most exciting trends and developments in brain imaging research in the past 10 years? It’s something that we call individualized or personalized fMRI. You know, just like you have personalized medicine. The trend in the field right now is to be able to say something about specific individuals, can we read this individual thought, can we read this individual’s perceptions, can we make predictions about this individual’s future behavior? That is definitely the direction that the field is heading towards.
Dr. Stieg: Speaking about this, the topic of unconscious prejudice comes up and do you envision the day when I get called in for jury duty and they give me a functional MRI study to determine whether I have some underlying prejudice that’s going to work to one person’s favor versus another’s?
Dr. Chun: it is actually already possible to measure unconscious prejudice. But it begs the question of, uh, you know, to the extent that people have control over their unconscious prejudices in many situations. For example, if you ask someone, consciously, are you prejudiced against a person? You know, people will report their conscious thoughts usually masking or overriding whatever prejudices they may have that are unconscious. And so to the extent that people have control over their thoughts and behaviors and judgment, we’re not sure whether a measurement of unconscious bias is necessary or has validity. Again, as long as people can consciously control their bias.
Dr. Stieg: As an individual who’s obviously thought about this, you’re living it, you’re studying it on a day in and day out basis. You alluded to it a little bit earlier about the ethical conundrum that this is going to give rise to from a legal standpoint. What are your thoughts? I mean is this going to be a good thing being able to predict somebody’s thought processes, being able to determine what their unconscious prejudice is? How, as a society, will this have impact?
Dr. Chun: I definitely like to think about these ethical concerns and in fact a course that I teach at Yale is something called neuroethics where we look at the implications for society based on these advances that are being made in, in brain imaging and in neuroscience in general. That’s that I’m not too concerned because I believe that the benefits far outweigh the potential misuse and abuse of these technologies. First, with regards to mind reading, we are protected by the Fifth Amendment, at least in this country, and so a person can always decline going into the scanner, or a court can block the use of such evidence of pain without the person’s consent. So I’m not too worried about misuse of the technology. At least in democratic societies like the United States. The benefits are huge. By being able to predict and quantify, for example, ADHD or depression or anxiety. The benefits of that to be able to make more accurate measurements of these mental illnesses or abnormal states I think was going to be very important for clinical practice. Another domain that I’m very excited about is quantifying and measuring concussion state. These will, I think, have huge clinical benefits that far outweigh the potential misuse and concerns that one can have about the technology.
Dr. Stieg: Marvin, I want to thank you again for taking the time to explain to me the utility of functional MRI and how this is going to help us understand the brain. I’m sure you do as do I believe that this will transform the way we are able to treat our patients and be able to teach us a little bit more about how the human brain works and makes ethical decisions. Thank you so much for being with me.
Dr. Chun: Thank you!