Scientists at the University of California (UC), Berkeley have successfully decoded, reconstructed and recorded movies from the human brain based on people’s “dynamic visual experiences.”
The researchers combined functional Magnetic Resonance Imaging (fMRI) with computer modeling to reconstruct movie trailers that people had previously viewed, paving the way for the eventual recording of internal images that currently nobody else can see, such as dreams, hallucinations, and memories.
“This is a major leap toward reconstructing internal imagery,” said Professor Jack Gallant, neuroscientist in a press release. “We are opening a window into the movies in our minds.”
Study lead author Shinji Nishimoto said in a press release that the natural experience of people is like “watching a movie.”
“In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences,” Nishimoto added.
Though far away, potential practical applications of the research include improving understanding of what goes on in the minds of stroke victims, coma patients and people with degenerative brain diseases. It could also allow for technology that would enable people with paralysis or cerebral palsy to control computers with their minds.
Nishimoto and two research team members participated in the experiment by watching sets of Hollywood movie trailers while fMRI was used to monitor blood flow through their visual cortex, the part of the brain that processes visual information.
Computer modeling was used to translate this information into a three-dimensional map of the brain constructed of tiny cubes called “voxels.” The model recorded how shape and motion information in a movie corresponded to brain activity.
Until now, research on reconstructing images using brain scans focused primarily on static images. This is due to the difficulty associated with reassembling moving images as blood flow signals measured using fMRI are slower than the neural signals that process dynamic information in movies.
“We addressed this problem by developing a two-stage model that separately describes the underlying neural population and blood flow signals,” Nishimoto said.
“We need to know how the brain works in naturalistic conditions,” he added. “For that, we need to first understand how the brain works while we are watching movies.”
The findings were published in the Sept. 22 edition of the online journal Current Biology and can be viewed here.
Friends Read Free