Have you ever wondered what really goes on in your mind while you sleep? Or wished you could clearly remember a fleeting dream? Imagine waking up and being able to see what you dreamt the night before, as if you had recorded a movie in your own mind. What until recently seemed like science fiction is now one step closer to reality thanks to a fascinating breakthrough in neuroscience and artificial intelligence.
A Japanese team at the ATR Computational Neuroscience Laboratory in Kyoto, led by Professor Yukiyasu Kamitani, has developed an experimental technology capable of capturing patterns of brain activity during sleep and reconstructing symbolic images of what we dream. They achieve this through a combination of functional magnetic resonance imaging (fMRI) and artificial intelligence.
Although the visual representations are still very basic, this achievement opens an unprecedented window into our understanding of the mind and the subconscious, promising to revolutionize the way we view dreams.
How do they “capture” dreams?
The process is an ingenious combination of neuroscience and artificial intelligence (AI). Scientists use a technique called functional magnetic resonance imaging (fMRI). Unlike a normal MRI that shows the structure of the brain, fMRI measures the small changes in blood flow that occur with brain activity. This allows researchers to see which areas of the brain are “on” at any given time.
How does MRI work and what is fMRI?
Magnetic Resonance Imaging (MRI) is a medical imaging technique that uses magnetic fields and radio waves to generate detailed images of internal organs, without the need for radiation. It is based on a physical principle: hydrogen atoms in the body respond to the magnetic field by aligning themselves; when the field is interrupted, they release energy that can be measured and transformed into computer images.Functional magnetic resonance imaging (fMRI) is a variant of this technology that makes it possible to observe brain activity in real time. It does not directly measure the activity of neurons, but rather changes in blood flow, through a signal called BOLD (Blood Oxygen Level Dependent). The idea is that active areas of the brain consume more oxygen, and fMRI detects these variations as brain “activation maps”.
How are the images captured? fMRI in detail
fMRI makes it possible to visualize the active areas of the brain, as shown in these maps of neuronal activation during REM sleep. These patterns are the key that AI interprets to reconstruct dream content. To achieve this feat, the scientists devised a meticulous protocol:
- Brain scanning during sleep Volunteers sleep inside an fMRI scanner, a kind of large magnetic tube, which measures the level of oxygenation in the blood (BOLD signal) as an indirect indicator of neuronal activity. As many REM (Rapid Eye Movement) phases as possible are recorded, since at that time dreams are most intense and vivid.
- Awakening and dream report As soon as the REM phase is detected, the volunteer is awakened to describe the dream. This process is repeated hundreds of times (more than 200 per participant), creating a robust database that links mental images with brain patterns, almost like a personalized dictionary for each dreamer.
- Additional recording during wakefulness Subjects then observe familiar images (such as everyday objects or scenes) while the fMRI scanner records the associated brain activity. These signals are crucial, as they serve to train the AI model with controlled, familiar examples.
The AI algorithm: from fMRI to dreamlike images
The real driver of this decoding lies in the sophistication of Artificial Intelligence algorithms. Machine learning models and, above all, deep neural networks (DNN) are used.
Training is done by mapping brain patterns observed during wakefulness (when volunteers look at real images and the fMRI records their brain) with the visual functions of a Deep Neural Network (DNN) that has already been ‘educated’ to recognize millions of real-world images. This is crucial: the AI first learns to ‘see’ how the brain represents familiar images.
When applied to data collected during sleep, the algorithm predicts which type of image (or visual category) matches the observed brain patterns. The predictions reach an astonishing 60% to 70% accuracy.
What machine learning models are used?
For this complex task, researchers use a combination of machine learning models:
- SVM (Support Vector Machines): This model is used to classify brain patterns. For example, it can learn to differentiate between when a person is dreaming of a “face” versus a “landscape”. SVM draws a line (or a surface) that separates the data into different categories, useful for determining to which category a brain pattern belongs.
- Linear Regression: When seeking to predict a continuous signal, such as the intensity of activation in a brain region associated with a visual object, linear regression is used. It is like drawing a line that best fits the observed points to predict new values.
- Deep Neural Networks (DNN): These networks, which attempt to simulate how the brain works, are the heart of decoding. They have many layers that extract features from the most basic (such as lines and edges) to the most complex (such as whole faces or scenes). In this experiment, a DNN already trained to recognize real-world images is critical. Brain signals are related to the internal representations of the network, allowing the system to “guess” what kind of image the subject saw (or dreamed).
- Generative Models (such as DGN or diffusion models): Once the content of the dream is guessed, generative networks can be used to create an approximate image. These networks not only recognize, but also know how to create new images based on learned patterns, which helps to visually shape the predictions.
Conceptual visual examples
Although the technology does not yet generate real photos, the results are symbolic or conceptual reconstructions:
- Fuzzy reconstructions: These are blurred images that suggest the presence of a person, object or landscape, rather than a photographic representation.
- Hierarchical simulation: This graph shows how the algorithm extracts visual features at different levels, from the most abstract (basic shapes in the initial layers of the DNN) to the most concrete and semantic (recognizable objects in the upper layers). It is as if the brain builds the dream layer by layer, and the AI decodes it in the same way. The algorithm associates brain patterns to visual representations at different levels (e.g., base to higher layers of a neural network).
- It represents the signal path: The brain produces an activity signal (captured by fMRI during sleep), this signal is translated into a ‘code’ or neural vector that is processed by the AI algorithm, and finally, the AI generates a reconstructed symbolic image that attempts to approximate the content of the dream.
- Generative networks: Some reconstructions use models such as DGN or diffusion to improve the visual coherence of the generated images.
🔬Anatomy of the technical process
| Phase | Detailed description |
| REM Phase | Recording just during REM phase to capture vivid dreams, when brain activity is similar to wakefulness. |
| BOLD signal | fMRI measures changes in blood oxygenation (BOLD signal) as a proxy or indirect indicator of neuronal activity in brain regions. |
| Database | Verbal descriptions of dreams are linked to corresponding fMRI activity patterns and, in later phases, to visual reference images. |
| IA training | The Deep Neural Network (DNN) correlates brain activity (recorded in both sleep and wakefulness) with hierarchical visual features learned from vast sets of images. |
| Reconstruction | A symbolic or conceptual image is generated by the algorithm, based on visual categories and features inferred from the brain’s sleep patterns. |
Quality and limitations.
It is important to understand that, for now, this technology has its limits. We are not dealing with a perfect “dream recorder”:
- Symbolic, not realistic: The technology infers the type of object or visual category (such as “face” or “car”), but does not generate a faithful and detailed image like a photograph or a video.
- Moderate accuracy: Predictions achieve remarkable, but still moderate accuracy (60%-70% correct), meaning that the algorithm does not always get it completely right.
- Personalized models: AI models are usually trained for each person, and do not generalize easily between individuals due to differences in brain activity.
- Unitary visual modality: At the moment, decoding focuses on visual information from dreams. It does not yet capture other elements such as sounds, smells, touch or emotions.
Beyond curiosity: What applications could it have?
While the idea of seeing our dreams is fascinating in itself, the true potential of this technology goes far beyond entertainment:
- Understanding mental health: By analyzing dream patterns and their relationship to mental imagery, we could gain a better understanding of conditions such as schizophrenia or post-traumatic stress disorder (PTSD). Could dreams reveal clues about a person’s mental state or how they process their traumas?
- More accurate diagnosis: This technology could offer an objective and novel tool for diagnosing neurological or psychiatric diseases based on how they manifest in dreams or the brain activity associated with imagination.
- New therapies: If we could better understand dreams and their content, new forms of therapy could be developed that use this information to help patients process emotions or memories more effectively.
- Brain-machine interfaces: This research lays the groundwork for future, more advanced brain-machine interfaces (BCIs), allowing, for example, people with severe paralysis to communicate or control devices simply by ‘thinking’ about images or concepts.
Ethical implications
As with any technology that delves into the innermost recesses of the human being, fundamental ethical issues arise that must be carefully debated and regulated:
- Mental privacy: Accessing a person’s dreams or thoughts can reveal extremely sensitive and private content, raising serious concerns about invasion of privacy.
- Consent: Ensuring informed and ethical consent from participants in subconscious research is essential.
- Regulation: Future commercial or clinical uses of this technology must be legally and ethically protected to prevent abuse or misinterpretation of brain data.
The future: A subconscious exposed?
Professor Yukiyasu Kamitani, leader of the study, explained that the breakthrough lies in being able to “identify the content of dreams” from brain activity, which is consistent with what the participants reported.
Currently, the images generated by the algorithm are simple, rather basic patterns or shapes that represent what was dreamt. They are not high-resolution recordings like a camera. However, the researchers are optimistic. With continued improvement of AI algorithms, increased data processing capacity and a deeper understanding of how the brain encodes visual information, it is very likely that in the future we will be able to visualize our dreams in much greater detail.
The question that remains up in the air is: will science be able to fully decode the human subconscious, or will dreams continue to hold a part of their mystery? What is certain is that, with each advance in neuroscience and artificial intelligence, the limits of what we thought possible are expanding.
What do you think of this fascinating breakthrough? Would you like to be able to see your own dreams on a screen, or do you think that certain aspects of the mind should remain undeciphered?
Bibliography:
https://neurotechjp.com/blog/kamitani-kyoto
https://www.open.edu/openlearn/body-mind/health/health-sciences/how-fmri-works
https://www.science.org/content/article/ai-re-creates-what-people-see-reading-their-brain-scans?
https://www.scientificamerican.com/article/ai-can-re-create-what-you-see-from-a-brain-scan/?
https://www.youtube.com/@ATRDNI (canal de youtube Kamitani Lab).
Translated with DeepL.com (free version).
