How Does the Brain Represent Meaning?
When you see an image or recall a memory, something remarkable happens in your brain. Complex visual information transforms into meaning: a rich, multi-layered representation that exists before you ever put it into words.
But how exactly does the brain encode and transform this meaning? What patterns of activity carry the essence of what you're experiencing?
How Does the Brain Represent Meaning?
When you see an image or recall a memory, something remarkable happens in your brain. Complex visual information transforms into meaning: a rich, multi-layered representation that exists before you ever put it into words.
But how exactly does the brain encode and transform this meaning? What patterns of activity carry the essence of what you're experiencing?
How Does the Brain Represent Meaning?
When you see an image or recall a memory, something remarkable happens in your brain. Complex visual information transforms into meaning: a rich, multi-layered representation that exists before you ever put it into words.
But how exactly does the brain encode and transform this meaning? What patterns of activity carry the essence of what you're experiencing?
These are the questions driving Dr. Tomoyasu Horikawa at NTT Communication Science Laboratories.
Through his research into Mind Captioning, he’s uncovering fundamental insights into how the brain represents meaning-rich visual information, revealing that our thoughts have a structure and language all their own.
These are the questions driving Dr. Tomoyasu Horikawa at NTT Communication Science Laboratories.
Through his research into Mind Captioning, he’s uncovering fundamental insights into how the brain represents meaning-rich visual information, revealing that our thoughts have a structure and language all their own.
These are the questions driving Dr. Tomoyasu Horikawa at NTT Communication Science Laboratories.
Through his research into Mind Captioning, he’s uncovering fundamental insights into how the brain represents meaning-rich visual information, revealing that our thoughts have a structure and language all their own.
The Question:
What is Meaning
in the Brain?
At the heart of this research lies a fundamental question: how does the brain represent meaning? When you look at a cat, your brain doesn't simply store the word "cat." Instead, it creates a rich, distributed pattern of activity that captures everything about that experience: the orange fur, the green eyes, the posture, the context.
This is semantic content, the underlying meaning of what you're perceiving or imagining, independent of any particular language or words.
The Question:
What is Meaning
in the Brain?
At the heart of this research lies a fundamental question: how does the brain represent meaning? When you look at a cat, your brain doesn't simply store the word "cat." Instead, it creates a rich, distributed pattern of activity that captures everything about that experience: the orange fur, the green eyes, the posture, the context.
This is semantic content, the underlying meaning of what you're perceiving or imagining, independent of any particular language or words.
The Question:
What is Meaning
in the Brain?
At the heart of this research lies a fundamental question: how does the brain represent meaning? When you look at a cat, your brain doesn't simply store the word "cat." Instead, it creates a rich, distributed pattern of activity that captures everything about that experience: the orange fur, the green eyes, the posture, the context.
This is semantic content, the underlying meaning of what you're perceiving or imagining, independent of any particular language or words.
For decades, neuroscientists have studied how individual brain regions process specific types of information. But meaning isn't confined to one area. It emerges from complex patterns spread across the entire brain, involving visual processing, memory, conceptual knowledge, and more.
Dr. Horikawa's research tackles this complexity head-on, asking: can we decode these distributed patterns of meaning directly from brain activity? And if so, what can this tell us about how the brain fundamentally organizes and represents our thoughts?
For decades, neuroscientists have studied how individual brain regions process specific types of information. But meaning isn't confined to one area. It emerges from complex patterns spread across the entire brain, involving visual processing, memory, conceptual knowledge, and more.
Dr. Horikawa's research tackles this complexity head-on, asking: can we decode these distributed patterns of meaning directly from brain activity? And if so, what can this tell us about how the brain fundamentally organizes and represents our thoughts?
For decades, neuroscientists have studied how individual brain regions process specific types of information. But meaning isn't confined to one area. It emerges from complex patterns spread across the entire brain, involving visual processing, memory, conceptual knowledge, and more.
Dr. Horikawa's research tackles this complexity head-on, asking: can we decode these distributed patterns of meaning directly from brain activity? And if so, what can this tell us about how the brain fundamentally organizes and represents our thoughts?
Translating the semantic content of visual experiences
directly from brain activity into text descriptions.
Translating the semantic content of visual experiences directly from brain activity into text descriptions.
The Method:
Decoding Semantic Content
Mind Captioning is a technique for translating the semantic content of visual experiences directly from brain activity into text descriptions. It works by mapping the relationship between what people see or imagine and the patterns of neural activity recorded by fMRI.
The process reveals something profound about how meaning is encoded in the brain. Rather than looking for specific words or relying on traditional language areas, the system interprets distributed patterns of activity that represent concepts, features, and relationships.
An AI model, trained to recognize these patterns, generates descriptions that capture what a person is experiencing; not by reading an inner monologue, but by decoding the semantic structure of the thought itself.
The Method:
Decoding Semantic Content
Mind Captioning is a technique for translating the semantic content of visual experiences directly from brain activity into text descriptions. It works by mapping the relationship between what people see or imagine and the patterns of neural activity recorded by fMRI.
The process reveals something profound about how meaning is encoded in the brain. Rather than looking for specific words or relying on traditional language areas, the system interprets distributed patterns of activity that represent concepts, features, and relationships.
An AI model, trained to recognize these patterns, generates descriptions that capture what a person is experiencing; not by reading an inner monologue, but by decoding the semantic structure of the thought itself.
The Method:
Decoding Semantic Content
Mind Captioning is a technique for translating the semantic content of visual experiences directly from brain activity into text descriptions. It works by mapping the relationship between what people see or imagine and the patterns of neural activity recorded by fMRI.
The process reveals something profound about how meaning is encoded in the brain. Rather than looking for specific words or relying on traditional language areas, the system interprets distributed patterns of activity that represent concepts, features, and relationships.
An AI model, trained to recognize these patterns, generates descriptions that capture what a person is experiencing; not by reading an inner monologue, but by decoding the semantic structure of the thought itself.
What makes this particularly revealing is what it shows us about the brain's representational code. The system can generate detailed descriptions (distinguishing between an orange cat and a grey one, noting whether it's sitting or standing) by analysing activity patterns that exist entirely outside the brain's canonical language network.
This demonstrates that rich, communicable meaning is represented in the brain in ways that transcend traditional linguistic processing.
What makes this particularly revealing is what it shows us about the brain's representational code. The system can generate detailed descriptions (distinguishing between an orange cat and a grey one, noting whether it's sitting or standing) by analysing activity patterns that exist entirely outside the brain's canonical language network.
This demonstrates that rich, communicable meaning is represented in the brain in ways that transcend traditional linguistic processing.
What makes this particularly revealing is what it shows us about the brain's representational code. The system can generate detailed descriptions (distinguishing between an orange cat and a grey one, noting whether it's sitting or standing) by analysing activity patterns that exist entirely outside the brain's canonical language network.
This demonstrates that rich, communicable meaning is represented in the brain in ways that transcend traditional linguistic processing.
Seeing and Imagining:
A Shared Neural Code
One of the most significant discoveries from this research is that the brain uses remarkably similar patterns to represent both what we perceive and what we imagine.
When you look at a photograph of a mountain, specific patterns of brain activity emerge. When you close your eyes and vividly imagine that same mountain, your brain generates similar patterns. This isn't just a loose similarity: the correspondence is strong enough that Mind Captioning can describe your mental imagery with nearly the same accuracy as it describes what you're actually seeing.
This finding has profound implications for our understanding of memory, imagination, and perception. It suggests that the brain has developed a flexible, abstract code for representing meaning that works independently of sensory input. Whether information comes from your eyes or from your memory, the brain translates it into a common representational format.
This shared code helps explain how we can think about past experiences, imagine future scenarios, and understand descriptions of things we've never seen. It reveals that the brain's system for representing meaning is fundamentally generative and flexible, not simply reactive to sensory input.
Seeing and Imagining:
A Shared Neural Code
One of the most significant discoveries from this research is that the brain uses remarkably similar patterns to represent both what we perceive and what we imagine.
When you look at a photograph of a mountain, specific patterns of brain activity emerge. When you close your eyes and vividly imagine that same mountain, your brain generates similar patterns. This isn't just a loose similarity: the correspondence is strong enough that Mind Captioning can describe your mental imagery with nearly the same accuracy as it describes what you're actually seeing.
This finding has profound implications for our understanding of memory, imagination, and perception. It suggests that the brain has developed a flexible, abstract code for representing meaning that works independently of sensory input. Whether information comes from your eyes or from your memory, the brain translates it into a common representational format.
This shared code helps explain how we can think about past experiences, imagine future scenarios, and understand descriptions of things we've never seen. It reveals that the brain's system for representing meaning is fundamentally generative and flexible, not simply reactive to sensory input.
Seeing and Imagining:
A Shared Neural Code
One of the most significant discoveries from this research is that the brain uses remarkably similar patterns to represent both what we perceive and what we imagine.
When you look at a photograph of a mountain, specific patterns of brain activity emerge. When you close your eyes and vividly imagine that same mountain, your brain generates similar patterns. This isn't just a loose similarity: the correspondence is strong enough that Mind Captioning can describe your mental imagery with nearly the same accuracy as it describes what you're actually seeing.
This finding has profound implications for our understanding of memory, imagination, and perception. It suggests that the brain has developed a flexible, abstract code for representing meaning that works independently of sensory input. Whether information comes from your eyes or from your memory, the brain translates it into a common representational format.
This shared code helps explain how we can think about past experiences, imagine future scenarios, and understand descriptions of things we've never seen. It reveals that the brain's system for representing meaning is fundamentally generative and flexible, not simply reactive to sensory input.
Beyond Language Areas:
Rethinking Communication
Perhaps the most striking insight from this research challenges long-held assumptions about the relationship between thought and language.
Traditional models suggested that for a thought to be communicated, it must be processed through the brain's canonical language areas (regions like Broca's and Wernicke's areas that we use for speaking and understanding speech). But Mind Captioning demonstrates that rich semantic content can be decoded from brain activity without relying on these language networks at all.
The system successfully generates descriptions from patterns of activity distributed across visual, conceptual, and memory-related regions. This means that meaning exists in the brain as something distinct from and prior to linguistic expression. You can think about a cat (representing its colour, posture, and features in your mind) without ever activating the word "cat" in your language centres.
Even more remarkably, the research has shown that accurate English descriptions can be generated from the brain activity of Japanese speakers who don't speak English. This cross-linguistic success suggests that while our spoken languages differ, the underlying semantic representations in our brains may share fundamental similarities: a kind of universal structure to how we represent meaning.
Beyond Language Areas:
Rethinking Communication
Perhaps the most striking insight from this research challenges long-held assumptions about the relationship between thought and language.
Traditional models suggested that for a thought to be communicated, it must be processed through the brain's canonical language areas (regions like Broca's and Wernicke's areas that we use for speaking and understanding speech). But Mind Captioning demonstrates that rich semantic content can be decoded from brain activity without relying on these language networks at all.
The system successfully generates descriptions from patterns of activity distributed across visual, conceptual, and memory-related regions. This means that meaning exists in the brain as something distinct from and prior to linguistic expression. You can think about a cat (representing its colour, posture, and features in your mind) without ever activating the word "cat" in your language centres.
Even more remarkably, the research has shown that accurate English descriptions can be generated from the brain activity of Japanese speakers who don't speak English. This cross-linguistic success suggests that while our spoken languages differ, the underlying semantic representations in our brains may share fundamental similarities: a kind of universal structure to how we represent meaning.
Beyond Language Areas:
Rethinking Communication
Perhaps the most striking insight from this research challenges long-held assumptions about the relationship between thought and language.
Traditional models suggested that for a thought to be communicated, it must be processed through the brain's canonical language areas (regions like Broca's and Wernicke's areas that we use for speaking and understanding speech). But Mind Captioning demonstrates that rich semantic content can be decoded from brain activity without relying on these language networks at all.
The system successfully generates descriptions from patterns of activity distributed across visual, conceptual, and memory-related regions. This means that meaning exists in the brain as something distinct from and prior to linguistic expression. You can think about a cat (representing its colour, posture, and features in your mind) without ever activating the word "cat" in your language centres.
Even more remarkably, the research has shown that accurate English descriptions can be generated from the brain activity of Japanese speakers who don't speak English. This cross-linguistic success suggests that while our spoken languages differ, the underlying semantic representations in our brains may share fundamental similarities: a kind of universal structure to how we represent meaning.
The Technical Foundation:
Why High-Resolution fMRI?
Understanding why this research requires high-resolution, whole-brain fMRI scanning reveals something important about how meaning is distributed in the brain.
When you process complex visual information, your brain doesn't just activate one region. Instead, meaning emerges from the coordinated activity of many different areas simultaneously: visual processing regions, conceptual areas, memory systems, and more. These patterns are spatially distributed and intricate, requiring a method that can capture activity across the entire brain with fine-grained resolution.
High-resolution fMRI provides exactly this: a comprehensive view of brain activity that can detect the subtle, distributed patterns carrying semantic information. While this requirement currently limits the research to laboratory settings with large, sophisticated equipment, it's precisely what allows the method to access the rich representational code underlying our thoughts.
This technical necessity also reveals an important characteristic of how the brain encodes meaning: it's not localized or simple, but distributed and complex. Understanding this distributed code is central to understanding how the brain fundamentally works.
This technical necessity also reveals an important characteristic of how the brain encodes meaning: it's not localized or simple, but distributed and complex. Understanding this distributed code is central to understanding how the brain fundamentally works.
The Technical Foundation:
Why High-Resolution fMRI?
Understanding why this research requires high-resolution, whole-brain fMRI scanning reveals something important about how meaning is distributed in the brain.
When you process complex visual information, your brain doesn't just activate one region. Instead, meaning emerges from the coordinated activity of many different areas simultaneously: visual processing regions, conceptual areas, memory systems, and more. These patterns are spatially distributed and intricate, requiring a method that can capture activity across the entire brain with fine-grained resolution.
High-resolution fMRI provides exactly this: a comprehensive view of brain activity that can detect the subtle, distributed patterns carrying semantic information. While this requirement currently limits the research to laboratory settings with large, sophisticated equipment, it's precisely what allows the method to access the rich representational code underlying our thoughts.
This technical necessity also reveals an important characteristic of how the brain encodes meaning: it's not localized or simple, but distributed and complex. Understanding this distributed code is central to understanding how the brain fundamentally works.
This technical necessity also reveals an important characteristic of how the brain encodes meaning: it's not localized or simple, but distributed and complex. Understanding this distributed code is central to understanding how the brain fundamentally works.
The Technical Foundation:
Why High-Resolution fMRI?
Understanding why this research requires high-resolution, whole-brain fMRI scanning reveals something important about how meaning is distributed in the brain.
When you process complex visual information, your brain doesn't just activate one region. Instead, meaning emerges from the coordinated activity of many different areas simultaneously: visual processing regions, conceptual areas, memory systems, and more. These patterns are spatially distributed and intricate, requiring a method that can capture activity across the entire brain with fine-grained resolution.
High-resolution fMRI provides exactly this: a comprehensive view of brain activity that can detect the subtle, distributed patterns carrying semantic information. While this requirement currently limits the research to laboratory settings with large, sophisticated equipment, it's precisely what allows the method to access the rich representational code underlying our thoughts.
This technical necessity also reveals an important characteristic of how the brain encodes meaning: it's not localized or simple, but distributed and complex. Understanding this distributed code is central to understanding how the brain fundamentally works.
This technical necessity also reveals an important characteristic of how the brain encodes meaning: it's not localized or simple, but distributed and complex. Understanding this distributed code is central to understanding how the brain fundamentally works.
Scientific Insights and
Future Possibilities
The primary value of this research lies in what it reveals about the brain itself. By demonstrating that rich semantic content can be decoded from distributed patterns of neural activity, this work advances our fundamental understanding of how the brain represents and transforms meaning.
What We're Learning
1
The brain represents meaning in distributed patterns that exist independently of language systems
2
Perceived and imagined content share common neural codes
3
Semantic representations may have universal structural properties across different languages and cultures
4
Complex visual meaning can be reconstructed from patterns of brain activity with surprising fidelity
Looking Ahead
While the current research focus is on understanding neural representation, these scientific insights naturally point toward potential future applications.
As we better understand how meaning is encoded in the brain, it's possible to imagine new approaches to supporting communication for individuals with language disorders like aphasia, where traditional speech and writing are impaired but semantic understanding remains intact.
However, any such applications would require substantial additional development, including new technologies that don't exist today. The current research is about building the foundational knowledge necessary to even begin exploring such possibilities.
Scientific Insights and
Future Possibilities
The primary value of this research lies in what it reveals about the brain itself. By demonstrating that rich semantic content can be decoded from distributed patterns of neural activity, this work advances our fundamental understanding of how the brain represents and transforms meaning.
What We're Learning
1
The brain represents meaning in distributed patterns that exist independently of language systems
2
Perceived and imagined content share common neural codes
3
Semantic representations may have universal structural properties across different languages and cultures
4
Complex visual meaning can be reconstructed from patterns of brain activity with surprising fidelity
Looking Ahead
While the current research focus is on understanding neural representation, these scientific insights naturally point toward potential future applications.
As we better understand how meaning is encoded in the brain, it's possible to imagine new approaches to supporting communication for individuals with language disorders like aphasia, where traditional speech and writing are impaired but semantic understanding remains intact.
However, any such applications would require substantial additional development, including new technologies that don't exist today. The current research is about building the foundational knowledge necessary to even begin exploring such possibilities.
Scientific Insights and
Future Possibilities
The primary value of this research lies in what it reveals about the brain itself. By demonstrating that rich semantic content can be decoded from distributed patterns of neural activity, this work advances our fundamental understanding of how the brain represents and transforms meaning.
What We're
Learning
1
The brain represents meaning in distributed patterns that exist independently of language systems
2
Perceived and imagined content share common neural codes
3
Semantic representations may have universal structural properties across different languages and cultures
4
Complex visual meaning can be reconstructed from patterns of brain activity with surprising fidelity
Looking
Ahead
While the current research focus is on understanding neural representation, these scientific insights naturally point toward potential future applications.
As we better understand how meaning is encoded in the brain, it's possible to imagine new approaches to supporting communication for individuals with language disorders like aphasia, where traditional speech and writing are impaired but semantic understanding remains intact.
However, any such applications would require substantial additional development, including new technologies that don't exist today. The current research is about building the foundational knowledge necessary to even begin exploring such possibilities.
Why This
Research Matters
Dr. Horikawa’s research changes our fundamental understanding of how the human mind works.
By revealing that thoughts carry rich, structured meaning independent of language, this work challenges us to rethink the relationship between perception, memory, imagination, and communication. It shows that the brain has developed sophisticated systems for representing semantic content, systems that operate through distributed patterns of activity across multiple regions.
These insights don't just advance neuroscience; they touch on profound questions about the nature of thought itself. What does knowing that our brains can represent complex visual meaning in ways that transcend language mean for us? What does the fact that perceived and imagined content share such similar neural codes tell us about human cognition?
Mind Captioning offers a window into these questions. Each description generated from brain activity isn't just a technical achievement, it's evidence of the brain's remarkable capacity to encode, transform, and communicate meaning through patterns of neural activity.
By continuing to investigate how complex, meaning-rich visual information is represented in the brain, this research is mapping the fundamental architecture of human thought. In doing so, it opens new ways of understanding what it means to think, to remember, to imagine, and ultimately, to be human.
Why This Research Matters
Dr. Horikawa’s research changes our fundamental understanding of how the human mind works.
By revealing that thoughts carry rich, structured meaning independent of language, this work challenges us to rethink the relationship between perception, memory, imagination, and communication. It shows that the brain has developed sophisticated systems for representing semantic content, systems that operate through distributed patterns of activity across multiple regions.
These insights don't just advance neuroscience; they touch on profound questions about the nature of thought itself. What does knowing that our brains can represent complex visual meaning in ways that transcend language mean for us? What does the fact that perceived and imagined content share such similar neural codes tell us about human cognition?
Mind Captioning offers a window into these questions. Each description generated from brain activity isn't just a technical achievement, it's evidence of the brain's remarkable capacity to encode, transform, and communicate meaning through patterns of neural activity.
By continuing to investigate how complex, meaning-rich visual information is represented in the brain, this research is mapping the fundamental architecture of human thought. In doing so, it opens new ways of understanding what it means to think, to remember, to imagine, and ultimately, to be human.
Why This
Research Matters
Dr. Horikawa’s research changes our fundamental understanding of how the human mind works.
By revealing that thoughts carry rich, structured meaning independent of language, this work challenges us to rethink the relationship between perception, memory, imagination, and communication. It shows that the brain has developed sophisticated systems for representing semantic content, systems that operate through distributed patterns of activity across multiple regions.
These insights don't just advance neuroscience; they touch on profound questions about the nature of thought itself. What does knowing that our brains can represent complex visual meaning in ways that transcend language mean for us? What does the fact that perceived and imagined content share such similar neural codes tell us about human cognition?
Mind Captioning offers a window into these questions. Each description generated from brain activity isn't just a technical achievement, it's evidence of the brain's remarkable capacity to encode, transform, and communicate meaning through patterns of neural activity.
By continuing to investigate how complex, meaning-rich visual information is represented in the brain, this research is mapping the fundamental architecture of human thought. In doing so, it opens new ways of understanding what it means to think, to remember, to imagine, and ultimately, to be human.
© 2026 Dr. Tomoyasu Horikawa
Design by Scrolly Science - powered by Animara Studios™
© 2026 Dr. Tomoyasu Horikawa
Design by Scrolly Science - powered by Animara Studios™
© 2026 Dr. Tomoyasu Horikawa
Design by Scrolly Science - powered by Animara Studios™