Skip to main content
Back to timeline
arXivSource publication:

Yang and six co-authors survey brain-to-language decoding, tracing a field that moved from constrained recognition and acoustic reconstruction to text generation, streaming personalised speech and facial animation

Synopsis

This survey by Yiqian Yang, Yiqun Duan, Chenyu Liu, Yiqi Wang, Xinliang Zhou, Chin-Teng Lin and Yu Zhang synthesises brain-to-language decoding, which translates neural activity associated with language production, internal speech and perception into linguistic or expressive outputs, across invasive and non-invasive measurements; it connects Articulated, Inner and Perceived tasks to the neural populations they engage, the representations available to decoders and the outputs those representations can support, examines model development, public resources and the evolution of evaluation, compares published performance and communication costs within their reported protocols, identifies phonetic, acoustic and semantic targets as preserving different aspects of a message, shared representations

Source-provided article image: Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond
Figure 1 ·

Figure 1: Literature retrieval, the updated registry and manuscript citation coverage. (a) The original retrieval and supplements produced 813 retained IDs and 795 canonical entries after publication-version consolidation. (b) Source-led additions bring the registry to 920 report/resource entries, including 246 with selected structured fields. (c) The active manuscript cites 448 distinct references: 317 in the main text and 131 only in appendices; 354 are linked to registry entries and 94 provide contextual reviews, foundations or adjacent evidence. The footer counts citations in the 8 Methods comparison figures separately from the communication-cost result rows. References, reports, resources, experimental conditions and participants are different counting units. Publication versions are linked; companion data releases and articles can describe the same acquisition.

arXiv

Interpretation

The survey organises the task space of brain-to-language decoding into Articulated, Inner and Perceived tasks, and links each to the neural populations it engages, the representations available to decoders and the outputs those representations can support. Where earlier accounts were often organised by method or by signal modality, this synthesis uses task as the axis that ties neural populations, representations and outputs into one thread. Built on a search without a lower year limit and source-led updates through September 2026, covering invasive and non-invasive measurements.

The survey records that the field has expanded from constrained recognition and acoustic reconstruction to text generation, streaming personalised speech and facial animation. It treats the widening of output forms as a signature change in the field rather than staying on a single decoding-accuracy axis. Derived from synthesis of published work, with performance and communication costs compared within their reported protocols.

The survey identifies three complementary routes to progress: phonetic, acoustic and semantic targets preserve different aspects of a message; shared representations support reuse across recording conditions and tasks; and online communication increasingly depends on calibration, feedback and user control alongside decoding accuracy. It places evaluation and use-level factors such as calibration, feedback and user control alongside representation-level factors as conditions for progress, rather than treating accuracy as the sole metric. Synthesises algorithmic comparisons enabled by shared benchmarks together with longitudinal studies that reveal the demands of sustained use.

The survey outlines a prospective five-level trajectory from commands and language to meaning, scenarios and bidirectional cognitive exchange. It offers the field a layered long-term roadmap that discusses present capabilities and farther goals within one frame. This is a forward-looking outlook, presented alongside the preceding discussion of existing developments and their remaining limitations.

Perspective

This survey is aimed at readers who want to understand brain-to-language decoding tasks, signals, methods, evaluation and practical use within a single frame; it covers invasive and non-invasive measurements, with a search without a lower year limit and source-led updates through September 2026. It connects Articulated, Inner and Perceived tasks to neural populations, available representations and supportable outputs, and compares published performance and communication costs within their reported protocols, so it is suited to locating where a given decoding target sits along which route, which representations it depends on, and what evaluation and use conditions it requires. The discussion of shared benchmarks and longitudinal studies offers reference points for reuse across recording conditions and tasks and for the demands of sustained use.

The loaded text is the arXiv abstract page and metadata, without body, figures or reference list, so specific performance values, the quantitative basis for communication costs, the individual entries of shared benchmarks and public resources, and the criteria for each of the five levels cannot be checked here. The search has no lower year limit and updates through September 2026, but its coverage and inclusion criteria are not expanded in the abstract. The complementary routes and the prospective trajectory are synthesis and outlook, and how testable they prove depends on later validation in shared benchmarks and longitudinal studies.

Sources