Academic

SENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrieval

arXiv:2603.17109v1 Announce Type: new Abstract: Decoding brain activity into natural language is a major challenge in AI with important applications in assistive communication, neurotechnology, and human-computer interaction. Most existing Brain-Computer Interface (BCI) approaches rely on memory-intensive fine-tuning of Large Language Models (LLMs) or encoder-decoder models on raw EEG signals, resulting in expensive training pipelines, limited accessibility, and potential exposure of sensitive neural data. We introduce SENSE (SEmantic Neural Sparse Extraction), a lightweight and privacy-preserving framework that translates non-invasive electroencephalography (EEG) into text without LLM fine-tuning. SENSE decouples decoding into two stages: on-device semantic retrieval and prompt-based language generation. EEG signals are locally mapped to a discrete textual space to extract a non-sensitive Bag-of-Words (BoW), which conditions an off-the-shelf LLM to synthesize fluent text in a zero-sh

A
Akshaj Murhekar, Christina Liu, Abhijit Mishra, Shounak Roychowdhury, Jacek Gwizdka · · 1 min read · 5 views
#cs.LG

arXiv:2603.17109v1 Announce Type: new Abstract: Decoding brain activity into natural language is a major challenge in AI with important applications in assistive communication, neurotechnology, and human-computer interaction. Most existing Brain-Computer Interface (BCI) approaches rely on memory-intensive fine-tuning of Large Language Models (LLMs) or encoder-decoder models on raw EEG signals, resulting in expensive training pipelines, limited accessibility, and potential exposure of sensitive neural data. We introduce SENSE (SEmantic Neural Sparse Extraction), a lightweight and privacy-preserving framework that translates non-invasive electroencephalography (EEG) into text without LLM fine-tuning. SENSE decouples decoding into two stages: on-device semantic retrieval and prompt-based language generation. EEG signals are locally mapped to a discrete textual space to extract a non-sensitive Bag-of-Words (BoW), which conditions an off-the-shelf LLM to synthesize fluent text in a zero-shot manner. The EEG-to-keyword module contains only ~6M parameters and runs fully on-device, ensuring raw neural signals remain local while only abstract semantic cues interact with language models. Evaluated on a 128-channel EEG dataset across six subjects, SENSE matches or surpasses the generative quality of fully fine-tuned baselines such as Thought2Text while substantially reducing computational overhead. By localizing neural decoding and sharing only derived textual cues, SENSE provides a scalable and privacy-aware retrieval-augmented architecture for next-generation BCIs.

Executive Summary

This article introduces SENSE, a novel framework for decoding brain activity into natural language from non-invasive electroencephalography (EEG) signals. SENSE decouples the decoding process into two stages: on-device semantic retrieval and prompt-based language generation. This approach allows for lightweight and privacy-preserving EEG-to-text translation without requiring the fine-tuning of Large Language Models (LLMs). The framework is evaluated on a 128-channel EEG dataset across six subjects, showing comparable or even superior generative quality to fully fine-tuned baselines while significantly reducing computational overhead. SENSE provides a promising solution for next-generation Brain-Computer Interfaces (BCIs), offering scalability and privacy awareness through localized neural decoding and textual cue sharing.

Key Points

  • ▸ SENSE is a lightweight and privacy-preserving framework for decoding EEG signals into natural language.
  • ▸ The framework decouples decoding into two stages: on-device semantic retrieval and prompt-based language generation.
  • ▸ SENSE does not require LLM fine-tuning and reduces computational overhead compared to existing approaches.

Merits

Strength in Decentralization

SENSE's decentralized approach ensures that raw neural signals remain local, preserving user privacy and reducing the risk of sensitive data exposure.

Demerits

Vulnerability to Sensor Noise

The framework's performance may be compromised by sensor noise or poor signal quality, which can hinder the accuracy of EEG-to-text translation.

Expert Commentary

While SENSE demonstrates impressive results in EEG-to-text translation, its limitations, such as vulnerability to sensor noise, must be addressed in future iterations. Additionally, the framework's potential for scaling and deployment in real-world applications is substantial, given its lightweight and privacy-preserving nature. Furthermore, the article's contribution to the broader field of BCIs is substantial, as it paves the way for more decentralized and user-centric approaches to neural decoding.

Recommendations

  • ✓ Future research should focus on optimizing SENSE's performance in the presence of sensor noise and developing strategies for mitigating its effects.
  • ✓ Developers and policymakers should consider the broader implications of SENSE's decentralized approach, including its potential impact on data protection and regulatory frameworks.

Sources

Related Articles