Language-EEG Aligned Foundation Model
for Brain-Computer Interfaces NeurIPS 2026

Muyun Jiang1,*, Shuailei Zhang1,*, Zhenjie Yang2,*, Wu Mengjun1, Wei Zhang1, Weibang Jiang3,
Chenyu Liu1, Zhiwei Guo1, Rui Liu1, Shangen Zhang4, Yong Li1,5, Yi Ding1,†, Cuntai Guan1,†

1Nanyang Technological University, Singapore
2The University of Hong Kong, Hong Kong   3Shanghai Jiao Tong University, China
4University of Science and Technology Beijing, China   5Southeast University, China
*Equal contribution   †Corresponding authors

From discrete task heads to a shared EEG–language space guided by natural-language instructions.

Abstract

Electroencephalography (EEG) foundation models learn transferable representations for brain–computer interfaces, but existing approaches treat tasks and labels as discrete targets and fail to use the semantic structure of natural-language task instructions to guide representation learning. We present LEAF, a foundation model for EEG–Language Alignment with Semantic Task Instruction and Querying. LEAF integrates task-aware semantic guidance to produce structured and linguistically aligned EEG embeddings, thereby enhancing decoding robustness and transferability. In the EEG pretraining stage, we introduce a joint Spectral–Temporal Reconstruction (STR) framework that captures the coupled spectral rhythms and temporal dynamics of EEG signals. STR applies randomized spectral perturbation to enhance frequency robustness and uses two complementary temporal objectives to learn both contextual and sequential structure. In the EEG–Language alignment stage, we propose the Instruction-conditioned Q-Former (IQF). This query-based cross-attention transformer injects instruction embeddings into EEG tokens and achieves semantic alignment with textual label embeddings through learnable queries. We evaluate LEAF on 16 downstream datasets spanning motor imagery, emotion recognition, steady-state visual evoked potentials, covert speech, and mental-workload decoding. Under supervised fine-tuning, LEAF achieves the best balanced accuracy on 12 of the 16 datasets and the strongest average performance across the evaluated paradigms. Further analyses show that instruction conditioning reduces intra-class variation, increases inter-class separation, and aligns EEG representations more closely with textual class prototypes. These findings support the use of task instructions as semantic priors for transferable EEG decoding. Code and pretrained checkpoints are available through the project links above.

Pre-training reconstructs frequency-aware contextual and causal EEG structure. Instruction tuning then conditions the learned representations on task descriptions and aligns them with textual label prototypes.

Task instructions progressively reshape the shared semantic space.

A generic placeholder is encoded without task or candidate-label semantics.

“None”

Interactive cluster plot for OpenBMI-MI. Loading precomputed embeddings…

Motor imagery (OpenBMI-MI)

“None”

Interactive cluster plot for SEED. Loading precomputed embeddings…

Emotion recognition (SEED)

Text label prototype. Points show the complete test sets; each panel uses one fixed UMAP coordinate system across all three levels. Both datasets were included in instruction tuning; this visualization is not a held-out-dataset experiment.

03

Results

01 · Supervised adaptation

Fine-tune on a target dataset

The instruction-tuned encoder and a new MLP learn from labeled target trials. This is the setting behind the main 16-dataset benchmark.

Compare the two fine-tuning paths →

02 · Direct inference

Decode with text prototypes

The frozen model scores candidate labels without further training. Standard evaluation uses held-out examples of tasks included in instruction tuning.

Explore direct inference →

03 · Held-out transfer

Test on unseen datasets

Dreyer2023 and Weibo2014 are excluded from both pretraining and instruction tuning. Evaluation uses no adaptation or labeled training trials from either dataset.

Explore held-out evaluation →

Cross-paradigm benchmark

Strong average performance across heterogeneous BCI tasks.

With supervised encoder + MLP fine-tuning, LEAF achieves the highest mean balanced accuracy on 12 of 16 datasets and an equal-weight average of 64.14%. These results measure supervised transfer of the instruction-tuned EEG encoder.

Supervised fine-tuning: two classification paths

Both paths start from the same instruction-tuned checkpoint and adapt to labeled training data from one target dataset. (A) Encoder + MLP, used for the main paper results, removes the Q-Former and jointly trains the EEG encoder and a new MLP on flattened contextual and causal tokens using cross-entropy. (B) Q-Former + cosine CE updates the existing encoder and Q-Former using cross-entropy on cosine similarities to fixed text prototypes, adding no trainable parameters.

Two supervised adaptation paths from a shared instruction-tuned LEAF checkpoint.

Training commands, settings, and evaluation protocol →

View all 16 dataset results
Fine-tuning comparison: mean ± standard deviation over three seeds, on the 0–1 scale. Average: equal weight across 16 datasets.
IDDataset(A) Encoder + MLP B.Acc(A) Kappa(B) Q-Former + cosine CE B.Acc(B) Kappa
1BCIC-IV-2a0.6381 ± 0.01200.5175 ± 0.01600.6620 ± 0.01360.5494 ± 0.0182
2OpenBMI-MI0.8134 ± 0.00310.6268 ± 0.00620.8017 ± 0.00180.6035 ± 0.0036
3BCIC-Upperlimb0.5483 ± 0.00380.3225 ± 0.00570.4828 ± 0.00380.2242 ± 0.0058
4SHU-MI0.6134 ± 0.00080.2268 ± 0.00160.5897 ± 0.00530.1792 ± 0.0106
5HighGamma0.7982 ± 0.00350.5964 ± 0.00700.7899 ± 0.01040.5797 ± 0.0208
6Cho20170.7908 ± 0.00600.5816 ± 0.01200.8030 ± 0.00560.6059 ± 0.0111
7Shin2017A0.7256 ± 0.00880.4512 ± 0.01760.7509 ± 0.03070.5019 ± 0.0614
8PhysioNet-MI0.6992 ± 0.00350.3984 ± 0.00700.7535 ± 0.00250.5070 ± 0.0050
9FACED0.5819 ± 0.00230.5293 ± 0.00260.4071 ± 0.00400.3292 ± 0.0050
10SEED0.7011 ± 0.00680.5511 ± 0.01020.6870 ± 0.00350.5347 ± 0.0045
11SEED-IV0.4630 ± 0.00340.2660 ± 0.00440.4511 ± 0.00480.2702 ± 0.0056
12SEED-V0.4126 ± 0.00230.2640 ± 0.00290.4010 ± 0.00760.2556 ± 0.0106
13SEED-VII0.3356 ± 0.00120.2246 ± 0.00140.3286 ± 0.00320.2175 ± 0.0038
14OpenBMI-SSVEP0.9462 ± 0.00430.9283 ± 0.00570.8971 ± 0.00470.8628 ± 0.0063
15BCIC-Speech0.5453 ± 0.00230.4316 ± 0.00290.5338 ± 0.00780.4172 ± 0.0098
16Mental Workload0.6493 ± 0.02110.2420 ± 0.03700.6296 ± 0.03340.3122 ± 0.0921
Average0.64140.44740.62310.4344

04

Inside the shared semantic space after alignment

Shared geometry

EEG embeddings and text prototypes share structure.

Under task-and-target instructions, motor-related datasets occupy broadly similar regions, while emotion datasets share another region. These UMAP projections describe datasets included in instruction tuning; their structure may reflect both EEG features and shared instruction and label conditioning.

These projections provide a qualitative view of the learned representations. Transfer to unseen datasets is assessed separately in the held-out experiments; UMAP alone does not establish generalization or rule out dataset-specific memorization.

Reproduce LEAF

Models, training code, and documentation.

The repository contains the full pretraining, instruction-tuning, direct-inference, and downstream evaluation pipeline.

05

Citation

BibTeX
@article{jiang2026leaf,
  title={LEAF: Language-EEG Aligned Foundation Model for Brain-Computer Interfaces},
  author={Jiang, Muyun and Zhang, Shuailei and Yang, Zhenjie and Wu, Mengjun and Zhang, Wei and Jiang, Weibang and Liu, Chenyu and Guo, Zhiwei and Liu, Rui and Zhang, Shangen and Li, Yong and Ding, Yi and Guan, Cuntai},
  journal={arXiv preprint arXiv:2509.24302},
  year={2026}
}
Expanded paper figure