“None”
Language-EEG Aligned Foundation Model
for Brain-Computer Interfaces NeurIPS 2026
Abstract
Electroencephalography (EEG) foundation models learn transferable representations for brain–computer interfaces, but existing approaches treat tasks and labels as discrete targets and fail to use the semantic structure of natural-language task instructions to guide representation learning. We present LEAF, a foundation model for EEG–Language Alignment with Semantic Task Instruction and Querying. LEAF integrates task-aware semantic guidance to produce structured and linguistically aligned EEG embeddings, thereby enhancing decoding robustness and transferability. In the EEG pretraining stage, we introduce a joint Spectral–Temporal Reconstruction (STR) framework that captures the coupled spectral rhythms and temporal dynamics of EEG signals. STR applies randomized spectral perturbation to enhance frequency robustness and uses two complementary temporal objectives to learn both contextual and sequential structure. In the EEG–Language alignment stage, we propose the Instruction-conditioned Q-Former (IQF). This query-based cross-attention transformer injects instruction embeddings into EEG tokens and achieves semantic alignment with textual label embeddings through learnable queries. We evaluate LEAF on 16 downstream datasets spanning motor imagery, emotion recognition, steady-state visual evoked potentials, covert speech, and mental-workload decoding. Under supervised fine-tuning, LEAF achieves the best balanced accuracy on 12 of the 16 datasets and the strongest average performance across the evaluated paradigms. Further analyses show that instruction conditioning reduces intra-class variation, increases inter-class separation, and aligns EEG representations more closely with textual class prototypes. These findings support the use of task instructions as semantic priors for transferable EEG decoding. Code and pretrained checkpoints are available through the project links above.
Task instructions progressively reshape the shared semantic space.
A generic placeholder is encoded without task or candidate-label semantics.
“None”
Emotion recognition (SEED)
Text label prototype. Points show the complete test sets; each panel uses one fixed UMAP coordinate system across all three levels. Both datasets were included in instruction tuning; this visualization is not a held-out-dataset experiment.
The interactive data could not be loaded. A static view is shown below.
03
Results
01 · Supervised adaptation
Fine-tune on a target dataset
The instruction-tuned encoder and a new MLP learn from labeled target trials. This is the setting behind the main 16-dataset benchmark.
Compare the two fine-tuning paths →02 · Direct inference
Decode with text prototypes
The frozen model scores candidate labels without further training. Standard evaluation uses held-out examples of tasks included in instruction tuning.
Explore direct inference →03 · Held-out transfer
Test on unseen datasets
Dreyer2023 and Weibo2014 are excluded from both pretraining and instruction tuning. Evaluation uses no adaptation or labeled training trials from either dataset.
Explore held-out evaluation →Cross-paradigm benchmark
Strong average performance across heterogeneous BCI tasks.
With supervised encoder + MLP fine-tuning, LEAF achieves the highest mean balanced accuracy on 12 of 16 datasets and an equal-weight average of 64.14%. These results measure supervised transfer of the instruction-tuned EEG encoder.
Supervised fine-tuning: two classification paths
Both paths start from the same instruction-tuned checkpoint and adapt to labeled training data from one target dataset. (A) Encoder + MLP, used for the main paper results, removes the Q-Former and jointly trains the EEG encoder and a new MLP on flattened contextual and causal tokens using cross-entropy. (B) Q-Former + cosine CE updates the existing encoder and Q-Former using cross-entropy on cosine similarities to fixed text prototypes, adding no trainable parameters.
Training commands, settings, and evaluation protocol →
View all 16 dataset results
| ID | Dataset | (A) Encoder + MLP B.Acc | (A) Kappa | (B) Q-Former + cosine CE B.Acc | (B) Kappa |
|---|---|---|---|---|---|
| 1 | BCIC-IV-2a | 0.6381 ± 0.0120 | 0.5175 ± 0.0160 | 0.6620 ± 0.0136 | 0.5494 ± 0.0182 |
| 2 | OpenBMI-MI | 0.8134 ± 0.0031 | 0.6268 ± 0.0062 | 0.8017 ± 0.0018 | 0.6035 ± 0.0036 |
| 3 | BCIC-Upperlimb | 0.5483 ± 0.0038 | 0.3225 ± 0.0057 | 0.4828 ± 0.0038 | 0.2242 ± 0.0058 |
| 4 | SHU-MI | 0.6134 ± 0.0008 | 0.2268 ± 0.0016 | 0.5897 ± 0.0053 | 0.1792 ± 0.0106 |
| 5 | HighGamma | 0.7982 ± 0.0035 | 0.5964 ± 0.0070 | 0.7899 ± 0.0104 | 0.5797 ± 0.0208 |
| 6 | Cho2017 | 0.7908 ± 0.0060 | 0.5816 ± 0.0120 | 0.8030 ± 0.0056 | 0.6059 ± 0.0111 |
| 7 | Shin2017A | 0.7256 ± 0.0088 | 0.4512 ± 0.0176 | 0.7509 ± 0.0307 | 0.5019 ± 0.0614 |
| 8 | PhysioNet-MI | 0.6992 ± 0.0035 | 0.3984 ± 0.0070 | 0.7535 ± 0.0025 | 0.5070 ± 0.0050 |
| 9 | FACED | 0.5819 ± 0.0023 | 0.5293 ± 0.0026 | 0.4071 ± 0.0040 | 0.3292 ± 0.0050 |
| 10 | SEED | 0.7011 ± 0.0068 | 0.5511 ± 0.0102 | 0.6870 ± 0.0035 | 0.5347 ± 0.0045 |
| 11 | SEED-IV | 0.4630 ± 0.0034 | 0.2660 ± 0.0044 | 0.4511 ± 0.0048 | 0.2702 ± 0.0056 |
| 12 | SEED-V | 0.4126 ± 0.0023 | 0.2640 ± 0.0029 | 0.4010 ± 0.0076 | 0.2556 ± 0.0106 |
| 13 | SEED-VII | 0.3356 ± 0.0012 | 0.2246 ± 0.0014 | 0.3286 ± 0.0032 | 0.2175 ± 0.0038 |
| 14 | OpenBMI-SSVEP | 0.9462 ± 0.0043 | 0.9283 ± 0.0057 | 0.8971 ± 0.0047 | 0.8628 ± 0.0063 |
| 15 | BCIC-Speech | 0.5453 ± 0.0023 | 0.4316 ± 0.0029 | 0.5338 ± 0.0078 | 0.4172 ± 0.0098 |
| 16 | Mental Workload | 0.6493 ± 0.0211 | 0.2420 ± 0.0370 | 0.6296 ± 0.0334 | 0.3122 ± 0.0921 |
| Average | 0.6414 | 0.4474 | 0.6231 | 0.4344 |
04
Inside the shared semantic space after alignment
Shared geometry
EEG embeddings and text prototypes share structure.
Under task-and-target instructions, motor-related datasets occupy broadly similar regions, while emotion datasets share another region. These UMAP projections describe datasets included in instruction tuning; their structure may reflect both EEG features and shared instruction and label conditioning.
These projections provide a qualitative view of the learned representations. Transfer to unseen datasets is assessed separately in the held-out experiments; UMAP alone does not establish generalization or rule out dataset-specific memorization.
Reproduce LEAF
Models, training code, and documentation.
The repository contains the full pretraining, instruction-tuning, direct-inference, and downstream evaluation pipeline.
05
Citation
@article{jiang2026leaf,
title={LEAF: Language-EEG Aligned Foundation Model for Brain-Computer Interfaces},
author={Jiang, Muyun and Zhang, Shuailei and Yang, Zhenjie and Wu, Mengjun and Zhang, Wei and Jiang, Weibang and Liu, Chenyu and Guo, Zhiwei and Liu, Rui and Zhang, Shangen and Li, Yong and Ding, Yi and Guan, Cuntai},
journal={arXiv preprint arXiv:2509.24302},
year={2026}
}