Held-out datasets
These motor-imagery datasets are excluded from configs/tasks.yaml and never appear in instruction tuning. They are reserved for zero-shot evaluation with the frozen instruction-tuned LEAF model.
MI_Weibo2014
- Role: held-out binary motor-imagery evaluation with 10 subjects.
- Preprocessing: read exported
.npzfiles, keep only event codes 1/2 for left/right hand, convert them to labels 0/1, band-pass filter at 0.3-40 Hz, resample to 200 Hz when necessary, and apply the shared pipeline. - Evaluation loading:
zero_shot_heldout.pyconcatenates all subjects unless exclusions are supplied and treats0=Left,1=Rightas verified.
MI_Dreyer2023
- Role: held-out binary motor-imagery evaluation.
- Expected input:
/media/public/LEAF/MI_Dreyer2023.h5with nestedtrain/X,Y,val/X,Y, andtest/X,Ygroups. - Expected shapes: 10,000/2,152/2,160 trials with shape
(N, 65, 801); evaluation keeps the first 800 samples.
Zero-shot evaluation
zero_shot_heldout.py evaluates Dreyer2023 and Weibo2014 with frozen MPNet or Qwen3 LEAF checkpoints at instruction levels 0, 1, and 2. It uses only the Left and Right text prototypes and reports balanced accuracy, Cohen's kappa, weighted F1, AUROC, confusion counts, and label-mapping provenance.