# EEGDash — full markdown corpus Concatenation of every Sphinx-rendered markdown page on eegdash.org, produced at build time for LLM ingestion. See the curated index at for navigation; this file is the raw corpus.

EEGDash: the Python library for 700+ BIDS-first EEG/MEG datasets.

Install with `pip install eegdash`, then load, preprocess, and train PyTorch models on open EEG/MEG data in minutes. Works hand-in-hand with MNE-Python and braindecode. [Browse datasets](dataset_summary.md) [Get started](install/install.md) Quickstart ### Install ```bash pip install eegdash ``` ### First search ```python from eegdash import EEGDash eegdash = EEGDash() records = eegdash.find(dataset="ds002718") print(f"Found {len(records)} records.") ``` Works with Python 3.10+. BIDS-first. Runs locally. [Run your first search](quickstart.md) [Read the Docs](api/api.md)

At a glance

Search-first discovery with reproducible pipelines and standardized metadata.

[![Test Status](https://github.com/eegdash/EEGDash/actions/workflows/tests.yml/badge.svg)](https://github.com/eegdash/EEGDash/actions/workflows/tests.yml)[![Doc Status](https://github.com/eegdash/EEGDash/actions/workflows/doc.yaml/badge.svg)](https://github.com/eegdash/EEGDash/actions/workflows/doc.yaml)[![PyPI](https://img.shields.io/pypi/v/eegdash?color=blue&style=flat-square)](https://pypi.org/project/eegdash/)[![Python Versions](https://img.shields.io/pypi/pyversions/eegdash?style=flat-square)](https://pypi.org/project/eegdash/)[![Downloads](https://pepy.tech/badge/eegdash)](https://pepy.tech/project/eegdash)[![Code Coverage](https://codecov.io/gh/eegdash/EEGDash/branch/main/graph/badge.svg)](https://codecov.io/gh/eegdash/EEGDash)[![License](https://img.shields.io/pypi/l/eegdash?style=flat-square)](https://github.com/eegdash/EEGDash/blob/main/LICENSE)[![GitHub Stars](https://img.shields.io/github/stars/eegdash/eegdash?style=flat-square)](https://github.com/eegdash/EEGDash) 700+ Curated and standardized metadata ready to explore. 5 EEG, MEG, fNIRS, EMG, and iEEG coverage. BIDS Interoperability and reproducibility baked in. GitHub Community-driven datasets, pipelines, and benchmarks. Build with the community Share datasets, contribute pipelines, and help define open standards for EEG and MEG. [GitHub](https://github.com/eegdash/EEGDash) [Join Discord](https://discord.gg/8jd7nVKwsc) Support Institutions ![UCSD](_static/logos/ucsd_white.svg)![UCSD](_static/logos/ucsd_dark.svg)![Ben-Gurion University of the Negev (BGU)](_static/logos/bgu_dark.svg)![Ben-Gurion University of the Negev (BGU)](_static/logos/bgu_white.svg) Funders ![National Science Foundation (NSF)](_static/logos/nsf_logo.png) AWS Open Data Sponsorship Program # Installation EEGDash requires Python 3.11 or higher. The package is on [PyPI](https://pypi.org/project/eegdash), and the source lives on [GitHub](https://github.com/eegdash/eegdash). Two install paths, depending on what you need: Install via `pip` For Beginners ```shell pip install eegdash ``` ![EEGDash Installer with pip](_static/eegdash_install.gif) [Installing from PyPI](install_pip.md#install-pip) Building from source code For Advanced Users ![Terminal Window](https://mne.tools/stable/_images/mne_installer_console.png) For Python users who want the development version. Follow the setup instructions for building from GitHub. [From Source Code](install_source.md#install-source) # Installing from PyPI Install EEGDash from [PyPI](https://pypi.org/project/eegdash) with pip: ```shell pip install eegdash ``` This pulls in the `eegdash` package and its dependencies. #### NOTE Use a recent `pip`. Older versions may resolve dependencies poorly. # Installing from sources This page covers installing EEGDash from source, which is what you want for contributing or for trying features that have not yet been released. For an overview of contributor workflows and project internals, see [Developer Notes](../developer_notes.md). #### NOTE If you only want to install a released version, see [Installing from PyPI](install_pip.md). ## Install a pre-release from PyPI ```shell pip install --pre eegdash ``` This installs the in-development version of `eegdash` from the main branch. It may not be stable. ## Install directly from GitHub Clone the repository and change into it: ```shell git clone https://github.com/eegdash/EEGDash && cd EEGDash ``` ## Install with pip For a one-off install straight from GitHub: ```shell pip install git+https://github.com/eegdash/EEGDash.git ``` From a local clone, install in editable mode so source edits are picked up without reinstalling: ```shell pip install -e . ``` Optional extras let you pull in test and documentation dependencies: ```shell pip install -e .[test,docs,dev] ``` Or install everything, which is what you want for contributing: ```shell pip install -e .[all] ``` ## Verifying the installation ```shell python -c "import eegdash; print(eegdash.__version__)" ``` # Datasets Catalog EEG-DaSh is a data-sharing archive for MEEG (EEG, MEG) recordings contributed by collaborating labs. It preserves publicly funded research data and exposes it in a form that machine learning and deep learning workflows can use directly. **864** **46,551** **92,070** **5** Browse the catalog interactively at [eegdash.org/dataset_summary.html](https://eegdash.org/dataset_summary.html). The same data is available programmatically in Python: ```default from eegdash import EEGDashDataset # List every dataset records = EEGDashDataset.list_datasets() # Filter by task, modality, subject count, … rest_datasets = EEGDashDataset.list_datasets(task="rest") ``` Or via the HTTP API at `https://data.eegdash.org` (see the `/docs` Swagger UI and the [api catalog](/.well-known/api-catalog)). The archive is still in beta testing mode, so be kind. # EEGDash objects: `EEGDash`, `EEGDashDataset`, `EEGChallengeDataset` The library exposes three top-level objects that look similar at first glance but answer very different questions. Picking the wrong one is the most common source of confusion for new users. This page explains what each one is, what it gives you back, and why each exists. In short: - `EEGDash` is a **catalogue client**. It talks to the metadata service and returns *records* (dicts of metadata). Nothing is downloaded. - `EEGDashDataset` is a **PyTorch-compatible dataset**. It turns a catalogue query into a list of recordings that can be loaded, preprocessed, windowed, and iterated over. - `EEGChallengeDataset` is a **frozen, derivative dataset**, used for shared-benchmark contexts (currently the EEG2025 Competition). It loads pre-resampled, pre-filtered, pre-cut data so that every participant is evaluated against an identical signal. ## Records vs. datasets A *record* is a metadata document for one BIDS recording: which dataset it belongs to, which subject, task, session, run, channel count, sampling frequency, the path on S3, and so on. A record does **not** contain the samples themselves. `EEGDash.find()` returns records. A *dataset* is a Python object that lazily resolves records into actual EEG recordings (typically wrapping `mne.io.Raw` via the `EEGDashRaw` adapter). `EEGDashDataset` returns datasets. The first time you access `.raw` on one of its entries, the underlying file is downloaded into the local cache; subsequent accesses are offline. This split exists because metadata is small and cheap (you can search 700+ datasets in seconds), but raw EEG is large and slow (one HBN session is hundreds of MB). You usually want to inspect metadata first, decide what to keep, and only then trigger downloads. ## Typical use of `EEGDash` Use `EEGDash` when you want to *browse* the catalogue without committing to a download. The result is a list of dicts; you can filter it in pure Python before doing anything heavyweight. ```python from eegdash import EEGDash client = EEGDash() # Discover datasets matching loose, human-friendly filters. datasets_df = client.search_datasets(modality="eeg", task="rest", n_subjects_min=20) print(datasets_df[["dataset", "n_subjects", "task"]].head()) # Drill into one dataset and look at individual recordings. records = client.find({"dataset": "ds002718", "task": "FacePerception"}) print(f"Found {len(records)} recordings.") print(records[0].keys()) # subject, session, run, sampling_frequency, ... ``` No EEG samples were downloaded by either of those calls. The catalogue is the API surface; downloads are explicit and live one layer deeper. ## Typical use of `EEGDashDataset` Use `EEGDashDataset` when you want a real, indexable dataset that you can hand to braindecode preprocessing or a PyTorch `DataLoader`. It accepts the same filter keywords as `EEGDash.find` plus a `cache_dir` and a small set of dataset-construction options. ```python from eegdash import EEGDashDataset ds = EEGDashDataset( cache_dir="./eegdash_cache", dataset="ds002718", task="FacePerception", subject=["sub-002", "sub-003"], description_fields=["subject", "session", "task", "age"], ) print(len(ds)) # number of recordings print(ds.description.head()) # tidy metadata table raw = ds[0].raw # triggers the first download raw.filter(0.5, 40) # mne.io.Raw operations ``` ## Lazy loading and caching The dataset is *lazy*. Construction merely resolves the metadata; the `raw` attribute on each entry is materialised on first access and then held in memory for the lifetime of that object. The sample bytes are written to `cache_dir / `, mirroring the BIDS layout, so a second run with the same query is offline. If you set `download=False` and the cache already has the data, the catalogue is bypassed entirely and `EEGDashDataset` reads the local BIDS tree directly. This makes it straightforward to share a cache between machines or to work without network access once the first run completes. The lazy mode also lets you pass an `on_error` policy: in pipelines that scan many recordings, `on_error="skip"` flags problem files via `ds._skipped` so you can filter them out with a list comprehension (`ds.datasets = [d for d in ds.datasets if not getattr(d, "_skipped", False)]`) when a few files in a release are known to be corrupt. ## When to use `EEGChallengeDataset` `EEGChallengeDataset` is a thin wrapper over the same machinery, but points at a frozen, preprocessed bucket: the data are downsampled to a fixed rate, filtered with a fixed band, and cut into the canonical task blocks used by the EEG2025 Competition. If you are participating in the competition, you **must** use `EEGChallengeDataset`; otherwise your local results are not comparable to the public leaderboard. If you are not participating, `EEGChallengeDataset` is still useful when you want a fully reproducible benchmark: every user sees identical bytes. The library prints a notice when you try to load competition releases through plain `EEGDashDataset` to nudge users away from this footgun. ## Choosing among the three A practical decision tree: - “I just want to know what is out there.” → `EEGDash`. - “I want a PyTorch-style dataset for my own analysis.” → `EEGDashDataset`. - “I am running EEG2025 Competition code, or I want strictly identical preprocessing across users.” → `EEGChallengeDataset`. Mixing modes inside a single experiment is fine: a common workflow uses `EEGDash.search_datasets` to find candidate datasets, then constructs `EEGDashDataset` instances for the few you actually want to model. ## Related tutorials - [Find a small cohort with the EEGDash API](../generated/auto_examples/tutorials/00_start_here/plot_00_first_search.md) walks through metadata-only catalogue exploration with `EEGDash`. - [Inspect your first EEG recording](../generated/auto_examples/tutorials/00_start_here/plot_01_first_recording.md) contrasts the catalogue view with a first `EEGDashDataset` load. - [From EEGDash to a PyTorch DataLoader](../generated/auto_examples/tutorials/00_start_here/plot_02_dataset_to_dataloader.md) builds a PyTorch `DataLoader` on top of an `EEGDashDataset`. - [Save and reload prepared EEG windows](../generated/auto_examples/tutorials/10_core_workflow/plot_13_save_and_reuse_prepared_data.md) shows how to persist a preprocessed dataset to disk and reload without re-running the catalogue query. ## Further reading - Pernet, C. R., et al. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. *Scientific Data*, 6(1), 103. [https://doi.org/10.1038/s41597-019-0104-8](https://doi.org/10.1038/s41597-019-0104-8) - Gramfort, A., et al. (2013). MEG and EEG data analysis with MNE-Python. *Frontiers in Neuroscience*, 7, 267. [https://doi.org/10.3389/fnins.2013.00267](https://doi.org/10.3389/fnins.2013.00267) - Cisotto, G., & Chicco, D. (2024). Ten quick tips for clinical electroencephalographic (EEG) data acquisition and signal processing. *PeerJ Computer Science*, 10, e2256. [https://doi.org/10.7717/peerj-cs.2256](https://doi.org/10.7717/peerj-cs.2256) # Features vs. deep learning A recurring question in EEG decoding is whether to engineer features — band power, common spatial patterns (CSP), Riemannian covariance — or hand the raw signal to a convolutional or transformer-style network and let it learn its own representation. The answer depends on three things: how much data you have, how stationary the signal is, and how much inductive bias you can afford to bake in. The honest summary is that neither family dominates universally. On small, single-task, single-cohort decoding problems, well-tuned feature pipelines are often competitive with — and frequently better than — deep nets trained from scratch. On large, heterogeneous corpora and on tasks where the relevant features are not known a priori, end-to-end deep models clearly win. Schirrmeister et al. (2017) [1](#id3) remains the clearest demonstration that ConvNets *can* match expert feature pipelines on motor-imagery decoding, but their result holds at scale and with careful regularisation. ## When handcrafted features tend to win Pick features when at least two of the following are true: - **You have less than ~50 subjects.** A logistic regression or SVM on 10 well-chosen features fits with minimal regularisation. A convolutional network has hundreds of thousands of parameters and needs orders of magnitude more data to avoid memorizing subject identity (see [Leakage and evaluation](leakage_and_evaluation.md)). - **The relevant rhythm is known.** If you are decoding alpha-band modulation, log-power in 8–13 Hz is a near-optimal feature; a deep net will rediscover it in the best case and miss it in the worst. - **You need interpretability.** Feature pipelines come with named, reportable inputs (“alpha at Pz”, “central-mu lateralisation”). Deep features come with saliency maps that are notoriously hard to read. - **You need cross-dataset transfer.** Riemannian and CSP-based pipelines have well-understood invariances (channel permutation, reference change). A vanilla ConvNet trained on one montage can fail on another for trivial reasons. - **You are CPU-bound.** Feature pipelines fit on a laptop in seconds. The [Extract features from real EEG trials](../generated/auto_examples/tutorials/40_features/plot_40_first_features.md) tutorial shows the simplest version of this argument: build a band-power table, fit a logistic regression, and read off the result. ## When deep learning tends to win Pick a deep model when at least two of the following are true: - **You have hundreds of subjects.** Foundation-model-style deep decoders need large, diverse training sets to learn subject-invariant filters. EEGDash’s HBN-derived corpora make this regime accessible for the first time. - **The relevant feature is not known.** Tasks like cognitive workload, emotional state, or fatigue rarely have a single canonical band; an end-to-end model can carve out a useful representation that no human has named. - **You can afford to combine multiple datasets.** Deep models gain more from data diversity than from data volume; one large dataset is worth less than three medium ones with different montages. - **You can apply augmentation.** Mixup, channel dropout, time masking, and frequency masking close most of the gap between deep nets and hand-tuned features on small data; without augmentation, the deep model is usually under-regularised. - **You will fine-tune downstream.** A pre-trained deep model is a reusable asset; a hand-tuned feature pipeline is bespoke per task. The [Train a baseline on real SSVEP trials](../generated/auto_examples/tutorials/10_core_workflow/plot_12_train_a_baseline.md) tutorial trains a small braindecode ConvNet on the same data the feature tutorials use, so that you can compare the two pipelines head to head. ## The “feature first” rule A reliable practical workflow is to make the feature pipeline mandatory. Before you commit to a deep architecture for a new dataset: 1. Build a one-page feature pipeline (band power, ratios, simple covariance summary). The [Extract features from real EEG trials](../generated/auto_examples/tutorials/40_features/plot_40_first_features.md) recipe is enough. 2. Fit a logistic regression or shallow tree on top. 3. Use exactly the same split (preferably subject-aware; see [Leakage and evaluation](leakage_and_evaluation.md)). 4. Record the score and the variance across folds. Now you have a baseline. Anything that costs ten times more compute should outperform it on more than just the headline number — it should beat it under cross-subject evaluation, with smaller variance, and on held-out cohorts. If it does not, the feature pipeline *is* your deliverable. You will save weeks of GPU time, and your paper will be honest. The [Share spectral preprocessing with a feature tree](../generated/auto_examples/tutorials/40_features/plot_41_feature_trees.md) and [Fit scikit-learn to the saved real feature table](../generated/auto_examples/tutorials/40_features/plot_42_features_to_sklearn.md) tutorials extend the feature baseline to richer models (gradient boosting, full scikit-learn pipelines) without leaving the feature-engineering regime. ## What the literature says Three observations recur across the EEG-deep-learning reviews: - **Architecture matters less than regularisation and split discipline.** Roy et al. (2019) survey 156 deep EEG papers and find no consistent architecture winner; what changes results is whether the split respected subject identity (it often did not). - **At small N, ConvNets and feature pipelines are within noise.** Schirrmeister et al. (2017) [1](#id3) explicitly tune their ConvNet to match FBCSP on motor imagery; both reach high accuracy and the gap is dataset-dependent. - **Subject-invariant claims require evidence, not only architecture language.** A model labelled “subject-invariant” must be evaluated cross-subject on a held-out cohort, which loops back to [Leakage and evaluation](leakage_and_evaluation.md). The takeaway is not “always use features” or “always use deep nets”. It is that the choice is an experiment in itself, and the only way to make it honestly is to run both pipelines under the same evaluation. ## Related tutorials - [Extract features from real EEG trials](../generated/auto_examples/tutorials/40_features/plot_40_first_features.md) — minimal band-power feature pipeline. - [Share spectral preprocessing with a feature tree](../generated/auto_examples/tutorials/40_features/plot_41_feature_trees.md) — tree-based models on the same features. - [Fit scikit-learn to the saved real feature table](../generated/auto_examples/tutorials/40_features/plot_42_features_to_sklearn.md) — full scikit-learn integration. - [Train a baseline on real SSVEP trials](../generated/auto_examples/tutorials/10_core_workflow/plot_12_train_a_baseline.md) — a deep braindecode baseline on the same data, suitable for a head- to-head comparison. ## Further reading * **[1]** Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., & Ball, T. (2017). Deep learning with convolutional neural networks for EEG decoding and visualization. *Human Brain Mapping*, 38(11), 5391–5420. [https://doi.org/10.1002/hbm.23730](https://doi.org/10.1002/hbm.23730) - Roy, Y., Banville, H., Albuquerque, I., Gramfort, A., Falk, T. H., & Faubert, J. (2019). Deep learning-based electroencephalography analysis: a systematic review. *Journal of Neural Engineering*, 16(5), 051001. [https://doi.org/10.1088/1741-2552/ab260c](https://doi.org/10.1088/1741-2552/ab260c) - Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018). EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. *Journal of Neural Engineering*, 15(5), 056013. [https://doi.org/10.1088/1741-2552/aace8c](https://doi.org/10.1088/1741-2552/aace8c) - Cisotto, G., & Chicco, D. (2024). Ten quick tips for clinical electroencephalographic (EEG) data acquisition and signal processing. *PeerJ Computer Science*, 10, e2256. [https://doi.org/10.7717/peerj-cs.2256](https://doi.org/10.7717/peerj-cs.2256) # Concepts These pages explain *why*, not *how*. They describe the ideas, design choices, and tradeoffs behind EEGDash so that you can reason about your own pipeline, debug surprising results, and read the rest of the documentation with the right mental model. For step-by-step recipes, see the [Tutorials](../generated/auto_examples/index.md) and the How-To guides. The split between “explanation”, “tutorials”, “how-to guides”, and “reference” follows the [Diataxis framework](https://diataxis.fr). Each part of the documentation answers a different question: - **Tutorials** answer *learning* questions. They are guided lessons. - **How-to guides** answer *task* questions. They are recipes. - **Reference** answers *information* questions. It is exhaustive lookup. - **Concepts (this section)** answer *understanding* questions. They explain the model, not the keystrokes. When you find yourself wanting to know “but *why* does it work this way?”, a concept page is the right place to look. When a tutorial says “we use a subject-aware split here”, the underlying argument lives in [Leakage and evaluation](leakage_and_evaluation.md). When a how-to says “set a montage”, the reasoning is in [Preprocessing decisions](preprocessing_decisions.md). ## Available concept pages * [EEGDash objects: `EEGDash`, `EEGDashDataset`, `EEGChallengeDataset`](eegdash_objects.md) * [Metadata and BIDS entities](metadata_and_bids.md) * [Leakage and evaluation](leakage_and_evaluation.md) * [Preprocessing decisions](preprocessing_decisions.md) * [Features vs. deep learning](features_vs_deep_learning.md) Summary of each page: - [EEGDash objects: EEGDash, EEGDashDataset, EEGChallengeDataset](eegdash_objects.md) — The three main objects (`EEGDash`, `EEGDashDataset`, `EEGChallengeDataset`), what each one returns, and when to reach for which. - [Metadata and BIDS entities](metadata_and_bids.md) — How BIDS entities (subject, session, task, run) map to EEGDash query keywords, why standardized metadata matters, and how participant-level descriptors flow through the dataset. - [Leakage and evaluation](leakage_and_evaluation.md) — Why subject-level data leakage destroys generalization claims in EEG decoding, why random window splits are unsafe, and how within-subject, cross-session, and cross-subject evaluation differ. - [Preprocessing decisions](preprocessing_decisions.md) — What changes when you pick a high-pass cutoff, a montage, or a reference scheme. Defaults are not neutral choices: they encode assumptions about the signal and the question. - [Features vs. deep learning](features_vs_deep_learning.md) — When handcrafted features (band power, CSP, Riemannian) outperform deep nets and vice versa. How to pick a baseline that is informative, not just easy. ## How to use these pages Read a concept page **before** you start a project so you know which questions to ask. Re-read it **after** something surprising happens — a suspiciously high accuracy, a result that disappears across subjects, a filter that changes class boundaries — so you can localise the assumption that broke. The pages are deliberately short on syntax. The tutorials contain runnable code; the how-to guides contain task recipes; this section contains the reasoning that connects them. ## Further reading - Diataxis documentation framework. [https://diataxis.fr](https://diataxis.fr) - Cisotto, G., & Chicco, D. (2024). Ten quick tips for clinical electroencephalographic (EEG) data acquisition and signal processing. *PeerJ Computer Science*, 10, e2256. [https://doi.org/10.7717/peerj-cs.2256](https://doi.org/10.7717/peerj-cs.2256) - Pernet, C. R., et al. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. *Scientific Data*, 6(1), 103. [https://doi.org/10.1038/s41597-019-0104-8](https://doi.org/10.1038/s41597-019-0104-8) - Gramfort, A., et al. (2013). MEG and EEG data analysis with MNE-Python. *Frontiers in Neuroscience*, 7, 267. [https://doi.org/10.3389/fnins.2013.00267](https://doi.org/10.3389/fnins.2013.00267) # Leakage and evaluation Subject-level data leakage is the single most common reason that an EEG decoder claims a high cross-validated accuracy and then collapses on a new participant. It is also the reason a tutorial that scores 0.95 on a random window split can score 0.55 on a subject-disjoint split using the *same* data and the *same* model. This page explains the failure mode, why it is specific to physiological signals, and how the within-subject / cross-session / cross-subject distinction maps onto the splitters EEGDash provides. The advice below is consistent with Tip 9 of Cisotto & Chicco (2024) [1](#id2), which explicitly identifies subject-aware cross-validation as the only defensible default for clinical EEG. Tutorials that demonstrate the problem live in [Split real windows without subject leakage](../generated/auto_examples/tutorials/10_core_workflow/plot_11_leakage_safe_split.md), [Within-subject decoding on real trials](../generated/auto_examples/tutorials/50_evaluation/plot_50_within_subject_evaluation.md), [Cross-subject decoding on real SSVEP recordings](../generated/auto_examples/tutorials/50_evaluation/plot_51_cross_subject_evaluation.md), and [Transfer a decoder between recorded sessions](../generated/auto_examples/tutorials/50_evaluation/plot_52_cross_session_evaluation.md). ## Why subject leakage destroys generalization claims EEG signals carry strong, idiosyncratic, *subject-specific* statistics: skull thickness, hair impedance, electrode placement, baseline alpha power, blink habits, posture. A neural network with a few thousand free parameters easily latches onto those identity features because they generalize perfectly within a subject — every window from subject A “smells like” subject A. Now imagine a binary decoder for “eyes open vs. eyes closed”. You shuffle all windows across all subjects and split 80/20 randomly. The classifier quickly discovers that a few windows from each subject are in the test set, and its best strategy is to memorize the spectral fingerprint of subject A and reuse it for the held-out windows from subject A. It then reports an apparent accuracy of, say, 0.94. Unfortunately, this number is a lower-bounded *identification* accuracy plus the actual condition classification — and on a fresh participant the model may do no better than chance. The cleanest demonstration of this is to train the same architecture on two splits of the same dataset: a leaky random window split and a subject-disjoint split. The accuracy gap is the leakage tax, and it is typically 0.20–0.40 in absolute accuracy on real EEG decoding problems. ## Why random window splits are unsafe A *window* is a short, overlapping slice of a recording. If you make 2-second windows with 50% overlap, neighbouring windows share a full second of samples; their feature vectors differ only by smoothing. When you assign one to “train” and the other to “test”, the test score is almost a noise estimate, not a generalization estimate. This problem exists on top of the subject leakage problem: even within a single subject, randomising windows leaks information across the train/ test boundary because the windows overlap in time. The mitigation is twofold: 1. Split at the **recording or session level**, not the window level. 2. If a single recording must be split, choose a splitter that respects temporal contiguity (e.g., the first 80% by time for train, the last 20% for test). EEGDash defers the actual splitting to braindecode and MOABB, but the conceptual rule is the same regardless of library: a window must inherit the train/test label of its parent recording, never get assigned independently. ## Within-subject vs. cross-session vs. cross-subject These three terms describe what kind of generalization you are claiming to measure. They differ in which axis the held-out fold spans: - **Within-subject** evaluation holds out *time* within a single participant. Train on the first portion of recording, test on the last portion. Answer: *can the model decode this person’s signal later in the same session?* This is the easiest setting and the one most clinical BCI demos report. - **Cross-session** evaluation holds out a *different session* of the same participant. Train on session 1, test on session 2 (typically collected on a different day, with re-applied electrodes). Answer: *does the model survive electrode re-application and day-to-day drift?* This is the relevant setting for repeated-use BCIs and for any real-world deployment where calibration is rare. - **Cross-subject** evaluation holds out *different participants*. Train on subjects A–T, test on subjects U–Z. Answer: *does the model generalize to a person it has never seen?* This is the standard for any “subject-invariant” or “foundation-model” claim. Each setting answers a different scientific question, so neither one is universally correct. The mistake is to *report* one and *implicitly claim* another. A paper that splits randomly and then advertises a “general-purpose decoder” is overstating the evaluation; a paper that holds out a session and accurately calls it cross-session is doing honest work even if the number is lower. ## Practical guidance 1. Always inspect `ds.description["subject"]` and `ds.description["session"]` before choosing a splitter. If any subject appears in more than one fold, the split is leaky by construction. 2. Treat the split function as part of the experiment, not a utility. Print, log, and version-control the participants in each fold. Tutorials such as [Split real windows without subject leakage](../generated/auto_examples/tutorials/10_core_workflow/plot_11_leakage_safe_split.md) include an audit step that verifies disjointness. 3. Pick a splitter that matches your scientific question — within-subject, cross-session, or cross-subject. If you cannot decide, default to cross-subject; it is the strictest of the three and rarely misleading. 4. Always include a chance-level baseline and, where possible, a simple feature baseline (see [Features vs. deep learning](features_vs_deep_learning.md)). A neural network that beats random by 3 points but loses to a logistic regression on band power has not learned the task. 5. Report variance across folds, not just the mean. Subject-level variance dominates EEG; a mean accuracy with no error bars is not a measurement. ## What “metric leakage” looks like in practice A few diagnostic patterns you should watch for: - **Suspiciously high accuracy on hard problems.** A decoder for emotional state from 30 seconds of resting EEG that scores 0.92 is almost certainly leaking subject identity. - **Accuracy drops on new subjects.** A 25-point drop between cross-validated and held-out cohorts is a leakage signal, not an overfitting signal. - **Random labels still score above chance.** If you shuffle the labels per recording but keep them constant within a recording, a leaky pipeline still scores well above chance because it is fitting recording identity. When you see one of those, re-read this page and re-check the splitter. ## Related tutorials - [Split real windows without subject leakage](../generated/auto_examples/tutorials/10_core_workflow/plot_11_leakage_safe_split.md) is the canonical demonstration of leaky vs. safe splits on EEGDash data. - [Within-subject decoding on real trials](../generated/auto_examples/tutorials/50_evaluation/plot_50_within_subject_evaluation.md), [Cross-subject decoding on real SSVEP recordings](../generated/auto_examples/tutorials/50_evaluation/plot_51_cross_subject_evaluation.md), and [Transfer a decoder between recorded sessions](../generated/auto_examples/tutorials/50_evaluation/plot_52_cross_session_evaluation.md) show the same dataset evaluated under each protocol. - [Learning curves from real training subjects](../generated/auto_examples/tutorials/50_evaluation/plot_53_learning_curves.md) illustrates how leakage interacts with sample-size effects. - [Compare pipelines on paired real subject folds](../generated/auto_examples/tutorials/50_evaluation/plot_54_compare_two_pipelines.md) shows how to compare pipelines once a defensible split is in place. ## Further reading * **[1]** Cisotto, G., & Chicco, D. (2024). Ten quick tips for clinical electroencephalographic (EEG) data acquisition and signal processing. *PeerJ Computer Science*, 10, e2256. [https://doi.org/10.7717/peerj-cs.2256](https://doi.org/10.7717/peerj-cs.2256) - Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C., & Kording, K. P. (2017). The need to approximate the use-case in clinical machine learning. *GigaScience*, 6(5), 1–9. [https://doi.org/10.1093/gigascience/gix019](https://doi.org/10.1093/gigascience/gix019) - Roy, Y., Banville, H., Albuquerque, I., Gramfort, A., Falk, T. H., & Faubert, J. (2019). Deep learning-based electroencephalography analysis: a systematic review. *Journal of Neural Engineering*, 16(5), 051001. [https://doi.org/10.1088/1741-2552/ab260c](https://doi.org/10.1088/1741-2552/ab260c) - Pernet, C. R., et al. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. *Scientific Data*, 6(1), 103. [https://doi.org/10.1038/s41597-019-0104-8](https://doi.org/10.1038/s41597-019-0104-8) # Metadata and BIDS entities EEG-BIDS is the contract that lets EEGDash advertise 700+ datasets as a single queryable catalogue. Every recording the library knows about has been described using the same vocabulary — *subject*, *session*, *task*, *run*, acquisition system, electrode montage, units, sampling frequency — so that filters work the same way regardless of the originating lab or repository. This page explains what those entities mean, how they map to EEGDash query keywords, and where participant-level metadata enters the picture. The EEG-BIDS specification was published as Pernet et al. (2019) [1](#id3); if you have not skimmed the paper, the short version is that BIDS standardizes file names, directory layout, and accompanying JSON/TSV sidecars so that any tool can walk a dataset without bespoke loaders. EEGDash reuses those entity names verbatim in its query API. ## BIDS entities at a glance A BIDS-compliant EEG file path encodes the following entities (with the most common ones in **bold**): - **subject** (`sub-XYZ`): one participant. - **session** (`ses-N`): one acquisition appointment for that participant. A subject can have multiple sessions when data are collected across days. - **task** (`task-ABC`): the experimental paradigm (resting state, oddball, steady-state visual stimulation, etc.). - **run** (`run-K`): a repetition of the same task within one session. - *acquisition* (`acq-XYZ`): an acquisition variant, such as a different amplifier or montage. - *processing* (`proc-XYZ`): a derivative pipeline tag. The entities are hierarchical: a recording is uniquely identified by the combination of dataset and the entities present in its filename. EEGDash’s catalogue stores these as separate fields in MongoDB, which is what makes queries like “all resting-state runs from subject sub-002 of ds002718” trivial to express. ## Mapping BIDS entities to EEGDash query keywords EEGDash’s filter keywords are intentionally identical to the BIDS entity names. The table below shows the mapping. A keyword accepts a single string or a list of strings; lists are interpreted as MongoDB `$in` queries. | BIDS entity | EEGDash query keyword | Notes | |----------------|-------------------------|-----------------------------------------------------------------------------------------------------------------------| | `sub-