Note
Go to the end to download the full example code or to run this example in your browser via Binder.
How do I evaluate participant-level p-factor regression?#
Estimate observed HBN p-factor from resting EEG using one row per participant. Six R5 mini participants keep acquisition bounded. These recordings are the 100 Hz challenge derivatives, already filtered at 0.5–50 Hz. First use downloads six recordings; the exact bytes depend on their duration. This small leave-one- participant-out exercise is not a challenge leaderboard estimate.
Before you start#
Use an installed EEGDash environment with MNE, NumPy, scikit-learn and
Matplotlib. No GPU is required. Set EEGDASH_CACHE_DIR to reuse the six
resting-state recordings; the subset needs roughly 100 MB on first download.
Cropping reduces processing time and memory, not the bytes needed to acquire
each recording.
The p-factor is an observed participant-level phenotype provided by the challenge. It is not a trial label, a diagnosis made from EEG, or a value to reconstruct from a participant’s identifier. A subject contributes exactly one feature row and one target, so long recordings cannot increase that subject’s weight simply by yielding more windows.
import os
from pathlib import Path
import matplotlib.pyplot as plt
import numpy as np
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import Ridge
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import LeaveOneOut
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from eegdash import EEGChallengeDataset
from eegdash.features import spectral_preprocessor, spectral_bands_power
from eegdash.const import SUBJECT_MINI_RELEASE_MAP
Load observed participant targets and recorded voltages. %% Load actual targets before computing features ———————————————
The sorted mini-release list provides a reproducible small subset rather
than selecting participants according to their outcomes. target_name names
the observed phenotype; description_fields makes the identity and target
available alongside each recording. The printed metadata lets you verify the
join between signal and participant.
A missing recording fails the coverage check. Missing or nonnumeric targets must be investigated at the source instead of filled with a group mean or a random number. For a larger cohort, specify missing-target exclusions before fitting a model and report the resulting number of people.
subjects = sorted(SUBJECT_MINI_RELEASE_MAP["R5"])[:6]
dataset = EEGChallengeDataset(
release="R5",
mini=True,
task="RestingState",
subject=subjects,
cache_dir=Path(
os.environ.get("EEGDASH_CACHE_DIR", "~/.eegdash_cache")
).expanduser(),
description_fields=["subject", "task", "p_factor"],
target_name="p_factor",
)
print(dataset.description.to_string(index=False))
assert len(dataset.datasets) == len(subjects)
╭────────────────────── EEG 2025 Competition Data Notice ──────────────────────╮
│ This object loads the HBN dataset that has been preprocessed for the EEG │
│ Challenge: │
│ * Downsampled from 500Hz to 100Hz │
│ * Bandpass filtered (0.5-50 Hz) │
│ │
│ For full preprocessing applied for competition details, see: │
│ https://github.com/eeg2025/downsample-datasets │
│ │
│ The HBN dataset have some preprocessing applied by the HBN team: │
│ * Re-reference (Cz Channel) │
│ │
│ IMPORTANT: The data accessed via `EEGChallengeDataset` is NOT identical to │
│ what you get from EEGDashDataset directly. │
│ If you are participating in the competition, always use │
│ `EEGChallengeDataset` to ensure consistency with the challenge data. │
╰──────────────────────── Source: EEGChallengeDataset ─────────────────────────╯
[09/16/26 20:48:45] INFO Auto-corrected misrouted dataset.py:561
storage.base for dataset
EEG2025r5mini: None ->
s3://nemar/EEG2025r5mini
subject task p_factor release_number sex age ehq_total commercial_use full_pheno attention internalizing externalizing restingstate despicableme funwithfractals thepresent diaryofawimpykid contrastchangedetection_1 contrastchangedetection_2 contrastchangedetection_3 surroundsupp_1 surroundsupp_2 seqlearning6target seqlearning8target symbolsearch
NDARAH793FBF RestingState 0.317 R5 M 9.3427 60.03 Yes No 0.819 0.485 -0.224 available available available available available available available available available available unavailable available available
NDARAJ689BVN RestingState 0.084 R5 F 5.4878 46.69 Yes No 1.227 0.271 0.115 available available available available available available available available available available available unavailable available
NDARAP785CTE RestingState -0.960 R5 M 8.7157 100.05 Yes No 1.556 -0.623 -0.685 available available available available available available available available available available unavailable available available
NDARAU708TL8 RestingState 0.710 R5 M 12.0751 100.05 Yes No -0.514 1.184 0.888 available available available available available available available available available available unavailable available available
NDARBE091BGD RestingState -0.332 R5 M 7.4098 -13.34 Yes Yes 1.408 -0.384 -0.077 available available available available available available available available available available available unavailable available
NDARBE103DHM RestingState 0.109 R5 M 16.5657 36.69 Yes No -0.252 0.678 -1.455 available available available available available available available available available available unavailable available available
Summarize a fixed resting interval#
crop(tmax=59) retains the recording from time zero through 59 seconds.
That fixed horizon prevents duration differences from deciding how much EEG
contributes to each row. It does not isolate a uniform eye condition: the
recorded resting instructions can occur within the interval.
EEGDash Welch spectra use non-overlapping 200-sample Hamming segments at
the source’s 100 Hz rate, giving 0.5 Hz frequency spacing. Features use four
ranges: 1–4, 4–8, 8–13 and 13–30 Hz. EEGDash’s spectral_preprocessor
computes the PSD and spectral_bands_power sums the selected bins.
Multiplying by their 0.5 Hz spacing approximates band power in V² before
log10. Each band includes its lower bound and excludes its upper bound,
so adjacent bands do not share bins. All participants use this convention.
The small floor
prevents undefined logarithms for flat channels such as the source reference;
it does not turn a flat electrode into an informative feature.
Concatenation is band-major, retaining channel order inside each band. The channel-order assertion keeps feature columns comparable across recordings.
features, targets, identities = [], [], []
channels = None
for recording in dataset.datasets:
raw = recording.raw.copy().pick("eeg").crop(tmax=59).load_data()
channels = raw.ch_names if channels is None else channels
assert raw.ch_names == channels
frequencies, psd = spectral_preprocessor(
raw.get_data(),
_metadata={"info": raw.info},
f_min=1,
f_max=30,
nperseg=200,
noverlap=0,
window="hamming",
)
powers = spectral_bands_power(
frequencies,
psd,
bands={"delta": (1, 4), "theta": (4, 8), "alpha": (8, 13), "beta": (13, 30)},
)
band_power = np.concatenate(list(powers.values())) * (
frequencies[1] - frequencies[0]
)
features.append(np.log10(np.maximum(band_power, 1e-30)))
targets.append(float(recording.description["p_factor"]))
identities.append(str(recording.description["subject"]))
print(
identities[-1],
len(raw.ch_names),
raw.info["sfreq"],
raw.annotations.description[:8],
)
Reading 0 ... 5900 = 0.000 ... 59.000 secs...
NDARAH793FBF 129 100.0 ['break cnt' 'resting_start' 'instructed_toOpenEyes']
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.bdf: 0%| | 0.00/12.9M [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.bdf: 0%| | 0.00/12.9M [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.bdf: 8%|▊ | 1.00M/12.9M [00:00<00:02, 4.98MB/s]
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.bdf: 100%|██████████| 12.9M/12.9M [00:00<00:00, 41.4MB/s]
Downloading sub-NDARAJ689BVN_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_channels.tsv: 100%|██████████| 3.34k/3.34k [00:00<00:00, 76.9kB/s]
Downloading sub-NDARAJ689BVN_task-RestingState_events.tsv: 0%| | 0.00/527 [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_events.tsv: 0%| | 0.00/527 [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_events.tsv: 100%|██████████| 527/527 [00:00<00:00, 13.9kB/s]
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARAJ689BVN_task-RestingState_eeg.json: 100%|██████████| 231/231 [00:00<00:00, 5.21kB/s]
Reading 0 ... 5900 = 0.000 ... 59.000 secs...
NDARAJ689BVN 129 100.0 ['break cnt' 'resting_start' 'instructed_toOpenEyes']
Downloading sub-NDARAP785CTE_task-RestingState_eeg.bdf: 0%| | 0.00/20.1M [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.bdf: 0%| | 0.00/20.1M [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.bdf: 5%|▍ | 1.00M/20.1M [00:00<00:04, 5.01MB/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.bdf: 84%|████████▍ | 17.0M/20.1M [00:00<00:00, 34.2MB/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.bdf: 100%|██████████| 20.1M/20.1M [00:00<00:00, 38.4MB/s]
Downloading sub-NDARAP785CTE_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_channels.tsv: 100%|██████████| 3.34k/3.34k [00:00<00:00, 73.8kB/s]
Downloading sub-NDARAP785CTE_task-RestingState_events.tsv: 0%| | 0.00/524 [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_events.tsv: 0%| | 0.00/524 [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_events.tsv: 100%|██████████| 524/524 [00:00<00:00, 13.3kB/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARAP785CTE_task-RestingState_eeg.json: 100%|██████████| 231/231 [00:00<00:00, 5.92kB/s]
Reading 0 ... 5900 = 0.000 ... 59.000 secs...
NDARAP785CTE 129 100.0 ['resting_start']
Downloading sub-NDARAU708TL8_task-RestingState_eeg.bdf: 0%| | 0.00/13.2M [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_eeg.bdf: 0%| | 0.00/13.2M [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_eeg.bdf: 8%|▊ | 1.00M/13.2M [00:00<00:03, 4.01MB/s]
Downloading sub-NDARAU708TL8_task-RestingState_eeg.bdf: 100%|██████████| 13.2M/13.2M [00:00<00:00, 32.9MB/s]
Downloading sub-NDARAU708TL8_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_channels.tsv: 100%|██████████| 3.34k/3.34k [00:00<00:00, 73.0kB/s]
Downloading sub-NDARAU708TL8_task-RestingState_events.tsv: 0%| | 0.00/523 [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_events.tsv: 0%| | 0.00/523 [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_events.tsv: 100%|██████████| 523/523 [00:00<00:00, 11.6kB/s]
Downloading sub-NDARAU708TL8_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARAU708TL8_task-RestingState_eeg.json: 100%|██████████| 231/231 [00:00<00:00, 5.11kB/s]
Reading 0 ... 5900 = 0.000 ... 59.000 secs...
NDARAU708TL8 129 100.0 ['break cnt' 'resting_start' 'instructed_toOpenEyes']
Downloading sub-NDARBE091BGD_task-RestingState_eeg.bdf: 0%| | 0.00/13.4M [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_eeg.bdf: 0%| | 0.00/13.4M [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_eeg.bdf: 22%|██▏ | 3.00M/13.4M [00:00<00:00, 14.4MB/s]
Downloading sub-NDARBE091BGD_task-RestingState_eeg.bdf: 100%|██████████| 13.4M/13.4M [00:00<00:00, 51.6MB/s]
Downloading sub-NDARBE091BGD_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_channels.tsv: 100%|██████████| 3.34k/3.34k [00:00<00:00, 70.9kB/s]
Downloading sub-NDARBE091BGD_task-RestingState_events.tsv: 0%| | 0.00/524 [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_events.tsv: 0%| | 0.00/524 [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_events.tsv: 100%|██████████| 524/524 [00:00<00:00, 6.91kB/s]
Downloading sub-NDARBE091BGD_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARBE091BGD_task-RestingState_eeg.json: 100%|██████████| 231/231 [00:00<00:00, 5.42kB/s]
Reading 0 ... 5900 = 0.000 ... 59.000 secs...
NDARBE091BGD 129 100.0 ['break cnt' 'resting_start' 'instructed_toOpenEyes']
Downloading sub-NDARBE103DHM_task-RestingState_eeg.bdf: 0%| | 0.00/14.1M [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_eeg.bdf: 0%| | 0.00/14.1M [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_eeg.bdf: 7%|▋ | 1.00M/14.1M [00:00<00:02, 5.11MB/s]
Downloading sub-NDARBE103DHM_task-RestingState_eeg.bdf: 100%|██████████| 14.1M/14.1M [00:00<00:00, 44.3MB/s]
Downloading sub-NDARBE103DHM_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_channels.tsv: 0%| | 0.00/3.34k [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_channels.tsv: 100%|██████████| 3.34k/3.34k [00:00<00:00, 41.7kB/s]
Downloading sub-NDARBE103DHM_task-RestingState_events.tsv: 0%| | 0.00/523 [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_events.tsv: 0%| | 0.00/523 [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_events.tsv: 100%|██████████| 523/523 [00:00<00:00, 13.1kB/s]
Downloading sub-NDARBE103DHM_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_eeg.json: 0%| | 0.00/231 [00:00<?, ?B/s]
Downloading sub-NDARBE103DHM_task-RestingState_eeg.json: 100%|██████████| 231/231 [00:00<00:00, 6.20kB/s]
Reading 0 ... 5900 = 0.000 ... 59.000 secs...
NDARBE103DHM 129 100.0 ['break cnt' 'resting_start']
Check the participant-level design matrix#
X has shape (participants, four bands × channels) and y has
one observed p-factor per row. With the current 129-channel recordings, this
means 516 predictors for only six participants. The identity and finiteness
assertions detect duplicated people, missing phenotypes and invalid features.
They do not test whether the EEG contains predictive information.
This high-dimensional, tiny-sample setting motivates regularization, but no penalty can make six participants sufficient for clinical inference. The page demonstrates a subject-independent evaluation boundary; it does not establish that the resulting features are invariant to subject identity.
X, y = np.asarray(features), np.asarray(targets)
assert len(set(identities)) == len(y) and np.isfinite(X).all() and np.isfinite(y).all()
print("Participant features:", X.shape, "observed targets:", y)
Participant features: (6, 516) observed targets: [ 0.317 0.084 -0.96 0.71 -0.332 0.109]
All scaling and baseline fitting occur inside the held-out participant fold. %% Fit inside each held-out participant fold —————————————–
Leave-one-out fits six models, each using five people and predicting the
remaining person once. StandardScaler is refitted inside each pipeline so
its means and spreads never include the held-out row. Ridge’s alpha=10 is a
fixed illustrative penalty, not a value selected from these six test errors.
The dummy model separately recomputes the training mean in every fold. Mean absolute error is reported in the provided p-factor scale, with equal weight per person. A smaller ridge error than the dummy error would be descriptive evidence on this subset only; a larger one is equally legitimate. The diagonal in the scatter denotes exact prediction, not a fitted regression line.
predicted, baseline = np.empty_like(y), np.empty_like(y)
for train, test in LeaveOneOut().split(X):
assert set(np.asarray(identities)[train]).isdisjoint(np.asarray(identities)[test])
model = make_pipeline(StandardScaler(), Ridge(alpha=10))
predicted[test] = model.fit(X[train], y[train]).predict(X[test])
baseline[test] = DummyRegressor().fit(X[train], y[train]).predict(X[test])
print("Participant MAE:", mean_absolute_error(y, predicted))
print("Training-mean MAE:", mean_absolute_error(y, baseline))
fig, ax = plt.subplots(figsize=(5, 4))
ax.scatter(y, predicted, label="held-out participant")
ax.plot([y.min(), y.max()], [y.min(), y.max()], "k--")
ax.set(xlabel="Observed p-factor", ylabel="Predicted p-factor")
ax.legend()
plt.show()
Participant MAE: 0.9230639824960019
Training-mean MAE: 0.5072
Separate model development from clinical interpretation#
First increase the number of independently held-out participants. If you want to choose the ridge penalty, channels, bands or resting interval, do so using an inner training-only validation split and keep the outer participant fold untouched. Reusing these outer errors to select features makes them model- development results rather than a final evaluation.
A useful extension is to compare observed EEG features with a prespecified metadata-only baseline, using the same people and folds. Report missing-target exclusions and participant-level uncertainty. Avoid interpreting one fitted coefficient as a clinical biomarker when predictors are correlated and the sample is this small.
Related evaluation example: Braindecode train, test and tune.
Total running time of the script: (0 minutes 4.556 seconds)
