Note
Go to the end to download the full example code or to run this example in your browser via Binder.
Predict observed p-factor from real EEG features#
Estimated reading time:8 minutes
Evaluate a Ridge baseline with one held-out participant at a time.
Use six explicitly selected participants from the real
HBN ds005505 release. Their
RestingState signal files total approximately 595.8 MB, cached under
EEGDASH_CACHE_DIR. This applied example is larger than the introductory
21 MB SSVEP subset. Cropping after loading reduces computation, not download.
Targets come from the observed participant metadata. Six participants are
sufficient to exercise the workflow, not to support clinical conclusions.
Before you start#
Install EEGDash with its EEGPrep tutorial dependencies and scikit-learn. Tutorials 02, 11 and 40 introduce windows, grouped evaluation and spectral features. This file runs independently: it predicts the observed p-factor score from one feature row per participant and prints each held-out prediction.
import os
from pathlib import Path
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from braindecode.preprocessing import (
create_fixed_length_windows,
preprocess,
Resampling,
RemoveDrifts,
)
from eegdash import EEGDashDataset
from functools import partial
from eegdash.features import (
FeatureExtractor,
extract_features,
spectral_bands_power,
spectral_preprocessor,
)
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import Ridge
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import LeaveOneOut
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
1. Load the named cohort and inspect observed participant metadata#
p_factor is a recorded participant-level research score. Inspect the
printed values before conversion; a missing score must not be replaced by a
label derived from EEG power or array order. This project predicts that
particular released score and does not interpret it as a diagnosis.
The two assertions require one selected recording per named participant.
description_fields makes the source attributes available alongside BIDS
identifiers; the EEG still comes from EEGDashDataset. These are the original
OpenNeuro recordings, not the separately filtered/downsampled challenge
release supplied by EEGChallengeDataset.
subjects = [
"NDARBH024NH2",
"NDARAM704GKZ",
"NDARAC904DMU",
"NDARAN385MDH",
"NDARAG143ARJ",
"NDARAP359UM6",
]
cache_dir = Path(os.environ.get("EEGDASH_CACHE_DIR", ".eegdash_cache"))
dataset = EEGDashDataset(
dataset="ds005505",
task="RestingState",
subject=subjects,
cache_dir=cache_dir,
description_fields=["subject", "task", "age", "sex", "p_factor"],
n_jobs=1,
)
assert len(dataset.datasets) == len(subjects)
assert dataset.description["subject"].nunique() == len(subjects)
print(dataset.description[["subject", "age", "sex", "p_factor"]])
╭────────────────────── EEG 2025 Competition Data Notice ──────────────────────╮
│ This notice is only for users who are participating in the EEG 2025 │
│ Competition. │
│ │
│ EEG 2025 Competition Data Notice! │
│ You are loading one of the datasets that is used in competition, but via │
│ `EEGDashDataset`. │
│ │
│ IMPORTANT: │
│ If you download data from `EEGDashDataset`, it is NOT identical to the │
│ official │
│ competition data, which is accessed via `EEGChallengeDataset`. The │
│ competition data has been downsampled and filtered. │
│ │
│ If you are participating in the competition, │
│ you must use the `EEGChallengeDataset` object to ensure consistency. │
│ │
│ If you are not participating in the competition, you can ignore this │
│ message. │
╰─────────────────────────── Source: EEGDashDataset ───────────────────────────╯
subject age sex p_factor
0 NDARAC904DMU 11.3386 F -0.603
1 NDARAG143ARJ 7.6648 M -0.258
2 NDARAM704GKZ 10.9449 M 0.062
3 NDARAN385MDH 10.7089 M 0.034
4 NDARAP359UM6 12.8422 F 0.467
5 NDARBH024NH2 8.6883 F -0.818
2. Prepare the first minute of recorded EEG#
The fixed first-minute interval and four named electrodes bound computation and establish a common channel order. They are not selected by test accuracy. The source reference channel can be flat; the chosen channels exclude that reference rather than pretending a zero-variance channel can be standardized.
Braindecode’s EEGPrep Resampling adapter reduces the rate to 100 Hz,
with anti-alias filtering, before RemoveDrifts applies its high-pass
transition from 0.5 to 1 Hz. Resampling first reduces the later filter’s
computation. These published components run on each recording separately;
no custom filtering function or cross-participant fit is needed.
A two-second window then contains 200 voltage samples. The source reference is retained: we do not average-reference an arbitrary four-channel subset. EEGPrep’s forward-backward drift filter is offline, not causal streaming preprocessing. This small recipe neither detects every artifact nor invokes channel rejection or ASR. The feature models below use only 1–30 Hz.
Fixed-length windows are used because the target belongs to a person, not an
event. They do not relabel eyes-open/closed intervals or remove every
instruction or artifact from the first minute. Equal size and stride avoid
overlap; a final short remainder is discarded. EEGDash reads the windows
in batches shaped (windows, 4 channels, 200 samples). Check the printed
window count rather than assuming each recording retains the same number.
channels = ["E11", "E62", "E75", "E22"]
for recording in dataset.datasets:
raw = recording.raw
print(
recording.description["subject"],
raw.info["sfreq"],
raw.ch_names,
"observed annotations:",
sorted(set(raw.annotations.description)),
)
assert set(channels).issubset(raw.ch_names)
raw.crop(tmax=59.99).load_data().pick(channels).reorder_channels(channels)
# Preserve annotation times in seconds across EEGPrep format conversions.
# Retain the measurement date too: it anchors annotations with absolute times.
annotations_before = [
recording.raw.annotations.copy() for recording in dataset.datasets
]
measurement_dates = [recording.raw.info["meas_date"] for recording in dataset.datasets]
durations_before = [
recording.raw.n_times / recording.raw.info["sfreq"]
for recording in dataset.datasets
]
preprocess(
dataset, [Resampling(sfreq=100), RemoveDrifts(transition=(0.5, 1.0))], n_jobs=1
)
for recording, annotations, duration, measurement_date in zip(
dataset.datasets, annotations_before, durations_before, measurement_dates
):
raw = recording.raw
assert raw.ch_names == channels and raw.info["sfreq"] == 100
assert abs(raw.n_times / 100 - duration) <= 1 / 100
raw.set_meas_date(measurement_date)
raw.set_annotations(annotations)
assert raw.annotations.orig_time == annotations.orig_time
np.testing.assert_array_equal(raw.annotations.onset, annotations.onset)
np.testing.assert_array_equal(raw.annotations.description, annotations.description)
windows = create_fixed_length_windows(
dataset,
window_size_samples=200,
window_stride_samples=200,
on_last_window="drop",
preload=True,
)
metadata = windows.get_metadata().reset_index(drop=True)
groups = metadata["subject"].astype(str).to_numpy()
assert len(windows) == len(metadata)
assert not metadata.duplicated(["subject", "i_start_in_trial"]).any()
print("Real two-second windows:", len(windows), "with", len(channels), "channels")
NDARAC904DMU 500.0 ['E1', 'E2', 'E3', 'E4', 'E5', 'E6', 'E7', 'E8', 'E9', 'E10', 'E11', 'E12', 'E13', 'E14', 'E15', 'E16', 'E17', 'E18', 'E19', 'E20', 'E21', 'E22', 'E23', 'E24', 'E25', 'E26', 'E27', 'E28', 'E29', 'E30', 'E31', 'E32', 'E33', 'E34', 'E35', 'E36', 'E37', 'E38', 'E39', 'E40', 'E41', 'E42', 'E43', 'E44', 'E45', 'E46', 'E47', 'E48', 'E49', 'E50', 'E51', 'E52', 'E53', 'E54', 'E55', 'E56', 'E57', 'E58', 'E59', 'E60', 'E61', 'E62', 'E63', 'E64', 'E65', 'E66', 'E67', 'E68', 'E69', 'E70', 'E71', 'E72', 'E73', 'E74', 'E75', 'E76', 'E77', 'E78', 'E79', 'E80', 'E81', 'E82', 'E83', 'E84', 'E85', 'E86', 'E87', 'E88', 'E89', 'E90', 'E91', 'E92', 'E93', 'E94', 'E95', 'E96', 'E97', 'E98', 'E99', 'E100', 'E101', 'E102', 'E103', 'E104', 'E105', 'E106', 'E107', 'E108', 'E109', 'E110', 'E111', 'E112', 'E113', 'E114', 'E115', 'E116', 'E117', 'E118', 'E119', 'E120', 'E121', 'E122', 'E123', 'E124', 'E125', 'E126', 'E127', 'E128', 'Cz'] observed annotations: ['break cnt', 'dot_no1_OFF', 'dot_no1_ON', 'dot_no2_OFF', 'dot_no2_ON', 'dot_no3_OFF', 'dot_no3_ON', 'dot_no4_OFF', 'dot_no4_ON', 'dot_no5_OFF', 'dot_no5_ON', 'dot_no6_OFF', 'dot_no6_ON', 'dot_no7_OFF', 'dot_no7_ON', 'dot_no8_OFF', 'dot_no8_ON', 'instructed_toCloseEyes', 'instructed_toOpenEyes', 'learningBlock_1', 'resting_start', 'seqLearning_start']
NDARAG143ARJ 500.0 ['E1', 'E2', 'E3', 'E4', 'E5', 'E6', 'E7', 'E8', 'E9', 'E10', 'E11', 'E12', 'E13', 'E14', 'E15', 'E16', 'E17', 'E18', 'E19', 'E20', 'E21', 'E22', 'E23', 'E24', 'E25', 'E26', 'E27', 'E28', 'E29', 'E30', 'E31', 'E32', 'E33', 'E34', 'E35', 'E36', 'E37', 'E38', 'E39', 'E40', 'E41', 'E42', 'E43', 'E44', 'E45', 'E46', 'E47', 'E48', 'E49', 'E50', 'E51', 'E52', 'E53', 'E54', 'E55', 'E56', 'E57', 'E58', 'E59', 'E60', 'E61', 'E62', 'E63', 'E64', 'E65', 'E66', 'E67', 'E68', 'E69', 'E70', 'E71', 'E72', 'E73', 'E74', 'E75', 'E76', 'E77', 'E78', 'E79', 'E80', 'E81', 'E82', 'E83', 'E84', 'E85', 'E86', 'E87', 'E88', 'E89', 'E90', 'E91', 'E92', 'E93', 'E94', 'E95', 'E96', 'E97', 'E98', 'E99', 'E100', 'E101', 'E102', 'E103', 'E104', 'E105', 'E106', 'E107', 'E108', 'E109', 'E110', 'E111', 'E112', 'E113', 'E114', 'E115', 'E116', 'E117', 'E118', 'E119', 'E120', 'E121', 'E122', 'E123', 'E124', 'E125', 'E126', 'E127', 'E128', 'Cz'] observed annotations: ['boundary', 'break cnt', 'instructed_toCloseEyes', 'instructed_toOpenEyes', 'resting_start']
NDARAM704GKZ 500.0 ['E1', 'E2', 'E3', 'E4', 'E5', 'E6', 'E7', 'E8', 'E9', 'E10', 'E11', 'E12', 'E13', 'E14', 'E15', 'E16', 'E17', 'E18', 'E19', 'E20', 'E21', 'E22', 'E23', 'E24', 'E25', 'E26', 'E27', 'E28', 'E29', 'E30', 'E31', 'E32', 'E33', 'E34', 'E35', 'E36', 'E37', 'E38', 'E39', 'E40', 'E41', 'E42', 'E43', 'E44', 'E45', 'E46', 'E47', 'E48', 'E49', 'E50', 'E51', 'E52', 'E53', 'E54', 'E55', 'E56', 'E57', 'E58', 'E59', 'E60', 'E61', 'E62', 'E63', 'E64', 'E65', 'E66', 'E67', 'E68', 'E69', 'E70', 'E71', 'E72', 'E73', 'E74', 'E75', 'E76', 'E77', 'E78', 'E79', 'E80', 'E81', 'E82', 'E83', 'E84', 'E85', 'E86', 'E87', 'E88', 'E89', 'E90', 'E91', 'E92', 'E93', 'E94', 'E95', 'E96', 'E97', 'E98', 'E99', 'E100', 'E101', 'E102', 'E103', 'E104', 'E105', 'E106', 'E107', 'E108', 'E109', 'E110', 'E111', 'E112', 'E113', 'E114', 'E115', 'E116', 'E117', 'E118', 'E119', 'E120', 'E121', 'E122', 'E123', 'E124', 'E125', 'E126', 'E127', 'E128', 'Cz'] observed annotations: ['boundary', 'break cnt', 'instructed_toCloseEyes', 'instructed_toOpenEyes', 'resting_start']
NDARAN385MDH 500.0 ['E1', 'E2', 'E3', 'E4', 'E5', 'E6', 'E7', 'E8', 'E9', 'E10', 'E11', 'E12', 'E13', 'E14', 'E15', 'E16', 'E17', 'E18', 'E19', 'E20', 'E21', 'E22', 'E23', 'E24', 'E25', 'E26', 'E27', 'E28', 'E29', 'E30', 'E31', 'E32', 'E33', 'E34', 'E35', 'E36', 'E37', 'E38', 'E39', 'E40', 'E41', 'E42', 'E43', 'E44', 'E45', 'E46', 'E47', 'E48', 'E49', 'E50', 'E51', 'E52', 'E53', 'E54', 'E55', 'E56', 'E57', 'E58', 'E59', 'E60', 'E61', 'E62', 'E63', 'E64', 'E65', 'E66', 'E67', 'E68', 'E69', 'E70', 'E71', 'E72', 'E73', 'E74', 'E75', 'E76', 'E77', 'E78', 'E79', 'E80', 'E81', 'E82', 'E83', 'E84', 'E85', 'E86', 'E87', 'E88', 'E89', 'E90', 'E91', 'E92', 'E93', 'E94', 'E95', 'E96', 'E97', 'E98', 'E99', 'E100', 'E101', 'E102', 'E103', 'E104', 'E105', 'E106', 'E107', 'E108', 'E109', 'E110', 'E111', 'E112', 'E113', 'E114', 'E115', 'E116', 'E117', 'E118', 'E119', 'E120', 'E121', 'E122', 'E123', 'E124', 'E125', 'E126', 'E127', 'E128', 'Cz'] observed annotations: ['boundary', 'break cnt', 'instructed_toCloseEyes', 'instructed_toOpenEyes', 'resting_start']
NDARAP359UM6 500.0 ['E1', 'E2', 'E3', 'E4', 'E5', 'E6', 'E7', 'E8', 'E9', 'E10', 'E11', 'E12', 'E13', 'E14', 'E15', 'E16', 'E17', 'E18', 'E19', 'E20', 'E21', 'E22', 'E23', 'E24', 'E25', 'E26', 'E27', 'E28', 'E29', 'E30', 'E31', 'E32', 'E33', 'E34', 'E35', 'E36', 'E37', 'E38', 'E39', 'E40', 'E41', 'E42', 'E43', 'E44', 'E45', 'E46', 'E47', 'E48', 'E49', 'E50', 'E51', 'E52', 'E53', 'E54', 'E55', 'E56', 'E57', 'E58', 'E59', 'E60', 'E61', 'E62', 'E63', 'E64', 'E65', 'E66', 'E67', 'E68', 'E69', 'E70', 'E71', 'E72', 'E73', 'E74', 'E75', 'E76', 'E77', 'E78', 'E79', 'E80', 'E81', 'E82', 'E83', 'E84', 'E85', 'E86', 'E87', 'E88', 'E89', 'E90', 'E91', 'E92', 'E93', 'E94', 'E95', 'E96', 'E97', 'E98', 'E99', 'E100', 'E101', 'E102', 'E103', 'E104', 'E105', 'E106', 'E107', 'E108', 'E109', 'E110', 'E111', 'E112', 'E113', 'E114', 'E115', 'E116', 'E117', 'E118', 'E119', 'E120', 'E121', 'E122', 'E123', 'E124', 'E125', 'E126', 'E127', 'E128', 'Cz'] observed annotations: ['boundary', 'break cnt', 'instructed_toCloseEyes', 'instructed_toOpenEyes', 'resting_start']
NDARBH024NH2 500.0 ['E1', 'E2', 'E3', 'E4', 'E5', 'E6', 'E7', 'E8', 'E9', 'E10', 'E11', 'E12', 'E13', 'E14', 'E15', 'E16', 'E17', 'E18', 'E19', 'E20', 'E21', 'E22', 'E23', 'E24', 'E25', 'E26', 'E27', 'E28', 'E29', 'E30', 'E31', 'E32', 'E33', 'E34', 'E35', 'E36', 'E37', 'E38', 'E39', 'E40', 'E41', 'E42', 'E43', 'E44', 'E45', 'E46', 'E47', 'E48', 'E49', 'E50', 'E51', 'E52', 'E53', 'E54', 'E55', 'E56', 'E57', 'E58', 'E59', 'E60', 'E61', 'E62', 'E63', 'E64', 'E65', 'E66', 'E67', 'E68', 'E69', 'E70', 'E71', 'E72', 'E73', 'E74', 'E75', 'E76', 'E77', 'E78', 'E79', 'E80', 'E81', 'E82', 'E83', 'E84', 'E85', 'E86', 'E87', 'E88', 'E89', 'E90', 'E91', 'E92', 'E93', 'E94', 'E95', 'E96', 'E97', 'E98', 'E99', 'E100', 'E101', 'E102', 'E103', 'E104', 'E105', 'E106', 'E107', 'E108', 'E109', 'E110', 'E111', 'E112', 'E113', 'E114', 'E115', 'E116', 'E117', 'E118', 'E119', 'E120', 'E121', 'E122', 'E123', 'E124', 'E125', 'E126', 'E127', 'E128', 'Cz'] observed annotations: ['boundary', 'break cnt', 'instructed_toCloseEyes', 'instructed_toOpenEyes', 'resting_start']
/tmp/tmpuj4sbwko.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpbk_fpdk1.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmp4psk433e.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpne0aj_6z.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmp07ptb74d.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpii0n58u3.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpbt2ftbvp.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpolvkbsla.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpc4kgdnpr.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmpa4atj8xx.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmp2em6dkbu.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
/tmp/tmp2txateuy.set
Field 'subject' is missing from the EEG dictionnary, adding it.
Field 'group' is missing from the EEG dictionnary, adding it.
Field 'condition' is missing from the EEG dictionnary, adding it.
Field 'session' is missing from the EEG dictionnary, adding it.
Field 'comments' is missing from the EEG dictionnary, adding it.
Field 'times' is missing from the EEG dictionnary, adding it.
Field 'icaact' is missing from the EEG dictionnary, adding it.
Field 'icachansind' is missing from the EEG dictionnary, adding it.
Field 'urchanlocs' is missing from the EEG dictionnary, adding it.
Field 'urevent' is missing from the EEG dictionnary, adding it.
Field 'eventdescription' is missing from the EEG dictionnary, adding it.
Field 'epoch' is missing from the EEG dictionnary, adding it.
Field 'epochdescription' is missing from the EEG dictionnary, adding it.
Field 'stats' is missing from the EEG dictionnary, adding it.
Field 'specdata' is missing from the EEG dictionnary, adding it.
Field 'specicaact' is missing from the EEG dictionnary, adding it.
Field 'splinefile' is missing from the EEG dictionnary, adding it.
Field 'icasplinefile' is missing from the EEG dictionnary, adding it.
Field 'dipfit' is missing from the EEG dictionnary, adding it.
Field 'history' is missing from the EEG dictionnary, adding it.
Field 'saved' is missing from the EEG dictionnary, adding it.
Field 'etc' is missing from the EEG dictionnary, adding it.
Field 'datfile' is missing from the EEG dictionnary, adding it.
Field 'run' is missing from the EEG dictionnary, adding it.
Field 'roi' is missing from the EEG dictionnary, adding it.
Real two-second windows: 180 with 4 channels
3. Average window band powers into one feature row per participant#
EEGDash’s FeatureExtractor shares one Welch spectrum between the four
band-power outputs. nperseg=200 spans the two-second window, giving a
0.5 Hz grid; four bands times four electrodes produce 16 named columns.
spectral_bands_power sums PSD values in each band. Its scale depends on
that fixed grid, so preserve the rate and segment length when reusing this
feature definition. No hand-written FFT or band-reduction function is needed.
Log compression acts on each trial’s feature values. We then group the
resulting table by subject and average within each person before fitting.
Thus X has six rows, and no participant gains weight merely because
more windows were retained. The same sorted index selects the observed
participant targets, preserving feature/target alignment.
bands = {"delta": (1, 4), "theta": (4, 8), "alpha": (8, 13), "beta": (13, 30)}
spectral = FeatureExtractor(
{"power": partial(spectral_bands_power, bands=bands)},
preprocessor=partial(
spectral_preprocessor, fs=100, nperseg=200, noverlap=0, f_min=1, f_max=30
),
)
feature_table = extract_features(
windows, spectral, batch_size=64, n_jobs=1
).to_dataframe()
assert feature_table.shape == (len(metadata), len(bands) * len(channels))
participant_features = (
np.log10(feature_table.clip(lower=1e-30))
.assign(subject=groups)
.groupby("subject")
.mean()
)
identities = participant_features.index.to_numpy()
X = participant_features.to_numpy()
participants = dataset.description.set_index("subject").loc[identities]
assert np.isfinite(X).all() and len(X) == len(subjects)
y = pd.to_numeric(participants["p_factor"], errors="raise").to_numpy(dtype=float)
assert np.isfinite(y).all()
Extracting features: 0%| | 0/6 [00:00<?, ?it/s]
Extracting features: 100%|██████████| 6/6 [00:00<00:00, 322.86it/s]
4. Fit only on training participants and measure held-out errors#
Each leave-one-out fold has five training people and one test person.
StandardScaler is fitted anew on the five training feature rows. Ridge
uses a fixed alpha=10 penalty to limit large coefficients in a setting
with more features than training participants; it is not a tuned optimum.
The separate mean predictor uses only those same training targets.
Mean absolute error is the average absolute participant prediction error, measured in the released p-factor scale. Lower is better. Compare the model’s MAE with the training-mean baseline rather than a classification chance line. The baseline can outperform the EEG model, and no assertion demands otherwise. The table lets you see whether the average error is driven by one individual; the diagonal in the scatter plot marks perfect prediction, not a fitted line.
predicted, baseline = np.empty_like(y), np.empty_like(y)
for train, test in LeaveOneOut().split(X):
assert set(identities[train]).isdisjoint(identities[test])
model = make_pipeline(StandardScaler(), Ridge(alpha=10))
predicted[test] = model.fit(X[train], y[train]).predict(X[test])
baseline[test] = DummyRegressor().fit(X[train], y[train]).predict(X[test])
print(
pd.DataFrame(
{
"subject": identities,
"observed": y,
"predicted": predicted,
"training_mean": baseline,
}
)
)
print("Participant MAE:", mean_absolute_error(y, predicted))
print("Training-mean MAE:", mean_absolute_error(y, baseline))
fig, ax = plt.subplots(figsize=(5, 4), layout="constrained")
ax.scatter(y, predicted, label="Held-out participants")
ax.plot([y.min(), y.max()], [y.min(), y.max()], "k--")
ax.set(xlabel="Observed p-factor", ylabel="Predicted p-factor")
ax.legend()
plt.show()
subject observed predicted training_mean
0 NDARAC904DMU -0.603 -0.198407 -0.1026
1 NDARAG143ARJ -0.258 -0.201047 -0.1716
2 NDARAM704GKZ 0.062 -0.371029 -0.2356
3 NDARAN385MDH 0.034 -0.555120 -0.2300
4 NDARAP359UM6 0.467 -0.761071 -0.3166
5 NDARBH024NH2 -0.818 -0.062721 -0.0596
Participant MAE: 0.5778409580091636
Training-mean MAE: 0.44839999999999997
What this participant split can establish#
The six held-out predictions exercise a leakage-safe analysis, but their folds share training participants and are not six independent experiments. A narrow age range, short recording segment or acquisition confound can strongly influence this small result. Add participants and define nested validation before comparing feature sets or regularization settings.
Total running time of the script: (0 minutes 2.858 seconds)
