Note
Go to the end to download the full example code or to run this example in your browser via Binder.
Track 3: predict time remaining until the first N2 epoch#
Use three recorded Sleep-EDF participants from EEGDash nm000185
(cassette63, cassette64, cassette65; night1), about 150 MB in total. Predict
once per five-second EEG window and evaluate on an unseen participant.
The official tracks page
describes the wearable task; consult it for current release availability.
This page uses the PSG seed corpus, not an assumed Muse configuration.
The NeuralBench Track 3 guide defines a capped time-to-onset target and binned mean absolute error (bMAE). We implement those target and metric conventions with real EEGDash data and a small spectral Ridge baseline. Our three-person LOSO split and model are not the competition’s frozen split or official trained baseline.
Before running, install EEGDash and its Braindecode/scikit-learn dependencies.
No NeuralBench installation, GPU or prior feature file is required. Retain
signals with EEGDASH_CACHE_DIR. The output contains each window’s actual
target, held-out predictions, per-bin errors and their equally weighted mean.
1. Load independent participants and inspect observed sleep stages#
A recording contains a sequence of scored stages. We need the full stage annotations to identify the first N2 onset, although only pre-onset voltage windows enter the model. This retrospective selection follows the task’s annotation-based evaluation region; it is not a prospective onset detector that can choose its analysis region without knowing the reference onset.
from functools import partial
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from braindecode.datasets import BaseConcatDataset
from braindecode.preprocessing import create_fixed_length_windows
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import Ridge
from sklearn.model_selection import LeaveOneGroupOut
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from eegdash.paths import get_default_cache_dir
from eegdash import EEGDashDataset
from eegdash.features import (
FeatureExtractor,
extract_features,
spectral_bands_power,
spectral_preprocessor,
)
subjects = ["cassette63", "cassette64", "cassette65"]
channels = ["EEG Fpz-Cz", "EEG Pz-Oz"]
dataset = EEGDashDataset(
dataset="nm000185",
subject=subjects,
session="night1",
task="sleep",
cache_dir=get_default_cache_dir(),
n_jobs=1,
)
print(dataset.description.to_string(index=False))
subject session task age sex study run gender
cassette63 night1 sleep 91.0 F cassette None None
cassette64 night1 sleep 85.0 F cassette None None
cassette65 night1 sleep 88.0 F cassette None None
2. Tile the last twenty pre-onset minutes into five-second windows#
The implementation of AddSleepOnsetTargets selects the earliest scored N2 start. It does not require a 60-second N2 run: the guide’s informal “stable” wording must not add a persistence rule. Here we fix a 1200-second horizon; the upstream transform is configurable. The region starts at max(recording_start, N2_onset - 1200 s) and ends at N2 onset. No-N2 records produce no official task windows; this subset requires an observed N2 onset in each recording.
Use the source 100 Hz sampling grid. A window is (2 bipolar derivations, 500 samples) in volts, covering [start, stop). Stop is the next sample boundary, not the timestamp of the last included sample. Target derivation therefore uses i_stop_in_trial / sfreq. Cropping after acquisition cannot reduce the full-file download, and trial metadata must retain both boundaries.
windowed_recordings = []
metadata_tables = []
for recording in dataset.datasets:
raw = recording.raw
n2_onsets = (
raw.annotations.onset[raw.annotations.description == "N2"] - raw.first_time
)
onset = float(n2_onsets.min())
region_start = max(0.0, onset - 1200.0)
region_stop = min(onset, raw.n_times / raw.info["sfreq"])
start_sample = int(np.ceil(region_start * raw.info["sfreq"]))
stop_sample = int(np.floor(region_stop * raw.info["sfreq"]))
raw.pick(channels).reorder_channels(channels)
windows = create_fixed_length_windows(
BaseConcatDataset([recording]),
start_offset_samples=start_sample,
stop_offset_samples=stop_sample,
window_size_samples=500,
window_stride_samples=500,
on_last_window="drop",
preload=True,
)
metadata = windows.get_metadata().reset_index(drop=True)
metadata["window_stop_s"] = metadata.i_stop_in_trial / raw.info["sfreq"]
metadata["n2_onset_s"] = onset
# This observed-annotation transformation matches SleepOnsetTargetExtractor.
metadata["target"] = np.clip(onset - metadata.window_stop_s, 0.0, 600.0)
windowed_recordings.append(windows)
metadata_tables.append(metadata)
[10/15/26 13:14:26] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/
"HTTP/1.1 200 OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/manifest.json "HTTP/1.1 200 OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette63/ses-night1/eeg/s
ub-cassette63_ses-night1_task-sleep
_channels.tsv "HTTP/1.1 200 OK"
[10/15/26 13:14:27] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette63/ses-night1/eeg/s
ub-cassette63_ses-night1_task-sleep
_events.tsv "HTTP/1.1 200 OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette63/ses-night1/eeg/s
ub-cassette63_ses-night1_task-sleep
_events.json "HTTP/1.1 200 OK"
[10/15/26 13:14:29] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette63/ses-night1/eeg/s
ub-cassette63_ses-night1_task-sleep
_eeg.json "HTTP/1.1 200 OK"
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 0%| | 0.00/48.0M [00:00<?, ?B/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 0%| | 0.00/48.0M [00:00<?, ?B/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 2%|▏ | 1.00M/48.0M [00:00<00:28, 1.75MB/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 8%|▊ | 4.00M/48.0M [00:00<00:08, 5.25MB/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 25%|██▌ | 12.0M/48.0M [00:01<00:02, 12.9MB/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 50%|█████ | 24.0M/48.0M [00:01<00:01, 22.4MB/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 79%|███████▉ | 38.0M/48.0M [00:01<00:00, 30.2MB/s]
Downloading sub-cassette63_ses-night1_task-sleep_eeg.edf: 100%|██████████| 48.0M/48.0M [00:01<00:00, 30.5MB/s]
[10/15/26 13:14:31] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/dataset_description.json
"HTTP/1.1 200 OK"
[10/15/26 13:14:32] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/participants.tsv "HTTP/1.1 200
OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/participants.json "HTTP/1.1 200
OK"
[10/15/26 13:14:33] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/README.md "HTTP/1.1 500
Internal Server Error"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/README.md "HTTP/1.1 200 OK"
[10/15/26 13:14:34] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/.bidsignore "HTTP/1.1 200 OK"
/home/runner/work/EEGDash/EEGDash/.venv/lib/python3.12/site-packages/braindecode/preprocessing/windowers.py:1370: UserWarning: Usage of offset_sample args in create_fixed_length_windows is deprecated and will be removed in future versions. Please use braindecode.preprocessing.preprocess.Preprocessor("crop", tmin, tmax) instead.
warnings.warn(
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette64/ses-night1/eeg/s
ub-cassette64_ses-night1_task-sleep
_channels.tsv "HTTP/1.1 200 OK"
[10/15/26 13:14:35] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette64/ses-night1/eeg/s
ub-cassette64_ses-night1_task-sleep
_events.tsv "HTTP/1.1 200 OK"
[10/15/26 13:14:36] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette64/ses-night1/eeg/s
ub-cassette64_ses-night1_task-sleep
_events.json "HTTP/1.1 200 OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette64/ses-night1/eeg/s
ub-cassette64_ses-night1_task-sleep
_eeg.json "HTTP/1.1 200 OK"
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 0%| | 0.00/46.6M [00:00<?, ?B/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 0%| | 0.00/46.6M [00:00<?, ?B/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 2%|▏ | 1.00M/46.6M [00:00<00:27, 1.74MB/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 6%|▋ | 3.00M/46.6M [00:00<00:11, 3.89MB/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 21%|██▏ | 10.0M/46.6M [00:01<00:03, 10.7MB/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 56%|█████▌ | 26.0M/46.6M [00:01<00:00, 23.5MB/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 82%|████████▏ | 38.0M/46.6M [00:01<00:00, 29.8MB/s]
Downloading sub-cassette64_ses-night1_task-sleep_eeg.edf: 100%|██████████| 46.6M/46.6M [00:01<00:00, 29.2MB/s]
/home/runner/work/EEGDash/EEGDash/.venv/lib/python3.12/site-packages/braindecode/preprocessing/windowers.py:1370: UserWarning: Usage of offset_sample args in create_fixed_length_windows is deprecated and will be removed in future versions. Please use braindecode.preprocessing.preprocess.Preprocessor("crop", tmin, tmax) instead.
warnings.warn(
[10/15/26 13:14:38] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette65/ses-night1/eeg/s
ub-cassette65_ses-night1_task-sleep
_channels.tsv "HTTP/1.1 200 OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette65/ses-night1/eeg/s
ub-cassette65_ses-night1_task-sleep
_events.tsv "HTTP/1.1 200 OK"
[10/15/26 13:14:39] INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette65/ses-night1/eeg/s
ub-cassette65_ses-night1_task-sleep
_events.json "HTTP/1.1 200 OK"
INFO HTTP Request: GET _client.py:1025
https://data.nemar.org/nm000185/v1.
0.0/sub-cassette65/ses-night1/eeg/s
ub-cassette65_ses-night1_task-sleep
_eeg.json "HTTP/1.1 200 OK"
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 0%| | 0.00/49.8M [00:00<?, ?B/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 0%| | 0.00/49.8M [00:00<?, ?B/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 2%|▏ | 1.00M/49.8M [00:00<00:27, 1.89MB/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 6%|▌ | 3.00M/49.8M [00:00<00:11, 4.15MB/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 20%|██ | 10.0M/49.8M [00:01<00:03, 11.3MB/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 50%|█████ | 25.0M/49.8M [00:01<00:01, 23.6MB/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 82%|████████▏ | 41.0M/49.8M [00:01<00:00, 32.8MB/s]
Downloading sub-cassette65_ses-night1_task-sleep_eeg.edf: 100%|██████████| 49.8M/49.8M [00:01<00:00, 31.8MB/s]
/home/runner/work/EEGDash/EEGDash/.venv/lib/python3.12/site-packages/braindecode/preprocessing/windowers.py:1370: UserWarning: Usage of offset_sample args in create_fixed_length_windows is deprecated and will be removed in future versions. Please use braindecode.preprocessing.preprocess.Preprocessor("crop", tmin, tmax) instead.
warnings.warn(
3. Compute EEGDash band powers without using the onset as a predictor#
The two source channels are recorded bipolar derivations, not independent Fpz/Cz/Pz/Oz electrodes. Preserve their reference. Five-second Hann-Welch segments give 0.2 Hz bins; one shared spectrum supplies four half-open bands. Multiplying summed PSD bins by their spacing approximates band power in V². Log power gives eight predictor columns. Neither window time, onset time, participant ID nor target enters X. No full-recording normalization or additional EEGPrep cleaning is fitted: this page isolates the target and evaluation contract rather than an artifact-removal protocol.
windows = BaseConcatDataset(windowed_recordings)
metadata = pd.concat(metadata_tables, ignore_index=True)
spectral = FeatureExtractor(
{
"power": partial(
spectral_bands_power,
bands={
"delta": (1, 4),
"theta": (4, 8),
"alpha": (8, 13),
"beta": (13, 30),
},
)
},
preprocessor=partial(
spectral_preprocessor,
fs=100,
nperseg=500,
noverlap=0,
window="hann",
f_min=1,
f_max=30,
),
)
feature_table = extract_features(
windows, {"spectral": spectral}, batch_size=64, n_jobs=1
).to_dataframe()
X = np.log10(np.maximum(feature_table.to_numpy() * 0.2, 1e-30))
y = metadata.target.to_numpy(dtype=float)
groups = metadata.subject.astype(str).to_numpy()
print("Feature matrix:", X.shape)
print(
metadata[["subject", "window_stop_s", "n2_onset_s", "target"]]
.groupby("subject")
.agg(["min", "max", "count"])
)
Extracting features: 0%| | 0/3 [00:00<?, ?it/s]
Extracting features: 67%|██████▋ | 2/3 [00:00<00:00, 16.84it/s]
Extracting features: 100%|██████████| 3/3 [00:00<00:00, 16.80it/s]
Feature matrix: (720, 8)
window_stop_s n2_onset_s ... target
min max count min ... count min max count
subject ...
cassette63 30305.0 31500.0 240 31500.0 ... 240 0.0 600.0 240
cassette64 23945.0 25140.0 240 25140.0 ... 240 0.0 600.0 240
cassette65 485.0 1680.0 240 1680.0 ... 240 0.0 600.0 240
[3 rows x 9 columns]
4. Predict every window from a participant held out of fitting#
A fresh scaler and Ridge model fit only the other two participants. Ridge uses fixed alpha=10; the dummy reference predicts the training-window mean. We clip both predictions to the known target range as a fixed output choice. This is a small CPU regression demonstration; it does not reproduce the guide’s neural model or regression-bin training sampler.
Every window for a test person stays out of training, including neighboring windows from the same night. Thousands of windows would still be only three independent people. Hyperparameter selection needs additional validation participants within training, not feedback from these outer test predictions.
predicted, baseline = np.empty_like(y), np.empty_like(y)
for train, test in LeaveOneGroupOut().split(X, y, groups):
model = make_pipeline(StandardScaler(), Ridge(alpha=10))
predicted[test] = np.clip(model.fit(X[train], y[train]).predict(X[test]), 0, 600)
baseline[test] = np.clip(
DummyRegressor().fit(X[train], y[train]).predict(X[test]), 0, 600
)
5. Compute bMAE using ground-truth time-to-onset bins#
NeuralBench’s BinnedMAE implementation uses [0,40), [40,90), [90,300), and [300,600]. Interior edge values enter the higher bin; 600 belongs to the final bin. First compute mean absolute error within each ground-truth bin, then average the nonempty bin means equally. Thus the many capped targets do not dominate the headline score. The metric does not average absolute recording-onset estimates.
Bin counts and per-person scores make the small evaluation inspectable. The pooled bMAE averages each bin’s held-out window errors before averaging bins, matching the metric’s aggregation. A mean of participant bMAEs is a different aggregation when their bin counts differ.
bin_edges = np.asarray([0.0, 40.0, 90.0, 300.0, 600.0])
bin_ids = np.searchsorted(bin_edges[1:-1], y, side="right")
errors = pd.DataFrame(
{
"subject": groups,
"bin": bin_ids,
"Ridge": np.abs(predicted - y),
"training mean": np.abs(baseline - y),
}
)
per_bin = errors.groupby("bin")[["Ridge", "training mean"]].mean().reindex(range(4))
bmae = per_bin.mean()
print("Held-out window counts by subject and bin:")
print(pd.crosstab(groups, bin_ids))
print("Per-bin MAE (seconds):\n", per_bin)
print("Pooled bMAE (seconds):\n", bmae)
print(
"Per-participant bMAE (seconds):\n",
errors.groupby(["subject", "bin"])[["Ridge", "training mean"]]
.mean()
.groupby("subject")
.mean(),
)
fig, axes = plt.subplots(1, 2, figsize=(11, 4), layout="constrained")
per_bin.plot.bar(ax=axes[0], rot=0)
axes[0].set(
xticklabels=["[0,40)", "[40,90)", "[90,300)", "[300,600]"],
xlabel="Observed time-to-onset bin (s)",
ylabel="Held-out MAE (s)",
)
axes[1].scatter(y, predicted, s=8, alpha=0.4)
axes[1].plot([0, 600], [0, 600], "k--")
axes[1].set(xlabel="Observed time remaining (s)", ylabel="Predicted time remaining (s)")
plt.show()
Held-out window counts by subject and bin:
col_0 0 1 2 3
row_0
cassette63 8 10 42 180
cassette64 8 10 42 180
cassette65 8 10 42 180
Per-bin MAE (seconds):
Ridge training mean
bin
0 253.870533 431.250000
1 242.253464 386.250000
2 155.973621 256.250000
3 194.286096 125.833333
Pooled bMAE (seconds):
Ridge 211.595928
training mean 299.895833
dtype: float64
Per-participant bMAE (seconds):
Ridge training mean
subject
cassette63 233.599238 299.895833
cassette64 139.014788 299.895833
cassette65 262.173759 299.895833
6. Extend the task without claiming a wearable benchmark#
A target of 600 means “at least ten minutes remaining,” so adding that prediction to the window stop does not recover a unique onset timestamp. For uncapped pre-onset targets, window_stop + prediction can be interpreted as an onset estimate; choosing windows using the true onset remains a retrospective evaluation convention, not a deployment-time selection rule.
Add sleepers and keep all nights of a person in one split. Verify the released wearable channels, timing, labels and official evaluation package before changing devices; no Muse dataset name or configuration is guessed here. For an official run, follow the linked NeuralBench guide and its released data/configuration rather than comparing this three-person result directly with a competition leaderboard.
Total running time of the script: (0 minutes 16.107 seconds)
