.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "generated/auto_examples/eeg2025/tutorial_eegdash_offline.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. or to run this example in your browser via Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_generated_auto_examples_eeg2025_tutorial_eegdash_offline.py: ============================ Working Offline with EEGDash ============================ Many HPC clusters restrict or block network access. It's common to have dedicated queues for internet-enabled jobs that differ from GPU queues. This tutorial shows how to use :doc:`EEGChallengeDataset ` offline once a dataset is present on disk. .. GENERATED FROM PYTHON SOURCE LINES 11-29 .. code-block:: Python from pathlib import Path import platformdirs from eegdash.const import RELEASE_TO_OPENNEURO_DATASET_MAP from eegdash.dataset.dataset import EEGChallengeDataset # We'll use Release R2 as an example (HBN subset). # :doc:`EEGChallengeDataset ` # uses a suffixed cache folder for the competition data (e.g., "-bdf-mini"). release = "R2" dataset_id = RELEASE_TO_OPENNEURO_DATASET_MAP[release] task = "RestingState" # Choose a cache directory. This should be on a fast local filesystem. cache_dir = Path(platformdirs.user_cache_dir("EEGDash")) cache_dir.mkdir(parents=True, exist_ok=True) .. GENERATED FROM PYTHON SOURCE LINES 30-38 Step 1: Populate the local cache (Online) ----------------------------------------- This block downloads the dataset from S3 to your local cache directory. Run this part on a machine with internet access. If the dataset is already on your disk at the specified ``cache_dir``, you can comment out or skip this section. To keep this example self-contained, we prefetch the data here. .. GENERATED FROM PYTHON SOURCE LINES 38-52 .. code-block:: Python ds_online = EEGChallengeDataset( release=release, cache_dir=cache_dir, task=task, mini=True, ) # Optional prefetch of all recordings (downloads everything to cache). from joblib import Parallel, delayed _ = Parallel(n_jobs=-1)(delayed(lambda d: d.raw)(d) for d in ds_online.datasets) .. rst-class:: sphx-glr-script-out .. code-block:: none ╭────────────────────── EEG 2025 Competition Data Notice ──────────────────────╮ │ This object loads the HBN dataset that has been preprocessed for the EEG │ │ Challenge: │ │ * Downsampled from 500Hz to 100Hz │ │ * Bandpass filtered (0.5-50 Hz) │ │ │ │ For full preprocessing applied for competition details, see: │ │ https://github.com/eeg2025/downsample-datasets │ │ │ │ The HBN dataset have some preprocessing applied by the HBN team: │ │ * Re-reference (Cz Channel) │ │ │ │ IMPORTANT: The data accessed via `EEGChallengeDataset` is NOT identical to │ │ what you get from EEGDashDataset directly. │ │ If you are participating in the competition, always use │ │ `EEGChallengeDataset` to ensure consistency with the challenge data. │ ╰──────────────────────── Source: EEGChallengeDataset ─────────────────────────╯ .. GENERATED FROM PYTHON SOURCE LINES 53-61 Step 2: Basic Offline Usage --------------------------- Once the data is cached locally, you can interact with it without needing an internet connection. The key is to instantiate your dataset object with the ``download=False`` flag. This tells :doc:`EEGChallengeDataset ` to look for data in the ``cache_dir`` instead of trying to connect to the database or S3. .. GENERATED FROM PYTHON SOURCE LINES 61-79 .. code-block:: Python # Here we check that the local cache folder exists offline_root = cache_dir / f"{dataset_id}-bdf-mini" print(f"Local dataset folder exists: {offline_root.exists()}\n{offline_root}") ds_offline = EEGChallengeDataset( release=release, cache_dir=cache_dir, task=task, download=False, ) print(f"Found {len(ds_offline.datasets)} recording(s) offline.") if ds_offline.datasets: print("First record bidspath:", ds_offline.datasets[0].record["bidspath"]) .. rst-class:: sphx-glr-script-out .. code-block:: none Local dataset folder exists: True /home/runner/.cache/EEGDash/ds005506-bdf-mini ╭────────────────────── EEG 2025 Competition Data Notice ──────────────────────╮ │ This object loads the HBN dataset that has been preprocessed for the EEG │ │ Challenge: │ │ * Downsampled from 500Hz to 100Hz │ │ * Bandpass filtered (0.5-50 Hz) │ │ │ │ For full preprocessing applied for competition details, see: │ │ https://github.com/eeg2025/downsample-datasets │ │ │ │ The HBN dataset have some preprocessing applied by the HBN team: │ │ * Re-reference (Cz Channel) │ │ │ │ IMPORTANT: The data accessed via `EEGChallengeDataset` is NOT identical to │ │ what you get from EEGDashDataset directly. │ │ If you are participating in the competition, always use │ │ `EEGChallengeDataset` to ensure consistency with the challenge data. │ ╰──────────────────────── Source: EEGChallengeDataset ─────────────────────────╯ Found 20 recording(s) offline. First record bidspath: ds005506/sub-NDARAB793GL3/eeg/sub-NDARAB793GL3_task-RestingState_eeg.bdf .. GENERATED FROM PYTHON SOURCE LINES 80-87 Step 3: Filtering Entities Offline ---------------------------------- Even without a database connection, you can still filter your dataset by BIDS entities like subject, session, or task. When ``download=False``, :doc:`EEGChallengeDataset ` uses the BIDS directory structure and filenames to apply these filters. This example shows how to load data for a specific subject from the local cache. .. GENERATED FROM PYTHON SOURCE LINES 87-105 .. code-block:: Python ds_offline_sub = EEGChallengeDataset( cache_dir=cache_dir, release=release, download=False, subject="NDARAB793GL3", ) print(f"Filtered by subject=NDARAB793GL3: {len(ds_offline_sub.datasets)} recording(s).") if ds_offline_sub.datasets: keys = ("dataset", "subject", "task", "run") print("Records (dataset, subject, task, run):") for idx, base_ds in enumerate(ds_offline_sub.datasets, start=1): rec = base_ds.record summary = ", ".join(f"{k}={rec.get(k)}" for k in keys) print(f" {idx:03d}: {summary}") .. rst-class:: sphx-glr-script-out .. code-block:: none ╭────────────────────── EEG 2025 Competition Data Notice ──────────────────────╮ │ This object loads the HBN dataset that has been preprocessed for the EEG │ │ Challenge: │ │ * Downsampled from 500Hz to 100Hz │ │ * Bandpass filtered (0.5-50 Hz) │ │ │ │ For full preprocessing applied for competition details, see: │ │ https://github.com/eeg2025/downsample-datasets │ │ │ │ The HBN dataset have some preprocessing applied by the HBN team: │ │ * Re-reference (Cz Channel) │ │ │ │ IMPORTANT: The data accessed via `EEGChallengeDataset` is NOT identical to │ │ what you get from EEGDashDataset directly. │ │ If you are participating in the competition, always use │ │ `EEGChallengeDataset` to ensure consistency with the challenge data. │ ╰──────────────────────── Source: EEGChallengeDataset ─────────────────────────╯ Filtered by subject=NDARAB793GL3: 1 recording(s). Records (dataset, subject, task, run): 001: dataset=ds005506, subject=NDARAB793GL3, task=RestingState, run=None .. GENERATED FROM PYTHON SOURCE LINES 106-116 Step 4: Comparing Online vs. Offline Data ----------------------------------------- As a sanity check, you can verify that the data loaded from your local cache is identical to the data fetched from the online sources. This section compares the shape of the raw data from the online and offline datasets to ensure they match. This is a good way to confirm your local cache is complete and correct. If you have network access, you can uncomment the block below to download and compare shapes. .. GENERATED FROM PYTHON SOURCE LINES 116-123 .. code-block:: Python raw_online = ds_online.datasets[0].raw raw_offline = ds_offline.datasets[0].raw print("online shape:", raw_online.get_data().shape) print("offline shape:", raw_offline.get_data().shape) print("shapes equal:", raw_online.get_data().shape == raw_offline.get_data().shape) .. rst-class:: sphx-glr-script-out .. code-block:: none online shape: (129, 40800) offline shape: (129, 40800) shapes equal: True .. GENERATED FROM PYTHON SOURCE LINES 124-129 Step 4.1: Comparing Descriptions, Online vs. Offline Data --------------------------------------------------------- If you have network access, you can uncomment the block below to download and compare shapes. .. GENERATED FROM PYTHON SOURCE LINES 129-137 .. code-block:: Python description_online = ds_online.description description_offline = ds_offline.description print(description_offline) print(description_online) print("Online description shape:", description_online.shape) print("Offline description shape:", description_offline.shape) print("Descriptions equal:", description_online.equals(description_offline)) .. rst-class:: sphx-glr-script-out .. code-block:: none subject task ... seqlearning8target symbolsearch 0 NDARAB793GL3 RestingState ... available available 1 NDARAM675UR8 RestingState ... unavailable available 2 NDARBM839WR5 RestingState ... available available 3 NDARBU730PN8 RestingState ... available available 4 NDARCT974NAJ RestingState ... available available 5 NDARCW933FD5 RestingState ... available available 6 NDARCZ770BRG RestingState ... available available 7 NDARDW741HCF RestingState ... unavailable available 8 NDARDZ058NZN RestingState ... unavailable available 9 NDAREC377AU2 RestingState ... available available 10 NDAREM500WWH RestingState ... unavailable available 11 NDAREV527ZRF RestingState ... available available 12 NDAREV601CE7 RestingState ... available available 13 NDARFF070XHV RestingState ... available available 14 NDARFR108JNB RestingState ... unavailable available 15 NDARFT305CG1 RestingState ... unavailable available 16 NDARGA056TMW RestingState ... available available 17 NDARGH775KF5 RestingState ... available available 18 NDARGJ878ZP4 RestingState ... unavailable available 19 NDARHA387FPM RestingState ... available available [20 rows x 25 columns] subject task ... seqlearning8target symbolsearch 0 NDARAB793GL3 RestingState ... available available 1 NDARAM675UR8 RestingState ... unavailable available 2 NDARBM839WR5 RestingState ... available available 3 NDARBU730PN8 RestingState ... available available 4 NDARCT974NAJ RestingState ... available available 5 NDARCW933FD5 RestingState ... available available 6 NDARCZ770BRG RestingState ... available available 7 NDARDW741HCF RestingState ... unavailable available 8 NDARDZ058NZN RestingState ... unavailable available 9 NDAREC377AU2 RestingState ... available available 10 NDAREM500WWH RestingState ... unavailable available 11 NDAREV527ZRF RestingState ... available available 12 NDAREV601CE7 RestingState ... available available 13 NDARFF070XHV RestingState ... available available 14 NDARFR108JNB RestingState ... unavailable available 15 NDARFT305CG1 RestingState ... unavailable available 16 NDARGA056TMW RestingState ... available available 17 NDARGH775KF5 RestingState ... available available 18 NDARGJ878ZP4 RestingState ... unavailable available 19 NDARHA387FPM RestingState ... available available [20 rows x 25 columns] Online description shape: (20, 25) Offline description shape: (20, 25) Descriptions equal: True .. GENERATED FROM PYTHON SOURCE LINES 138-144 Notes and troubleshooting ------------------------- - Working offline selects recordings by parsing BIDS filenames and directory structure. Some DB-only fields are unavailable; entity filters (subject, session, task, run) usually suffice. - If you encounter issues, please open a GitHub issue so we can discuss. .. rst-class:: sphx-glr-timing **Total running time of the script:** (1 minutes 14.010 seconds) **Estimated memory usage:** 1023 MB .. _sphx_glr_download_generated_auto_examples_eeg2025_tutorial_eegdash_offline.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/sccn/EEGDash/main?urlpath=lab/tree/notebooks/generated/auto_examples/eeg2025/tutorial_eegdash_offline.ipynb :alt: Launch binder :width: 150 px .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: tutorial_eegdash_offline.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: tutorial_eegdash_offline.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: tutorial_eegdash_offline.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_