eegdash.downloader module#

File downloading utilities for EEG data from cloud storage.

This module provides functions for downloading EEG data files and BIDS dependencies from AWS S3 storage, with support for caching and progress tracking. It handles the communication between the EEGDash metadata database and the actual EEG data stored in the cloud.

eegdash.downloader.download_dependencies(s3_bucket: str, bids_dependencies: list[str], bids_dependencies_original: list[str], cache_dir: Path, dataset_folder: Path, record: dict[str, Any], s3_open_neuro: bool) → None[source]

Download all BIDS dependency files from S3.

Iterates through a list of BIDS dependency files, downloads each from the specified S3 bucket, and caches them in the appropriate local directory structure.

Parameters:

  • s3_bucket (str) – The S3 bucket to download from.

  • bids_dependencies (list of str) – A list of dependency file paths relative to the S3 bucket root.

  • bids_dependencies_original (list of str) – The original dependency paths, used for resolving local cache paths.

  • cache_dir (pathlib.Path) – The root directory for caching.

  • dataset_folder (pathlib.Path) – The specific folder for the dataset within the cache directory.

  • record (dict) – The metadata record for the main data file, used to resolve paths.

  • s3_open_neuro (bool) – Flag for OpenNeuro-specific path handling.

eegdash.downloader.download_s3_file(s3_path: str, local_path: Path, s3_open_neuro: bool) → Path[source]

Download a single file from S3 to a local path.

Handles the download of a raw EEG data file from an S3 bucket, caching it at the specified local path. Creates parent directories if they do not exist.

Parameters:

  • s3_path (str) – The full S3 URI of the file to download.

  • local_path (pathlib.Path) – The local file path where the downloaded file will be saved.

  • s3_open_neuro (bool) – A flag indicating if the S3 bucket is the OpenNeuro main bucket, which may affect path handling.

Returns:

The local path to the downloaded file.

Return type:

pathlib.Path

eegdash.downloader.get_s3_filesystem() → S3FileSystem[source]

Get an anonymous S3 filesystem object.

Initializes and returns an s3fs.S3FileSystem for anonymous access to public S3 buckets, configured for the ‘us-east-2’ region.

Returns:

An S3 filesystem object.

Return type:

s3fs.S3FileSystem

eegdash.downloader.get_s3path(s3_bucket: str, filepath: str) → str[source]

Construct an S3 URI from a bucket and file path.

Parameters:

  • s3_bucket (str) – The S3 bucket name (e.g., “s3://my-bucket”).

  • filepath (str) – The path to the file within the bucket.

Returns:

The full S3 URI (e.g., “s3://my-bucket/path/to/file”).

Return type:

str