Discover and inspect data packages

This article shows how to choose a Global Light Commons package and understand its contents before downloading or importing measurements. Every example uses the validated MELIDOS IZTECH package, whose current passing revision uses schema 3.0.2. Remote examples run on the pkgdown website but remain unevaluated in ordinary package and CRAN builds. Set GLCDP_SKIP_LIVE=true to request an offline website build.

Understand the registry

glc_packages() returns one row per registered repository. The most useful fields distinguish the repository’s current revision from its most recent passing revision:

current_status and current_commit describe the configured current revision;
latest_pass_commit and has_latest_pass identify the last validated revision available for reproducible use; and
attestation_verified reports whether the registry attestation was verified.

packages <- glc_packages()
packages[, c(
  "id", "repository", "current_status", "has_latest_pass",
  "attestation_verified"
)]

Registry results are cached for the R session. Set refresh = TRUE only when you need to fetch the registry again.

packages <- glc_packages(refresh = TRUE)

Searches are fixed and case-insensitive by default:

glc_search_packages("iztech", packages)
glc_search_packages(packages = packages, status = c("pass", "fail"))
glc_search_packages(packages = packages, has_pass = FALSE)

Choose a revision deliberately

Opening a registered package with the default ref = "latest_pass" selects an exact passing commit, not a moving branch:

iztech_repository <- "tscnlab/melidos-iztech-glc-dataset"
iztech <- glc_open(iztech_repository)
iztech

Use ref = "current" when you explicitly need the registry’s current revision. If that revision is not passing, glcdp warns. You can also provide an exact 40-character commit SHA; commits that are not selected through a registry record are marked as unverified.

current <- glc_open(
  iztech_repository,
  ref = "current"
)
current

registry_row <- glc_search_packages("melidos-iztech", packages)
registry_row$repository[[1]]
registry_row$latest_pass_commit[[1]]

pinned <- glc_open(
  registry_row$repository[[1]],
  ref = registry_row$latest_pass_commit[[1]]
)
pinned

For private repositories, pass token directly or define GITHUB_PAT or GITHUB_TOKEN. Do not put tokens in scripts, vignettes, or package options.

Start with a package summary

glc_summary() reports the schema version and counts of studies, datasets, participants, devices, file groups, files, and variables. It also summarizes modalities, time zones, and primary variables.

glc_summary(iztech)

Explore inventories

The inventories are tibbles, so they can be printed, filtered, or joined using ordinary data-frame tools.

glc_resources(iztech)

glc_datasets() describes logical datasets and their associations. Once you have a dataset id, reuse it to narrow the other inventories.

iztech_dataset <- "MELIDOS_IZTECH_S001"
iztech_demographics <- "MELIDOS_IZTECH_S001:4"
iztech_chest_light <- "MELIDOS_IZTECH_S001:17"

glc_datasets(iztech)

glc_files(iztech, dataset_id = iztech_dataset)
glc_files(
  iztech,
  # dataset_id = iztech_dataset,
  modality = "light",
  available = TRUE
) |>
  dplyr::filter(device_location == "eye level")

File inventories expose both declared and resolved paths, format, encoding, time zone, role, data state, device, storage type, expected size, and availability. Git LFS-backed files are identified without requiring a local Git LFS installation.

Variable inventories can be narrowed by dataset, file group, semantic term, or primary status:

demographic_variables <- glc_variables(
  iztech,
  file_group = iztech_demographics
)
demographic_variables[, c(
  "name", "type", "factor_values", "primary"
)]

glc_variables(
  iztech,
  file_group = iztech_chest_light,
  primary = TRUE
)
glc_variables(iztech, term = "melanopic_edi")

The type and factor_values columns are not merely descriptive: glc_read() uses them to construct the corresponding R columns and factor levels in schema-declared order. Use source variable names with its variables argument and semantic terms with its terms argument.

Load and search metadata

With no resources argument, glc_metadata() loads the core resources that the package declares. Requesting resources explicitly is often faster and makes dependencies clearer.

metadata <- glc_metadata(
  iztech,
  resources = c("study", "participants")
)
names(metadata)
metadata$study
metadata$participants

JSON objects remain lists, tabular resources become tibbles, and directory resources become named lists keyed by package-relative path.

Search traverses nested metadata and reports the resource, record, complete field path, value, and context for each match:

glc_search_metadata(iztech, "light exposure")

glc_search_metadata(
  iztech,
  "age",
  resources = "participants",
  search_in = "fields"
)

glc_search_metadata(
  iztech,
  "meq_type",
  resources = "participant_characteristics"
)

glc_search_metadata(
  iztech,
  "Izmir",
  resources = "study",
  fields = "study_geographical_location"
)

Work with local packages

Local packages use the same public interface, which makes them useful for development, validation follow-up, and offline analysis:

iztech_local <- glc_open("path/to/iztech-subset", quiet = TRUE)
glc_summary(iztech_local)
glc_files(iztech_local, available = FALSE)

The directory must contain a datapackage.json descriptor and all paths needed by the selected operation. A subset created by glc_download() can be reopened in exactly the same way.

After selecting and collecting compatible light data, use the LightLogR function reference for downstream quality checks, summaries, metrics, and visualizations.