---
title: "Discover and inspect data packages"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Discover and inspect data packages}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
live <- identical(Sys.getenv("IN_PKGDOWN"), "true") &&
  !identical(Sys.getenv("GLCDP_SKIP_LIVE"), "true")
```

This article shows how to choose a Global Light Commons package and understand
its contents before downloading or importing measurements. Every example uses
the validated MELIDOS IZTECH package, whose current passing revision uses
schema 3.0.2. Remote examples run on the pkgdown website but remain unevaluated
in ordinary package and CRAN builds. Set `GLCDP_SKIP_LIVE=true` to request an
offline website build.

```{r load}
library(glcdp)
```

## Understand the registry

`glc_packages()` returns one row per registered repository. The most useful
fields distinguish the repository's current revision from its most recent
passing revision:

- `current_status` and `current_commit` describe the configured current
  revision;
- `latest_pass_commit` and `has_latest_pass` identify the last validated
  revision available for reproducible use; and
- `attestation_verified` reports whether the registry attestation was
  verified.

```{r registry, eval = live}
packages <- glc_packages()
packages[, c(
  "id", "repository", "current_status", "has_latest_pass",
  "attestation_verified"
)]
```

Registry results are cached for the R session. Set `refresh = TRUE` only when
you need to fetch the registry again.

```{r registry-refresh, eval = live}
packages <- glc_packages(refresh = TRUE)
```

Searches are fixed and case-insensitive by default:

```{r registry-search, eval = live}
glc_search_packages("iztech", packages)
glc_search_packages(packages = packages, status = c("pass", "fail"))
glc_search_packages(packages = packages, has_pass = FALSE)
```

## Choose a revision deliberately

Opening a registered package with the default `ref = "latest_pass"` selects an
exact passing commit, not a moving branch:

```{r open-latest-pass, eval = live}
iztech_repository <- "tscnlab/melidos-iztech-glc-dataset"
iztech <- glc_open(iztech_repository)
iztech
```

Use `ref = "current"` when you explicitly need the registry's current
revision. If that revision is not passing, `glcdp` warns. You can also provide
an exact 40-character commit SHA; commits that are not selected through a
registry record are marked as unverified.

```{r open-revisions, eval = live}
current <- glc_open(
  iztech_repository,
  ref = "current"
)
current

registry_row <- glc_search_packages("melidos-iztech", packages)
registry_row$repository[[1]]
registry_row$latest_pass_commit[[1]]

pinned <- glc_open(
  registry_row$repository[[1]],
  ref = registry_row$latest_pass_commit[[1]]
)
pinned
```

For private repositories, pass `token` directly or define `GITHUB_PAT` or
`GITHUB_TOKEN`. Do not put tokens in scripts, vignettes, or package options.

## Start with a package summary

`glc_summary()` reports the schema version and counts of studies, datasets,
participants, devices, file groups, files, and variables. It also summarizes
modalities, time zones, and primary variables.

```{r summary, eval = live}
glc_summary(iztech)
```

## Explore inventories

The inventories are tibbles, so they can be printed, filtered, or joined using
ordinary data-frame tools.

```{r resources, eval = live}
glc_resources(iztech)
```

`glc_datasets()` describes logical datasets and their associations. Once you
have a dataset id, reuse it to narrow the other inventories.

```{r datasets, eval = live}
iztech_dataset <- "MELIDOS_IZTECH_S001"
iztech_demographics <- "MELIDOS_IZTECH_S001:4"
iztech_chest_light <- "MELIDOS_IZTECH_S001:17"

glc_datasets(iztech)

glc_files(iztech, dataset_id = iztech_dataset)
glc_files(
  iztech,
  # dataset_id = iztech_dataset,
  modality = "light",
  available = TRUE
) |>
  dplyr::filter(device_location == "eye level")
```

File inventories expose both declared and resolved paths, format, encoding,
time zone, role, data state, device, storage type, expected size, and
availability. Git LFS-backed files are identified without requiring a local
Git LFS installation.

Variable inventories can be narrowed by dataset, file group, semantic term, or
primary status:

```{r variables, eval = live}
demographic_variables <- glc_variables(
  iztech,
  file_group = iztech_demographics
)
demographic_variables[, c(
  "name", "type", "factor_values", "primary"
)]

glc_variables(
  iztech,
  file_group = iztech_chest_light,
  primary = TRUE
)
glc_variables(iztech, term = "melanopic_edi")
```

The `type` and `factor_values` columns are not merely descriptive:
`glc_read()` uses them to construct the corresponding R columns and factor
levels in schema-declared order. Use source variable names with its `variables`
argument and semantic terms with its `terms` argument.

## Load and search metadata

With no `resources` argument, `glc_metadata()` loads the core resources that
the package declares. Requesting resources explicitly is often faster and
makes dependencies clearer.

```{r metadata, eval = live}
metadata <- glc_metadata(
  iztech,
  resources = c("study", "participants")
)
names(metadata)
metadata$study
metadata$participants
```

JSON objects remain lists, tabular resources become tibbles, and directory
resources become named lists keyed by package-relative path.

Search traverses nested metadata and reports the resource, record, complete
field path, value, and context for each match:

```{r metadata-search, eval = live}
glc_search_metadata(iztech, "light exposure")

glc_search_metadata(
  iztech,
  "age",
  resources = "participants",
  search_in = "fields"
)

glc_search_metadata(
  iztech,
  "meq_type",
  resources = "participant_characteristics"
)

glc_search_metadata(
  iztech,
  "Izmir",
  resources = "study",
  fields = "study_geographical_location"
)
```

## Work with local packages

Local packages use the same public interface, which makes them useful for
development, validation follow-up, and offline analysis:

```{r local, eval = FALSE}
iztech_local <- glc_open("path/to/iztech-subset", quiet = TRUE)
glc_summary(iztech_local)
glc_files(iztech_local, available = FALSE)
```

The directory must contain a `datapackage.json` descriptor and all paths needed
by the selected operation. A subset created by `glc_download()` can be reopened
in exactly the same way.

After selecting and collecting compatible light data, use the
[LightLogR function reference](https://tscnlab.github.io/LightLogR/reference/index.html)
for downstream quality checks, summaries, metrics, and visualizations.