Distributional Regression K > 1 and Residual Diagnostics with DHARMa

Tutorial for fitting per-slot AMM with custom families and validating fits via G1 / G2 / G3 residuals (Path 1)

José Mauricio Gómez Julián

2026-07-06


1. What this vignette covers

gdpar supports per-slot AMM canonical decomposition for distributional regression — that is, fitting one AMM per parameter of a distribution (K > 1). Sub-phases 8.3.4 through 8.3.7 brought online the following library of K > 1 likelihoods:

This vignette covers two complementary topics:

  1. The API for declaring and fitting a K > 1 model.
  2. The residual / posterior-predictive workflow that complements the fit, including the optional integration with the DHARMa package.

For the intermediate AMM specifications (B-spline W bases, heterogeneous families per slot), see vignette("vop04_amm_intermediate", package = "gdpar").


2. The K > 1 API

2.1. Three equivalent input forms

gdpar() accepts three syntactically equivalent ways of declaring a K > 1 distributional regression. All three canonicalise to the same internal gdpar_formula_set object (sub-phase 8.3.3, decision E):

library(gdpar)

# (a) brms-style `bf()` sugar
fit <- gdpar(
  gdpar_bf(y ~ a(x1), sigma ~ a(x2)),
  data = d, family = gdpar_family("gaussian")
)

# (b) Named list of formulas
fit <- gdpar(
  list(mu = y ~ a(x1), sigma = ~ a(x2)),
  data = d, family = gdpar_family("gaussian")
)

# (c) Named list of amm_spec (low-level, bypasses formula parsing)
fit <- gdpar(
  list(
    mu    = amm_spec(a = ~ x1),
    sigma = amm_spec(a = ~ x2)
  ),
  data = d, family = gdpar_family("gaussian")
)

Three contract notes:

2.2. Choosing K

The number of slots K is determined by the input. K = 1 retains the legacy path; K = 2 adds dispersion / scale modelling; K = 3 adds shape / weight modelling. The minimum K per family is enforced by .gdpar_guard_K_below_family_min:

Family min_K
gaussian, poisson, bernoulli, neg_binomial_2 1
beta, gamma, lognormal_loc_scale 2
student_t, tweedie 3
zip, hurdle_poisson 2
zinb, hurdle_neg_binomial_2 3

A K = 1 fit on a beta family aborts with gdpar_input_error pointing to elevation to K = 2.

The pattern name lognormal_loc_scale is not part of the enum of gdpar_family(name): the package registers it as a K = 2 custom-family pattern (canonised in Sub-phase 8.3.4), accessed via gdpar_family_custom_K(stan_lpdf_id = "lognormal_loc_scale", ...). See §2.4 below for the literal recipe.

2.3. End-to-end example: Gaussian K = 2

set.seed(2026L)
n <- 100L
x1 <- rnorm(n); x2 <- rnorm(n)
mu_true       <- 0.4 + 0.6 * (x1 - mean(x1))
log_sigma_eta <- -0.2 + 0.4 * (x2 - mean(x2))
y <- rnorm(n, mu_true, exp(log_sigma_eta))
d <- data.frame(y = y, x1 = x1, x2 = x2)

library(gdpar)
fit_K2 <- gdpar(
  gdpar_bf(y ~ a(x1), sigma ~ a(x2)),
  data   = d,
  family = gdpar_family("gaussian"),
  chains = 2L, iter_warmup = 400L, iter_sampling = 400L,
  refresh = 0L
)

co <- coef(fit_K2)
co$mu
co$sigma

coef.gdpar_fit for K > 1 returns a named list of gdpar_coef objects (decision E4.A, sub-phase 8.3.10). Each entry follows the scalar gdpar_coef contract: posterior summaries of theta_ref, the additive a, the multiplicative b/c_b, and the modulating W. The modulating block is globally shared across slots (replicated identically in every slot’s W component).

2.4. Custom K > 1 families via gdpar_family_custom_K()

The constructor gdpar_family_custom_K() exposes the K-family custom-pattern registry opened in Sub-phase 8.3.4 of Block 8. Each registered pattern is identified by a stan_lpdf_id (the name of a pre-validated Stan _lpdf function shipped with the package) and carries its own minimum K. The first pattern registered is lognormal_loc_scale (min_K = 2); subsequent sub-phases extend the whitelist.

Signature:

gdpar_family_custom_K(
  name,                 # character scalar; must not collide with a built-in
  stan_lpdf_id,         # character scalar; key in the registry
  did_holds     = TRUE, # logical; user declaration of D-ID
  did_condition = NULL, # character scalar describing any conditional D-ID
  did_reference = NULL  # citation supporting did_holds
)

Literal recipe for lognormal_loc_scale (a K = 2 location-scale family on the log scale; slot 1 carries the location and slot 2 carries the log-scale):

my_lognorm <- gdpar_family_custom_K(
  name          = "my_lognormal_K2",
  stan_lpdf_id  = "lognormal_loc_scale",
  did_holds     = TRUE,
  did_reference = "User declaration"
)

fit_lognorm <- gdpar(
  gdpar_bf(y ~ a(x1), sigma ~ a(x2)),
  data   = d,
  family = my_lognorm,
  chains = 2L, iter_warmup = 400L, iter_sampling = 400L,
  refresh = 0L
)

The user is responsible for asserting that the chosen pattern is identifiable in its parameter (did_holds = TRUE); the package does not test identifiability from data, only registers the declaration. Attempting to use an unregistered stan_lpdf_id aborts with a gdpar_input_error that enumerates the allowed patterns. The general (K = 1) custom-family constructor gdpar_family_custom() is documented in §6 below.

2.5. Prediction

# In-sample prediction (theta_i_k draws)
pred_in <- predict(fit_K2, summary = "mean_se")
str(pred_in, max.level = 1L)

# Out-of-sample prediction on new covariates
new_d <- data.frame(x1 = c(-1, 0, 1), x2 = c(-1, 0, 1))
pred_new <- predict(fit_K2, newdata = new_d, summary = "mean_se")
str(pred_new, max.level = 1L)

Three points of contract:


3. Residual diagnostics: G1 / G2 / G3

gdpar provides three complementary layers of residual diagnostics (sub-phase 8.3.9, decision D4 ranqueada por máxima robustez):

3.1. API

# G1: deviance and Pearson (frequentist canonical)
r_dev  <- residuals(fit_K2, type = "deviance")
r_pear <- residuals(fit_K2, type = "pearson")

# G2: Bayesian quantile residuals (Dunn-Smyth)
r_q <- residuals(fit_K2, type = "quantile", randomize_seed = 1L)

# Response residuals (y_obs - mean of y_pred draws)
r_resp <- residuals(fit_K2, type = "response")

head(data.frame(deviance = r_dev, pearson = r_pear,
                quantile = r_q, response = r_resp))

The signature residuals.gdpar_fit(object, type, coord = NULL, randomize_seed = NULL, ...) lets the user pin the randomisation seed for reproducible G2 residuals across runs. For multi-coordinate fits (p > 1), coord selects which coordinate is summarised.

3.2. Posterior predictive draws and PPC

# Posterior-predictive draws (S x n matrix for K=1 or K>1 with p=1)
pp <- gdpar_posterior_predict(fit_K2)
dim(pp)

# Visual PPCs via bayesplot::pp_check generic
if (requireNamespace("bayesplot", quietly = TRUE)) {
  pp_check(fit_K2, type = "dens_overlay", ndraws = 30L)
}

gdpar_posterior_predict is the exported posterior-predictive draws extractor; pp_check.gdpar_fit is an S3 method off the bayesplot::pp_check generic and supports five PPC types: dens_overlay, hist, ecdf_overlay, stat, intervals. Loading bayesplot makes pp_check(fit_K2) work directly; without it the user can still call pp_check.gdpar_fit(fit_K2) if bayesplot is installed.


4. DHARMa integration (optional)

DHARMa (Hartig 2024) is a popular R package for residual diagnostics that simulates from the fitted model and constructs scaled residuals on \([0, 1]\) for diagnostic plots and formal tests (uniformity, dispersion, outliers, zero-inflation). gdpar integrates with DHARMa via the gdpar_dharma_object() exported function, which constructs a DHARMa simulationOutput from a gdpar_fit. Two points of contract:

4.1. API

dh <- gdpar_dharma_object(fit_K2)
class(dh)
#> [1] "DHARMa"
DHARMa::testResiduals(dh)

#> $uniformity
#> 
#>  Asymptotic one-sample Kolmogorov-Smirnov test
#> 
#> data:  simulationOutput$scaledResiduals
#> D = 0.085, p-value = 0.7789
#> alternative hypothesis: two-sided
#> 
#> 
#> $dispersion
#> 
#>  DHARMa nonparametric dispersion test via sd of residuals fitted vs.
#>  simulated
#> 
#> data:  simulationOutput
#> dispersion = 1.0076, p-value = 0.86
#> alternative hypothesis: two.sided
#> 
#> 
#> $outliers
#> 
#>  DHARMa outlier test based on exact binomial test with approximate
#>  expectations
#> 
#> data:  simulationOutput
#> outliers at both margin(s) = 0, observations = 60, p-value = 1
#> alternative hypothesis: true probability of success is not equal to 0.004987531
#> 95 percent confidence interval:
#>  0.00000000 0.05962949
#> sample estimates:
#> frequency of outliers (expected: 0.00498753117206983 ) 
#>                                                      0

The returned object is a standard DHARMa::createDHARMa() simulationOutput with:

All DHARMa post-processing functions (testUniformity, testDispersion, testOutliers, testZeroInflation, plotResiduals, plotQQunif) work off this object.

4.2. When to use DHARMa vs the built-in G2

The two paths agree on methodology (Bayesian randomized quantile residuals à la Dunn-Smyth 1996). They differ in scope:

Both paths are reproducible: pass randomize_seed to residuals() or set set.seed() before gdpar_dharma_object().


5. Worked example: zero-inflated negative binomial (K = 3)

This example exercises both the mixture-likelihood path of sub-phase 8.3.6 and the residual / DHARMa workflow on a tri-parametric K = 3 family.

set.seed(515L)
n <- 120L
x1 <- rnorm(n); x2 <- rnorm(n); x3 <- rnorm(n)
mu_eta    <- 1.0 + 0.5 * (x1 - mean(x1))
log_phi   <- -0.3 + 0.2 * (x2 - mean(x2))
logit_pi  <- -1.0 + 0.6 * (x3 - mean(x3))
mu_true   <- exp(mu_eta)
phi_true  <- exp(log_phi)
pi_true   <- 1 / (1 + exp(-logit_pi))
zero_struc <- rbinom(n, 1, pi_true)
y_count    <- rnbinom(n, size = phi_true, mu = mu_true)
y <- ifelse(zero_struc == 1L, 0L, y_count)
d <- data.frame(y = y, x1 = x1, x2 = x2, x3 = x3)

fit_zinb <- gdpar(
  gdpar_bf(y ~ a(x1), phi ~ a(x2), pi ~ a(x3)),
  data   = d,
  family = gdpar_family("zinb"),
  chains = 2L, iter_warmup = 600L, iter_sampling = 600L,
  refresh = 0L
)

# Per-slot coefficient summary
co <- coef(fit_zinb)
names(co)
co$mu
co$pi
# G2 quantile residuals — robust to mixture structure when jittering
# discrete responses is enabled (default for ZIP/ZINB/hurdle).
r_q <- residuals(fit_zinb, type = "quantile", randomize_seed = 99L)
hist(r_q, breaks = 20L,
     main = "Bayesian quantile residuals — ZINB K=3",
     xlab = "residual")

# DHARMa-side diagnostics if available
if (requireNamespace("DHARMa", quietly = TRUE)) {
  dh <- gdpar_dharma_object(fit_zinb)
  DHARMa::testZeroInflation(dh)
}

For ZIP / ZINB / Hurdle families, gdpar documents the parametrization of pi (zero-inflation / hurdle probability) in the logit scale and the default vectorised prior normal(0, 2.5) per the canonical decision D6 of sub-phase 8.3.6. The pi slot’s coef() output reports the per-term posterior of the AMM acting on logit_pi.


6. Custom family registry: gdpar_family_custom() (K = 1)

The complement to gdpar_family_custom_K() of §2.4 is the K = 1 constructor gdpar_family_custom(): it builds a fully user-defined family for the legacy single-slot path, where the user supplies the Stan likelihood, the log_lik block (consumed by gdpar_loo()), and the posterior-predictive block (consumed by PPC utilities). Unlike the K-side custom registry, the K = 1 constructor does not draw from a curated whitelist of patterns: any mathematically valid likelihood can be passed verbatim, and the user assumes responsibility both for correctness of the Stan code and for the declaration of identifiability.

Signature:

gdpar_family_custom(
  name,                 # character scalar; must not collide with a built-in
  link,                 # one of "identity", "log", "logit"
  did_holds,            # logical; explicit user declaration of D-ID
  did_condition,        # character scalar (NA_character_ if unconditional)
  stan_loglik_block,    # Stan snippet for the model block (per-observation
                        # target += ... ; references eta[i] and y_real[i] or
                        # y_int[i] per y_type)
  stan_log_lik_block,   # Stan snippet for generated quantities log_lik[i]
  stan_y_pred_block,    # Stan snippet for generated quantities y_pred[i]
  y_type,               # one of "real", "integer"
  did_reference         # citation supporting did_holds
)

Literal recipe for a custom log-Normal K = 1 family (a degenerate one-slot mirror of the lognormal_loc_scale pattern of §2.4):

my_family <- gdpar_family_custom(
  name               = "my_log_normal",
  link               = "log",
  did_holds          = TRUE,
  did_condition      = NA_character_,
  stan_loglik_block  =
    "target += normal_lpdf(log(y_real[i]) | eta[i], sigma_y[1]);",
  stan_log_lik_block =
    "log_lik[i] = normal_lpdf(log(y_real[i]) | eta[i], sigma_y[1]);",
  stan_y_pred_block  =
    "y_pred[i] = exp(normal_rng(eta[i], sigma_y[1]));",
  y_type             = "real",
  did_reference      = "User declaration"
)

The package emits an informational message restating the two user responsibilities (likelihood correctness and identifiability) every time a custom family is constructed. See ?gdpar_family_custom for the full Roxygen and Lemma 1B in Block 1 (§6.4) for the methodological backing of the D-ID declaration.


7. Known limitations and future work


References

mirror server hosted at Truenetwork, Russian Federation.