Introduction to statwitness

statwitness creates model-aware validation certificates across computational integrity, numerical stability, design adequacy, assumption screening, and influence stability. The package changes inputs in ways that should have known consequences, refits the model, and compares the observed response with the registered expectation.

Linear model

library(statwitness)
dat <- transform(mtcars, transmission = factor(am))
audit <- statwitness(mpg ~ transmission + wt + hp, data = dat, focus = "transmission")
audit
#> STATWITNESS VALIDATION CERTIFICATE
#> ============================================================
#> Model type:   linear_model
#> Formula:      mpg ~ transmission + wt + hp
#> Focus:        transmission1
#> Observations: 32
#> Audit level:  standard
#> 
#> PRIMARY RESULT
#> ------------------------------------------------------------
#> Estimate:     2.084
#> 95% CI:      [-0.7358, 4.903]
#> p-value:      0.1413
#> 
#> AUDIT DOMAINS
#> ------------------------------------------------------------
#> [PASS] Computational integrity - 12 passed, 0 review item(s)
#> [PASS] Numerical stability - 2 passed, 0 review item(s)
#> [PASS] Design adequacy - 1 passed, 0 review item(s)
#> [PASS] Assumption screening - 2 passed, 0 review item(s)
#> [REVIEW] Influence stability - 0 passed, 1 review item(s)
#> 
#> FINDINGS REQUIRING REVIEW
#> ------------------------------------------------------------
#> [WARN] Case-level influence screening [Influence stability]
#>   maximum Cook's distance = 0.3339; cases above 4/n = 4 of 32
#>   Potentially influential observations were detected; inspect case-deletion estimates and data quality.
#> 
#> CERTIFICATE STATUS: WARNING
#> Certificate ID:     SW-3CD9D5D893CD
#> Data fingerprint:   49488cfd52cfedbaa0b88aad2c35b232
#> Analysis fingerprint: 3cd9d5d893cd921c4c17186fa7a1dd2f
#> ------------------------------------------------------------
#> The certificate evaluates computational behavior and selected diagnostics under registered transformations.

Mixed-effects model

The following example runs only when lme4 is installed.

dat <- lme4::sleepstudy
dat$criterion <- factor(rep(c("A", "B"), length.out = nrow(dat)))
dat$model <- factor(rep(c("M1", "M2"), each = 2, length.out = nrow(dat)))
dat$rater <- dat$Subject

audit <- statwitness(
  Reaction ~ criterion * model + (1 | rater),
  data = dat,
  focus = "criterion:model",
  exclude = "optimizer_agreement",
  audit_level = "core"
)
audit
#> STATWITNESS VALIDATION CERTIFICATE
#> ============================================================
#> Model type:   linear_mixed_model
#> Formula:      Reaction ~ criterion * model + (1 | rater)
#> Focus:        criterion:model
#> Observations: 180
#> Audit level:  core
#> 
#> PRIMARY RESULT
#> ------------------------------------------------------------
#> Omnibus term: criterion:model
#> Test method:  full-versus-reduced likelihood-ratio test
#> Statistic:    0.001484
#> Degrees df:   1.000
#> p-value:      0.9693
#> 
#> AUDIT DOMAINS
#> ------------------------------------------------------------
#> [PASS] Computational integrity - 8 passed, 0 review item(s)
#> [PASS] Numerical stability - 4 passed, 0 review item(s)
#> [REVIEW] Design adequacy - 1 passed, 1 review item(s)
#> [N/A] Assumption screening - 0 passed, 0 review item(s)
#> [N/A] Influence stability - 0 passed, 0 review item(s)
#> 
#> FINDINGS REQUIRING REVIEW
#> ------------------------------------------------------------
#> [WARN] Random-effects design audit [Design adequacy]
#>   criterion varies within rater for 100% of groups but has no registered random slope; model varies within rater for 100% of groups but has no registered random slope
#>   This is a design review item, not an automatic instruction to add every possible random slope. Compare scientifically defensible random-effects structures.
#> 
#> CERTIFICATE STATUS: WARNING
#> Certificate ID:     SW-A8882C836FF5
#> Data fingerprint:   056c99dd6636cb2ce8b41605b317461e
#> Analysis fingerprint: a8882c836ff5d23f9ea677faa92d1eb6
#> ------------------------------------------------------------
#> The certificate evaluates computational behavior and selected diagnostics under registered transformations.

The interaction is treated as an omnibus term. Standard mixed-model audits also inspect fixed-design conditioning, convergence, singularity, random-effect boundaries, alternative-optimizer agreement, possible omitted within-rater random slopes, residual behavior, and feasible leave-one-rater-out influence.

Repeated-measures ANOVA

Use statwitness_repeated() with long-format data. It requires afex and applies the same audit domains with repeated-design-specific checks.

statwitness_repeated(
  data = dataset,
  outcome = "score",
  id = "rater",
  within = c("criterion", "model"),
  focus = "criterion:model",
  audit_level = "standard"
)

Scope

A passing certificate validates behavior under registered computational tests. It does not prove that the design, estimand, causal interpretation, or data-generating assumptions are scientifically correct.