mfrmr Visual Diagnostics

Establish what higher scores mean

A logit value needs a rubric direction and a model role. For comprehensibility, 1 = difficult / 4 = easy makes higher Person locations mean easier speech; 1 = easy / 4 = difficult makes them mean more difficult speech. In both cases, higher Person locations predict higher coded scores. With the default negative Rater orientation, higher Rater locations predict lower coded scores, so “stricter” is appropriate only when higher scores are more favorable.

Record both endpoint labels with the analysis. positive_facets changes the orientation of the selected facet parameters; it does not reverse the outcome categories. Renaming categories in a figure also leaves the model unchanged. The visual-diagnostics help explains recoding, anchors and the terms used below. The package can report the fitted signs but cannot infer the substantive meaning of your rubric.

Wright map

Use the Wright map first when you want one shared logit view of persons, facet levels, and step thresholds.

wright <- plot(fit, type = "wright", preset = "publication", show_ci = TRUE)

Interpretation:

• Compare person density on the left to facet and step locations on the right.
• Large gaps suggest weaker targeting in that logit region.
• Wide overlap in marginal confidence whiskers suggests imprecision; estimate the relevant pairwise contrast directly before claiming that two levels are separated or indistinguishable.

The locations used in the figure remain available as a table:

head(plot_data(wright, component = "locations"))

Read the accompanying definitions and fitted directions outside the figure:

knitr::kable(plot_data(wright, "notes"), row.names = FALSE)

To find a particular judge, use their ID from the fitted data. The > prefix marks your selection in color or monochrome; it does not flag a statistical problem. Use another facet name in the list to select its levels instead.

judge <- as.character(toy$Rater[1])
focused <- plot(fit, type = "wright", highlight = list(Rater = judge),
                draw = FALSE)
subset(plot_data(focused, "locations"), Highlighted)

Selected levels remain visible even if the compact top_n selection would omit them. They are added to the usual selection, so inspect plot_data(focused, "retention") for the displayed and omitted counts. top_n = Inf retains all levels. Selection does not recenter or refit the model, and the automatic logit range is determined before compaction.

Here Step means a transition between adjacent score categories: 1 to 2, 2 to 3, or 3 to 4. At a transition’s crossing, those two categories are equally probable; other categories remain possible, so this does not mean 50 percent each. The displayed reference sets other additive facet effects to zero. For a particular judge and criterion, their effects shift the crossing along the Person scale. A Step is not a developmental stage or a contest pass mark.

The native view above remains the recommended analytic figure because it keeps facet uncertainty and fitted step locations visible. For a closer FACETS-facing handoff, switch the renderer and leave show_ci at its default FALSE. The fitted coordinates do not change.

Every retained native facet level is labelled. Only label coordinates are displaced for collision avoidance, with leader lines back to the fitted points. Step thresholds form one vertical ladder and their labels include the fitted transition logits.

plot(
  fit,
  type = "wright",
  renderer = "facets",
  category_labels = c(
    `1` = "Beginning", `2` = "Developing", `3` = "Secure", `4` = "Advanced"
  )
)

This renderer uses one common logit ruler, a * person-frequency column, signed facet headers, every facet level, and short labeled score-transition lines. It is FACETS-style visual correspondence, not a claim that mfrmr and FACETS produce numerically identical estimates. A numerical comparison requires output from a documented FACETS version and aligned estimator, identification, score, and orientation settings. Set show_ci = TRUE only when you deliberately want a hybrid FACETS-style ruler with the mfrmr uncertainty extension. Use draw = FALSE and inspect the facets_style tables when rebuilding the display with ggplot2 or another graphics system.

Next, use the pathway map when you want to see how expected scores progress across theta.

plot(fit, type = "pathway", preset = "publication")

Interpretation:

• Steeper rises indicate stronger score progression.
• Dominant-category strips show where each category is most likely to govern the score.
• Flat or compressed regions suggest weaker category separation.

Fit pathway

The expected-score pathway is not a fit pathway. To review measure against Infit, place Infit on the horizontal axis. This focused view retains all facet levels and the 12 persons furthest from expected fit. It is a review screen, not a representative sample of person fit. Labels are limited to flagged facet levels; the table below identifies every displayed row.

fit_pathway <- plot(
  fit,
  type = "fit_pathway",
  diagnostics = diag,
  fit_stat = "Infit",
  fit_scale = "mnsq",
  include_person = TRUE,
  top_n_person = 12,
  person_labels = "none",
  facet_labels = "flagged",
  show_ci = TRUE,
  preset = "publication"
)

Interpretation:

• The vertical axis remains the fitted measure in logits.
• The horizontal axis is Infit MnSq; the 1.0 line is the model-expectation reference.
• Vertical whiskers show measure uncertainty; person and non-person rows have distinct uncertainty bases, recorded in the draw-free payload metadata.
• Treat displaced or flagged rows as review prompts, not automatic exclusions.

Overfit means less residual variation than the model expects (mean square below 1); underfit means more (above 1). The selected bands determine screening flags, so not every deviation from 1 triggers a flag. Infit and Outfit weight residuals differently; ZSTD has a zero-centered reference. Calibrated speech-contest judges can agree or have similar severity without overfit. If low fit values occur, inspect actual residual patterns, category use and rating independence before attributing them to training. Shared discussion/copying and consistent independent rubric use are different processes. Neither a low nor a high value alone establishes the cause or justifies removing a judge. See the linked help for the definitions and sources.

fit_values <- plot_data(fit_pathway, component = "table")
fit_values[, c("Facet", "Level", "FitValue", "Measure", "CIEligible")]

CIEligible = FALSE means that a location interval is unavailable under the stored inference rules, not that its uncertainty is zero. Inspect CIBasis and CIUse in the same table before interpreting whiskers. Set top_n_person = Inf to retain all persons, or include_person = FALSE to focus on facet levels. A crowded view can be read through its table without changing the model or the fit-screening rules.

Category probabilities

Category curves ask which ordered score is most probable at each ability. They complement the expected-score pathway: a single expected score can hide which categories contribute to it.

category_curves <- plot(fit, type = "ccc", preset = "monochrome")

These are reference-profile probabilities, not observed category frequencies or uncertainty bands. For PCM/GPCM, select and interpret the relevant step/slope owner. A rarely dominant category is a reason to inspect counts and rubric meaning, not automatic evidence that categories should be merged.

head(plot_data(category_curves, component = "probabilities"))
# Optional customization without refitting:
# as_ggplot(category_curves)

For observed counts and stored step points, save rating_scale_table(fit, diag) and use plot(result, type = "counts") or type = "thresholds"; both support as_ggplot(). Counts in that figure are pooled: inspect owner_category_table for each step owner’s denominator and category support. Stored step SEs are not curve or gap intervals. The separate category_curves_report() bundle uses base graphics for its multi-family overview and selected curve views; custom graphics from plot_data() must retain the curve family and group.

Strict marginal follow-up

When you need the package’s latent-integrated follow-up path, switch to MML and request diagnostic_mode = "both" so the legacy and strict branches stay visible side by side. The chunk below uses compact quadrature for a shorter runtime; final reporting should be based on a refit with the package default or a higher quadrature setting.

fit_strict <- fit_mfrm(
  toy,
  person = "Person",
  facets = c("Rater", "Criterion"),
  score = "Score",
  method = "MML",
  model = "RSM",
  quad_points = 7,
  maxit = 40
)

diag_strict <- diagnose_mfrm(
  fit_strict,
  residual_pca = "none",
  diagnostic_mode = "both"
)

strict_checklist <- reporting_checklist(fit_strict, diagnostics = diag_strict)
subset(
  strict_checklist$checklist,
  Section == "Visual Displays" &
    Item %in% c("QC / facet dashboard", "Strict marginal visuals"),
  c("Item", "Available", "NextAction")
)

plot(diag_strict, type = "marginal_fit", top_n = 12, preset = "publication")

Interpretation:

• Treat strict marginal plots as exploratory corroboration screens, not as standalone inferential tests.
• Use the checklist rows to confirm that the current run actually supports the strict branch before routing figures into a report.
• When pairwise follow-up is needed, continue with plot(diag_strict, type = "marginal_pairwise", preset = "publication").

Checking one substantive dimension without fitting a multidimensional model

A single intended construct is a substantive hypothesis, not a result of choosing a one-dimensional estimator. Ask whether one ability provides a useful account of the ratings for the intended score and decision. Also ask whether remaining associations arise from common tasks, performances, assessors or category use. Several slopes do not create several abilities, and one ability plus a testlet effect is not the same hypothesis as two substantive abilities.

Current diagnostic entry points are available, but there is no dedicated API that certifies unidimensionality or selects a latent dimension count:

A supported first-pass workflow, with the paired facet chosen to match the question, is:

diag <- diagnose_mfrm(fit, residual_pca = "none", diagnostic_mode = "both")
pca <- analyze_residual_pca(diag, mode = "both", parallel = TRUE,
                           parallel_reps = 200, seed = 9028)
summary(pca)
# Replace Criterion with the fitted facet relevant to the question.
q3 <- q3_statistic(fit, diagnostics = diag, facet = "Criterion")
q3$pairs
q3$notes

plot(pca, type = "scree") draws the saved overall PCA; plot(pca, type = "loadings", mode = "facet", facet = "Criterion") uses its saved facet analysis. Preserve the full pca result with saveRDS() so its reference, unavailable scopes and warnings accompany the figure. There is no dedicated plot(q3) method: obtain the companion heatmap from fit with the same diagnostics, facet and min_pairs, and save its draw-free payload. as_ggplot() can then convert that payload without recalculating residuals; plot_data(p, "notes") retains the qualifications outside the figure.

Two hundred permutations are an example setting, not a qualification threshold. Check missing comparisons, overlap counts, aggregation and the reported reference method before interpreting a cutoff. A small first component or few flags can reflect low power or restricted exposure. A large component does not identify its cause. Neither counting eigenvalues above one/two nor adding one to the number of residual components estimates the number of substantive abilities. The extended testlet/shared-rater models use their own posterior response diagnostics; the ordinary residual-PCA/Q3 contract is not inherited.

For a supported RSM/PCM MML fit, a second reference asks: would similar residual structure arise when this fitted one-ability model generates the ratings and is estimated again? The Person’s generated ability is shared across their rating rows; it is not redrawn separately for every rating.

# fit must be an RSM/PCM MML fit with fixed standard-normal population,
# fixed quadrature, additive facets, unit weights, and no anchors or shrinkage.
reference <- analyze_residual_pca(
  fit, mode = "both", parallel = TRUE,
  parallel_method = "model_bootstrap", parallel_reps = 200, seed = 9028
)
reference$bootstrap_settings
reference$bootstrap_trials       # All attempts, including failures and warnings
reference$parallel_status        # Availability separately for each PCA scope
plot(reference, type = "overall_parallel_scree")
# If the reference is available:
# as_ggplot(plot(reference, type = "overall_parallel_scree", draw = FALSE))

This is more expensive than permuting residuals: 200 replicates mean 200 model refits, shared across the requested overall/facet analyses. It is an example, not a universal recommended budget or an error-rate guarantee. The method currently excludes GPCM, JML, estimated population models, adaptive quadrature, anchors and extended testlet/shared-rater fits. Generating parameters are fitted point estimates; refitting propagates their sampling variation under that null.

The existing analyzed assignment is held fixed. Missing scores and unassigned ratings are not filled, so this does not test or correct a missingness mechanism. All source/replicate fits must pass numerical convergence checks. Every planned replicate is recorded; if a scope cannot be calculated in any replicate, its reference cutoff is withheld. Inspect the recorded reason instead of dropping failed replicates. Undefined or indefinite sparse correlation matrices remain unavailable. Increasing the replicate count does not repair them.

The plotted cutoff is a componentwise exploratory reference, not a multiplicity-adjusted decision across components or facets. A component above it motivates investigating task dependence, rater effects, assignment and substantive dimensions; it does not identify which explanation is correct. The first component and an any-component rule answer different questions: one can miss a departure that the other flags. Choosing whichever rule looks most striking after viewing the results changes the decision procedure. General false-flag rates and sensitivity across those scenarios still require qualification. Q3 and residual-network references are not supplied by this PCA option.

reference$parallel_status
# For a prespecified criterion-level question, if that reference is available:
plot(reference, type = "facet_parallel_scree", facet = "Criterion")

Residual networks, EGA and the assignment graph answer different questions

A graph consists of nodes and edges, but their meanings must be specified. The assignment graph records who rated what; its connectivity concerns linking. A score-correlation graph describes observed associations. A residual graph instead describes associations left after a fitted response model. A residual Pearson correlation edge and a partial-correlation edge conditional on all other residual variables are different estimands. None is automatically causal.

Exploratory Graph Analysis (EGA; Golino and Epskamp, 2017) uses estimated networks and community detection to explore dimension structure. Applying the idea to fitted residuals asks about remaining structure and is not the same procedure as EGA on the original item responses. The existing mfrm_network_analysis(), rater_network_analysis() and rater_halo_network_analysis() do not implement a residual EGA procedure. mfrm_residual_network() now supplies a native signed correlation network and optional hierarchical clustering, using standardized residual means per Person and facet level. It is not EGA, which uses a different network and community estimation procedure.

# Reuse a fitted model and its matching diagnostics.
net <- mfrm_residual_network(fit, diagnostics = diag, facet = "Criterion",
  missing = "complete", clusters = 2, min_abs = .2)
net$settings
net$pairs                    # include unavailable pairs and shared-person N
net$person_usage             # complete-person inclusion, not rating-row N
plot(net, type = "network")  # sign/strength; circular layout is not a distance
plot(net, type = "matrix")   # all correlations, before the display threshold
if (net$clustering$status == "available") plot(net, type = "dendrogram")
report <- mfrm_report(net)
export_summary_appendix(report, output_dir = "residual-review")
saveRDS(net, "residual-review/network.rds")

Clustering uses sqrt(2*(1-r)) distances between centered unit-length residual profiles on a common complete-person sample. Average or complete linkage and the supplied number of groups define an exploratory partition. Strong negative correlations are dissimilarity, not similarity by absolute value. The display threshold never changes the groups. Without a common usable sample or variation at every level, clustering is unavailable; the network retains that reason. Use missing = "pairwise" without clustering when pair-specific overlap is the intended descriptive analysis. Neither route estimates group stability.

Random forests belong to a separately designed prediction stage. Wesolowski (2019) formed rater groups by k-means on differential-rater-functioning indices and then predicted those labels with random forests. Predicting exploratory labels does not validate latent classes. The function help explains unsupervised learning, supervised prediction, leakage, resampling and why out-of-bag accuracy alone cannot validate labels discovered on all data.

For example, residual associations among accuracy criteria across different tasks and assessors may motivate a second substantive-domain hypothesis. Associations confined to one performance may instead motivate a testlet model. Associations organized by assessor or assignment panel require review of those effects. These patterns guide follow-up; the graph alone cannot distinguish all competing explanations. Communities, centrality and edge density are not proof of dimensions, rater quality or an omitted causal mechanism. No edges can also reflect regularization or weak data rather than independence.

For any residual-network workflow, specify nodes (criterion, task-by-criterion, or another justified unit), residual type, aggregation and pairwise person counts. Sparse pairwise correlations may be undefined or form an invalid correlation matrix. Do not encode unassigned cells as zero, treat a missing edge as a zero association, or silently repair a matrix and then claim the same estimand. Record regularization and community choices; examine edge and community stability with resampling appropriate to the shared effects. A single nominal sample size can misrepresent strongly unequal pair overlap.

bootEGA offers network/dimension stability analyses, but applying it to a fixed residual matrix does not propagate uncertainty from fitting the MFRM. Its empirical-correlation bootstrap is not a one-dimensional MFRM null simulation. Finally, residual network modeling (Epskamp, Rhemtulla and Borsboom, 2017) models residual dependencies jointly with latent variables. Drawing a graph from fitted residuals does not fit such a joint model or produce its likelihood. A future mfrmr network view should reuse the checked residual/overlap outputs and provide a signed heatmap/table alongside the network, with the same edge values and uncertainty. It should remain exploratory until its statistical reference is qualified.

For interaction screening, use the packaged bias example.

Here bias means an additional departure for a particular combination, such as one judge scoring Content, after accounting for the fitted baseline. It is distinct from overall rater severity or proof of unfairness. A positive Bias Size increases expected coded scores; with high-is-good scoring this favors the combination, whereas a positive default Rater severity reduces scores. Reverse-coded rubrics reverse that substantive interpretation. The profile below uses mean absolute bias, so it describes magnitude; inspect signed rows in bias$table to see direction. Its uncertainty holds the baseline calibration fixed.

bias_df <- load_mfrmr_data("example_bias")

fit_bias <- fit_mfrm(
  bias_df,
  person = "Person",
  facets = c("Rater", "Criterion"),
  score = "Score",
  method = "MML",
  model = "RSM",
  quad_points = 7
)

diag_bias <- diagnose_mfrm(fit_bias, residual_pca = "none")
bias <- estimate_bias(fit_bias, diag_bias, facet_a = "Rater", facet_b = "Criterion")

bias_figure <- plot(bias, type = "bias", view = "facet_profile", preset = "publication")

The definition, sign, magnitude-only profile and uncertainty qualifications are retained in plot_data(bias_figure, "notes") and shown by print(bias_figure).

Interpretation:

• Facet profiles are useful for seeing whether a small number of levels drives most flagged interaction cells.
• Treat these plots as screening evidence and inspect the corresponding tables, cell counts and calibration limitations. Several views of the same estimates are not independent confirmation of bias.

For repeat plotting, save bias_interaction_report(bias, ...) after choosing the review thresholds. plot(result, type = "scatter") compares the observed-minus-expected score average with conditional bias in logits; type = "ranked" shows saved interaction estimates. These two views support as_ggplot(), including conditional bounds when show_ci = TRUE. They hold calibration fixed. The other bias views use base or custom graphics.

Rater centrality and halo have different targets. rater_centrality_table() returns indirect indices in a plain list; save the list and review source status and unavailable-index reasons alongside its data frames. rater_halo_network_analysis() returns observed-score profile correlations. Its edge_distribution, selected halo_summary metric and retained-edge matrix views support as_ggplot(); use base graphics for the network. Save the whole halo result to retain its pair counts, edge filters and caveats. Neither result is automatically added to an aggregate mfrm_report(). These views do not distinguish halo from horn effects: a negative impression can lower several criterion ratings together without creating a negative correlation. For a beginner explanation of severity, centrality, halo/horn and local dependence, see help("mfrmr_methods_reference").

Save a figure with its values and notes

Saving only a PNG loses the underlying values and interpretation notes. Save the complete returned object as well. This example uses a temporary filename; choose a project filename to keep it beyond this R session. Keep the rubric alongside the plot: the labels below are the illustrative labels used earlier, not wording that mfrmr inferred from the scores. Use your actual rubric wording.

figure_file <- tempfile(fileext = ".rds")
figure_record <- list(plot = wright,
  rubric = c(`1` = "Beginning", `2` = "Developing", `3` = "Secure", `4` = "Advanced"))
saveRDS(figure_record, figure_file)
saved_record <- readRDS(figure_file)
saved_figure <- saved_record$plot
saved_record$rubric
print(saved_figure)
identical(plot_data(saved_figure, "locations"), plot_data(wright, "locations"))

For the supported Wright-map conversion, redraw the saved values without refitting or recalculating uncertainty:

as_ggplot(saved_figure)

as_ggplot() requires the optional ggplot2 package. Save the fit and matching diagnostics too if you want to choose a different analysis view later. The plot guide above distinguishes dedicated conversions, different table-based views and plots without automatic conversion. This example does not promise the same layout for every renderer or conversion of every plot family.

Set an appearance once for several plots

Use options(mfrmr.plot_preset = "monochrome") to set a common default in the development version. This example restores the previous option when it ends:

local({
  old <- options(mfrmr.plot_preset = "monochrome")
  on.exit(options(old))
  saved <- plot(fit, type = "ccc", draw = FALSE)
  special <- plot(fit, type = "ccc", preset = "publication", draw = FALSE)
  c(session_default = saved$data$preset, explicit_override = special$data$preset)
})

The choices are "standard", "publication", "compact" and "monochrome". Explicit preset wins over the option; omitting both retains "standard". Explicit preset = NULL also selects the package default. Remove the session setting with options(mfrmr.plot_preset = NULL).

A supported as_ggplot(saved) conversion uses the saved preset, whereas a new plot(fit, ...) call uses the current option. Set the option in a script or use explicit arguments to reproduce its appearance in another session. This applies to the common preset controls, including category, coverage and network report plots. Extended-model plots with separate palette controls retain their own settings. It does not change estimates, confidence levels, screening thresholds or the user’s global ggplot theme.

Use a common title argument

In the development version, eight existing helpers accept title: plot(), plot(), plot(), plot(), plot(), plot(), plot() and plot().

plain <- plot(fit, type = "bubble", diagnostics = diag, view = "infit_outfit", title = NULL, preset = "monochrome", draw = FALSE)
as_ggplot(plain)
# Reuse the saved values, screening references and interval settings.
plain$data$reference_lines

Omit title for the default heading, supply title = "Scoring patterns" to replace it, or use title = NULL (or "") to hide it. This does not remove subtitles, legends, screening settings or interpretation notes. The dashboard method also accepts plot(dashboard, title = NULL). Keep review-only limitations in the figure caption or accompanying report if its heading is hidden.

Existing main calls and positional arguments still work without deprecation warnings. main = NULL retains its earlier meaning of “use the default heading”. Do not supply both main and title, even with the same value. These aliases do not imply that every plot has an as_ggplot() converter; consult mfrmr_output_guide("plots") for the supported route.

Fair Scores and annotations outside the figure

Use view = "measure" to relate measures to fair scores, "scatter" to compare observed and fair scores, or "difference" to rank their gaps. FairM uses mean reference measures; FairZ uses zero reference measures. FairZ is an expected score on the internal score scale, not a z-score. Observed-minus-fair gaps also reflect assignment and person mix, so they do not by themselves establish rater bias.

p_fair <- plot(fit, type = "fair_average", diagnostics = diag, facet = "Rater", metric = "FairZ", view = "measure", show_ci = TRUE, preset = "monochrome", show_title = FALSE, show_notes = FALSE, draw = FALSE)
p_fair$data$notes
p_fair$data$plot_data
p_fair$data$excluded
# Requires ggplot2:
g_fair <- as_ggplot(p_fair)
g_fair
attr(g_fair, "mfrmr_notes")

Use draw = TRUE to draw the base-R version. Titles and notes remain in the returned object when hidden in the figure. The same annotation controls apply to Wright, expected-score pathway, and CCC plots; they do not change readiness or interval eligibility. Keep the applicable notes in the surrounding caption or report. Other plotting helpers have their own documented options.

Fair-score intervals remain diagnostic-only (CI_Eligible = FALSE), with full-refit coverage unverified. RSM/PCM plots propagate only the focal measure SE, holding thresholds and reference measures fixed; they require a fitted model, not just a stored fair-average bundle. GPCM-MML plots use available structural delta-method SEs for non-Person rows, conditioning on Person EAP/reference means. The table option fair_se = TRUE supplies the GPCM route, not RSM/PCM fair-score SEs. Historical table SE columns describe measures. Difference-view whiskers hold observed means fixed and are not confidence intervals for the observed-minus-fair gap.

Expected-score and CCC curves use six line types: solid, dashed, dotted, dotdash, longdash, and twodash. Fair-score plots use six point shapes. These encodings repeat for larger sets; select facets or use panels to keep a grayscale figure readable. Check labels and line separation at the intended output size rather than relying on color or gray levels alone.

Save the agreement result and its review rules

Keep the agreement table when drawing several views. Its matched-context counts and saved thresholds explain the figure; correlation describes score ordering, while exact agreement describes equal scores. Neither supplies an automatic rater-quality decision. Model-expected agreement is a fitted-model baseline, not chance agreement.

agreement <- interrater_agreement_table(fit, diagnostics = diag, rater_facet = "Rater")
p_agreement <- plot(agreement)
# Alternative saved view: plot(agreement, type = "agreement", metric = "correlation")
# Optional ggplot conversion: as_ggplot(p_agreement)

Save agreement with saveRDS() to choose another pair view after reopening, or save p_agreement to retain the selected view and its appearance. Inspect plot_data(p_agreement, "notes") for the qualifications outside the figure. Bar colors follow the combined exact/correlation flag, so a bar above its displayed threshold can still be flagged by the other rule. To change the rules, recreate agreement from the existing fit with explicit exact_warn and corr_warn; changing a reference line alone would contradict the flags.

Response-time QC context

If your rating-event data include response times, review them separately from the MFRM likelihood. Rapid and slow response-time flags are descriptive quality-control prompts; they do not change measures and should not be treated as proof of disengagement, cheating, or speededness.

toy_rt <- toy
toy_rt$ResponseTime <- 12 + (seq_len(nrow(toy_rt)) %% 7) +
  as.numeric(toy_rt$Score)
toy_rt$ResponseTime[1] <- 2
toy_rt$ResponseTime[2] <- 38

rt <- response_time_review(
  toy_rt,
  person = "Person",
  facets = c("Rater", "Criterion"),
  score = "Score",
  time = "ResponseTime",
  rapid_quantile = 0.10,
  slow_quantile = 0.90
)

summary(rt)
plot(rt, type = "distribution", preset = "publication")
plot(rt, type = "person", preset = "publication")

Interpretation:

• Start with the distribution plot to see whether the rapid/slow thresholds are sensible for this administration.
• Inspect person and facet summaries for concentrated rapid or slow rates rather than isolated events.
• Keep timing flags separate from fit, bias, and validity claims unless the study design explicitly supports stronger speed-accuracy modeling.

Small-N shrinkage with uncertainty

When a non-person facet has few levels or sparse observations, a large raw severity estimate can be a noisy estimate rather than a stable facet signal. The shrinkage funnel shows how far empirical-Bayes pooling moved each level toward the facet mean and whether the uncertainty remains wide after pooling.

fit_eb <- apply_empirical_bayes_shrinkage(fit)

shrink <- plot(fit_eb, type = "shrinkage", view = "funnel", show_ci = TRUE, ci_level = 0.95, preset = "publication", draw = FALSE)

head(shrink$data$table[, c(
  "Facet", "Level", "RawEstimate", "RawCI_Lower", "RawCI_Upper",
  "ShrunkEstimate", "ShrunkCI_Lower", "ShrunkCI_Upper",
  "ShrinkageFactor"
)])

plot(fit_eb, type = "shrinkage", view = "funnel", show_ci = TRUE, ci_level = 0.95, preset = "publication")

Interpretation:

• Long raw-to-shrunken segments identify levels most affected by the partial-pooling prior.
• Wide raw whiskers that narrow after pooling indicate estimation instability, not automatic rater-quality failure.
• Report the shrinkage method and keep this display separate from bias, fit, or validity claims.

External-feature hierarchy

A hierarchy can summarize raters’ backgrounds before discussing training. These eight fictional raters illustrate the mechanics; their groups are not estimates of severity, ability or rater quality. Choose attributes and their coding for the substantive question before interpreting a tree.

backgrounds <- data.frame(
  Rater = paste0("R", 1:8),
  Years = c(1, 2, 4, 6, 8, 10, 12, 15),
  Training = ordered(c("Basic", "Basic", "Advanced", "Basic",
    "Advanced", "Advanced", "Specialist", "Specialist"),
    levels = c("Basic", "Advanced", "Specialist")))
features <- mfrm_features(backgrounds, "Rater", c("Years", "Training"))
hierarchy <- mfrm_cluster_hierarchical(features, k = 3, linkage = "average")

hierarchy_plot <- plot(hierarchy, preset = "monochrome")

Three groups were requested; the diagram does not select that number or establish that the groups are real. Feature weights, coding and linkage can change the branches. The full tree and memberships in leaf order are available for inspection. The development version supports both the base plot and a dedicated ggplot conversion of this saved tree.

plot_data(hierarchy_plot, component = "table")
tree <- plot_data(hierarchy_plot, component = "tree")

if (requireNamespace("ggplot2", quietly = TRUE)) {
  hierarchy_figure <- as_ggplot(hierarchy_plot, component = "tree") +
    ggplot2::labs(title = NULL, subtitle = NULL)
  print(hierarchy_figure)
}

The conversion keeps the saved label setting and preset. Use labels = FALSE when creating a crowded tree to hide IDs without removing leaves. Where heights tie, boxes follow the stored merge-order groups; they need not correspond to a unique horizontal height cut. Removing titles or captions with ggplot2::labs() leaves the tree and memberships in plot_data().

For imputed features, group comparisons and PCA, continue with Exploring person, rater, and task attributes.

D-study planning

For data preparation, interpretation of plan differences and saved reopening, start with Plan an assessment with G theory. For several score criteria in a single panel, use plot(d, score = c("V", "W"), metric = "Phi"), with names from your result. Select named composites with composite = c("Equal", "Difference"). For two-facet results, add x_var = "Tasks", panel_by = "Raters", ncol = 2 to separate the rater counts into panels with shared axes and score colors; reverse x_var and panel_by to separate task counts. as_ggplot() retains this layout using facet_wrap(). Panels retain unavailable results and nested count labels such as “Raters per Task”. Add reference = 0.8 only if that is your chosen criterion; guides are not universal adequacy rules. Both plot families omit reference lines by default; plot(main_effects_d, reference = c(0.7, 0.8)) restores its earlier guides.

Use three complementary views: plot(g, weights = ...) shows the G-study’s raw sources of score variation; plot(d) and plot(d, type = "sem") show future plan projections; plot(comparison, view = "differences") shows the eligible paired differences with uncertainty. Raw components are in squared score units and retain negative estimates. They are not percentages or future-plan error contributions. G-study component and D-study projection plots support as_ggplot() for editing; retrieve cautions with plot_data(p, "notes") and preserve them outside the figure.

A D-study asks how a specified measurement design would perform if we changed facet counts. The packaged MGENOVA manual example has two scores, V and W, observed for Persons crossed with Tasks. Here the target is their equal-weight composite. There is no rater facet in this example.

tasks <- read.csv(system.file("extdata", "mgenova-table12.csv", package = "mfrmr"))
gstudy <- mfrm_g_study(tasks, c("V", "W"), rater = NULL)
planned <- mfrm_d_study(gstudy,
  data.frame(Tasks = c(3, 6, 12)), weights = c(V = 0.5, W = 0.5))

planning_plot <- plot(planned, type = "coefficients", preset = "monochrome")

G concerns relative comparisons among Persons; Phi concerns absolute score interpretation. More tasks reduce the modeled task-related error while holding the estimated covariance components and composite weights fixed. These are observed-score projections, not MFRM ability reliability or a guarantee for a new task population. Uncertainty in the estimated components is omitted.

plot_data(planning_plot, component = "table")
# Customizable conversion of this selected score/composite:
# as_ggplot(planning_plot)
# Error in score units is a different view:
# plot(planned, type = "sem")

Use ?mfrm_d_study for score/composite selection and supported crossed or nested designs. An ordinary mfrm_d_study object has its own base plot; its automatic ggplot conversion is currently unavailable.