---
title: Loop over all stages aka loopy
---

We have a big ol loop at the heart of `tune_grid()` (and thus the rest of the tuning/resampling functions). That can be a lot to hold in your head at once, so here's a memory aid.

## Overview

We need to work our way through a whole lot of resamples and tuning parameter combinations. We could take the whole grid of tuning parameters, splice them into the workflow, and fit all those workflows on all resamples. This repeats potentially costly calculations, so we have carefully crafted the loop to avoid redundant computation.

The loop runs over a single resample. There are two paths depending on whether metrics are **prediction-based** (standard yardstick metrics) or **model-based** (metrics computed directly from the fitted workflow, e.g. clustering).

### Prediction-based metrics path (default)

- For each preproc parameter combination
	- Fit preprocessor
	- Apply it to analysis set
	- Apply it to the assessment set
	- Apply it to the calibration set
	- For each (non-sub) model parameter combination
		- Fit the model
		- Predict for all submodel parameters at once (via multi_predict)
		- Predict on calibration data for all submodels at once
		- For each submodel parameter value
			- Filter to this submodel's predictions
			- Filter to this submodel's calibration predictions
			- For each post parameter combination
				- Fit the postprocessor on the calibration predictions
				- Apply the postprocessor to the predictions (on the assessment set)
				- Combine (post-processed) predictions with the grid to `final_pred`
				- Save `final_pred` by appending it to `pred_reserve`
				- Do the extracts
- Compute the metrics from `pred_reserve`

### Model-based metrics path (e.g. clustering)

Used when `static$metric_type == "model"`. Metrics are computed directly from the fitted workflow and the assessment data, bypassing the predict/submodel/post stages entirely.

- For each preproc parameter combination
	- Fit preprocessor
	- Apply it to analysis set
	- For each model parameter combination
		- Fit the model
		- Compute metrics directly: `metrics(workflow, new_data = pred_data)`
		- Save metrics to `metrics_reserve`
		- Do the extracts
- Collect metrics from `metrics_reserve`

## Inputs

### The schedule

The schedule is a nested tibble created by `schedule_grid()` that organizes tuning parameters by stage. Each stage contains the next stage in a list-column:

```
sched
├── [preproc params]
└── model_stage (list-col)
    └── tibble
        ├── [model params]
        └── predict_stage (list-col)
            └── tibble
                ├── [submodel params]
                └── post_stage (list-col)
                    └── tibble
                        └── [post params]
```

The four stages are:

- **pre**: preprocessing via recipes
- **model**: the model fit via parsnip (submodel parameters are collapsed via `min_grid()`)
- **predict**: prediction (submodel parameters are expanded)
- **post**: postprocessing via tailor

To access the next stage, extract with `[[1]]`:

- `current_sched_pre$model_stage[[1]]` → tibble of model param combinations
- `current_sched_model$predict_stage[[1]]` → tibble of submodel param values
- `current_sched_pred$post_stage[[1]]` → tibble of post param combinations

For model-based metrics (e.g. clustering), the predict and post stages are never entered.

### The `static` object

The `static` list contains everything that stays constant throughout the loop:

- `wflow` - the original workflow (template for finalization)
- `param_info` - parameter set info from `tune_args()`
- `configs` - tibble mapping parameter values to `.config` labels
- `metrics` - the metric set
- `metric_type` - `"prediction"` for standard yardstick metrics, `"model"` for metrics computed directly from the fitted workflow (e.g. clustering)
- `pred_types` - prediction types needed (e.g., "class", "prob", "numeric"); empty for model-based metrics
- `eval_time` - evaluation times for survival models
- `control` - control options
- `data` - list with `fit`, `pred`, and `cal` data partitions (added after setup)
- `y_name` - outcome column name(s); not set for model-based metrics

### Data partitions

The `static$data` list contains three partitions (set up once per resample):

- `fit` - training data for preprocessor and model
- `pred` - assessment data for predictions (used to compute metrics)
- `cal` - calibration data for postprocessors that need fitting (e.g., probability calibration)

When there's no postprocessor requiring calibration, `cal` is NULL and `fit` uses the full analysis set. When calibration is needed, the analysis set is further split into `fit` and `cal`.

## Reading the code

### Naming conventions

**Loop variables:**

- `iter_{stage}` - iteration counter (e.g., `iter_pre`, `iter_model`, `iter_pred`, `iter_post`)
- `num_iterations_{stage}` - total iterations for that stage

**Schedule objects:**

- `sched` - the full nested schedule tibble
- `current_sched_{stage}` - current row of the schedule at each stage

**Grid objects:**

- `current_grid` - progressively accumulates tuning params as we descend into loops
- `grid_with_pre` - snapshot with pre params (before model loop)
- `grid_with_pre_model` - snapshot with pre + model params, without submodel col (before pred loop)
- `grid_with_pre_model_pred` - snapshot with pre + model + pred params (before post loop)

**Workflow snapshots** (saved to allow re-finalization in inner loops):

- `current_wflow` - the workflow being modified
- `wflow_with_fitted_pre` - snapshot after fitting preprocessor
- `wflow_with_fitted_pre_and_model` - snapshot after fitting model

**Prediction objects (prediction-based metrics path):**

- `pred_data` - processed prediction data (features + outcomes)
- `pred_all_submodels` - batched predictions for all submodel values (source for filtering)
- `current_pred` - predictions for current pred iteration (filtered from `pred_all_submodels`)
- `final_pred` - predictions after postprocessing (ready to save)
- `pred_reserve` - accumulator for all final predictions

**Metric objects (model-based metrics path):**

- `metrics_reserve` - accumulator for metrics computed directly per model fit

**Calibration objects:**

- `cal_pred_data` - processed calibration data (features + outcomes), analogous to `pred_data`
- `cal_pred_all_submodels` - batched calibration predictions for all submodel values
- `current_cal_pred` - calibration predictions for current pred iteration (filtered from `cal_pred_all_submodels`)

**General conventions:**

- `static` - things that don't change during the loop
- `current_*` - value for the current iteration
- `*_all_submodels` - batched values for all submodel params (e.g., `pred_all_submodels`)
- Stage suffixes: `_pre`, `_model`, `_pred`, `_post`

### Key helper functions

**Finalization** (splice tuning params into workflow and fit):

- `finalize_fit_pre()` - finalize recipe params, fit preprocessor
- `finalize_fit_model()` - finalize model params, fit model
- `finalize_fit_post()` - finalize tailor params, fit postprocessor on pre-computed predictions
- `fit_post_from_predictions()` - fit the postprocessor from predictions (instead of raw data)

**Grid helpers:**

- `extend_grid()` - extend a grid with params from a schedule row (strips `*_stage` columns)
- `remove_stage()` - remove nested stage columns from a schedule row

**Prediction:**

- `process_prediction_data()` - apply fitted preprocessor to data (`source = "pred"` for assessment, `source = "cal"` for calibration)
- `predict_all_types()` - generate all needed prediction types (`source = "pred"` for assessment, `source = "cal"` for calibration)

### Error handling

The loop uses a consistent error handling pattern:

```r
result <- .catch_and_log(
  some_operation(),
  control = static$control,
  split_labels = split_labs,
  location = location,
  notes = notes
)

if (is_failure(result)) {
  next
}
result <- remove_log_notes(result)
```

- `.catch_and_log()` wraps operations to capture errors/warnings without stopping
- `is_failure()` checks if the operation failed
- `next` skips to the next iteration (the failed config won't have results)
- `remove_log_notes()` strips logging metadata from successful results

## Efficiency

The nested structure avoids redundant computation:

### Prediction-based metrics

| What | Computed | Reused for |
|------|----------|------------|
| Preprocessor fit | Once per preproc param combo | All model params below it |
| Processed prediction data | Once per preproc param combo | All predictions below it |
| Model fit | Once per model param combo | All submodel predictions |
| Submodel predictions | Once per model (batched) | All submodel × post combos |
| Calibration predictions | Once per model (batched) | All submodel × post combos |
| Postprocessor fit | Once per post param combo | That specific config |

Submodel parameters (like `penalty` in glmnet) are predicted all at once using `multi_predict()`, which is much faster than predicting one at a time.

### Model-based metrics

| What | Computed | Reused for |
|------|----------|------------|
| Preprocessor fit | Once per preproc param combo | All model params below it |
| Model fit | Once per model param combo | Metric computation for that config |

There are no predictions, submodels, calibration, or postprocessing steps. Metrics are computed immediately after each model fit from the assessment data (`static$data$pred$data`).

## Debugging

Set `control = control_grid(allow_par = FALSE)` to run sequentially with `lapply()` so you can see output and use `browser()`.

For parallel debugging, `options(future.debug = TRUE)` can help.
