From raw sample to final weights

New here? This is the 15-minute tour: what problem weightflow solves and the shortest recipe that produces analysis weights. For the statistical reasoning behind each adjustment, read Staged survey weighting: the adjustment logic.

The problem is not the weights

Every survey organization has weighting scripts. They grow over time, absorb new adjustments, and eventually become difficult to understand, maintain and reproduce. Two analysts can implement the same weighting strategy in different ways, which turns audits, updates and methodological reviews into archaeology.

The weights themselves are rarely the hard part. The hard part is that the strategy (the sequence of methodological decisions that produced the weights) is scattered across files, intermediate objects and the memory of whoever ran them last. When the responsible methodologist moves on, the knowledge often leaves with them.

A typical pipeline looks like this:

Weighting spread across scripts and intermediate files Sampling feeds script1.R, which writes weights1.csv, read by script2.R, which writes weights2.csv, read by script_final_v7.R, which writes final_weights_NEW.csv. The steps zigzag between code files and intermediate CSVs. sampling script1.R weights1.csv script2.R weights2.csv script_final_v7.R final_weights_NEW.csv Logic scattered across files and run order, not kept in one place.

Anyone who has worked in a statistical office recognizes this immediately. Along the way you typically reach for many different packages, spread the logic across several scripts, and generate hundreds of lines of code. The methodology is real, but it is implicit: hidden in file order, column names, intermediate files and undocumented tweaks.

A map of what the steps are for

Before we look at any code, one picture. Weights exist because the realized sample differs from the population we want to describe, in specific, nameable ways. Each region below is corrected by a specific step.

From target population to respondents Two overlapping ellipses show the target population and the sampling frame, with undercoverage, an in-scope overlap, and out-of-scope units. The in-scope area is sampled; the sample splits into unknown eligibility, ineligible and eligible, and eligible splits into nonrespondents and respondents. A legend maps each region to a weightflow step. Target population Frame Undercoverage (missing from frame) In scope Out of scope (ineligible on frame) draw sample (design weight 1/π) Sample (known inclusion probability π) Unknown eligibility Ineligible Eligible Non- respondents Respondents Unknown eligibility → step_unknown_eligibility(): redistribute their weight Ineligible → step_drop_ineligible(): remove from the sample Nonrespondents → step_nonresponse(): respondents absorb their weight Respondents carry the final analysis weight Coverage & known totals → step_calibrate(): align to the population

The frame and the population overlap rather than nest: some of the target population is missing from the frame (undercoverage) and some frame units are out of scope (overcoverage). From the sample downward it is pure nesting (sample ⊃ eligible ⊃ respondents), and each split is a step. The weights are not conjured; they are the consequence of this sequence of decisions.

The same strategy as one object

weightflow expresses the whole strategy as a single, explicit recipe. Nothing is hidden in a script; nothing lives in an intermediate CSV. Every methodological decision is a step_*() you can read top to bottom:

The whole strategy as one recipe object A single container holds weighting_spec() and a stack of step functions: unknown eligibility, drop ineligible, select within, nonresponse, calibrate and trim. An arrow leaves the container to prep(), which produces the final weights. One object: the whole strategy, defined lazily, read top to bottom. weighting_spec(data, base_weights) # design weights 1/π step_unknown_eligibility() # resolve unknown eligibility step_drop_ineligible() # remove out-of-scope units step_select_within() # within-household selection step_nonresponse() # correct for nonresponse step_calibrate() # align to population totals step_trim() # tame extreme weights prep() final weights one output of the object

weightflow doesn’t just compute weights: it documents how they are constructed.

It does compute the weights, of course. But a weighting strategy is a sequence of methodological decisions, and weightflow turns those decisions into an explicit object that can be inspected, reproduced, modified and reused. The final weights are only one output of that object, and usually the least valuable one. The recipe is the institutional asset.

If you know the recipes/tidymodels world, the analogy is deliberate: a recipe describes how a dish is produced, not the dish itself. A weightflow recipe describes how the weights are produced, not the weights themselves.

The shortest real recipe

Enough theory. weightflow ships a small bundled example (a stratified household sample, sample_survey, drawn from a known population), so every line below actually runs.

str(sample_survey[, c("pw", "region", "sex", "unknown_elig", "responded")])
#> 'data.frame':    467 obs. of  5 variables:
#>  $ pw          : num  12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 12.5 ...
#>  $ region      : Factor w/ 4 levels "North","South",..: 1 1 1 1 1 1 1 1 1 1 ...
#>  $ sex         : Factor w/ 2 levels "F","M": 1 1 2 2 2 2 1 1 1 1 ...
#>  $ unknown_elig: int  0 0 0 0 0 0 0 0 0 0 ...
#>  $ responded   : num  1 0 0 0 1 1 0 1 1 1 ...

A minimal but complete cascade: start from the design weights, resolve unknown eligibility, correct for nonresponse, and calibrate to the population totals of region and sex. We pass those totals in the tidy form: a small data frame per margin with the categories and a counts column. table() already produces exactly that shape (its counts column is named Freq):

m_region <- as.data.frame(table(region = population$region))
m_sex    <- as.data.frame(table(sex    = population$sex))
m_region
#>   region Freq
#> 1  North 1570
#> 2  South 1250
#> 3   East  927
#> 4   West  748
fit <- weighting_spec(sample_survey, base_weights = pw) |>
  step_unknown_eligibility(unknown = unknown_elig, by = "region") |>
  step_nonresponse(respondent = responded, method = "weighting_class",
                   by = "region") |>
  step_calibrate(method = "raking",
                 totals = list(m_region, m_sex), count = "Freq") |>
  prep()

prep() is where the recipe is actually estimated. Everything before it only records the strategy. collect_weights() returns the data with the final weight attached as .weight:

w <- collect_weights(fit)
summary(w$.weight)
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>   10.69   12.85   18.07   16.65   19.35   20.94

The object is the documentation

Because the strategy is an object, it explains itself. summary() walks the cascade step by step, reporting how many units each stage touched, how weights changed, and diagnostics such as the design effect:

summary(fit)
#> 
#> == Weighting specification (weightflow) ==
#> Data    : 467 cases
#> Base wts: pw
#> Steps   :
#>   1. unknown eligibility
#>   2. nonresponse (weighting class)
#>   3. calibration (raking)
#> Status  : estimated (prep)
#> 
#> Stage summary:
#>                             stage n_active sum_wts cv_wts deff_kish n_eff
#>                              base      467    4371  0.236     1.056   442
#>  stage_1_step_unknown_eligibility      449    4371  0.250     1.063   423
#>          stage_2_step_nonresponse      270    4371  0.144     1.021   265
#>            stage_3_step_calibrate      270    4495  0.211     1.045   258
#> 
#> deff_kish = 1 + CV^2 (Kish design effect from unequal weighting);
#> n_eff = n_active / deff_kish. Both worsen with each adjustment and
#> improve with trimming.
#> 
#> --- Step 1: unknown eligibility ---
#>   cell  level n_known n_unknown   factor
#>   East person      96         0 1.000000
#>  North person     113         6 1.053097
#>  South person     113         8 1.070796
#>   West person     127         4 1.031496
#> Kish deff: 1.056 -> 1.063   |   n_eff: 442 -> 423
#> 
#> --- Step 2: nonresponse (weighting class) ---
#>   cell n_respondents n_nonresponse   factor
#>   East            52            44 1.846154
#>  North            78            35 1.448718
#>  South            72            41 1.569444
#>   West            68            59 1.867647
#> Kish deff: 1.063 -> 1.021   |   n_eff: 423 -> 265
#> 
#> --- Step 3: calibration (raking) ---
#>  variable category target achieved
#>    region     East    927      927
#>    region    North   1570     1570
#>    region    South   1250     1250
#>    region     West    748      748
#>       sex        F   2311     2311
#>       sex        M   2184     2184
#> (converged/iterated in 5 iterations)
#> Kish deff: 1.021 -> 1.045   |   n_eff: 265 -> 258
#> 
#> R-indicator (representativity of response): 0.869  (on region)

That printout is the methodological record. You did not write a separate memo describing what the scripts did; the recipe and its summary are the memo. Re-running it next year, or handing it to a colleague, reproduces the exact same weights and the exact same explanation.

Where to go next

You now have runnable weights and a mental map. To go deeper: